from Inquiry, Volume 1, Number 1, Spring 1997, 49-57
© Copyright 1997 Virginia Community College System
Brief Abstract
Six faculty members at Germanna Community College conducted a pilot study of an assessment approach that provides immediate feedback on what is working in the classroom and what is not. Background on Cross and Angelo’s Classroom Assessment Techniques is included.
In 1989, the members of the Education Commission of the States, meeting on the topic of “Assessment and Accountability in Higher Education,” concluded:
The biggest shortcoming of so many state assessment initiatives is their failure to communicate a compelling set of reasons for the mandate and doing of [assessment]. . . Why are institutions being asked to do this? What real problem prompts it? Just what is supposed to happen as a result? (Assessment, 1990, p. 14)
Faculty have been asking the same question: “Just what is supposed to happen as a result of assessment?” The members of the 1989 Education Commission of the States provoide a clear response: ”Assessment has less to do with measurement and data than with a systematic asking of questions about student learning [that leads to] new evidence useful to the improvement of learning" (Assessment, 1990, p. 13). This leads to another issue: How do we effectively carry out this “systematic asking of questions”?
With support from a VCCS Professional Development Research Award, I began a research project in the spring of 1995 to respond to the issue: How do we effectively carry out this “systematic asking of questions”? From my research, I concluded that an effective means to achieve instructional improvement was through faculty-focused assessment, a concept I defined as follows:
Faculty-focused assessment assumes that faculty have primary responsibility for determining the assessment objectives that will be used to evaluate the program, for carrying out the assessment, for analyzing the results, and for documenting the process, including the results and recommendations.
While faculty-focused assessment does respond to some administrative needs, this process is actually faculty-centered. The purpose of faculty-focused assessment is to provide immediate feedback to faculty on what is working in the classroom and what is not. It is individual, personalized, and confidential to the extent desired by the individual faculty member. The primary benefit of faculty-focused assessment lies in establishing pedagogical research as an essential element of a college mission for all academic areas. The most effective means of carrying out pedagogical research is through what K. Patricia Cross and Thomas Angelo call classroom assessment and classroom research.
To begin the process of implementing faculty-focused assessment at Germanna Community College (GCC), I determined that the most effective means would be through a pilot project. To this end, I wrote a grant proposal and received funding from a Spring 1996 VCCS Professional Development Research Award to carry out the pilot project. The proposed project involved six GCC faculty who agreed to 1) improve their understanding of classroom research; 2) assume responsibility for carrying out and documenting a classroom assessment activity in a class of their choosing; and 3) discuss the experience/results with colleagues. These six faculty members are Dr. Rodwell Catoe, Assistant Professor/Coordinator Administration of Justice; Mr. William Daley, Adjunct Instructor of ; Mr. David Fama, Associate Professor of Mathematics; Ms. Cathy Jones, P.A.S.S. Center Coordinator (Tutor Center); Mr. Gerald Slusser, Associate Professor of Business Management; and Dr. Gayle Wolfe, Professor of Psychology.
The Pilot Project - An Overview
On 29 January 1996, Dr. Karen Gentemann, Director Institutional Assessment at George Mason University, conducted a “kick-off” workshop for the six faculty members at Germanna Community College. During the workshop, Dr. Gentemann first reviewed and discussed the context in which classroom assessment is an appropriate activity. Using Cross and Angelo's Classroom Assessment Techniques: A Handbook for College Teachers as a guide, she then .reviewed specific assessment techniques for first-time practitioners. Finally, she helped the participants begin the process of selecting a class and determining an appropriate classroom assessment technique (CAT) or techniques to use with that class. [See Appendix A for a complete list of these Classroom Assessment Techniques.] Dr. Gentemann suggested the following procedure for carrying out classroom assessment.
1) Select a class.
2) Determine specific goals.
3) Select appropriate CATs using the Cross & Angelo book
(CATs should be linked to the goals).
4) Prepare the class for classroom assessment; there should be no
surprises, no grades. The student responses should be anonymous.
5) Analyze responses quickly and simply, for example, determine
the number who got the point in question, the number who were
close, and the number who missed it.
6) Share the results of the CATs with students as soon as
possible and as simply as possible. Tell students, for example,
that because of the results of the CAT, “this is what will
happen.”
7) Share the results with colleagues.
To expedite the process, I created a form to help the participants organize the results of their classroom assessments more effectively. I then asked the participants to critique the form, and distributed revised copies to each one of them. The form requested input on the purpose for using the technique (“what you wanted to learn from the students’ responses”), the findings, and an assessment of the results.
After the workshop, I met individually several times with the participants to discuss their progress and to answer questions they had about the project. During the semester, I also scheduled informal “brown-bag” lunch sessions to give the participants a chance to share the results of their classroom assessment activities, as well as to discuss problems, if any, and to let me know how I might help them, These sessions were held on 22 February; 21, 29 March; and 4 April. In addition, I met with each participant on a semi-regular basis during the semester
The Results
By the end of the 1996 spring semester, all six participants were able to apply one or more classroom assessment techniques in at least one class. Two participants were able to involve two classes in classroom assessment. Five out of six participants used the Minute Paper (CAT #6), the most frequently employed technique . Four participants used the Muddiest Point (CAT #7), while one participant used the Misconception/Preconception Check (CAT #3), and another used the One-Sentence Summary technique (CAT #13). [See Appendix B for an explanation of the four CATs mentioned above.]
During the meetings held throughout the semester and in individual comments made to me, the faculty involved in the pilot study had nothing but positive comments about the impact the classroom assessment techniques had on their students and on the classes. At the end of the semester, I asked each participant to critique the pilot project, asking them the following questions.
1. What classroom assessment techniques or aspects of the project worked particularly well for you?
2. Did you have any difficulties as a participant in the project? If yes, please indicate below.
3. Assuming Germanna plans to continue to promote formative classroom assessment, what suggestions would you make for more effectively carrying out classroom assessment?
4. What insights did you gain regarding teaching/learning or about your particular class as a result of your participation in the project?
5. With respect to the form I created to help you document the results of your classroom assessments, what suggestions can you make to help me improve the form?
In response to question #1, most participants simply noted their preferred classroom assessment technique. One participant indicated that he liked the Minute Paper which asked students to name the most important thing learned in the class. He added that this technique worked particularly well when he also asked the students to name the least important thing learned in the class. Another participant stated the following:
Participation itself was beneficial because I learned a great deal about classroom assessment — the “how to” and the”'why.” The group meetings were valuable. I found it interesting to see how others carried out their assessment and documentation.
Question #2 asked if the participants had any difficulties as a participant in the project. One participant made note of her personal situation which made it difficult for her to meet with the group. Several participants mentioned limited time as an obstacle. One participant stated that “teaching five different courses, doing the facilitator work for the division, and working on the Self-Study just didn't leave a lot of time.” The most of the other participants had a similar responsibility loads. (Note: The participants' responses to question #3 are included in Section IV — Recommendations.)
Question #4 asked the participants to indicate any insights they had gained regarding teaching/learning or about their particular class as a result of their participation in the project. One participant noted that he was dissatisfied with the Minute Paper technique as he used it. He felt that the answers he received did not respond fully to the question and “did not provide the information I was seeking.” He added that he would have to reconsider use of this technique for the future. An insight of another participant was that “students frequently have as difficult a time expressing what is 'muddy' to them as they have understanding something in the first place.” Another participant responded simply that “students like the idea of engaging in classroom research/assessment." On a somewhat broader consideration, a third participant noted the following:
Certainly, the instructor cannot assume that clarity of expression and good oral skills lead to meaningful communication for the students in the classroom. While this is frustrating, it also has opened up opportunities to improve the communications with the very same students.
Similarly, another participant in her response to Question #4 indicated that the main insight she had gained from the project was “how important it is to get on-going feedback from [her] students regarding their understanding of the material. [She] had mistakenly assumed that if [she] discussed a topic in class that all the students comprehended what [she] said!” She added that the Minute Paper and One-Sentence Summary are “simple ways to do a reality check and to correct (hopefully) any misunderstandings.” A concluding remark was offered that, through the project, the value of formative assessment was made very clear to her. “I think students would also see the benefits and appreciate the focus on the quality of learning as opposed to adherence to a syllabus.”
Only two participants responded to question #5, which asked for suggestions to help improve the form, Documentation Form for Learning Assessment Technique. One suggested that it would be helpful to indicate the sequence of instruction up to [the point of the CAT], i.e., “is it a newly introduced skill or review of previous material?”. The other participant recommended including information about how the results were used to improve teaching. While a third participant did not include any specific suggestions in response to question #5, he made several effective additions to his own documentation of what he had done. These additions included the name of the class involved in the classroom assessment and a brief course description. He also noted the number of students participating in the particular CAT, as well as the specific question asked or subject area for which he had used the CAT. In a revision of the form, I have incorporated the ideas of all three participants. (See copy of revised form in Appendix C at the end of this report.)
Recommendations
In question #3 of the Participant Reactions & Suggestions form, I asked the participants to make suggestions for more effectively carrying out classroom assessment at Germanna Community College. The following is a list of recommendations made by the participants.
Another recommendation was given related to what happens after the classroom assessment technique is applied. The participant noted:
I think it would be interesting to investigate what happens after assessment (re-teach class, individual instruction referral to PASS Center, academic advising). I'm sure there will be questions about the rationale of re-teaching. . . Unless faculty are already concerned about whether or not students are “getting it,” they won't see the need for the formative assessment process.
Two participants made strong recommendations related to the time factor. One of them stated:
I believe that summative and formative classroom assessment should occur. In order to develop the formative assessment strategies, we have to have the commitment from the College. Many of us, because of our teaching schedules and administrative expectations, will be limited as to how [effective] we can be [without this commitment].
The other participant was more specific in her recommendation related to time as a factor:
One suggestion that I would make . . . is to stop asking us to serve on committees that do not serve to improve teaching and to give us time to meet with BOTH students and faculty to discuss TEACHING. This only time this seems to happen is through a faculty initiative such as this grant.
Finally, a recommendation was made which supports my strong feeling that faculty-focused assessment is critical to the institution. The participant suggested that “it would also help if our regular assessment process was somehow connected to classroom assessment.”
Conclusions
As documented above in Section IV, one participant asserted that “unless faculty are already concerned about whether or not students are 'getting it,' they won't see the need for the formative assessment process.” This “getting it” is what Barr and Tagg mean when they talk about the college as “an institution that exists to produce learning [the Learning Paradigm], not simply to provide instruction [the Instruction paradigm] (Barr & Tagg, as quoted in ”The Learning Paradigm," 1996, p. 1). In their article “From Teaching to Learning — A New Paradigm for Undergraduate Education” published in Change (November/December 1995), Barr and Tagg note that the Learning Paradigm “supports any learning method and structure that works, where 'works' is defined in terms of learning outcomes, not as the degree of conformity to an ideal classroom archetype” (Barr & Tagg, as quoted in “The Learning Paradigm” 1996, p. 1). There must also be effective assessment to support the Learning Paradigm.
I believe that as responsible and concerned educators, we must continue to narrow the chasm that exists between what, in many cases, is the ivory tower of standard assessment practices and what the faculty needs to know and do in order to improve the learning environments for their students. I believe faculty-focused assessment, as I define it, does just that. Faculty must have primary responsibility for determining the assessment objectives that will be used to evaluate the academic areas of the college, for carrying out the assessment, for analyzing the results, and for documenting the process. It is the results of these evaluative assessments, both formative and summative assessments together, that should form the basis of the evaluation of the program and that lead to improvements in instruction.
In conclusion, it is clear from the results of the pilot project that faculty participants and students alike benefited from the classroom assessment experience. In order for Germanna Community College to continue to become more learning centered, the entire faculty must be involved in all aspects of the learning environment, particularly in assessment. Faculty-focused assessment promotes this involvement, and the use of classroom assessment and classroom research promotes not only the involvement of the faculty but also the students.
References
Assessment and accountability in higher education. (1990). [Proceedings Document. December 5-7, 1989. Santa Fe, NM] ESC Working Papers. Denver: Education Commission of the States.
Angelo, T. A. & Cross, K. P. (1993). Classroom Assessment Techniques: A Handbook for College Teachers. San Francisco: Jossey-Bass Publishers.
“The Learning Paradigm.” (1996, February). The Teaching Professor. p. 1.
Julie R. Adams is Assistant Professor of Composition and Developmental English at the Locust Grove campus of Germanna Community College.
Appendix A
Classroom Assessment Technique and CAT Number
Analytic Memos, 12
Annotated Portfolios, 18
Applications Cards, 24
Approximate Analogies, 15
Assignment Assessments, 49
Audio- and Videotaped Protocols, 22
Background knowledge Probe, 1
Categorizing Grid, 8
Chain Notes, 41
Classroom Assessment Quality Circles, 45
Classroom Opinion Polls, 28
Concept Maps, 16
Content, Form, and Function Outlines, 11
Course-Related Self-Confidence Surveys, 32
Defining Features Matrix, 9
Diagnostic Learning Logs, 40
Directed Paraphrasing, 23
Documented Problem Solutions, 21
Double-Entry Journals, 29
Electronic Mail Feedback, 42
Empty Outlines, 4
Everyday Ethical Dilemmas, 31
Exam Evaluations, 50
Focused Autobiographical Sketches, 33
Focused Listing, 2
Goal Ranking and Matching, 35
Group Instructional Feedback Technique, 44
Group-Work Evaluations, 47
Human Tableau or Class Modeling, 26
Interest/Knowledge/Skills Checklists, 34
Invented Dialogues, 17
Memory Matrix, 5
Minute Paper, 6
Misconception/Preconception Check, 3
Muddiest Point, 7
One-Sentence Summary, 13
Paper or Project Prospectus, 27
Pro and Con Grid, 10
Problem Recognition Tasks, 19
Process Analysis, 39
Productive Study-Time Logs, 37
Profiles of admirable Individuals, 30
Punctuated Lectures, 38
Reading Rating Sheets, 48
RSQC2 (Recall, Summarize, Question Comment, and Connect), 46
Self-Assessment of Ways of Learning, 36
Student-Generated Test Questions, 25
Teacher-Designed Feedback Forms, 43
What’s the Principle?, 20
Word Journal, 4
(Angelo & Cross, 1993, p. 109)
Appendix B
Explanations of Four
Classroom Assessment
Techniques (CATs #3, #6, #7 & #13)
Misconception/Preconception Check (CAT #3)
This CAT “assesses students’ prior knowledge. . . that may hinder or block further learning. . . Because assessment activities such as this CAT identify misconceptions and preconceptions early on and help students explicitly recognize and understand them, students stand a much greater change of learning new material correctly and integrating it into their ‘revised’ and often ‘transformed’ knowledge of structures” (Angelo & Cross, 1993, p. 132).
The Misconception/Preconception Check asks the instructor first to create a list of troublesome common misconceptions or preconceptions of students in previous classes. Then, in order to determine his/her present students’ beliefs and ideas in these areas, the instructor surveys the students using a simple, anonymous questionnaire (multiple-choice or short answer). The instructor should explain his/her reasons for using this CAT to the students (Angelo & Cross, 1993, p. 135-136).
Minute Paper (CAT #6)
This CAT, “also known as the One-Minute Paper and the Half-Sheet Response — provides a quick and extremely simple way to collect written feedback on student learning. To use the Minute Paper, an instructor stops class [five to ten] minutes early and asks students to respond briefly [and anonymously] to some variation on the following two questions: ‘What was the most important thing you learned during this class?’ and ‘What important question remains unanswered?’ Students then write their responses on index cards or half-sheets of scrap paper. . . and hand them in” (Angelo & Cross, 1993, p. 148).
Note: The results of the instructor’s tabulation/analysis of the One-Minute Papers should be shared with the students as soon as possible, preferably at the next class meeting. Mechanisms should be provided by the instructor for clarification of students’ questions.
Muddiest Point (CAT #7)
“The Muddiest Point is just about the simplest Classroom Assessment Technique imaginable. It is also remarkably efficient, since it provides a high information return for very low investment of time and energy. The technique consists of asking students to jot down a quick response to one question: ‘What was the muddiest point in ______?’ The focus of the Muddiest Point assessment might be a lecture, a discussion, a homework assignment, a play or a film” (Angelo & Cross, 1993, p. 154).
Note: The instructor should respond to the students’ feedback to the Muddiest Point as soon as possible, preferably at the next class meeting.
One-Sentence Summary (CAT #13)
“This simple technique challenges students to answer the questions ‘Who does what to whom, when, where, how, and why?’ (represented by the letters WDWWWWHW) about the given topic, and then to synthesize those answers into a single informative, grammatical, and long summary sentence. . . The One-Sentence Summary enables teachers to find out how concisely, completely, and creatively students can summarize a large amount of information on a given topic” (Angelo & Cross, 1993, p. 183).
Appendix C
Information for Documenting
Classroom Assessment Techniques (CATs)
INSTRUCTOR’S NAME
DATE
COURSE including section (OPTIONAL - a brief course description, at least the first time with each class)
NUMBER of students involved in the specific classroom assessment technique
A brief DESCRIPTION of what happened instructionally that led up to the application
of the classroom assessment technique (CAT)
CLASSROOM ASSESSMENT TECHNIQUE USED
THE SPECIFIC QUESTION(S) ASKED
YOUR PURPOSE FOR USING THE TECHNIQUE (what you wanted to learn from the students’ responses)
THE RESULTS/FINDINGS (in brief form):
YOUR OVERALL ANALYSIS/ASSESSMENT OF THE RESULTS
(CONCLUSIONS/DISCUSSION)
HOW YOU USED THE RESULTS TO IMPROVE INSTRUCTION
[Created July 1996]