Using Analogies to Assess Student Learning

By Callan Bentley

from Inquiry, Volume 13, Number 1, Spring 2008, 26-35

© Copyright 2008 Virginia Community College System

Abstract
The author offers his experiences in using Classroom Assessment Techniques (CATs) in his geology courses to gauge his students’ grasp of course concepts.

One of the most powerful pieces of knowledge that students can gain from the study of geology is an understanding of the immense scale of geologic time. In my introductory-level physical geology course at Northern Virginia Community College in Annandale, we discuss geologic time about one-third of the way through the semester, after a thorough review of plate tectonics and the rock cycle. Because this subject is of foundational importance in all of my geology classes, I chose it as the focus for a Classroom Assessment Technique (CAT). Angelo and Cross (1993) present an extensive user’s guide to many different CATs, but the key idea with any classroom assessment is to try to figure out if students are learning what we want them to learn. Based on the feedback an instructor gets from a CAT, he or she can decide whether to move on or re-examine a particular issue.

My physical geology course is a 16-week introduction to Earth products and processes. Meeting twice a week, we begin with an overview of Earth’s place in the solar system, followed by a discussion of plate tectonics and then several weeks examining the rock cycle. Traditionally, I spend one lecture discussing geologic time immediately following our first lecture exam. During our second lecture that same week, I traditionally move on to begin a unit on the Earth’s surface. The first lecture in this surficial unit covers mass wasting, the down-slope movement of sediment and rock under the influence of gravity (e.g., landslides). As concepts, geologic time and mass wasting are not connected to one another, but traditionally my syllabus follows them in that order. During the same week in lab, students complete a series of exercises on geologic dating, practicing both relative dating and simple calculations of absolute rock ages.

In the example I am about to give, a total of 52 students were enrolled in my physical geology course, and their ages, backgrounds, academic ability, and motivation were diverse.  In the past, it has been difficult for me to determine whether my students were “getting it” as a class prior to testing them.

For my CAT, I chose approximate analogies to examine how well my students were learning about geologic time. I delivered my usual geologic time lecture, including examinations of both relative dating and absolute dating. (Relative dating is a comparison between two rock units, determining which of the two is older compared to the other. Absolute dating, on the other hand, assigns a precise numerical age to a rock unit, based on the decay of radioactive isotopes in the rock’s component mineral crystals.) Once the principals of isotopic dating had been established, I concluded with a summary diagram that showed how geologists have determined that the Earth is as old as we claim. Measurements of three different isotopes of radiogenic lead (Pb), produced by three independent isotopic decay systems, all agree that the planet congealed about 4.65 billion years ago (Figure 1). I then drove the point home by writing out that number on the blackboard: 4,650,000,000. Then I wrote 70 – the average lifespan for a human – below that larger number so that tens and ones aligned. I pointed out the vast difference in magnitude of these two numbers. “Geologic time is much vaster than human time,” I told the students. “It has been described as Deep Time. It’s so deep, it is hard for us to wrap our minds around its real magnitude.” For emphasis, I then said, “The Earth is really, really, really, really, really, really, really, really, really, really, really, really, really, really, really, really, really, really old,” which earned me a few indulgent smiles.

 At the end of the lecture, I introduced the CAT with a reminder of how they had provided me with important feedback during previous CATs. I then passed out slips of paper and asked students to take one minute and complete an approximate analogy that required them to fill in the blanks in this statement: “Geologic time is to human time as ____ is to ____.” The students reacted well, efficiently following my instructions. They turned in their analogies, and class was dismissed.

Analyzing the Data

After class, I tallied up the results of the approximate analogy and grouped the 42 responses into clusters according to whether I considered them “correct,” “sort of correct,” or “wrong.” According to this grouping, there were 18 correct responses, 17 sort of correct responses, and 7 incorrect responses. I compiled a summary sheet to distribute to students which included all the responses, grouped into these three categories, as well as my comments as to why they qualified as correct, sort of correct, or incorrect.

Students came up with some terrific analogies for comparing geologic time’s immense scale to the relatively tiny amount of time in a human lifespan. Answering correctly, students gave responses such as “an elephant is to a peanut” and “the Earth is to an ant.” Several students used water as their analogy: “Geologic time is to human time as the ocean is to a drop of water.” One particularly evocative analogy was “Geologic time is to human time as the human population is to your fingernail.” Another was “Geologic time is to human time as a blanket is to a thread.” I like these last three because they are inclusive: they deal with different aspects of the same entity. A fingernail is part of a human, which is part of a population. A thread is one of many that are woven together into a larger blanket. Likewise, humanity is a part of Earth history, not separate from it.

A final student made a correct response with an implicit connection to the geologic record (an important aspect in the lecture’s earlier discussion of relative dating). The student said that “Geologic time is to human time as writing a library’s worth of books is to flipping a page.” Of all the responses, this one was my favorite because it emphasized the energy intensive process (researching, writing, erasing, revising, editing, publishing, binding, shelving, more research) that goes into producing a book, as compared to the reader’s literal flippancy of turning one page. To me, this was an eloquent analogy for the various processes of the rock cycle (weathering, erosion, transportation of sediments, deposition, lithification, metamorphosis, intrusion by igneous bodies, uplift, more weathering) that have yielded the information-rich geologic record as compared to our relative ease in interpreting small parts of that geologic story.

In the sort-of-correct category, I had several clusters of responses. The most common deficiency in these responses was a diminished sense of the magnitude that separates geologic time from human time. For instance, several students completed the analogy with “as an hour is to a second.” There are 3,600 seconds in an hour, but 70 can be divided into 4.6 billion almost 66 million times. There are four orders of magnitude difference between these two comparisons. Another couple of students used dog years to complete the analogy. The standard conversion between a dog’s lifespan and a human’s lifespan posits seven dog years in each human year. Of course, this analogy misses the mark even further: geologic time is much more than seven times as vast as human time. (Maybe I should have put a few more “reallys” into my “really, really old” spiel.)

There were also two sort-of-correct responses that added an additional dimension to misunderstanding the relationship between human time and geologic time. One student responded that “Geologic time is to human time as a baby is to an embryo.” While an embryo is smaller than a baby, that is not the best summary of their relationship. The relationship between an embryo and a baby is a developmental one: an embryo becomes a baby with time. Human time does not, however, become geologic time.

Among the analogy responses I considered to be incorrect, one student gave an analogy which apparently took the relationship between geologic time and human time as being an unlikely or unique event. They responded with “Geologic time is to human time as the Cubs winning the World Series is to baseball.” Others compared different technologies (cell phone versus pager) or opposites (prison versus freedom). Some of the responses totally mystified me. The most startling of these was “Geologic time is to human time as a Christian losing his/her virginity is to confessing.”  

Overall, I noticed one major trend that ran through the answers. Whether right, wrong, or somewhere in between, many students transposed the sense of large and small implied by the analogy. Geologic time is the larger of the initial pair; human time is small by comparison. In the second pair, which completes the analogy, the larger item must also be placed in the first blank so that the sense of the analogy is maintained. I found that many students reversed the sense of the analogy by putting the smaller item first. For example, the thread/blanket analogy mentioned earlier was originally written as “Geologic time is to human time as a thread is to a blanket.” This implies that geologic time is but a small part of human time, which is the opposite sense of perspective from what I was trying to impart. In the correct category, 10 of the 18 responses reversed the sense of the analogy. In the sort-of-correct category, six of the 17 responses reversed the sense.

Developing Strategies to Increase Student Learning

Using the information revealed by the approximate analogies CAT, I decided that an instructional intervention was warranted, as the data showed that I had not made the difference in the magnitude of geologic time clear enough to the whole class. I also wanted to make sure the students understood how these analogies were supposed to work. I designed a plan to make these two issues clearer. At the beginning of the next class meeting, I set aside ten minutes of instructional time to accomplish three things.

First, I prepared the summary handout sheet, which I distributed to students. This handout showed the students where their responses were grouped and attempted to discern the logic underlying the responses. I also wrote some general comments about reversing the sense of the analogy and highlighted particularly good examples. I distributed this handout at the beginning of class.

Figure 2. Examples of PowerPoint slides used to illustrate correct, sort-of-correct, and incorrect
analogy responses. The slides on the left are examples of responses I considered to be correct and
evocative. Those on the right are examples of a developmental response and an opposite response,
neither of which captures the true relationship between geologic time and human time.

Third, I discussed how analogies like these work, and I drew the students’ attention to the difference between “Geologic time is to human time as a blanket is to a thread” and “Geologic time is to human time as a thread is to a blanket.” I emphasized that when analogies like this are presented to them, their first task is to discern the relationship between the pair of items in the first part of the analogy and then to think of two items for the second half of the analogy which share the exact same relationship.

My instructional intervention concluded, I went on to deliver my usual lecture on mass wasting, a completely unrelated topic to geologic time. However, I wanted to make sure my lesson on analogy construction had sunken in, so I planned a second round of approximate analogies. I explained landslides, creep, mudflows, solifluction, and quicksand but left several minutes at the end for another edition of the CAT. At the conclusion of class, I distributed sheets of paper with a new analogy. This time, I sought to focus their responses by filling in one of the second blanks for them. I did this because I wanted to experiment with different variations of the CAT, and the one-blank version was suggested in Angelo and Cross (1993) as a possible way to extend the CAT. The sheets of paper I distributed said “Gravity is to mass wasting as subduction is to ___.” Gravity is the cause of mass wasting, so I was looking for students to fill in the blank with something that has subduction as its cause. (Subduction is the tectonic process whereby a slab of oceanic crust is shoved underneath another plate. I chose subduction because it is a concept on which they had recently been tested, and I felt they understood it well.) I admonished the students to think carefully about the relationship between gravity and mass wasting and then to apply that same relationship to subduction. Again, the students filled in their responses and then left the class.

I again compiled the responses, grouping them into categories based on whether they were correct, correct-but-transposed (the sense of the analogy had been reversed), sort of correct, or wrong. There were 36 total responses, with nine of them correct, four sort of correct, 13 that were reversed, and ten which were incorrect for some reason.

Correct responses included melting crust, volcanoes, earthquakes, and deep sea trenches. Subduction causes all of these, just as gravity causes mass wasting.

There were several sort-of-correct responses. One student supplied “continental crust” in the blank. While subduction has an important role to play in generating continental crust, it is not its sole creator. The surficial processes of weathering and sedimentation also have an important role to play in giving continental crust its structure and composition. This answer is therefore only partially correct. Similarly, “rock cycle” does not fully complete the analogy. While subduction can be the driver of some portions of the rock cycle like melting and metamorphism, it is not responsible for other portions like weathering or erosion. Again, the analogy does not fully capture the direct causative relationship between gravity and mass wasting. I was astonished at the number of  reversed analogies. Over a third of the students returned responses that detailed causes of subduction, rather than its effects. I found this very frustrating since I had just gone over the importance of maintaining the same analogous sense in the second half of the analogy as demonstrated in the first half. The most frequently repeated response by far was “plate tectonics,” which is the cause of subduction, not one of the effects of subduction.

Lastly, there were three main sub-groupings of incorrect responses. First, there were three students who linked subduction with divergent tectonic boundaries and the related process of sea-floor spreading. Tectonically, this is the opposite situation from where subduction occurs. Second, there was a group of five students who responded with an answer directly derived from the mass wasting lecture, in spite of the lack of any real connection between mass wasting and subduction. These responses were especially disappointing. For instance, one student wrote “Gravity is to mass wasting as subduction is to quicksand.” Another completed the analogy with “wavy lobes.” I found myself perplexed by these responses. Third, there were two responses which appeared to be providing human analogues for the motion of subduction (“scuba diving,” “drowning”) without addressing the cause/effect relationship at all.

Figure 3. Examples of PowerPoint slides used to illustrate correct, sort-of-correct, and incorrect analogy responses. The slide at upper left was a comprehensive illustration of the effects of subduction. The slide at lower left was an illustration of the importance of maintaining the sense of analogies, as plate tectonics causes subduction, but subduction does not cause plate tectonics. At the upper right is a slide illustrating a partially correct answer. At the lower right is an example of an analogy that missed the point completely. Figure 3. Examples of PowerPoint slides used to illustrate correct, sort-of-correct, and incorrect analogy responses. The slide at upper left was a comprehensive illustration of the effects of subduction. The slide at lower left was an illustration of the importance of maintaining the sense of analogies, as plate tectonics causes subduction, but subduction does not cause plate tectonics. At the upper right is a slide illustrating a partially correct answer. At the lower right is an example of an analogy that missed the point completely.

Continuing the Discussion

As a side project in classroom assessment, I recently opened up a new, anonymous discussion thread in Blackboard. In an open discussion area, I asked for any feedback students might have about how the course was being taught. Out of curiosity, I did not publicize the discussion thread, preferring to see first how many students stumbled onto it on their own. During the same week as my second in-class CAT assessment, a student post appeared on the discussion thread. The anonymous student wrote, “I was kind of disappointed when the last analogy you asked us to fill out only had one blank entry. It felt like it was a quiz because you had a predetermined answer you expected from us. I understand that some students (maybe I am included in this group) were not able to be creative enough to provide adequate answers but that’s a key part of the learning process: making mistakes, finding out why we weren’t correct, and learning from them.”

I responded to this posting with a post of my own: “You’re right that I was trying to restrict the answers a bit… Gravity causes mass wasting. Mass wasting is the result of gravity’s pull. The second part of the analogy puts ‘subduction’ in the first blank. Essentially, then, what I was hoping for is students saying ‘Okay, what does subduction cause?’ And here, you’ll note, there is not just one answer. Subduction causes earthquakes, metamorphism, partial melting, distillation of the crust towards a more felsic composition, volcanic arcs (both island and continental), everything that goes along with volcanoes (nuee ardentes, lahars, death and destruction, etc.) and oceanic trenches. There’s a whole host of different things/processes that stem from subduction.”

I also uploaded a PDF copy of the student responses to the second run of the Analogy CAT. It would be several days until we had class again, and I wanted this student to have access to the results as soon as possible.

When the next lecture day arrived, I presented my PowerPoint slides, and the students and I discussed each in turn. I emphasized more vehemently the importance of maintaining the proper sense of the analogy.

After this assessment to improve learning, I was ready to see how effective my intervention and follow-up analogy generation had been. I had assessed my students and gotten feedback, responded to that feedback, solicited a second round of feedback, and then responded to it. Finally, I was ready to formally evaluate the students. I announced that we would have a quiz the following class meeting.

The quiz consisted of five questions, one of which was an analogy. I chose the “Geologic time is to human time as ___ is to ___” analogy because I felt like that was a more important conceptual point to evaluate than any of the details of mass wasting. I also wanted something familiar, that students were familiar with seeing in analogy form.

There were 35 students in class on the day I gave the quiz. Of those, 25 scored a correct answer on the analogy quiz question. A further six received half credit for a reversed analogy, two supplied incorrect answers, and two showed up late (and didn’t have time to finish the quiz).  

Values and Claims

The geologic time and mass wasting approximate analogies CAT assessments went as planned. I feel the arc of all five activities (geologic time analogy, intervention, mass wasting analogy, intervention, and quiz) was a particularly strong method for keeping my finger on the pulse of student learning. Though I feel somewhat chagrined that there were still so many reversed analogies on the quiz, I feel pleased at my own thought about how best to address the issue, which is certainly far deeper than before I began practicing classroom assessment. I am certain that increased learning took place as a direct result of the CAT and its follow-up activities. This is evidenced by the number of correct responses on the quiz (25/35) as opposed to the initial CAT (18/42), and far fewer wrong answers on the quiz (2/35) as opposed to the initial CAT (7/42).

One thing I would change for the next time I run this CAT would be to do analogy composition in groups or to start off having students make their own individual analogies and then discuss them in groups. They could then debate the analogies’ merits on their own and learn from one another. I feel that some group work would be a better way to handle the approval process: that way it would be less about the professor deciding which analogies are correct (as essentially a quiz that doesn’t count) and more about students considering and articulating relationships.

As another result of this exercise, I have gained familiarity with a new piece of technology, the online discussion boards in Blackboard. Because I have grown accustomed to using online discussion as a means of learning through my enrollment in a graduate-level education course, I have imagined its potential as a source of constant, asynchronous feedback from my own students. I had not promoted the discussion boards as a way to get feedback, but the small trickle of responses so far from students has indicated it has strong potential for assessment purposes. Promoting the discussion board to the whole class will be the next step.

I was also pleased to get the anonymous discussion board post that expressed the student’s opinion about the assessment cycle as I had been practicing it. This is an affirmation that the system is working, and it made me feel positive about the whole process. This feedback indicates that my students and I have achieved a culture of assessment in the classroom, to our mutual benefit. It is extremely gratifying.

Reference