I’ve spent the past few days reading Clifford Hill’s work on how children interpret questions on reading tests, and I can’t help but be struck by the comparisons to what Ethan is writing about his students in Cameroon.
Hill demonstrates again and again that children generate their answers not just from the text of the very brief, decontextualized passages which show up on the tests, but from a good deal of extra cognitive work they do in order to generate a mental system in which the passages make sense, or connect these passages to the world they know. Here is one of his examples of a test passage and questions, from the Lawrence Cremin Lecture he delivered at Teachers College this past March:
Raisins are made from sweet varieties of grapes. The ripe fruit is usually placed on trays right in the vineyard. There, it dries in the sun. Drying may take several weeks.
A. Raisins are made from grapes that have a lot of
water varieties
skin sugar
B. What kind of climate is best for making raisins?
warm and dry warm and wet
cool and dry cool and wet
(If you’re not yet reminded of how soul-crushingly insipid most multiple-choice tests are, Hill’s written whole books on why they make for bad assessments. And actually, reading chapters upon chapters of children’s justifications for wrong answers is pretty entertaining, and will bolster your faith in the ingenuity and resourcefulness of your fellow person.)
Of questions like these, he says,
tasks which force children to manipulate polarities often engender a confusion that they would not experience when reading an ordinary text in which polarities are not overworked and are embedded in a naturally occurring context.
He provides a few examples of students whose thinking about these questions led them to choose a distractor answer:
“Look, first I didn’t understand so much so I picked warm and dry be-cause of this [points to “dries” in the text]. But then when I read it more, I think—uhm—warm and wet, ’cause here it says, “There, it dries in the sun. Drying may take several weeks.” Weeks. I think weeks are very long so I don’t think dry is very good.”
“Just as the extended drying was used to justify the choice of warm and wet in Task (B), so it was used to justify the choice of water in Task (A). As one girl put it with great confidence, “It has to be water because of all that drying in the sun.” Another girl read out loud the final sentence in the passage to justify her choice of water. “It was water, because here it says, ‘Drying may take several weeks.’” “
(As Hill notes elsewhere, correct answers like “varieties” are kind of puzzling in and of themselves; the word is pretty much peripheral to the whole point of the passage. I think this is what he was calling “acommunicative,” as opposed to “communicative,” questions.)
Hill concludes:
These girls, like the boy described in the previous paragraph, adopt a dynamic stance toward textual information, which clearly runs counter to the static orientation that testing calls for. This stance leads them to project from one lexical pole to its opposite.
These examples clarified for me one recent post of Ethan’s. He reports the following exchange:
Teacher: OK, class. What color is the sky?
Student, raising hand and being called upon, says: Air!
Teacher: OK, good, the sky has air in it. But what color is it?
Different student, too excited to wait to be called upon: Monsieur! It’s air!
Ethan’s been frustrated by what he has called the “illogical” nature of answers like this. But in this answer, as in some of the others he’s blogged about, there is a certain logic at work. The students did not excitedly raise their hands to call out, “Monsieur! It’s octopus!” They gave an answer related to the topic he was asking questions about, in a way that recalls Hill’s students generating a context for their answers which just happens to not be the one called for by the test makers.
One possible imagining of the students’ imagined context in this case goes like this:
“I am in computer class with Msr. Ethan. Computers are a difficult subject. Msr. Ethan is from America. He is a teacher, and very smart, because he understands things like computers and science.
“Msr. Ethan has asked us a question about the sky. Teachers like him often ask difficult questions about science. I need to give the most erudite answer I can to impress him, because heaven knows a teacher wouldn’t ask something completely stupid and obvious to all of us, like what color the sky was. Who even cares what color the sky is? The context of this question is school, not the world out in the fields.”
This is, of course, not taking into account any possible linguistic confusion. I have no idea if the local dialect’s word for “blue” happens to be the same word for “air.”
(Ethan was, of course, really focusing more in that post about the theme of re-trying strategies that have already proven not to work; but then, Cameroonians don’t have a lock on that market. And Ethan would probably say, as usual, that I don’t understand what’s going on there; but then, by his own admission, he doesn’t either.)
Does this give any guidance to Ethan so that he might elicit the answers he wants next time? Maybe not, except that it might be a good time to set a rule like “before you answer this a question, I want you to listen closely to exactly what I am saying, think, and then wait five seconds before you raise your hand.”
But Hill’s work goes a long way towards demonstrating the logic of wrong answers. My particular interest has been the way he illuminates fundamental linguistic differences in cultures’ ways of understanding simple spatial concepts like “in front of,” “behind,” “at,” “in,” and “on.” While he explains these primarily in terms of the ways it leads African and Chinese students to answer test questions differently, I think it can also tell us a good deal about how people understand where they are on the Internet.
Comments 4