The article makes much of the idea that this goes against conventional pedagogical wisdom, according to which concept mapping is thought to be the "gold standard" of learning methods.
But what's seriously puzzling to me is that the authors and experts quoted in the article find these conclusions the slightest bit surprising. There's no puzzle here: One thing testing does is give students feedback concerning their performance, particularly negative evidence regarding their performance. The struggle to recall material compels effort, whereas other methods without feedback tend to generate the illusion of comprehension. Quoting me quoting Marilia Svinicki:
Students must learn better ways to monitor their own understanding while they're learning, and we must structure our class time so that these false senses of understanding will not survive. The first step in combating students' illusion of knowing is to confront them on a regular basis with evidence of their knowing or lack of it. (Learning and Motivation in the Postsecondary Classroom, 120)The NYT article at least grasps this implicitly:
It may also be that the struggle involved in recalling something helps reinforce it in our brains.Add that students who took tests already had experience with being tested on the material, and it seems no surprise at all they'd outperform those who studied, did concept mapping, etc.
Maybe that is also why students who took retrieval practice tests were less confident about how they would perform a week later.
“The struggle helps you learn, but it makes you feel like you’re not learning,” said Nate Kornell, a psychologist at Williams College. “You feel like: ‘I don’t know it that well. This is hard and I’m having trouble coming up with this information.’ ”
By contrast, he said, when rereading texts and possibly even drawing diagrams, “you say: ‘Oh, this is easier. I read this already.’ ”
Some thoughts about implications and limitations:
Group work As reported here last week, studying in groups seems especially ineffective. My own experience is that for moderate to well-prepared students, studying in groups helps them, but for less capable or well-prepared students, group study probably hurts them. Students in the latter category give each other feedback in their groups, but the feedback is likely to reinforce their collective misunderstandings rather than correct them. The Dunning-Kruger effect is amplified in group settings.
What is 'studying'? One issue not discussed in the article (or the studies) is exactly what students who 'studied' instead of taking tests did. As discussed here before, how students study is rather opaque to us instructors, and my guess is that most students (to the extent they're studying at all) probably study in ways that encourage superficial, rather than deep, learning, and are particularly unhelpful in trying to learn philosophy. I'd love to see the authors replicate their experiments with more tracking of how the students 'studied.'
Testing retrieval The obvious limitation of these experiments is that the students were later tested for the very thing — information retention — that the earlier tests helped them prepare for. Might the students who used concept mapping, who engaged in traditional 'studying,' etc. achieved deeper understanding even if their information retention was weaker?
Is it *test taking* per se vs studying, or is it actual practice in working through the answer as opposed to just reading about the answer?
ReplyDeleteI think of logic, for instance. Studying for a logic exam by reading the book is one thing, but actually doing large numbers of logic exercises is another. And testing is really a matter of doing exercises. So I wonder if it's not so much "test taking" but rather "hands on practice".
Chris, sure. Again, we don't know much about what the 'studying' was. It may or may nor include what you describe, self-testing.
ReplyDeleteI had the same reaction to this article that Chris did. Based on the Times article, it sounds to me like students in the "study" condition just read and re-read the material, while students in the "testing" condition wrote essays on the material without having the material in front of them. I don't see why this is about testing at all, as opposed to different ways of studying.
ReplyDeleteIn particular, I don't see any evidence that the students who "took the test" got feedback on the test. So, I don't think that's why the "testers" did better.
My wife, though, had the same question that Michael did: Testing may have helped students remember more, but which method actually helped students learn more (and more deeply)?
Anyway, what does this tell us about teaching? Somewhere (possibly on this blog) I remember hearing about someone who has students write a five-minute summary of the day's lecture at the end of class, without looking at their notes. The instructors collects the summaries and scans a few of them to see what students got out of the lecture. Perhaps this study is evidence in favor of that method? (It also has the nice benefit of providing an unobtrusive attendance check that's less prone to gaming than a sign-in sheet is.)
There's a nice set of letters to the editor about this article on the Times' web site.
ReplyDeleteDavid, you raise some good points. What I had in mind by "feedback" was not necessarily instructor feedback but simply information about one's own performance. And tests can give you that, since you will experience questions as more or less difficult, etc. Reading the material again and again might do that, but my guess is that it won't correct fundamental misunderstandings that stand in the way of student learning. But I think I agree that 'taking tests is better than studying' is not a conclusion we can easily draw from these studies without a lot more detail (and probably more studies!).
ReplyDeleteInteresting point, Michael. It hadn't thought about the act of writing about the material or taking a test as providing feedback about how well you know the material. I suppose, though, that it can only tell you what you know you don't know. If you are confident in a false belief, the test will appear easy to you.
ReplyDeleteTo be Rumsfeldian about it, an ungraded test can reveal known-unknowns, but it can't distinguish between known-knowns and unknown-unknowns. (What about unknown-knowns? I guess the test could reveal them, too, even if that category is unknown to Rumsfeld.)