Tuesday, April 19, 2011

In favor of prospective exams

I'd like to share a positive experience I had with final exams.

Last quarter I gave what I'll call a prospective final exam in my introductory ethics sections. The idea here is that though the exam was intended to test the skills, etc., that the students were expected to acquire in the course, they had to display these skills with totally unfamiliar material. (I've been kicking around this idea for a while — see this post from 2008 — though the format differs from what I described there.)

Here's how this worked:
The course content is split about evenly between ethical theory and current ethical controversies (abortion, same sex marriage, capital punishment, etc.). About a week prior to the exam, I informed the students of the exam format: answer two essay questions out of five, with each question pertaining to one of the articles in our anthology that they had not been assigned during the term.

Here's what I noticed: I think I got a much better sense from these exams of how well students had actually acquired the relevant philosophical skills. My guess is that a purely retrospective exam is likely to collapse or blur differences in how well students can read philosophical texts, extract arguments, etc. With a retrospective exam, students can get pretty far by memorizing arguments, etc., that we discussed in class. With this exam, my sense was that I could discern sharper differences in student abilities. So I think it's probably superior as a diagnostic or evaluative tool.

Second, I think it compelled students to actually study (which, as we know from previous discussions here at ISW, mysterious to students and to us as instructors). They couldn't just sit down and "review" the material and feel ready. My guess is that though most students did not read all five articles — they probably skimmed them and then chose two or three to focus on in preparation for the exam — they probably read those two or three articles more carefully than anything else they read that quarter.

Third, this kind of exam encourages (I hope) metacognition. I've become a big believer in the notion that genuine, sustained learning requires learners to be metacognitive. When I told the students about the exam format, I reminded them of the course learning objectives and told them that the exam will require them to demonstrate their mastery of these objectives, albeit with new content. And without me to guide them, the students had to monitor their own learning. Have I really understood this argument? What are some assumptions the author is making? Has the author considered the best objections to her position? During the term, those are questions the students can answer indirectly through my feedback and assistance. But here they have to pose those questions for themselves.

Lastly, the most surprising thing: Several students said they liked this exam format. Part of this is that I selected provocative articles (Harris' "The survival lottery," Tooley on infanticide). But I suspect that for some students, they liked the sense of autonomy that such an exam communicates. Those essays were their work in a way that essays they would have written on material addressed in the course would not have been. As one student told me, "I liked that we got to take off our training wheels."

In any case, I think for introductory courses in particular, which (in my opinion) should be mostly about skill-building rather than long-term retention of content, this sort of exam has definite advantages. I suspect that in logic courses, the prospective exam is the norm. It might be tougher to implement in a history of philosophy course, but it could work even there.

What do folks think? Have you tried anything like this yourself?

7 comments:

  1. Maybe I'm missing this in my quick reading, but how did the students do, overall?
    Nathan

    ReplyDelete
  2. Nathan, I don't have exact numbers, but eying it, I'd say the average grade was about the same but the distribution was wider than it's typically been for final exams in this course in the past. Again, my suspicion is that since recall of past material plays little role in these exams, the student performances will be more divergent.

    ReplyDelete
  3. I like the idea, but I'll have to think about how to apply it in, say, my intro to philosophy courses, where the range of topics covered is rather diverse.

    ReplyDelete
  4. I'd been planning to introduce something like this in my contemporary moral issues course in the fall: the exams will involve "ethics bowl" type questions on issues that we haven't discussed in class. I'm glad to hear that you've had a positive experience with such exams. With that encouragement, I might try something similar in an upper-level ethics course, too.

    My thinking is that this sort of exam requires a course that is specifically designed to prepare students for that sort of task. In the contemporary moral issues case, this strikes me as fairly straightforward. We can focus on argumentation and ways of arguing in ethics, and we can do plenty of practice activities. I don't feel much pressure to cover the "classics" of applied ethics.

    In a different course, like Intro to Philosophy (to answer Matthew's question), I think I'd want to focus the course on building specific skills, such as argument analysis or the framing of objections, and have students apply those skills to articles or excerpts that were new to them. You might say, "We've covered twelve topics. 48 hours before the exam, you'll be given articles on eight of those topics. You'll have to answer questions about any five of them on exam day." If you choose articles cleverly, knowing the course material well could be of help in understanding the articles. If you convince the students of that, they might study all twelve topics prior to the announcement of the eight articles.

    Michael, did you do anything different in this course to prepare students for this type of exam?

    ReplyDelete
  5. David, no, nothing special. They knew what was coming, but I didn't reveal the format until the final week of the term.

    ReplyDelete
  6. Just a thought on the nature of in-class testing and ethics:
    Do we really want to train our philosophy students to be able to create quick arguments for or against some topic? This seems counter to philosophy. One could study law for this type of education. While it is easy to create an argument for or against in a given ethical framework, it seems this is almost more lawyerly than philosophical. I participated in ethics bowl myself and our team was in the top 10 at nationals. I know that my team members and I, not only used our knowledge and experience of ethical systems to create arguments, but thought and discussed in great detail every aspect of the cases. We were not only trying to win the argument, but come to grips with the philosophical issues deeply embedded in the cases as well as our own approaches. The sort of "on the fly" timed essays, talked about here, seem to negate philosophical thinking rather than encourage it.
    Anyone who can come up with an decent argument in three hours with unfamiliar material is either a genius, overly confident, or only dealing with the issue superficially (most falling into the last case).

    I would hope that in an ethics class, the focus should be on the importance of ethics and promoting the thoughtful consideration of an issue before coming to a decision.

    I do think that this could be a great way of testing a student's ability to apply ethical systems, or timed critical thinking skills. Perhaps this should be used as only part of the final in conjunction with a paper in which the student has time to properly digest all the implications of the issue at hand.

    Any thoughts?

    ReplyDelete
  7. William, I don't see this sort of exam as contrary to the aim of "thoughtful consideration" of an ethical issue. The students are directed to the relevant articles about a week before the exam, so this is not an exam that tests 'cold,' so to speak. They have plenty of time to familiarize themselves with the articles, and I encourage students to do precisely what you have to do in Ethics Bowl: study the material and be able to anticipate what questions you might be asked about. So I don't think it tests cleverness. It tests depth and richness of understanding, as well as the ability to anticipate issues and questions.

    ReplyDelete

If you wish to use your name and don't have a blogger profile, please mark Name/URL in the list below. You can of course opt for Anonymous, but please keep in mind that multiple anonymous comments on a post are difficult to follow. Thanks!