One of our loyal readers, Gary Bartlett (aka 'Gazza'), wrote me last week about my post on a more retrospective essay exam idea I was tinkering with. Gary confessed to being skeptical about essay exams in general, and his thoughts were provocative enough that I thought we'd post them here (starting below the fold).
GARY: I started to write the following thoughts in response to Michael’s post about exams last week, but I realized they were rather too tangential to his post. So Michael kindly offered me this chance to air them independently. I’d like to hear what others think.
I’m against essay exams in philosophy (and in others disciplines, I think, though I’ll stick to philosophy here). I think exams of any kind have only limited use in philosophy. I use them only in introductory classes, and I ask only short-answer and multiple-choice questions. (Multi-choice questions are an issue on their own, of course. I won’t address that here except to say that I used to think they were inappropriate for philosophy, but I changed my mind. With care and attention put into their construction, and used judiciously, I believe they can be very effective for certain purposes.)
My first kind of reason for thinking that essay exams aren't a good fit for philosophy classes is similar to some of the worries that Michael voiced in his post last week. Even given the revised format he suggests, though, we’re still asking students to produce the essay or essays in a matter of 2-3 hours. And that doesn’t fit well, I think, with the goal we have for our classes. We want to see students engaging with texts and ideas in a thoughtful, reflective way. Asking students to turn out an essay within an hour or two is not very conducive to finding out whether they can be thoughtful or reflective about the material, because such things take time. Essay exams are more conducive to finding out how much of the course material students have memorized. But although memorization of material is perhaps some part of what we want students to get out of our classes, it is only a small part, and it’s only there at all in the service of the more important goal of reflection, thoughtfulness, and so on.
Open-book essay exams would be better, since they would reduce the emphasis that the format places on being able to remember material. But still, in an exam situation it’s not really going to be possible for students to engage in careful and critical reading of and writing about texts.
A second kind of worry I have is that essay exams foster in students a damaging attitude toward academic writing: the idea that writing an essay is something one does relatively quickly. Students already tend to think that papers get written ‘in one fell swoop’, in some sense: that you start at the start, you write for a while, and then you’re done. I don’t suppose that giving essay exams will make students think that we write papers in an hour or two; but it probably does reinforce their idea that papers don’t involve much in the way of reflection, re-writing, and so on.
What do others think about this? Are there benefits to essay exams that I’m missing? (I do, of course, assign essays for my classes – just not in exams.)
I agree with your points about what essay exams communicate to students about philosophical writing. If we want students to reflect, revise etc, we ought not put them under time pressure and ask the to write from memory.
ReplyDeleteI also think that short-answer/muliple choice etc.. kinds of assessments are of limited value, as are reading quizzes.
On the other hand, if there isn't the threat of an exam with some sort of time pressure, they simply won't do the reading.
My solution, although imperfect, is to give a limited-time take home exam. The idea is that they get the questions on Tuesday in class, it is due on Thursday in class. They'll typically have 10-15 "terms" to define and maybe a choice of one out of two possible long essays. The idea is to write an exam that would be quite difficult for students who haven't been keeping up and that is only moderately challenging for those who have been doing the reading and coming to class etc.
THe other alternative is to release four exam questions in advance, telling them that they'll be expected to write on one. I've allowed them to write the answers in advance and simply turn in the one I ask for at the beginning of the exam. If they choose not to write in advance, they can write it in he class period.
One positive about the essay exam is that it is one essay assignment you can be pretty sure was actually written by the student. Plagiarism on an in-class exam is hard (I suspect, though I don't speak from experience), especially when the professor knows your name and what you look like.
ReplyDeleteI agree with much of what has been said about the disadvantages of examination essays, particularly the problem of time constraints. However, I don't see the point of an examination essay as being a test of memory. It is a chance to see how well students can think when they have nobody else to help them. For this reason, I do not give my students the titles in advance, nor do I allow them to bring in any material to help them. This is not so much because I want to prevent plagiarism, as because any books that they brought in with them would most probably be a distraction anyway.
ReplyDeleteSome sample questions from my Introduction to Philosophy class:
"Should religious believers be embarrassed when people say ‘Yes, but who made God?’"
"If, as many people say, computers can play chess but cannot think, does it follow that a human could learn how to play chess to a good standard without thinking about the game?"
"Compose a letter to Sir Peter Strawson in which you help him overcome his difficulties in understanding what the thesis of determinism is."
These are related to topics we had examined in class, but my aim was not to see whether they could reproduce a summary of Aquinas or Searle, but to see whether they could understand these questions well enough to offer a reasonable response.
We had spent time in class discussing Strawson's Freedom and Resentment, and I commented that Strawson's statement that he was unsure what 'determinism' really means was a typical strategy for analytical philosophers. The point of that question was to see whether students could propose any good reasons why the definition of 'determinism' might not be straightforward.
I only expect one essay in a two hour examination. I provide five titles for them to choose from, and encourage them to spend time planning their answer. I hope this means that the time-constraint does not become a source of stress.
Philosophy Factory: I agree that short answer & multi-choice do have limited value, but not (as you perhaps mean?) zero value. In an Intro class there are some basic concepts and methods that students should be familiar with and know how to manipulate, and I use SA/MC questions in exams to test that. As for getting them to do the reading, I deal with that by having a daily question drawn from the reading for that day. (I keep these questions exceedingly simple, requiring at most a few words in answer; they don't have to be, and usually aren't, 'philosophical'.)
ReplyDeleteKevin: One way to exercise some control over plagiarism is to require students to submit a draft. Admittedly that's probably not as sure-fire as an exam, but I guess I think the downside of essay exams outweighs the plagiarism risk.
Ben: I like your questions. But I would still rather give students time and resources to write their answers rather than asking them to do it on their own in 2 hours.
I used to be skeptical of essay exams, but a friend in Religious Studies persuaded me otherwise and I have gotten really good results.
ReplyDeleteHere is what I do: The students receive six to ten essay questions at least two weeks in advance. From those I choose a subset on the day of the exam, and students are allowed to choose a smaller number from that subset.
I do this now because I like to give a variety of assignments - some people are better at exams than they are at papers and vice versa. I like to maximize the possibility that some assignments will play to various student's strengths.
I got a really surprising result: the writing in the essays was usually of a much higher quality than their papers. Once I noticed this, I did two things. First I showed them examples of the differences between the two pieces of writing and used this as an opportunity to teach some writing skills. Second, I turned the essays into drafts for a separate paper assignment.
As ever, the ability to get the most out of this project depends heavily on class size.
Ben - I'm with you - I give two in-class essay assignments - a mid-term and a final. The questions are driven by the discussions and materials covered in class. Students are, thereby, forced to attend class to be part of the discussions that show up on the essays. In addition, my students write four out of class essays. These essays are critical essays.
ReplyDeleteI don't like multiple choice - anyone can memorize anything for a brief time (hence the propensity to cram for a test). These tests do nothing to promote the ideal lesson of an introductory course - learn how to aquire information, learn how to assess information, learn how to apply information.
I do exactly what Becko does. I've been pleased with the results, and I think the method undercuts Gazza's objection to essay tests.
ReplyDeleteWe agree, I take it, that an exam (of any kind) would be good if it required students to think carefully about the relevant topic, develop a coherent position on it or a clear view of someone else's position, and learn to articulate that position clearly and concisely. The objection is that can't achieve these objectives in two hours. I agree. But Becko's method gives them two weeks to achieve them at home. And even better, students need to solidify their understanding enough to reproduce a semblance of it in the classroom under pressure. Doing that is, I think, good evidence that they understand the material well.
However, I've also been persuaded (mostly by people on this blog) of the virtues of multiple choice tests (even over short answer). When well constructed, they require much more than memorization of material.
While I don't think I've had as much success as Becko has had, I'm also someone who has grown somewhat disillusioned with essay exams over the years. For probably all of grad school and my first year in this job, I gave all essay exams. Students did well. Then I thought I'd try out multiple choice questions one semester. I was shocked that students weren't able to answer even simple multiple choice questions about the same topics as the essays.
ReplyDeleteSo I refined both my multiple choice techniques (if you know it, the answer should pop out at you) and my essay exams. I give them a small number of possible essay questions for the exam, but I also allow them to bring in outlines for their essays (NOT fully written out essays) to the exam. This (I think) forces them both to know the topic well enough to fill in an outline and it helps them write fairly well-structured essays in their own time.
I've found this works pretty well and takes a lot of anxiety out of the essay exams.
Gazza's post has generated some great discussion. I don't think it makes sense to be dogmatic about exam formats. Essay exams can be constructed well or badly, as can multiple choice, etc. I also think that some of Gazza's initial worries are mitigated once we consider that whatever we ask students to do in exams is accompanied by many other learning tasks that complement the exams. So, e.g., I'd be more worried about Gazza's second worry if I didn't also have other writing assignments that did demand reflection, careful textual engagement, etc. And one of the aims of the exam format I described in the 'Please. Do go on' post is to have an essay exam that actually demands some measure of reflection and pre-writing.
ReplyDeleteAnd I think I agree with Ben that memorization is not what a good essay exam tests. It tests how much conceptual and logical mastery a student has acquired, and more specifically, whether she can think in new ways about the material she has mastered. Just to give a simple example: You can't successfully discuss objection O to S's argument for thesis T unless you have a fairly rich understanding of T, S's argument for T, and why O is relevant to T and to S's argument for it (and perhaps the context in which S makes the argument for T). In other words, good essay exams tell us a lot about how far up a curve of understanding a student has climbed.
And one last small point: Granting that we want students to produce thoughtful philosophical writing, timed essay exams encourage another learning goal I think is important, learning to think on your feet in a pressure situation. Yes, we should want to emphasize the value of reflection, but it's worth keeping in mind that a lot of our students' future lives will not afford them as much reflection and will ask them to show their competence on demand, so to speak.
Thanks for all the comments folks! You've given me some things to think about. I think my core problem with essay exams was that I couldn't think of any benefit that they offered that was not also offered, and usually offered more effectively, by a normal (non-exam) essay assignment. But folks have now mentioned at least two possible benefits of essay exams that non-essay exams might not offer:
ReplyDelete1). The possibility of students producing better writing in an exam. I can sort of see how that might be the case. I often observe that students are their own worst enemies in essays, as tie themselves up in knots trying to do too much: they tend to over-reach in content and/or form (grammar and organization). The limited time format of an exam might help them avoid that problem.
2). Encouraging the ability to perform under pressure. I'm less sure that this is really valuable, though. Is it really important that students learn to write under pressure? (Maybe we'd just be encouraging them to procrastinate on their assignments...) I'm more inclined to think the value here is that of (1) again, where the pressure might force students to focus on what's actually important and not get onto the sidetracks they often get stuck on in their regular papers.
Gazza, just to clarify: My point was that essay exams compel thinking and expressing oneself under pressure. The writing part isn't what I had in mind.
ReplyDeleteI was revisiting this discussion, and it occured to me that I would like to know more about successful multiple choice exams - that is - can someone provide me with an example of a question that requires more than memorization? I use tests so infrequently, I guess I don't know how to construct a good one. Thanks
ReplyDeleteGail--I was just poking around and saw your question. I really like this discussion of multiple-choice exams: Testing for Deep Understanding
ReplyDelete