I cannot think that the habit of evaluating one's teacher can encourage a young person to long for the truth, to aspire to achievement, to emulate heroes, to become just, or to do good. To have one's opinions trusted utterly, to deliver them anonymously, to have no check on their truth, and no responsibility for their effect on the lives of others are not good for a young person's moral character. To have one's opinions taken as knowledge, accepted without question, inquiry, or conversation is not an experience that encourages self-knowledge.
Wednesday, November 5, 2008
Are student evaluations bad for students' moral development?
I'd be interested (to rip off Brian Weatherson) thoughts, arguments and rants about the following quotation from Michael Platt. True or false?
Subscribe to:
Post Comments (Atom)
I think it's probably true as far as it goes, but then, no one thinks student evaluations are there in order to encourage the pursuit of justice or the development of self-knowledge; and students have no power to affect whether their opinions are "take as knowledge, accepted without question, inquiry, or conversation". Indeed, I rather suspect that most students aren't ever been given a clear idea what's done with their evaluations, anyway; they have a tendency to treat them like those customer service evaluations you get in hotels and restaurants. No one ever gives them a good idea why it's important or a precise notion of what will be done with it (I think at many institutions their professors are kept somewhat in the dark, too, never knowing more than the general outlines of the process); it's just put in front of them and they are told to do it. I don't think they can really be blamed for any of that; and it's not the habit of evaluating one's teacher that's the real problem.
ReplyDeleteI think we should also consider the complementary argument:
I cannot think that a habit of not being evaluated by one's students could encourage a teacher to pursue truth, aspire to achievement, to be just or to do goo. To do a job without being evaluated by those with whom it is most intimately concerned, to give students no definitely anonymous and thus safe way to voice any complaints, to have no check on any lapses or excesses, and not to be held accountable for their effect on the lives of others are not good for a teacher's moral character. A teacher is in a position of power, students are to some extent at the teacher's mercy, and without something to compensate for this fact, this is not a situation that encourages self-knowledge.
This reminds me of an upper division course I took in my junior year called "Multicultural America" in which I wrote on the evaluation form, "This is the worst professor I have ever had." Another student in the class who was walking past me noticed what I was writing, and she said, "I wrote the same thing!" That is why the course evaluation forms are done. If the whole class thinks you're good, you might be good. If the whole class thinks you're bad, you're probably bad. If the class, as a whole, thinks you're average, then you're doing an acceptable job.
ReplyDeleteMost of Michael Platt's comments could - mutatis mutandis - neatly map on the little election we had (in the US) the other day. It seems like we raise our youth in such a way that we deem it appropriate that they can make private decisions in a sensible way. It's true that our students might not have been expressly told how to assess but the parallel with voting seems to still apply. My guess is that the skills required for completing these tasks are part of the "tacit given."
ReplyDeleteThere are many, many problems with students evaluations. Platt's worries are not among mine. The goals he spells out are the work of the classroom, not the work of a few minutes filling out the very problematic forms. Brandon points out one of my worries, i.e., that such evaluation reinforce the corrosive notion that students are consumers and that we are a business.
ReplyDeleteI think this is a useful discussion, so allow me to point out some of my worries:
Small institutions with classrooms from between 7 to 40 students are not receiving statistically useful information from these forms. That would be all well and good if the institution is using them to weed out the very worst, as Kevin points out. But that is in fact not how they are used. Indeed, very fine-grained differences in percentage points are used to make decisions regarding employment and salary when such fine-grained distinctions map on to nothing statistically significant.
At small institutions, I suspect - and this is pure speculation - that there may be student evaluation inflation. This makes these forms even less informative.
Too many institutions that make teaching one of its fundamental measures of excellence - and thus a measure used to decide on employment and salary - use student evaluations and evaluations alone as a measure of this element. By contrast, consider the various elements used to measure scholarly success, e.g., external letters, publications, presentations, editorial board positions, etc. That an instructor's work as a teacher be evaluated along a single, defective dimension is very poor practice, and diminishes undergraduate education.
There are less worthy worries that I think undergird a lot of the griping about evaluations. Allow me to use a syllogism to capture one. A person is able to evaluate a teacher only if she is an expert in the field being taught. No student, by definition, is an expert in the field being taught. Therefore, no student, by definition, is able to evaluate a teacher. While I sympathize with the sentiment because I have had students who have written "She doesn't know what she is talking about," I disagree with the syllogism. I think students are in a very good position to tell us whether they learned and why they learned. I know I need this feedback.
But I think we are in a good position as well. We can compete with this defective practice by making clear the learning outcomes we expect our assignments to achieve and by documenting whether and to what degree such learning outcomes are met. It's a lot of work, but it might help when the cash-strapped administration comes knocking on the door talking about us missing the threshold by .5 points.
To end this too long post: the problem is not the evaluations. It is the use to which they are put. We desperately need feedback from our students. It is a shame that such a vital need is used as an often single measure of whether we deserve to remain in the field or receive a salary adjustment.
Great comments everyone!
ReplyDeleteKevin - I believe there's a lot of research to support your claim that student evals can ID poor teaching but are pretty weak at distinguishing quality above that. This in turn echoes Becko's worries about small differences being statistically noisy.
John Sager - I'm not sure I follow the analogy to the election, but I wonder if you're interested in what we might call the epistemology of student evals. The epistemology of student evaluations is very atomistic rather than social. Like voting, the student offers her evaluation of the teacher by herself, filling in bubbles on her eval form. I can understand wanting to give students autonomy, but shouldn't we also want the students' beliefs and perceptions to be checked against others? I'm guessing that a few people walk into the voting booth having not given a moment's thought to their vote, students almost never talk seriously about teachers' performance. This may have the result that snap or extreme reactions show up in student evals. I wonder if there's a way to keep student evals in the picture (like Becko, I think they're an important source of feedback) but make them less atomistic. A couple of times during my career, I've had what's known as a Small Group Instructional Diagnosis (SGID) done in my courses. In a SGID, an outsider comes to your class and collects the consensus student response to three questions:
1. What is most helping you learn in this class?
2. What is most keeping you from learning in this class?
3. What suggestions do you have to improve the learning experience in this class?
I've found the results of a SGID far more valuable than normal student evals. First, since only the consensus is reported, I don't have to work out on my own what the consensus is. Second, I can direct my efforts where they're likely to do the most good, rather than responding to the concerns of a particular student that may be idiosyncratic. Third, the SGID is social rather than atomistic in nature. In a conversation focused on these three questions, students see that their own experiences in the classroom may have more to do with their own habits or efforts than with the instructor's methods. The SGID thus illustrates to students that they have duties in the collaborative enterprise of learning. Following Brandon, it'd be a shame to reject student evals altogether, since instructors need the self-knowledge that students provide no less than the students need to be encouraged to have self-knowledge. But I think worth thinking creatively about different models for student evaluation.
One more thought: Students are at the 'back end' of the educational experience. I think we have, or are developing, a pretty good idea of the principles, practices, etc., that are conductive to student learning. The "trouble" here is the students themselves. Not all students will put in the effort, etc., which (when it occurs in the context of teaching in accordance with the aformentioned principles or practices) is very likely to produce learning. Teachers aren't miracle workers. The best we can do is design effective learning experiences, and after that, the students, with all their variations in knowledge, motivation, and so on, take over. So one reason for reluctance about student evals is that we fully control the inputs, but the students control a lot of the outputs. This fact leads me to think that evaluating teaching performance should focus mainly on the inputs rather than the 'back end' outputs. Terry Doyle (Ferris State University) says it well:
"Teachers also have limited control over many of the most important factors that impact students’ learning, including students’ attitudes, background knowledge of the course content, study and learning skills, time students will spend on their learning, their emotional readiness to learn, and on and on. Since there is clearly a shared responsibility between the teacher and the student as to what that student learns, and because many students are able to learn in spite of the teacher, while others fail despite all of the best efforts of a skilled practitioner, the definition of “teacher effectiveness” appears to be, as Derek Bok put it, “an act of faith” on the part of students and teachers to do their best."
Sorry for my lack of brevity -- this is a great topic.
Michael,
ReplyDeletePlatt's comments seem largely hyperbolic. This is close to a rant, isn't it? I have never seen student evaluations considered "knowledge" or "accepted without question". They are in general given a statistical weighting depending on lots of factors including the student's preparation for the course, the students desire to take the course, the student's major, rank, the class size, etc. I'm not suggesting that, even with good instruments for statistical weighting, the evaluations are very reliable, but they certainly don't negatively affect students' moral character (yeesh, what a stretch) and they are much more reliable than peer review.
Mike -- Its rantiness is kind of why I wanted to discuss it. I agree it's hyperbolic, but even hyperbole can contain glimmers of insight. In this case, I'd say the insight may be misexpressed in terms of "moral development." I too doubt that student evals of the usual sort make students morally worse. But on the other hand, they also don't seem to encourage a very self-conscious attitude toward one's own learning and seem to endorse, however subtly, an adversarial or non-collaborative picture of learning.
ReplyDeleteI do think that Becko's worries that they are accepted as "knowledge" at some institutions is worrisome. And I've not taught anywhere where the statistical weight you describe is applied to student evaluation data:
"a statistical weighting depending on lots of factors including the student's preparation for the course, the students desire to take the course, the student's major, rank, the class size, etc."
Would that it were so!
"a statistical weighting depending on lots of factors including the student's preparation for the course, the students desire to take the course, the student's major, rank, the class size, etc." Would that it were so!
ReplyDeleteBut it is so! For at least 7 years, we have been using the IDEA survey for course evaluations that has multiple statistical dimensions and is scored outside the university. We get comparisons to similar courses offered at universities nationally, and specific comparative scores on the aspects of the course instructors list as essential. In addition, IDEA allows specific departments to add their own discipline specific questions. In further addtion, each department offers students the opportunity for a separate narrative assessment of the their course.
There are nonetheless (the inevitable) complaints when raw scores are higher than adjusted scores. The IDEA survey is a very well-tested assessment tool, though it has the limitations of all such tools.
I don't understand how student evaluations could be taken at face value at any institution that has actually taken 10mins to think about it.
I know it is hard to believe, but they are taken at face value at many institutions. I know this from an informal survey of friends of mine at institutions where teaching is an important factor in employment and salary considerations. In addition, at such places the evaluation is an in-house product, often designed by committee. I'm glad, Mike, that your institution is more responsible and more aware of the unscientific nature of evaluations. But your experience is far from representative.
ReplyDeleteI fear, Mike, that most institutions are not terribly sophisticated, or even especially reflective, about what the student evals are supposed to signify. It's great that you're somewhere that puts the numbers in context and invites a careful discussion, but I have little evidence that this is typical.
ReplyDeleteThanks Becko and Mike. I was more or less trying to illustrate that even (what I think is) a very good instrument gets complaints from faculty. I'm prepared to concede that every approach is going ot have some serious shortcomings. The dreaded alternative (sometimes discussed here) is peer evaluation. Anyway, there it is.
ReplyDeleteAs a student I've always hated having to fill in professor evaluations as well. I've always felt them to be a pointless waste of time and only when I have a really atrocious professor do I put effort into filling out the evaluations properly. I know I shouldn't have this attitude towards the evaluations because it does affect whether or not the profs I like get tenure, or get raises and such, but one thing that must be kept in mind is that unless I really care about a professor I'm not going to bother filling it out properly (often everything is just neutral).
ReplyDeleteI acknowledge that this attitude is due to how my generation have been brought up, and I myself have had some bad experiences with anonymity in regards to what I have written on student evaluations. This generation has been filling out teacher evaluations since Primary School, and often we have been lied to as to whether the evaluations would be confidential or not. For example, drawing on personal experience I have been put on report twice (once in primary school and once in secondary school) for evaluating a teacher as terrible - I was deemed as having an attitude problem towards authority figures.
Many of my fellow students have had similar experiences with student evaluations before even stepping foot inside an establishment like an University. I cannot speak for all students, only the small number that I know, but for us, we're wary of student evaluations because we have a history of being lied to by authority figures in educational institutions. We doubt the sentiment behind student evaluations and as such the only time we put effort into filling one out correctly is when the prof is atrocious.
On another note: I don't remember the source for this information, but I remember one of my professors actually bringing it up: apparently there was a study done that found that the results of a group of students when shown a 1 minute clip of lecturer and then asked to fill out an evaluation form was remarkably similar to the actual evaluation forms from the class. The only point at which the results began to diverge was when the clip was 10 seconds long or shorter.