Monday, May 4, 2009

Lang's On Course: Student evaluations

Lang's chapter 13 deals with a perennially controversial teaching topic: student evaluation of teaching. Here I'll lay out some of his main claims about student evaluations and talk a bit about his suggestions for improving the results on one's evaluations.


First, let me say that my own reading of the literature on student evaluation supports Lang's conclusion that much of the conventional skepticism about student evaluation is misplaced. By and large, student evaluations validly measure certain aspects of instructor performance (whether students have learned from the course, their enthusiasm, how well the exams or other evaluative instruments corresponded with the course content). At the same time, there are elements of instructor performance that students are not well positioned to evaluate: the choice of textbooks or course content, the instructor's knowledge or mastery of the course content, etc. My own sense is that Lange may underestimate how many bad student evaluation questionnaires are out there: Many that I've seen ask students questions that are outside their competence and fail to ask questions that fall within their competence. And many have basic methodological problems (for example, students are asked to consider double-barreled statements such as "The instructor's lectures were organized and informative").

Indeed, this may be one area where faculty autonomy is overrated. Colleges and universities vary in this regard, but at most of the places I have taught, departments are given some vague guidelines as to how to develop their questionnaires but are left to their own devices beyond that, with the result that different disciplines have questionnaires that vary dramatically in length, content, and so on. I wouldn't suggest absolute uniformity across an institution with respect to these questionnaires, but I wonder if having an outside expert develop the questionnaires might be a good idea. In any event, I'd be curious to know from commenters whether my remarks reflect their own experience of student evaluation in different institutions.

Since Lang's book is directed at new teachers, he gives some valuable advice as to how to improve your student evaluation results.
  1. Know the form and what's asked.
  2. Get additional feedback from students earlier in the term, using techniques such as minute papers, 'muddiest point' exercises, and questionnaires.
  3. Be transparent as you teach, explaining to students how the various activities you provide them promote their learning, etc.

I think Lang is right on target with these suggestions, but let me echo and underscore a few points.
  • When you get additional feedback, do what Lang proposes and discuss the feedback with the students. Students are willing to forgive our missteps and will tolerate high academic standards if they see that we are acting in good will. Hence, when students give us feedback and we not only acknowledge it but also indicate how we intend to incorporate that feedback in the future, we engender that good will -- and better evaluations result.
  • Constantly remind students of the course learning objectives. One challenge for students, especially in philosophy courses, is that they are often uncertain as to what they are supposed to learn. If so, then student evaluation becomes a misguided exercise: Students are being asked whether they learned X but their attention is focused on whether they learned Y.
  • Foster the habit of student self-evaluation. Many of us are nervous about student evaluation because it may not do a good job prying apart our contribution to student learning from students' own contribution to their learning. Clearly, the interaction between these contributions is complex: Sometimes great teaching produces great learning, but it may not always do so. In light of this, I've found it important for students to get in the frequent habit of evaluating their own performance and efforts. This encourages a more mature attitude to instructor evaluation because students are more aware of their role in the collaborative learning process and so are more able to focus their attention specifically on how the instructor helped or hindered learning.
Finally, I think one thing that bothers many teachers about student evaluation of their performance is that the information gathered is often not put into context. I'm familiar with institutions where student evaluations are the only data used to evaluate teaching performance. Student evaluations are valid but limited, and need to be supplemented by other kinds of information: peer visits (though these don't take place often enough to be of great value), expert consultations, instructors' own self-evaluations, etc. Student evaluations measure the outputs of an instructor's teaching efforts, but these other sources of information can help measure the inputs to determine if the instructor's teaching is basically sound, regardless of whether it happens to be effective in producing student learning in a particular case. Teaching is, as Lang reminds us in this chapter, as much art as science.

5 comments:

  1. This sounds about right. The most important aspect is that too many institutions use teaching evaluations as the sole method of evaluation. One extra piece of advice, especially to those starting out: in your reviews, acknowledge any patterns in negative comments (look for patterns - not anomalies) and set out a specific set of changes you will make to address these comments. For example, I had a pattern of not turning material back promptly. Over time I tried several different changes, few of which worked. Finally, I realized I was giving too many assignments. The point is that a good review committee isn't looking for perfection - they are looking for you to acknowledge weaknesses and to have a plan for addressing them. It is easy to get defensive (again ignore the strange anomalous negative comments) but if you try to maintain a practical attitude it can help your teaching and your review process.

    ReplyDelete
  2. I guess I'm not surprised, but some of the myths and irrational suspicion still linger: Check out the discussion of student evaluations at Philosophers Anonymous:
    http://philosophersanon.blogspot.com/2009/05/teaching-evaluations-ignore-them.html

    ReplyDelete
  3. I've never taught at an institution that had different evals for different departments. In fact, every place I've been has had a very rigorous process by which the evals were created, with a lot of input from the faculty. I think that this is a good thing, as the faculty were able to raise concerns with loaded questions or questions that presupposed knowledge that the students will not have.

    Also, at every place I have taught (except for an adjunct job at a CC when I was working on my dissertation) the evals are the only measure of teaching used. The CC had classroom visits, but only for the adjuncts. My current chair reads over syllabi and things like that, but that does very little to assess teaching ability.

    ReplyDelete
  4. I haven't read the chapter because my book is actually in China, but thanks for the summary. I think you're definitely right about discussing the feedback. I've developed a midterm evaluation form that is ridiculously more informative than the final evaluation form at our university to get feedback mid-semester.

    One other thing I'd like to urge is mentioning (to current courses) that certain changes are the result of past teaching evaluation input. I sometimes get dismayed when I make a big change to a course in response to student feedback, but the students don't see it as a change (and how much better the current policy/assignment is!) Constantly mentioning last semester's teaching evaluations goes a little way towards fixing it. I'm also hoping that it gets them to take this year's evaluations more seriously.

    Right now I'm on the committee to reform teaching evaluations at our school. It's a daunting task. When I asked students what would get them to take the form more seriously the number one thing they mention is visible results. Of course, unless they take multiple courses from the professor this is rarely possible. But mentioning changes might go a little way towards satisfying this request.

    ReplyDelete
  5. Michael, I would be very interested to hear more information about how you do this:

    I've found it important for students to get in the frequent habit of evaluating their own performance and efforts.Re: Adam's comment--I think just letting them know that: (1) you have listened to students in the past, and (2) you have reasons for doing what you do, goes a long way to help students understand what you are doing and have some respect for it. I think they see our choices as a series of random events if we don't make a point of making the process transparent.

    I used to frequently ask for feedback, including asking for it on exams (that is, there was a section on an exam asking for some feedback, ungraded or worth a point for at least filling out a likert scale, with a disclaimer that it wouldn't affect their grade.)

    I would then make a point of addressing what they said. I didn't always change what I was doing, but I would explain why I was doing something in a particular way even though several students asked for something else. The students seem to appreciate that I listened to them, even when I didn't accept what they wanted. My end-of-semester evaluations went up when I engaged in this, but even better, the rapport with the students throughout the semester was better.

    ReplyDelete

If you wish to use your name and don't have a blogger profile, please mark Name/URL in the list below. You can of course opt for Anonymous, but please keep in mind that multiple anonymous comments on a post are difficult to follow. Thanks!