Wednesday, September 14, 2011

Evaluating student learning without student evaluations

Even though I'm less skeptical than others about the validity of standardized student evaluations, I share many of the common reservations about them: that they are often poorly designed, asking students what they are in no position to evaluate; that they reflect, to some extent, students' grade expectations; that small differences in instructor evaluations probably do not measure significant differences in instructors' effectiveness; that they are influenced by irrelevant factors like instructor gender; that students are actually not honest when they complete them; that their usual timing (at the end of the term) influences the results.

But one way to counteract this is to develop your own instruments for evaluating student learning that are more informative and appropriate. This post at Faculty Focus describes some methods, and I mention some here. For example, I give out a midterm self-evaluation (available here) that students are required to turn in. It asks, among other things, how challenging the course is, whether it meets their expectations, how well they're meeting the course learning objectives, what helps and hinders their learning, what changes they'd suggest, etc.

I've found this to be an enormously useful tool. Let me point to four reasons why:
  1. It's required, which the 'official' student evaluations aren't. So I get a high rate of return. 
  2. The 'indirection' has a positive effect on the quality of the feedback I receive. I call it a 'self-evaluation' rather than a course evaluation, but in evaluating their own learning, the students are obliquely evaluating the course. My observation has been that, for whatever reason, when students are compelled to think about themselves in the context of a learning environment, they are actually more thoughtful than when they are asked directly about the learning environment.
  3. Since it's in the middle of the term, I can actually put the feedback to use during the course. And since I share the feedback with the students, this helps to remind them that I'm listening and we're on the same page. Discussing Lang's On Course, I wrote: "Students are willing to forgive our missteps and will tolerate high academic standards if they see that we are acting in good will. Hence, when students give us feedback and we not only acknowledge it but also indicate how we intend to incorporate that feedback in the future, we engender that good will -- and better evaluations result." The effect of showing that you're listening is a powerful one.
  4. Students react positively to this because it seems customized for the learning experience they're actually in, instead of being a standardized institutional form. They seem less skeptical that the information I gather won't just end up being tabulated and forgotten.
Long story short: I've found that creating your own instruments to evaluate student learning can have a very positive effect on one's teaching. Does anyone else have 'custom made' learning evaluation instruments they use or advice about creating them?

1 comment:

  1. I've used something like this method in all my courses, and I've had success with it. I've also begun writing a letter (email, of course) to students reporting the results and commenting on some of their comments. That way, the evaluation process becomes more of a conversation, and they know for sure that I've read what they've taken the time to write. It also gives them a feel for what other students think about the class. For example, someone who thinks the class is too hard might realize that he has the minority view, and that might prompt him to change his expectations.

    Of course, I don't know how effective this method is for improving teaching or student satisfaction or student learning or whatever. But my end-of-semester evaluations have improved since I started doing this.

    ReplyDelete

If you wish to use your name and don't have a blogger profile, please mark Name/URL in the list below. You can of course opt for Anonymous, but please keep in mind that multiple anonymous comments on a post are difficult to follow. Thanks!