Wednesday, December 22, 2010

Quality teaching: Just add seasoning?

This article from the Journal of Political Economy got my attention with its provocative conclusion about the relationship between instructor experience and student learning. The researchers, Scott Carrell and James West, studied calculus courses at the U.S. Air Force Academy, a nice controlled setting: All students have to take the course, and instructors use an identical syllabus and administer identical exams. They then considered how students fared in related courses over the long-term, comparing the subsequent academic performance of students taught by more credentialed instructors to those taught by those with less seasoning. Drum roll...

Results show that there are statistically significant and sizable differences in student achievement across introductory course professors in both contemporaneous and follow‐on course achievement. However, our results indicate that professors who excel at promoting contemporaneous student achievement, on average, harm the subsequent performance of their students in more advanced classes. Academic rank, teaching experience, and terminal degree status of professors are negatively correlated with contemporaneous value‐added but positively correlated with follow‐on course value‐added. Hence, students of less experienced instructors who do not possess a doctorate perform significantly better in the contemporaneous course but perform worse in the follow‐on related curriculum.
Student evaluations are positively correlated with contemporaneous professor value‐added and negatively correlated with follow‐on student achievement. That is, students appear to reward higher grades in the introductory course but punish professors who increase deep learning (introductory course professor value‐added in follow‐on courses). 

The more experienced instructors' ability to foster deep learning didn't show up in student evaluations of teaching.  In fact, the less experienced instructors tended to have better student evaluations but their students did poorer in the long run.

Carrell and West draw the obvious conclusion about student evaluations:
Since many U.S. colleges and universities use student evaluations as a measurement of teaching quality for academic promotion and tenure decisions, this latter finding draws into question the value and accuracy of this practice.
Putting aside the student evaluation issue: Does this conclusion resonate with others' experience? And how might we explain this phenomenon? The study suggests that untenured or adjunct faculty have incentives to teach in ways "that have great value for raising current scores, but may have little value for lasting knowledge."

That strikes me as correct, but as only part of a very complex story. Some not terribly unified thoughts:
  1. When I've observed younger teachers, I've noticed a strong orientation toward the immediate mastery of course content. In my own teaching, I now worry less than I once did about content mastery. I know that the details of, say, Mill's argument for the principle of utility aren't likely to linger in my students' minds for long. What might linger are the skills that result from repeated practice at extracting arguments out of complex texts, figuring out the logical relations among the arguments' part, assessing the arguments for their soundness, etc. I see this as a maturation process: As we teach, we become clearer as to why we teach, and for the most part, we philosophy teachers know that only a handful of our students will major in philosophy, and fewer still will study philosophy post-college. Learning is the lasting transformation of one's cognitive repertoire. For the vast majority of our students, what they can learn from us is not the details of particular arguments, positions, schools of thought. Best to teach them what can be learned — and what can be reinforced by subsequent academic and life experience. They're not going to be exposed further to virtue ethics or to the ontological argument. So I 'teach' those things as the vehicle for what I actually want to teach: the set of dispositions or aptitudes that constitute philosophical thinking. Which brings me to a second thought...
  2. As my career has proceeded, I've been more mindful of the fact that I teach in an institutional context. I won't be the only faculty member responsible for improving a student's writing skills, critical thinking, etc. Indeed, I think more about how what I do in introductory philosophy prepares the students for other academic settings. And I think of myself not as teaching philosophy qua content, but as leading students along one path to the body of knowledge that their college education is supposed to provide them. I doubt if many new instructors see their teaching in a cross-institutional way.
  3. Lastly, academic freedom and tenure often get discussed in the context of research. But I wonder if the conclusion of the study highlights an unappreciated advantage of academic freedom and tenure: They free us to teach in ways that foster deep learning by inoculating us from the fears of challenging students in the ways necessary to foster deep learning. You can't learn what you already know, and learning is change. These two facts make teaching a challenge. But if we want to teach something that might last, we have to embrace that challenge and challenge our students in turn. For myself, I find myself more willing to let students experience the strife associated with learning. I still maintain the high expectations I had early in my career, but I come to their rescue less. I do less of their thinking for them. I let them experience the discomfort that learning anything worth knowing necessarily entails. I'm less afraid of students being disoriented. And I have to say that being tenured helps here. I'm not worried about student complaints about the difficulty of my classes, say. (I don't mean that such complaints are necessarily illegitimate. I now see the rare complaint as evidence that I've discombobulated their intellectual universes a bit — exactly my intent!) In short, I'm now more willing to do the ugly Socratic midwifery of genuine learning than I once was.


  1. Ugh, what does "contemporaneous" mean??

  2. Very interesting thoughts. I think you are on to something here. Watching my partner (a full professor) teach intro to politics I also wonder if more experienced teachers select the content to maximize the kinds of learning you talk about. He is very explicitly focused on teaching concepts that will form a solid foundation for future deeper learning about politics, for example, rather than on details of particular political systems.

    In my own past teaching of intro to sociology, I was also explicit in teaching the kinds of skills that you mention.

    I think it is important that we reclaim the notion of "skills" to include precisely the things you raise -- "extracting arguments out of complex texts, figuring out the logical relations among the arguments' part, assessing the arguments for their soundness, etc."

    I also think there is scope for institutions (or parts of them like departments) to make the institutional location of specific courses more obvious. I taught in the UK (which might be relevant in terms of academic culture) and we collectively discussed the content of core courses like intro and 2nd year theory and method courses even if only one person actually taught a particular course. We discussed things like what we expected students to know when they arrived in (what you would call) upper division courses with a more specialist focus. Although we didn't always agree, at least the conversation among faculty was open and all parties knew what was going on. Of course we also treated those core courses as common property in the sense that no individual should be "stuck" with one forever. We assigned them in a 3 year rotation (roughly) and the idea was that it should be possible for any of us to teach them.

    I think there is a lot to ponder in the common curriculum and assessment for big core courses, but I'd like to see it developed in a way that integrated those courses with a set of institutional offerings, which might mean involving those casual staff hired to teach (some of) the sections into at least part of the culture of the department.

  3. Hm. This strikes me as a very unfortunate result. From lower education, we know there's a big difference between the teaching of a good teacher and a bad teacher, and that identifying and firing the bad teachers is probably the most effective strategy for improving public school and education in general.

    This is the motivation behind recording value-added for various teachers, like the Los Angeles database we all read so much about:

    But this result seems to indicate that year-to-year is badly incomplete and actually gives opposite results!

    It may not generalize from higher education to lower, but then again, it may...

    So how are we supposed to judge teachers?


If you wish to use your name and don't have a blogger profile, please mark Name/URL in the list below. You can of course opt for Anonymous, but please keep in mind that multiple anonymous comments on a post are difficult to follow. Thanks!