Quality teaching: Just add seasoning?

This article from the Journal of Political Economy got my attention with its provocative conclusion about the relationship between instructor experience and student learning. The researchers, Scott Carrell and James West, studied calculus courses at the U.S. Air Force Academy, a nice controlled setting: All students have to take the course, and instructors use an identical syllabus and administer identical exams. They then considered how students fared in related courses over the long-term, comparing the subsequent academic performance of students taught by more credentialed instructors to those taught by those with less seasoning. Drum roll...

Results show that there are statistically significant and sizable differences in student achievement across introductory course professors in both contemporaneous and follow‐on course achievement. However, our results indicate that professors who excel at promoting contemporaneous student achievement, on average, harm the subsequent performance of their students in more advanced classes. Academic rank, teaching experience, and terminal degree status of professors are negatively correlated with contemporaneous value‐added but positively correlated with follow‐on course value‐added. Hence, students of less experienced instructors who do not possess a doctorate perform significantly better in the contemporaneous course but perform worse in the follow‐on related curriculum.

Student evaluations are positively correlated with contemporaneous professor value‐added and negatively correlated with follow‐on student achievement. That is, students appear to reward higher grades in the introductory course but punish professors who increase deep learning (introductory course professor value‐added in follow‐on courses). 

The more experienced instructors' ability to foster deep learning didn't show up in student evaluations of teaching.  In fact, the less experienced instructors tended to have better student evaluations but their students did poorer in the long run.

Carrell and West draw the obvious conclusion about student evaluations:

Since many U.S. colleges and universities use student evaluations as a measurement of teaching quality for academic promotion and tenure decisions, this latter finding draws into question the value and accuracy of this practice.

Putting aside the student evaluation issue: Does this conclusion resonate with others' experience? And how might we explain this phenomenon? The study suggests that untenured or adjunct faculty have incentives to teach in ways "that have great value for raising current scores, but may have little value for lasting knowledge."

That strikes me as correct, but as only part of a very complex story. Some not terribly unified thoughts:

1. When I've observed younger teachers, I've noticed a strong orientation toward the immediate mastery of course content. In my own teaching, I now worry less than I once did about content mastery. I know that the details of, say, Mill's argument for the principle of utility aren't likely to linger in my students' minds for long. What might linger are the skills that result from repeated practice at extracting arguments out of complex texts, figuring out the logical relations among the arguments' part, assessing the arguments for their soundness, etc. I see this as a maturation process: As we teach, we become clearer as to why we teach, and for the most part, we philosophy teachers know that only a handful of our students will major in philosophy, and fewer still will study philosophy post-college. Learning is the lasting transformation of one's cognitive repertoire. For the vast majority of our students, what they can learn from us is not the details of particular arguments, positions, schools of thought. Best to teach them what can be learned — and what can be reinforced by subsequent academic and life experience. They're not going to be exposed further to virtue ethics or to the ontological argument. So I 'teach' those things as the vehicle for what I actually want to teach: the set of dispositions or aptitudes that constitute philosophical thinking. Which brings me to a second thought...
2. As my career has proceeded, I've been more mindful of the fact that I teach in an institutional context. I won't be the only faculty member responsible for improving a student's writing skills, critical thinking, etc. Indeed, I think more about how what I do in introductory philosophy prepares the students for other academic settings. And I think of myself not as teaching philosophy qua content, but as leading students along one path to the body of knowledge that their college education is supposed to provide them. I doubt if many new instructors see their teaching in a cross-institutional way.
3. Lastly, academic freedom and tenure often get discussed in the context of research. But I wonder if the conclusion of the study highlights an unappreciated advantage of academic freedom and tenure: They free us to teach in ways that foster deep learning by inoculating us from the fears of challenging students in the ways necessary to foster deep learning. You can't learn what you already know, and learning is change. These two facts make teaching a challenge. But if we want to teach something that might last, we have to embrace that challenge and challenge our students in turn. For myself, I find myself more willing to let students experience the strife associated with learning. I still maintain the high expectations I had early in my career, but I come to their rescue less. I do less of their thinking for them. I let them experience the discomfort that learning anything worth knowing necessarily entails. I'm less afraid of students being disoriented. And I have to say that being tenured helps here. I'm not worried about student complaints about the difficulty of my classes, say. (I don't mean that such complaints are necessarily illegitimate. I now see the rare complaint as evidence that I've discombobulated their intellectual universes a bit — exactly my intent!) In short, I'm now more willing to do the ugly Socratic midwifery of genuine learning than I once was.