Academically Adrift: Three Easy Questions; Three Not So Easy Answers

Question 1: What is the CLA

The CLA is the “Collegiate Learning Assessment” of The Council for Aid to Education and is the primary tool for the findings of Academically Adrift (AA). The Council for Aid to Education (CAE) is a national, nonprofit organization established in 1952 designed to advance the corporate support of education, to conduct policy research on higher education, to improve the quality of and access to education, and to conduct research on and promote policy reforms related to education. The Collegiate Learning Assessment (CLA) is a national attempt “to assess the quality of undergraduate education by directly measuring student learning outcomes.” The CLA is a performance-based assessment model, developed by CAE, that directly measures the quality of learning in undergraduate education “that all of the major stakeholders - university administrators, faculty, students, parents, employers, and policy makers - can use as part of their evaluation of academic programs.” The CLA is in many regards a considerable resource for gathering information regarding student learning.

The difficulty with the CLA, upon which Arum and Roksa base their conclusions, is that it is in many ways a limited tool for assessment. The test seeks to determine gains in three key academic areas: “[F]aculty members across subjects overwhelmingly agree that critical thinking, complex reasoning, and writing are key skills to be taught in higher education.” (AA 108) The authors, however, readily admit that “The CLA measures a specific set of skills…that is far from the totality of learning or the full repertoire of skills acquired in higher education. As students both select and are socialized into specific ways of knowing and thinking, they will perform well on the CLA to the extent that their disciplines emphasize the skills assessed.” (AA 108) Robert Sternberg at Inside Higher Education further points out, “These are all rather valid and reliable tests, insofar as they go, but they are narrow in what they measure. They achieve their reliability in part because they focus their assessments so narrowly. (So-called “internal-consistency reliability” rises to the extent that a test narrowly measures just a single construct.) So psychometrically, the tests are reasonably good ones. But the issue…is not how “good” the tests are, but rather, how well they are used--whether they have sufficient breadth adequately to serve as measures of learning in college.”

The result then is that, while the CLA is reliable for what it is, the CLA offers only a moderate and somewhat skewed version of what constitutes learning in college. It is important, I think, not to overstate the objection, since the findings of Arum and Roksa are instructive, but the difficulty now is in determining just how instructive the findings are, and what decisions we should make, if any, based on them.

A not so easy answer.


Question 2: What is the purpose of education?

Most of us became philosophers because of our affection and predilection for learning, particularly the learning of philosophy. In genuine Socratic imitation, then, to become a philosopher meant to become a teacher, a midwife for learning, and so we did. According to William Damon, as cited in AA 126-127, the purpose of education is to develop in others “a love of learning for learning’s sake—a love that will then lead to self-maintained learning throughout the lifespan.” I have yet to meet a philosopher who disagrees. Yet, for all the Socratic posturing about a life not worth living, many of us, and many of the policies of our institutions, continue to act contrary to the stated purpose of education.

At the institutional level, “[s]tanding in the way of significant reform efforts are, of course, a set of entrenched organizational interests and deeply ingrained institutional practices.” This is particularly true when we conceive of the profession solely in terms of research and publishing; this is particularly true when we penalize teachers for spending time thinking about their teaching; this is particularly true when we allow markets to dictate “best practices”, such as class sizes and acceptances. What seems worse is that “[l]imited learning is in no way perceived as a formidable crisis that threatens the survival of organizational actors, institutions, or the system as a whole…Students, parents, faculty, and administrators are not overly concerned with the lack of academic learning currently occurring in colleges and universities, as long as other organizational outcomes more important to them are being achieved.” (AA 143) Whatever the organizational outcomes are, they seem to be contrary, in practice if not in principle, to the purpose of education.

It is no better at the individual level. Many college professors don’t know how to teach, and to say so strikes at the heart of a hubris peculiar to academics: “We are the ones who know, because we have done the hard research, and our understanding makes us the ones best suited to teach that knowledge.” Yet, Ph.Ds. are rarely ever taught how to teach, and often times they are uninterested in learning how. There is, in addition, hardly any incentive for college professors to adopt “best practices” in teaching, which is different than in college prep schools. In pre-college education “elementary and secondary school teachers have understood the importance of high expectations for all students, and have been institutionally encouraged to demonstrate them. Unlike elementary and secondary school teachers, however, college professors have typically not received formal training in instruction that has emphasized the pedagogical functions of educational expectations.” (AA 130)

The unintended result of both the institutional and individual emphasis away form teaching has been a professional, academic culture dismissive of, even sometimes hostile toward, learning about teaching. Since academic institutions have gradually nudged professors away from thinking about teaching, often penalizing them for doing so, professors have embraced the culture academic institutions have created for them. This culture has nudged teaching, and therefore learning, out of the classroom. As Arum and Roksa put it, “[w]ith regard to the quality of research, we tend to evaluate faculty the way the Michelin Guide evaluates restaurants…We ask, ‘How high is the quality of this cuisine relative to the genre of food” How excellent is it?’ With regard to teaching, the evaluation is done more in the style of the Board of Health. The question is, ‘is it safe to ear here?’ (AA 121)

If the academic culture of higher education nudges teaching and learning out of the classroom, what becomes of the purpose of education? What Arum and Roksa recommend is that, “[i]nstitutions need to develop a culture of learning if undergraduate education is to be improved.” (AA 127)

A not so easy answer.


Question 3: What is the relationship of teaching to learning?

Finally, the relationship of teaching to learning is ambiguous. Above I argued that the institutional, academic culture nudges teaching and learning out of the classroom, yet it is false to say that where no teaching occurs, necessarily no learning occurs. In fact, much learning often still occurs, even with little to no teaching, perhaps not the learning that was intended, but learning nevertheless. As Arum and Roksa contend, if we are going to see improvement in collegiate education, we need to begin“…shifting attention from faculty teaching to student learning.” (AA131)

Such a shift, I believe, is a noble cause, but the ambiguity between teaching and learning is difficult to untangle. In conversations with my colleagues about topics such as poor student writing one of my faculty members made the distinction between “teaching writing and assigning writing”. Many faculty members, he argued, think that assigning writing is the same as teaching writing. After all, we correct errors, make comments and give grades, right? The problem is that there is a huge difference between teaching writing and assigning writing, but since writing (and critical thinking and research) come more or less naturally to faculty, we fail to fully understand how to learn to do it.

At the same time, many of us uphold academically rigorous standards in the classroom, and see positive results. Arum and Roksa explain, “[o]ur findings provide clear empirical evidence that academically rigorous instruction is associated with improved performance on tasks requiring critical thinking, complex reasoning, and written communication. Spending time studying, having faculty who hold high expectations, and offering courses that require reasonable amounts of reading and writing are associated with students’ learning during the first two years of college. These practices focus attention on the fact that students benefit when they are in instructional settings where faculty demand and students engage in rigorous academic endeavors.” (AA 129)

Further, they contend, “having demanding faculty who include reading and writing requirements in their courses…is associated with improvement in students’ critical thinking, complex reasoning, and writing skills.” (AA 93) It doesn’t seem to get any simpler: demand more of your students and you will get more. Yet, the ambiguity between teaching and learning persists. If the nature of increasing demand in the classroom means to assign more work (and thus grade more work), and the increase has an impact on student learning, we might be tempted to believe that the learning was correlated to the increased demands, which we often call “teaching”.

Herein lies a difficulty I have with the findings of Arum and Roksa. They tie learning to teaching, and have argued that with greater demandingness, the result will be greater student learning. Yet, they offer on tap only a model of teaching that is purely quantitative, not qualitative—assigning more, not teaching better. I agree with them when they reverse the emphasis from faculty teaching to student learning, but if we take seriously student learning, it should have real world implications for better faculty teaching, not just an increase of demandingness. There are real world disincentives to greater demandingness (it takes away from publications, committee work, tenure review), and if that is our only model of better teaching, then I’m afraid we shall remain academically adrift even if the test scores increase. If, however, we accept the purpose of education, and we take seriously student learning, we should uphold the same willingness to learn that we demand of our students. We should be more demanding, and hold students more accountable, but in turn we owe it to them to learn how to teach better.

A not so easy answer.