My department just got a small grant to work on assessment -- developing some mechanisms for working out what our students learn, and whether what they learn is what we want them to. The grant is for a pilot project that will focus specifically on two of our three large enrollment courses -- Intro to Philosophy (most of the students are first years, in Letters and Science) and Contemporary Moral Issues (most of the students are juniors and seniors, mostly from Business, or Letters and Science). We're going to do focus groups with faculty who regularly teach each class, to discuss what the course objectives are and how they generally assess whether the students meet those objectives. Then, with those objectives in mind, we plan to design a pre- and post-test (to be given very early in the semester and very late in the semester), which we'll use in all sections of the course in question. We are not aiming to use this to evaluate the professors -- but to find out whether what the students learn matches what we think we are teaching them. Because it is a pilot, of course, we'll be testing and getting ready to refine the instrument itself.
We also plan to gather together syllabuses, assignments, and run focus group discussions around grading practices (eg, by getting faculty to read several artifacts and assign grades, and discuss why they gave the grade they did).
It occurs to me that some of our readers might have experience -- and might even have existing examples of pre- and post-tests that they use in Intro Philosophy or courses similar to Contemporary Moral Issues. Any and all input on how we should go about this is welcome.
This is a great idea Harry! I really would love to see copy of the tests you use. I have considered doing something like that for our own assessment.
ReplyDeleteOne imperfect measure I have used is to survey students in upper-level philosophy courses about their experience in Intro to philosophy. I ask them about how well that course prepared them for the upper-level course they are taking and I break it down in terms of our learning objectives: reading philosophy, writing philosophy, defending a position, etc. One advantage of doing it this way is that you can find out whether students take themselves to have learned skills in Intro to Philosophy that enable them to do better in the upper-level courses.
Hello Harry,
ReplyDeleteWhile I practice and teach a different discipline than yours - architecture - I have followed your blog for some time, enjoying your posts that highlight the similarities more than the differences in our professions.
A few years back as the department chair I was presented a similar assessment scenario, and would like to share the results of our process, which resulted in a paper presented at our national annual conference. The paper titled a Collective Theory of Architectural Education, [Dirlam, D. K. and Singeisen, S. R. (2009). Collaboratively Crafting a Unique Architecture Education through MODEL Assessment. In P. Crisman and M. Gillem (Eds.) The Value of Design (pp. 445-455), Washington, DC: ACSA Publishing.], outlines how we structured the process by beginning with interviews of faculty members; included the law of succession, which has been shown to apply to historical and developmental strategies as well as ecosystem changes; improved assessment reliability; and helped students set goals.
I'd be happy to send you the full paper if you want to email me. ssingeisen@gmail.com
Best,
Scott