In my 15 years of teaching, I've always given my students a set of grading criteria so they could see the criteria I use to assess work as deserving different grades.Grading on factors besides 'end products' (attitudes, behaviors, etc.) has attracted our interest before here at ISW (Chris on "comportment," Mike on in-class behavior, me on pass/fail writing assignments). I've got my own thoughts on Jennifer's proposal and question, but do other readers have some thoughts as to how much 'effort' should count, if it is to count at all?
After reading a significant amount about the perils of grading performance (namely that students focus on performing well instead of learning and in so focusing become very risk averse). One idea I'm toying with (because some folks I respect have used it) is to grade students based on effort (and I'll give them a set of what I take to be indicative of A effort throughout the semester, B effort, etc. and also give them an opportunity a number of times to make the case to me that they are putting in X amount of effort).
My question, then, is if students belong in a class (i.e., aren't in a class too difficult for them) what minimum grade should they be able to get in the course with 100% effort assuming that the teacher is doing a good job teaching. That is,if a student puts in 100% effort (rewriting, working with teacher, working with classmates outside of class, pays attention to and works with comments, etc) and gets less, say, than an A (or B?) is this grade an indication that the teacher is failing?
Put another way, what grade, minimum (if any), would be reasonable to guarantee a student if the student can demonstrate putting in 100% effort (however one might choose to measure it).
Put yet another way, assuming the teacher isn't at fault, at what point is a certain grade due to the student not putting in the effort and to what extent is it due to something else (since I don't really believe in 'raw ability' much I don't know what that something else might be but others may have a good case for 'raw ability' - and then there's the question of whether it's fair to grade based on 'raw ability' but then we've moved to a different concern)
The motivation in all of this is to enhance student learning and, thus, performance in the long run with the logic being that grading performance does not, in fact, enhance performance in the long run.
Monday, July 5, 2010
Reader query: Evaluating student effort
ISW reader Jennifer McCrickerd offers the following query about the 'grading ceiling' if students are graded by effort:
Subscribe to:
Post Comments (Atom)
Grading on effort might be a good idea, but it would be enormously difficulty to tell who is genuinely putting in effort. After all, how do you tell the difference between someone who is quite in class due to shyness (or just thinks slowly and carefully, and so rarely has comments at the ready), and someone who is quite because they haven't done the reading, or aren't paying attention? How can you tell when poor results are due to slapdash effort versus real difficulty with the subject matter? Maybe some people find it easy to tell the difference (but how can they check if they are right?), but I do not find it so easy. Any advice on how to tell how much effort is being put in?
ReplyDeleteAnother worry, if we come to treat "effort" as equivalent to "time spent on the subject", by requiring multiple revisions and out of class collaboration and so on, mightn't we end up accidentally discriminating against the less well off, who may have other time commitments to work and family.
I don't understand how this is grading "effort." You write that 100% effort would be rewriting, working with teacher, working with classmates outside of class, pays attention to and works with comments, etc.
ReplyDeleteThat's about 5 things. Some students might be able to do all these things well, but easily, with little "effort" (i.e., time and struggle?). Other students might require a lot of "effort" and still not do very well on some or all of these 5 things.
So, it seems to me like you'd be, in the end, grading on product(s), not "effort."
This comment has been removed by a blog administrator.
ReplyDeleteBen has an excellent point -- perhaps one of the central ones to hit at here. How can you tell? in my own experience, whereas students will certainly complain about a test or paper they've received a bad/mediocre grade on in order to try to get a better grade, the amount of attempts to argue for a better grade go up considerably when "effort grading" is a component. And why not? How can you assess effort? How will you justify the grade? Moreover, it could be a student' perception of effort is very different from the instructor's.
ReplyDeleteOne way to possibly get around some of this worry (perhaps) is to link effort to something tangible, but not necessarily linked to improvement of content knowledge -- like sheer number of attempts to engage in something related to the course.
So, say, a student who actually does the rewrite for a paper might get a better grade simply because they did it (and actually followed the instructors for the rewrite), even if it really isn't much better (or at all) in the end.
Or: say you have a unit on X in the course. If X lasts a month, you could have questions sheets that can be answered and turned in each week. A student who does more of them gets a default better grade than another student whose understanding of the material is the same but completed less question sheets.
I'm not sure specifically how to link all this in to the grade, but basically it would allow for some component of effort grading to take place, and the evidence would be pretty tangible and easily pointed to.
I didn't notice Nathan's comment above before writing mine. I think he's right -- even my own patch has this problem. For some students, the effort require to complete a task may be half that of another student. Of course, you could always (in my patch) just call it "iteration grading component" or something, and just not call it effort! :)
ReplyDeleteLet me clarify my thinking a bit. My focus here is to get students to the point where they are improving upon skills (thinking, writing, speaking, reading). My presumption is that regardless of where the student begins in terms of skill level that they can improve, probably significantly. So, I’m really not interested in them learning content per se, I’m interested in them improving upon their skills and, presumably in so doing, learning content.
ReplyDeleteIn addition to improving upon where they are, I also want them to learn that working hard (and well) is what's, ultimately, important. Not getting 'the right answer.' Of course right answers are important, but are typically gained only through hard, good work.
Most of the concern folks have noted, so far, are on the issue of whether grading based on effort is possible or fair.
I take it that, to at least some extent, effort is more obviously important when it comes to, say, learning how to play instruments, write creatively, or participate in any fine arts. To at least some extent effort is assessed in classes in these areas and so it can’t be that effort is impossible to ascertain.
Students routinely assess their own effort and I take it to be my job, as teacher, to help them so that effort is directed in a fruitful direction (that is, if their effort is needless expenditure of energy, I should be working with them to help them use their time and energy well instead of letting them continue to go down the wrong track). By grading them on effort I’m giving them a reason to work hard and lean on me to learn to work well.
It seems to me that effort is measurable by whether students pay attention, participate (students have opportunities for participating either by speaking in class or through writing that is shared with the class), respond to feedback for ways to improve writing, speaking, etc. I don’t see how any of these are not indicative of effort. And, of course, doing all these things all the time isn’t necessary but if these are set as the targets and students get to, multiple times during the year, make the case that they are putting in a certain level of effort (pointing perhaps to improvement in work as evidence) then these things combined would seem to be decent ways to determine effort. Further, if someone is able to do these things well without putting in much effort, then I have no problem at all saying that they ought to be putting in more effort. Putting in enough effort to get by is not what learning is and this is exactly what focusing on performance emphasizes.
Whether it’s fair or not, I’m not sure how measuring effort is less fair then measuring output insofar as for at least some students’ output and effort (or time) are going to be connected and for different life reasons may not have the ability to put in the time or effort to get the desired output. Grading output instead of effort isn’t any more fair, at least not that I can see. And part of the motivation for grading this way is that a significant number of people believe that someone is good at something or isn’t and that it just comes easily to those who are good at it, without realizing that being good at something take lots of work……continued on next comment
This comment has been removed by the author.
ReplyDeleteBut all this is irrelevant to my primary question: assuming (a) there was a way of knowing that a student was putting in 100% (or 90% effort), (b) the course is not over the head of the student, (c) the teacher is good, then what grade (assuming grading is done by assessing produced artifacts), should a student who is putting in 90-100% effort be able to get? I include (a) for obvious reasons, (b) to circumvent the student who takes an upper level Greek class without having ever taken Greek before. I admit that in such a case all the effort in the world isn’t going to make much of a difference. And I include (c) because, again, it seems to me that it’s the teacher’s responsibility to make sure that the student learn how to direct his or her effort and that a good teacher would do this effectively.
ReplyDeleteSo, assuming (a)-(c), what grade would you expect a student to be able to get on a artifact based grading scheme?
I know that in K-12, alternative schools don’t focus on effort per se but they focus on skill building and these students do better on standardized tests than their counterparts at traditional schools (and many of these students are not better in virtue of home environments).
I asked this of some of my colleagues and one said the entry level course in his department is such that putting in 100% effort would not guarantee a student an A (or even a B). From my perspective, this seems unfair to an entry level student — that a course is taught where it is known that regardless of effort some students will not get better than a C (and some may not even get that). This seems like a really good way to get all but the students already good in a field to really despise the field and to believe that effort is meaningless and all that really matters is ‘talent.’
J:
ReplyDeleteSorry if I didn't answer your question, and instead answered my own (inserted) one! It could well be that we turn quickly to the "is this possible or fair?" question because we've had little success in getting this to work correctly ourselves. At least I'll admit as much in my own case.
I don't doubt for a second that your goal is right minded. I too think of philosophy more like a fine art, where skill acquisition is way more important than actual content acquisition, and this is likely true even more so for non-majors (who presumably make up the majority of our students).
My own concern isn't the fairness of it (seems fine to me, actually, but I tend to not see the fairness of grading as a comparative thing). Instead, my worry is the time intensive nature of doing this right, and (b) the amount of flack you will receive from students as a result of grading this way, because it will be difficult to "point" in an easy way to how well their effort matches up to the criteria.
One thing I think we agree on, though - to grade in this way, I do think you'll need to have very regular reviews with students about their progress. As you likely know, the fewer the times they are assessed, the more they will tend to complain about how or why they are being graded in the way they are. As well, regular assessment gets students clear on exactly what you want, and that helps them to know how to modify their own activity in the classroom.
Okay, but onto your actual question about the grade. I think I'm a bit unclear about exactly what you're asking. When you say "what grade" do you mean "assuming that a student puts in 100% effort (say) but makes no real gains with respect to content"? Also, I'm curious here whether your intention is to make this criteria 100% of the grading in the course. It might matter if you had 50% allocate to effort, and 50% to tests, or just 100% to effort.
Can you clarify that a bit (you may have already done so, but I'm not reading effectively)?
Chris,
ReplyDeleteThanks for the question - I'm working out my thoughts still, hence the lack of clarity.
What I'm thinking may up being more a theoretical than a practical question though it started as a practical question.
Practical: If I were to make a guarantee to students of "if you put in 100% effort, you are guaranteed an grade of ___" What grade should that be? Do I really want to say effort is *all* I'm measuring? I want to emphasize the importance of effort, but I'm not sure that grading solely on effort does this. Nonetheless, I think that guaranteeing a certain grade with massive effort (independent of product) may have the consequence of alleviating problems of having students only worrying about producing the right product.
My typical grading is that C=adequate understanding with no radical misunderstanding of material and then B's and A's include particular clarity, originality and creativity (with significant emphasis on the last two). What should a student reasonably expect with 100% effort?
Theoretical #1: My colleague who says that 100% effort in an intro level course for his department would in no way guarantee an A...possibly a C, but even that doesn't always happen. Is this fair? What should I be *aiming* at. If I'm not grading based solely on effort, what should be *possible* for students who work 100%?
Theoretical #2: If a student puts in 100% effort and doesn't get an A (B? C?), is this *my* fault as the teacher? For either not providing enough guidance or not catching that the student was ill-prepared and suggesting they get out of the class? Or is there something else going on? And if there is something else going on, is it conducive to learning to grade that thing? In the end, is it fair to grade students (for learning purposes) on anything *other than* effort?
I realize that this (a) is still muddled and (b) has seriously mutated from my initial question, but it's getting closer to my concerns.
Thanks for letting me think 'out loud'
I think that if I see a student has received a grade of A in a course titled "Introduction to Philosophy," I should be able to infer from this that the student has demonstrated promise as a philosophical thinker (not necessarily as a professional Philosopher, it's just an intro course after all).
ReplyDeleteIf the professor was grading on effort, then this inference from a grade of A becomes a weak one.
For that reason, I'm not a fan of grading on effort.
But if you really think that you should do so, I'd say it should be worth no more than 50% of the final course grade at the most.
Regarding your Theoretical #1: If a student could get an A just from working hard, even when the student turned in very bad work, then that grade is not very meaningful. In fact, if you grade in a way that results in a "natural" who doesn't have to work very hard to write good work receiving a poor grade for lack of effort, then when I see you've given a student an A, all I can infer from this is that the student is a hard worker!
Regarding Theoretical #2: Students bring different life circumstances and different levels of (yes) natural talent to the table. If a student puts in 100% effort and still only gets a C, this doesn't necessarily reflect on you as a teacher. The student's past circumstances, present circumstances, or natural talents may be such that the student simply needs a lot more help than you could offer as a teacher of a course for a single semester.
Achievement = Effort + Ability
ReplyDeleteLet's say that ability goes from 0-100, like % effort. If my ability is at 50 and I put in 100% effort, my top achievement will be 150. If my ability is 100 and I put in 100% effort, my achievement can be 200.
You would have to decide if the A is 150, 200, or both. It sounds to me like your dilemma is that you don't want to reward or punish people for their native ability...but achievement does in part depend upon ability...so what to do? Especially because part of the point is increasing ability through effort!
I can see making this work, but you would have to have 1) a baseline measure to assess ability, and 2) an agreement on what steps constitute 100% effort. The steps can't be just checking off boxes, either, because of the problem that Chris identified (steps taking different effort for different people).
Hmm....this seems a lot like the concept of "contract grading."
To address your original original concern, "namely that students focus on performing well instead of learning and in so focusing become very risk averse," could you add "taking chances" to your grading criteria? Make the risk part of the achievement?
My word verification is porke, which has nothing to do with anything but struck me as amusing for some reason.
This is a very interesting question. One element that I think is missing is the amount of interest the student has that will affect effort and may impact ability. So I would modify Victoria's equation to Achievement = Interest + ability + effort.
ReplyDeleteMany students, at least in the intro level courses, seem to have a very low interest in the subject - they are there to earn gen ed credits and this course fulfills this need and fits their schedule. As teachers we try to make the course as interesting as possible so as to increase the students level of interest in the material, but often this will remain low for some students.
As educators we directly impact their ability by increasing and improving their skill sets, but I do not think we need to work directly on effort. If we increase interest in the material and develop their ability for dealing with the material by giving them the tools to improve their skill sets then their effort ought to also improve thereby improving their achievement. But, this does not guarantee a certain grade. A student can earn a C and use max effort in doing so and be interested in the material, but simply understand the material at a C (average) level. I have definitions for each grade that I put in my syllabus. I should also note that I tell students that I assume they are average and that it is their job to convince me otherwise. (Come on - lets us admit it - we are not all 'A' philosophers. I am certainly no Parfit or Quine or Rawls. I am average at best!)
So, I do not grade for effort. When students gets lower grades then they thought they were going to get, or deserve, and end up by telling me that they worked hard on the assignment, I tell them to work harder next time. When your students got lower grades then they thought they would/should tell you they put in max effort are they the ones the show up at office hours for further discussion on the material, or send e-mails asking for clarification, or participate in class? My guess is that they are not.
This is a very interesting question. One element that I think is missing is the amount of interest the student has that will affect effort and may impact ability. So I would modify Victoria's equation to Achievement = Interest + ability + effort.
ReplyDeleteMany students, at least in the intro level courses, seem to have a very low interest in the subject - they are there to earn gen ed credits and this course fulfills this need and fits their schedule. As teachers we try to make the course as interesting as possible so as to increase the students level of interest in the material, but often this will remain low for some students.
As educators we directly impact their ability by increasing and improving their skill sets, but I do not think we need to work directly on effort. If we increase interest in the material and develop their ability for dealing with the material by giving them the tools to improve their skill sets then their effort ought to also improve thereby improving their achievement. But, this does not guarantee a certain grade. A student can earn a C and use max effort in doing so and be interested in the material, but simply understand the material at a C (average) level. I have definitions for each grade that I put in my syllabus. I should also note that I tell students that I assume they are average and that it is their job to convince me otherwise. (Come on - lets us admit it - we are not all 'A' philosophers. I am certainly no Parfit or Quine or Rawls. I am average at best!)
So, I do not grade for effort. When students gets lower grades then they thought they were going to get, or deserve, and end up by telling me that they worked hard on the assignment, I tell them to work harder. next time. When your students got lower grades then they thought they would/should tell you they put in max effort are they the ones the show up at office hours for further discussion on the material, or send e-mails asking for clarification, or participate in class? My guess is that they are not.
The most important thing for a philosophy student to do is spend time *thinking*. In thinking, he/she needs to be willing to confront and reject his/her current views, be open to new ideas, have humility, trying to reason etc..
ReplyDeleteBut there is *absolutely no* way for you to measure whether – or to what extent – they have done this, other than through looking at the quality of the end product. You can look at two students sat in front of their laptop, who both appear identical, and one is thinking in this way and the other is daydreaming or following his/her usual mental habits.
Redrafting, talking to teacher etc is nothing without engaging in such thinking, and nothing in comparison with it.
You risk giving a good mark to someone who has spent time redrafting, talking to teacher etc and giving a poor mark to someone who produces a much better essay through sitting around thinking. This would be morally wrong.
Let's assume I agree that the most important thing for a philosophy student (a student taking a philosophy class?) to do is spend time thinking, I don't understand why looking at the quality of their product is the only or best way to determine time spent thinking. Or, for my purposes, why grading them based on this will actually *motivate* them to spend time thinking.
ReplyDeleteThe research I've read about motivating people suggests that giving people goals *slightly* ahead of what they are currently capable of is what is most likely to improve performance. This means that one would have to individualize grading - the question is how to do this without actually individualizing grading standards. Or, perhaps the question is whether we are supposed to be motivating.
Some of the question on this thread comes down to the purpose of grading. Kris notes that getting an A in an Intro to Philosophy class should be indicative of having promise as a philosophical thinker. I'm not sure I agree with this (nor am I sure I disagree).
My interest, at this point in my career, is motivating students to do push beyond what they think they can do and I'm trying to work with the evidence on motivation and K-12 successful practices to find a way to do this. Currently, I'm *really* good at motivating the already high achieving students - but this seems like a fairly uneventful accomplishment. I fear that what I do now is contributing to the "Matthew Effect" (good students getting better while poorer students fall behind for various reasons).
I'll figure out what I'm going to use for grading (how much or if effort factors in) and then report back here on how it worked.
Any progress or activity to report on this topic?
ReplyDeleteMy administrator recently introduced at a faculty meeting a plan to incorporate evaluating student's effort into their total grade. This has met with a lot of questions, and some outright resistance.
For example: how do you sell to parents "Your son's performance earned a 90% but I have lowered his grade to an 88% because of my perception of lack of effort on his part."?
Obviously, this is not what is intended.
My questions are: How do we honestly achieve objectivity when assessing effort? What exactly are we trying to measure? Is rewarding effort merely a way of allowing the teacher a way to inflate to a grade that wasn't earned. (Assessing based on effort could certainly degenerate to that eventually.)
Btw...I Googled my way here.
ReplyDeleteIs this a blog about teaching "philosophy", or "teaching philosophy"?
I'm a science teacher, so I may be lost...