My Problem with Assessment: A Rant

On a plane flight earlier this summer I was trying to get this out of my system. I am absolutely bedeviled by assessment.


I don’t like grading. Well, no one enjoys marking papers. It is tedious and deflating work—endless stacks of repetition makes the tedium; focusing on errors, misconceptions, and failures leaves a sour taste in your mouth.

But I don’t really want to complain about marking papers, per se. Much good can come from interacting thoughtfully with written student work. A well-crafted assignment and thoughtful comments from an instructor provide students with the feedback required to generate reflection and intellectual growth.

The part I take issue with is the grading by assigning some sort of score. One might use points, one might use letters: I want neither.

I suppose that what would make me happiest is some sort of narrative dialogue of evaluation. That way, the students and I can keep our eyes on what is important—striving for personal improvement.

What is it about grades that I dislike? They poison the enterprise. Focus shifts from learning for its own sake to “making a grade.” Students perceive school as a game and then they “play it” instead. This leads to point grubbing. I am too lazy to look for the links: but I have heard of studies demonstrating (1) that students will ignore comments if a paper has any kind of summative mark on it, and (2) that enjoyment in an activity dissipates after being paid to do that activity.

The problem remains that everyone expects grades. Whole subsystems of modern tertiary education depend upon the A-F system we use: scholarships, graduation requirements, grad school apps, the dean’s list, etc…

So, what can be done? How do I turn the system on itself to my advantage?

Some Ideas

  1. Provide only narrative evaluations unless pressed.
  2. Avoid legalistic “contract-type” language on a syllabus.
  3. Give meaningful feedback often and in person.
  4. Share with students my goals for their personal growth.
  5. Design a system where students can see this in a positive light. Sell it.
  6. Ensure my “standards” cannot be gamed. That is, if a student really sees the external motivation of a certain grade as primary, then an attempt to structure work to improve to that grade should lead to behaviors for personal growth and authentic learning. (Whatever that means.)

So, I’ll have to put together something coherent in this direction. Soon, I’ll share what I have been doing.


Added After Publication: Well, I didn’t expect this to be the most popular item today, but twitter seems to like this one. I did get this

I admit, there is a valid point here. I don’t actually have a problem with the fact that we do assessment. But I still have a problem with how I do assessment, so I am going to leave the title alone.

Advertisements

15 thoughts on “My Problem with Assessment: A Rant

  1. I have been thinking about these same things too. I feel boxed in by The System, and momentum certainly pushes me to keep doing things the way they’ve always been done. I think to give the kind of feedback I’d like to give to students would require either working twice as many hours per week, or drastically fewer students. My typical semester usually involves instruction of between 120 and 160 “entry-level” students (i.e., 100-level students taking first year courses like calculus). Department policies require I comply with a common, panel written final examination (worth at least 25% of a student’s final course grade) and three or four mid-semester tests.

    But supposing I were given complete freedom and a much smaller student count, I think I might prefer to give biweekly “assessment checks” in my office hours. I’d have each student bring three or four prepared solutions off of some predetermined problem list, and they would get to choose which one to present on my office chalk board for us to talk about. The purpose of the chat would not be to “assign a grade” but instead to help guide students to think about questions like, “How could we rewrite this part to make it more clear?” or “Are there any important details we skipped?” or “Is this solution generalizeable, or will it only work for this specific problem?”

    My dissertation advisor has an interesting grade scheme. His calculus syllabus is available at http://www.math.sc.edu/~mcnulty/141/sylab.pdf ; see page 2 for the grade scheme. He is doing some standards-based grading and does not assign “points” per problem. Part of what he does is he prints a midterm exam specific to each student: Student X may have problems 1, 2, 5, 9, and 11; while Student Y may have problems 5, 6, 7, 9, and 10. I have no idea how he manages the administrative task of tracking this data, but it is surely helped by his student cap of 50-or-so.

    I hope to come up with some ideas about these things over the next several months. The next time I’ll be back in my own classroom will be January 2014, so I have lots of time to come up with really great ideas, procrastinate hugely, and then not implement them. 😉

  2. I really like where you are going with this, and I remember similar arguments with myself on the issue of assessment. There are a lot of people who are on the same page as you; we love seeing students learn, we hate reducing that learning to numbers.

    #6 is the huge challenge in your list, I think. Even written feedback can be gamed. I worked in a school where students were assessed on their evaluation of their thinking, which was typically represented as a series of paragraphs near the end of their work. One student I worked with would copy, verbatim, his evaluation from each project to the next project, and adjust his evaluation based on the feedback he had received on the previous assignment. In other words, he wasn’t doing any evaluating of his thinking (except at a superficial level) on each project at all, he was focused on achieving the most positive feedback on his evaluation section of his project that he could.

    I would see #6 as an ideal to work toward though. It would be nice to develop a system that was not gamed (or at least if was gamed, that gaming by students led to learning).

  3. I’d love to see something like this:
    – During the semester, give a variety of assessments with non-numerical feedback given to students for the sole purpose of improving areas of need.
    – Students submit a portfolio of their best work toward the end of the semester.
    – Then, instead of grades for the course, each student gets a performance evaluation that is a written narrative of their abilities and areas for growth, based on instructor observations and especially the portfolio.
    – The semester ends with a series of exit interviews with the students to gather feedback and give students an opportunity to ask questions about the performance evaluation.

    Then the performance evaluation goes into an electronic portfolio that is issued by the university upon graduation instead of a transcript.

    Something like this would capture the nuances of student performance a lot better than grades. It is also a lot more similar to how students will be evaluated on the job later in life. I would also add that it encourages students to form positive relationships with the professor early and nurture them often.

  4. Kate:

    I am not so “boxed in.” At least, not right now. Thanks for the link. I will check it out.

    And I am quite certain that I do not have the administrative powers to pull of that scheme of giving individualized exams. But fortunately, something similar is in the structure of how my class runs on a daily basis. Instead, my biggest problems come from students *not taking opportunities afforded them.*

  5. David:

    You know, I feel like I have #6. Except that it is necessarily vague. But I though about that part a lot last summer, so I have something I am willing to work with for now.

    I am more challenged by doing something honest but achievable at the same time. I don’t want to lose every waking minute to doing assessment drudgery.

  6. On the flip side, how might a student go about providing better feedback to their professor earlier in the semester than the course evaluations? While these evaluations might help the next class, they’re too late for us. Is there a subtle and upbeat way to ask someone to, for example, stop reading directly from their notes and make eye contact with the class?

  7. I know of no subtle way to ask someone to engage in teaching rather than reading aloud to a class.

    Even a colleague would have to be brave to approach with this concern. “Um. Hi. I would like to talk to you about the way you are wasting everyone’s time three hours a week…”

  8. Here are some of the references that you mention:

    Butler, R. (1987). “Task-Involving and Ego-Involving Properties of Evaluation.” Journal of Educational Psychology 79: 474-482.

    Butler, R. (1988) Enhancing and Undermining Intrinsic Motivation.” British Journal of Educational Psychology 58 (1988): 1-14.

    Butler, R., and M. Nissan. (1986). “Effects of No Feedback, Task-Related Comments, and Grades on Intrinsic Motivation and Performance.” Journal of’ Educational Psychology 78: 210-216.

    And I like everything that was written by everyone here. I have no solutions to offer, other than give a lot formative assessment during the semester, and then somehow come up with a letter grade at the very end. And try to make the assessment good enough so that, even though the formative assessment didn’t directly mention it, the student is not surprised by the letter grade at the end.

    I haven’t figure out how to do this yet.

  9. I’m not that comfortable with number 4:
    “4.Share with students my goals for their personal growth.”
    Perhaps this might be preferable:
    “4. Talk with students about their goals for their personal growth (if any), and discuss how this course might fit with those ideas.”
    Many of your students may be taking your class for reasons other than intellectual interest, and they might not be looking for “personal growth and authentic learning” from this particular class. And they may have strong, rational reasons for it.

  10. Bill– I take your point. It is important to be mindful of your audience.

    But it remains that my course is designed for a particular audience, with specific educational goals in mind. From the point of view of our secondary math ed program, these students are pretty homogeneous. They tend to be at a particular (early) stage of development as mathematicians, and my course is designed for this population. I think more than 90% of my students are sophomores or juniors who wish to be high school math teachers and are taking a first proof intensive class.

    Every once in a while I get a student who is further along in their development, and I have a trick or two for differentiating the course to everyone’s advantage.

    If I were teaching something larger, or with a broader audience (say, my liberal arts course) I would have to give a wider interpretation like you suggest.

  11. I really like the general thrust of this for younger students as well. I find one-on-one contact powerful for 80% of students in grade 2+. I’d like to hear someone from the establishment defend the current system. Perhaps their defence will be based on time management of the instructor?

  12. I’m pretty late to the party, but I’ve been thinking a lot about how to do assessment (and grading) for the second semester calculus course this fall. This blog post is really about a large issue, and I feel like I’ve been grappling with a much smaller issue: homework, under the assumption that exams will be done like they are in most calculus courses. I’d be interested to hear if anyone has any thoughts on my proposed system (particularly any ways you think students might try to game the system).

    Most days in class I’ll give a homework assignment. There will be a single problem that the students should write a careful solution to. There will be a collection of standard problems (some easy, some difficult) that represent the core of the course. On some days there will be a starred question (one that is particularly interesting or challenging, but outside the core of the course).

    One of the purposes of the graded homework problem is to help the students to learn how to write a good solution to a problem (this is surely relevant to your intro to proofs course for prospective high school teachers). Having a single problem encourages the student to do their best work in writing a solution and allows the grader to give more substantial feedback than would be possible if more problems were graded. The graded problem will receive 3 points if it is clear, complete, and correct. It will receive 2 points if there are minor problems with clarity, completeness, or correctness, and 1 point if there are major problems. If students would like to redo that homework problem and submit an improved solution, a new score would be given and would replace the old score.

    One of the purposes of the ungraded homework is to give the students a clear idea of what they are expected to be able to do on exams. In fact, most exam questions will come directly from the homework assignments. This should provide at least some motivation to take the ungraded homework seriously. Also, there will be some time at the beginning of each class for questions related to the ungraded homework that was assigned during the previous class (this also should help with motivation, as long as there is a good classroom culture).

    The starred questions will hopefully pique the interest of students who are interested in going beyond the fundamentals of the course.

    The majority of a students grade in the course will be determined by their exam grades. I hope that I’ll be able to convince the students that the best way to do well on those exams is put significant effort in the graded and ungraded homework. If the homework problems are well written, I think there is a good chance that a student working for a good grade will go down the path that entails “personal growth and authentic learning.”

    By the way, another thing I will do at the end of the semester (I did the last time I taught calc 2 and thought it went very well) is this: I’ll let the students know that sometimes grades don’t clearly reflect how well a student understands the material. I’ll say that if any of them feel like they understand the course material better than their current grade indicates, then they should do 2 things. First, write out careful solutions to a review packet (that I’d give them) that cover the core material of the course (last time I did this the packet contained about 100 questions). Then schedule a time to meet with me for 45 minutes. I would ask them questions connected to the review packet and do my best to clearly see what they understood and what they did not understand. If they could show me that they understood all the material of the course I would be delighted to give them an A. If they showed me they understood the most fundamental things in the course I’d be happy to give them at least a C. When I did this last time, I had (I think) 14 students accept this invitation, and every single “interview” was a good experience.

  13. I have had some success in “ungraded” homework. I make explicit that the homework is “for the students”, that it is intended to help prepare them, that I expect them to do it, and I am happy to discuss it at appropriate times if they feel they want help. But I have only tried this with a fairly “mature” audience that can handle the responsibility and independence… That group did buy into the fact that this kind of work was good for them and so they would do it. I can also see it cratering if I fail to make that case.

Comments are closed.