An Approach to Specifications Grading: Guest Post by John Ross

I have been involved in a lot of discussions about assessment strategies lately. There is a bit of a swell of young faculty who are rethinking their assessment strategies carefully. For some, this is a first serious step to rethinking their jobs as educators, and for others it is further step into the details of how to be effective.

Today we have a guest post by John Ross of Southwestern University. I met John at the Legacy of R.L. Moore meeting this summer, so I already know he is interested in effective teaching methods. This past weekend he mentioned lightly on twitter that he is using a new assessment setup. I wanted to hear the details, so I invited him to write about it. I am very pleased that he accepted my challenge.


My Version of Specs-Based Grading

by John Ross, Southwestern University
This semester I am running my calculus class using a specifications-based grading system. The decision to do this was made after discovering Robert Talbert’s blog and reading the many informative things he had to say about specs grading. If you’re unfamiliar with this style of grading, I’d recommend starting there (http://rtalbert.org/blog/2015/Specs-grading-report-part-1/).

Continue reading

A reflection on “Assessment Interviews, Phase 2”

I have spent a large portion of today in one-on-one conversation with the students in my Euclidean Geometry course. To prepare the students for these meetings, I asked them to complete a one page reflection paper, with this prompt. If you don’t want to click through, I basically ask the students to read through the “standards for assessment,” which is just a fancy name for my student learning goals, and do a self-assessment. Then I want them to make a plan of action for improvement during the next three weeks.

The striking part is the strength of the negative correlation between student self-assessment and my assessments.

Students who I recognize as having developed strong skills come it with focused critiques and tight plans for how to improve.

Students who I recognize as having not yet demonstrated many of our foundational skills show up with some confidence that they are doing everything just fine, and weak plans for self-improvement.

(This relationship is not perfect. Some students were spot on, of course.)

I have enough experience that I expected this, but to watch in unfold all day was really something.

Opening Week for a Moore Method Course: Getting Comfortable

I am teaching iteration number ten of my Modified Moore Method Euclidean Geometry course. This semester I am making an effort to refocus on the basics: managing and mentoring the students as much as I can.

At this point, my theorem sequence is very stable. (I am no longer surprised much by what happens in this course.) This allows me to work on the other aspects of the course. I feel like I have started to let some important things go in the last year or so, and now I want to sharpen up. What has been lacking? I don’t think I have kept on top of the students to keep them engaged as well as I might. And I don’t think I have done a good job selling the method of instruction, either.

So, I was much more deliberate about introducing myself to each of my sixteen students on the first day. I have been very explicit about my expectations and my willingness to help them meet those expectations (which are rather high). And I will be making a conscious effort to check in with as many students as possible each day.

The first week was a rousing success, I think. Each day we got at least one theorem. We have already set the expectation for what counts as an argument. (Well, surely, there is still some work to be done.) The class has made two conjectures. We took some time to discuss some basic points of what acceptable writing will look like. I even successfully navigated our first potential difficult situation and found something positive in it. All in all, I am feeling pretty good about this.

I think our next test comes when we have to finish conjecture 1.1. They haven’t addressed the second statement in that, yet.

And sometime next week I will have to steal ten minutes to talk with them about my Standards Based Assessment experiment for the term.

Meditations on Feedback

I am taking part in a “Talking Teaching and Learning” group, and my homework this week was to think about the last few details of my new assessment structure for Euclidean Geometry. In particular, how will I handle the “regular, daily feedback” part of the process?


So, if I am to provide regular feedback to my students at “assessment opportunities” they take, how shall I do it? I want this to be meaningful and effective. And it would be nice if it didn’t consume my working time.

I think I will try a format beloved by politicians: I will ask and answer my own questions. If any of you wishes to play investigative journalist and ask other questions that I should be forced to answer, go hit the comment box. I would like to play.

What counts as an assessment opportunity?

Any student presentation, meaningful engagement in class discussion, a discussion with me outside of class where I learn something. Those things count as opportunities for me to assess student performance that don’t necessarily have written feedback attached to them. In each case there is plenty of verbal feedback from classmates–but I don’t always participate. In fact, I prefer to leave it to the students.

Why are written papers not on this list?

Students will get feedback in the form of a referee report on each paper. I am not as concerned about providing more structured feedback here because I feel it is adequately covered.

Why do you prefer to leave the process of verbal feedback to the students?

One of the skills I am trying to encourage is the ability to evaluate arguments critically and thoughtfully.

If there is a reason to leave the verbal feedback to the students, might written feedback from the instructor corrupt this process?

Oh, yes. That is my main worry.

How can you avoid this trouble?

um. uh. [blink. blink.]

I hope this writing will spur me to some ideas about that…

What are the goals for this written feedback?

I want to focus student attention on some aspects of what they did. Ideally, quality feedback should help speed up a student’s process of improvement by directing his or her attention to something concrete.

What kind of constraints are you going to impose upon yourself?

I am a constructivist at heart. The student must come to grips with the material and how to do it. Each one should do this on his or her own terms. One idea would be to give feedback by asking questions.

I am just not sure what kinds of questions I would ask that are detached from the process of running a class meeting. We handle lots of things in class, and I almost always do it by asking questions. Maybe I will just reiterate some of the unanswered questions. That doesn’t feel like a very good answer.

Another idea is to use a sandwich approach: mention something positive, make a suggestion for improvement, reiterate the positive outcomes. And be relentlessly optimistic.

Now I’ve run out of questions. So.

I think whatever I do will have to play to my strengths. I am at my best when I split my time as a cheerleader, mentor, and coach. Students are capable of amazing things, and sometimes they just need for me to believe in them and expect it out of them. Sometimes they just need a little bit of commiserating about how frustrating it is to do mathematics. Sometimes they need a concrete suggestion of what to do when they are stuck and at their wit’s end.

That was unsatisfying.

Here we are. 500+ words in, and no answers I feel wonderful about.

Never mind, time for some unbridled confidence.

When I got into IBL teaching, I recognized that a major asset I had was hubris. I just believed that I could do this. Usually it works.

What? So I will have to help each student in as individual a way as possible, thinking on
my feet and being careful about everyone’s feelings? Why should I worry? I can do that.
I’ll think about this some more, and just try to roll with it.

I don’t feel like I finished my homework.

Standards Based Assessment for a Moore Method Course

Motivation

I have been working on developing a reasonable assessment model for my IBL Euclidean Geometry Course for a while now. I have several reasons for this:
1. It would be more fair, and better for my students, if I found a way to communicate with them about their progress. At the very least, I need to open the line of communication, so students feel they can have a conversation with me about how things are going.
2. So far, I have been going with a “you will have to trust me” approach. I have gotten away with it. But someone who wants to raise hell will make my undocumented life difficult.
3. This class is conducted as a lightly modified Moore Method course. Standard assessment with homework, quizzes and exams just doesn’t feel right.
4. The accountability movement is coming. Sooner or later, I will have to deal with a top-down mandate to deal with how I assess my students, and how I assess my teaching. I choose to start, on my own, with the parts I can control before that pressure gets here. First up: how I assess students.

The Main Idea

I will try to use a Standards Based Assessment scheme. I will attempt to focus on this mainly as a feedback mechanism. Grades will only happen to the minimal extent that is required.

What didn’t work well enough, and why.

I tried to implement a simple SBG/SBAR scheme in each of the last two semesters. Neither worked because I had not found a method of dealing with the administrative details. At first, I asked too much of myself. Then, I asked even more of myself, but on deadlines. Ugh.

What is working

I am happy with my set of standards (read that as learning goals). I am very proud that they are weighted toward process goals: what one does and how one behaves as a mathematician. This is intentional—I want students to become acculturated to doing mathematics, and to acquire some of a mathematicians habits of mind.

A New Attempt

For next semester, I have devised a two-prong approach to administering a standards based assessment mechanism.

The First Prong: Face to face meetings

In order to make for better communication about expectations, I will meet with each student individually every three weeks. This will involve splitting the class. I will meet with half one week, half the next, and then take a week break.

Before each meeting, I expect the student to write a one page reflection about their progress in the course. To tighten this process up, I have written specific prompts to which the students must respond. This must be done before the meeting. It can either be sent to me electronically, or it can be brought to the meeting on paper, but it has to be done before the conversation. Really, the paper is not important. But the time for reflection is crucial. The meeting could too easily be wasted without it.

Second Prong: Professional Feedback at each Assessment Opportunity

Each time a student participates in some sort of assessment opportunity (a presentation at the board, turning in a written paper), I will provide feedback. I have a little electronic system built (with the help of my friend Stephen Hughes) using a Google Docs form/spreadsheet/script combo. I have a web form into which I will type comments. When I click the “enter” button, my comments are saved in a spreadsheet, emailed to me, and emailed to the students.

It is too much to manage class and write out feedback at the same time, so I will be doing this during the hour after my class meeting. I normally take time to convert my notes into a blog post for the students anyway. Now I will just add a little bit to the “post meeting decompression” that I do.

What is left to do?

I need to think some more about how I will provide feedback. I want this to be a narrative process, but what are my aims? What constraints should I observe?

That should be my next post. 🙂

Where is all of my stuff?

Well, I keep a blog for the students, and it has a page all about assessment. Go have a look. Not all of the links are live, yet, but they will be at the appropriate time of the semester.

In the end, what about grades?

Here, I have no substantive changes, but Ed Parker has pushed me a bit…

My Problem with Assessment: A Rant

On a plane flight earlier this summer I was trying to get this out of my system. I am absolutely bedeviled by assessment.


I don’t like grading. Well, no one enjoys marking papers. It is tedious and deflating work—endless stacks of repetition makes the tedium; focusing on errors, misconceptions, and failures leaves a sour taste in your mouth.

But I don’t really want to complain about marking papers, per se. Much good can come from interacting thoughtfully with written student work. A well-crafted assignment and thoughtful comments from an instructor provide students with the feedback required to generate reflection and intellectual growth.

The part I take issue with is the grading by assigning some sort of score. One might use points, one might use letters: I want neither.

I suppose that what would make me happiest is some sort of narrative dialogue of evaluation. That way, the students and I can keep our eyes on what is important—striving for personal improvement.

What is it about grades that I dislike? They poison the enterprise. Focus shifts from learning for its own sake to “making a grade.” Students perceive school as a game and then they “play it” instead. This leads to point grubbing. I am too lazy to look for the links: but I have heard of studies demonstrating (1) that students will ignore comments if a paper has any kind of summative mark on it, and (2) that enjoyment in an activity dissipates after being paid to do that activity.

The problem remains that everyone expects grades. Whole subsystems of modern tertiary education depend upon the A-F system we use: scholarships, graduation requirements, grad school apps, the dean’s list, etc…

So, what can be done? How do I turn the system on itself to my advantage?

Some Ideas

  1. Provide only narrative evaluations unless pressed.
  2. Avoid legalistic “contract-type” language on a syllabus.
  3. Give meaningful feedback often and in person.
  4. Share with students my goals for their personal growth.
  5. Design a system where students can see this in a positive light. Sell it.
  6. Ensure my “standards” cannot be gamed. That is, if a student really sees the external motivation of a certain grade as primary, then an attempt to structure work to improve to that grade should lead to behaviors for personal growth and authentic learning. (Whatever that means.)

So, I’ll have to put together something coherent in this direction. Soon, I’ll share what I have been doing.


Added After Publication: Well, I didn’t expect this to be the most popular item today, but twitter seems to like this one. I did get this

I admit, there is a valid point here. I don’t actually have a problem with the fact that we do assessment. But I still have a problem with how I do assessment, so I am going to leave the title alone.

Standards Based Assessment for my Euclidean Geometry Course

So, I am trying again to implement a standards based assessment communication scheme in my Euclidean Geometry course. I failed pretty miserably at this last term because I designed something with too much paperwork.

This term, I have streamlined things considerably. The list of standards is shorter, the standards themselves are broader, and the number of official reporting points is smaller.

Now that we are a bit into the course and our classroom culture is basically set where I want it, it is time for me to give these details to my students. So I wrote something to let them in on my thinking. Since most of my audience consists of pre-service teachers, I feel it is my responsibility to engage them just a little about my practice. This means that I say a bunch more in this document than I would if I were doing this in a Calculus I class. I just “lift the curtain” a little.

This is a long way from perfect, I am sure. I believe that it will be workable. This owes a lot to many other people, whom I listed on this Google+ post.