Originally Posted 06-05-2012
The Problem of Outcomes Assessment
Student Outcomes Assessment (SOA) is a major pain in the ass, and I dislike it.
OK. Now that I have done that, I feel better and I can get on with my work. Today I am trying to figure out how to appropriately handle SOA for my Euclidean Geometry course. I recognize the importance of this type of reflective practice, and I implicitly do it, of course.
The trouble is finding a way to assess and document if UNI students are learning what the department wants, without negatively impacting the learning process. Also, whatever I use has to be workable if someone else gets a chance to run the course, and has to pass muster with the UNI administration and the accrediting bodies it
kow-tows to, um, uses for cover, no…uh, asks for certification. (Yeah, that’s the ticket!)
Also, the process really has to be something manageable. I do a lot of work as it is, and so this can’t be too big a structure on top of everything else. I would like to find a way to either integrate this into my standard practice, or extract it from what I am already doing. The distinction there is really minor. I will certainly have to do something new. I just don’t want it to be huge.
As long as we are talking about assessment
I have been thinking for a while that I want to try a Standards Based Assessment experiment in this course. For one thing, my Euclidean Geometry is a course that is running so well that I am comfortable running experiments in it. And for another, it really needs some kind of paper trail to avoid problems.
Why does it need tightening up?
I run my Euclidean Geometry course as a Modified Moore Method experience. This is a pretty old-fashioned version of Inquiry Based Learning. If you know about the traditional Moore Method, then my major adaptations are
* I use Euclid as a text book. (In particular, I use the Green Lion Press edition as it is Heath’s translation without the commentary and is a beautifully constructed book.) My task seqence is “around and nearby” Euclid.
* I don’t usually cold-call students.
Now, such a class is all assessement, all the time. The line between formative and summative assessments is completely obliterated. Students work hard and give presentations on their ideas. Each presentation and conversation, the student bares his or her soul upon the chalkboard, and I simultaneously (1) update my opinion of their capabilities and performance, (2) communicate to the student what they can do to improve and where I think their work as fallen down, (3) mentally adjust the rest of the semester to help address what the student and the class need to work on.
That is a lot to juggle. So much so, that I can take only cursory notes. But at the end of the semester deciding letter grades for the registrar and provost is easy: I just know.
The thing is, I am getting away with it. I know I am getting away with it. The students trust me enough (so far) that they haven’t complained. But if a student complained, I have nothing to say except, “This is my professional opinion.” I need something more concrete to
use for cover um, … document my opinions.
That need to document student progress really doesn’t just come from above. I want it to make sure I try to be fair. I mean, I try, but I need to make a larger effort to be objective.
Past SOA Approach
Shortly after starting at UNI, I got asked to do a very simple SOA for geometry. The Euclidean Geometry course fits in our department’s SOA plan as a (the?) designated place that we instruct our teaching majors on what a proof is and how to construct one. I was asked to make an appropriate problem for my final exam that I could reuse and a rubric for scoring it so that we could track progress.
What Am I Trying to Achieve?
The department’s Student Learning Objectives statement includes the following relevant language. (I’m taking this from the SLO for the liberal arts major, but it applies about as well to the pre-service teaching track, I think.)
Analytical reasoning and problem solving skills specification:
Students will state problems and definitions carefully, modify problems when necessary to make them tractable, articulate assumptions, reason logically to conclusions, and interpret results intelligently. Students will approach problem solving with a willingness to try multiple approaches, persist in the face of difficulties, assess the correctness of solutions, explore examples, pose questions, and devise and test conjectures.
Proof and Argument specification:
Students will be able to compose and explain proofs in clear mathematical style, both orally and in writing, and to critically evaluate mathematical arguments made by others. Students will be able to use a variety of techniques of proof, including direct proof, proof by contradiction, and mathematical induction.
Now, as a class we address all of those except for mathematical induction.
My Magical Exam Question
Here is the question from my final exam, with set-up.
Definition: A quadrilateral is said to be circumscriptable when there exists a circle inside the quadrilateral which is tangent to each of the four sides.
Task: Settle the following conjecture as completely as you can.
Conjecture: Let ABCD be a quadrilateral. Then ABCD is circumscriptable if, and only if, AB and CD taken together are congruent to AC and BD taken together.
I don’t know why I felt the need to write that out, except that I am proud of this one. It hits all of those specifications as best I can manage.
My stupid “rubric”
I split the assessment into two phases: The first is “mathematics” like recognizing the bidirectional statement, noticing that one direction is false, construction of a counter-example for this direction, noticing that the false direction can be repaired with the addition of a hypothesis (hint: we talk about convexity alot in this course), and basic correctness of the arguments for each direction. The second phase is more about “writing and communication” like clarity and precision. Each part was judged on a 0–4 scale.
Looking back, I seriously don’t know what any of that is supposed to mean.
Why I Think That My Magical Exam Question is Inadequate
The more I think about repairing the rubric (or tossing it out and writing a new one), the more I think that the whole enterprise is doomed to failure.
This assessment happens once. It happens during a week long take-home final exam with five of six questions. Not every student turns in an attempt at that question.
How do I meaninfully separate the mathematics from the communication standards? How do I assess those students who spent six hours and tried the kind of strategies they should be learning, but didn’t close the circuit on this particular problem? How do I assess the partial progress that most students make?
How do I take data from the last five years and use it to either defend what I am doing, or inform me about the kinds of changes I need to make? As far as I can tell, the little columns of numbers don’t mean anything to me, and I made them.
The Big Reveal
So, the rest of today (and probably tomorrow) I will spend thinking about how to design and implement a Standards Based Assessment regime for this course. I have to meet the following guielines:
* minimaly intrusive
* fits my current practice and set of goals
* better reporting of expecations and progress to students
* ease of extraction for SOA purposes
So, anyone got any advice?