Serendipity35

Back in January at FETC 2008, Chris Dede asked his audience that question as his presentation intro. (Chris Dede teaches at Harvard's Graduate School of Education.)

His vision for the future of assessment? Two main points: 1) formative assessment (the diagnostic type) provides more valuable information about students' abilities than summative assessment (those final exams) 2) technology gives schools access an amazing amount of data that can gauge your students' understanding of the key concepts you would typically include in those summative tests. He's not addressing here other assessment types like objective & subjective, referencing (criterion-referenced, norm-referenced, & ipsative AKA "forced choice") and informal & formal.

I agree with his first point, but (though I don't find it a new or radical point of view) it is not an idea that has been widely implemented in education. We are still heavy on the summative. Much of that is probably still driven by the demands of larger agencies (accrediting, governmental).

His second point is a good one and more important to the focus of this blog. Unfortunately, using technology is even harder to implement than a formative philosophy.

He gave the example of technology mediated interaction (webinar, video conference, chat session) and how logging this interaction could be analyzed later to mine information about your students' thought processes. Of course, this type of formative assessment could be used to revise instruction.

The example is used in his talk is the River City project which I wrote about in more detail in an earlier post. Briefly, it's a multi-user virtual environment (MUVE). Students are immersed in an online scenario and asked to apply scientific inquiry skills to solve a problem.

When students interact with residents of this MUVE, the interaction is logged in a database. The project has researchers who can use this data (where students went, what they examined, who they talked with, and what they said). No lack of data. Of course, as you would guess, the problem is what to do with all this information.

Dede admits that educators can see easily things like what scientific data was contributed by a student and time on task. Again, this project (unlike your classroom - unless you have some grad students at your beck & call) has researchers who are looking at data-mining techniques to help use that data.

If you were asked that opening question, what would these 21st century tools (ones that exist or that you want) DO for you? How much of the assessment equation is in collecting data versus interpreting data? Do you feel the problems occur after the data is analyzed and you try to implement change in instruction?

Though I'm actually more interested in your answers, I'll answer my own questions for one project. Right now, the grant I'm directing has as one of assessment tools a writing exam that the college requires a passing grade on for graduation. They have tons of data going back a decade. It has been analyzed and interpreted in different ways. The tools that have been used are 20th century - Excel and Access. The interpretation has been human-powered and inconclusive. It has actually led to some change - the grant itself is part of that change - but it will be at least a year before we can expect anything to emerge from the exam data.

I would love a tool to help automate the collection of data about the students taking the test and correlating it with the results. The interpretation is human-powered which means there will alwyas be arguments. I would like good machine scoring of the essays so that we don't need to corral readers to go through the essays, and especially to get objective scoring. But I feel those are 20th century desires. A new century desire would be speed. Student does essay online, clicks submit and gets their score and I get an analysis of why they got that score and how they could improve. I want the only problem to be implementing the changes. And that is plenty of work.