Serendipity35

This essay is about holistic scoring. I see connections all around me. It's one of those blessing/curse qualities. As an undergraduate English major, it was very useful for literature classes. I think it helps students in my courses. But, there are times when the connections that I see are not so clear to others.

Case in point: I'm reading an article about a project to try to classify insects using barcodes. Matching living things to their names is difficult. So difficult, that it can be called the taxonomic impediment.

"Biological classification may seem like an esoteric problem better left to librarians than field researchers, but it is reaching unprecedented importance as discoveries swell the existing rolls of some 1.8 million known species, and prominent scientists such as E.O. Wilson throw their backing behind an ambitious project to make taxonomic data for all of life on Earth accessible online. Classification systems, meanwhile, have themselves become the subject of intensive study, thanks to the explosion in data-labeling and -sorting procedures allowed by digital media."     "What's In a Name? The Future of Life"  WIRED

Last weekend, I'm watching CBS-TV's Sunday Morning and David Pogue does a piece on that very project, the Encyclopedia of Life, and I realize the connection to a current project at the college to create a rubric for scoring the required writing exam.

There are been heated discussion on what we mean when we say holistic scoring. A web search will bring you about 30,000 hits. Try define:"holistic scoring" in Google and you'll get a page full of definitions. There are certainly official definitions, but I find that readers have a wide range of personal definitions. Those have been emerging in the norming sessions at the college in preparation for using a "new" rubric for the October test date.

I say "new" rubric because the old one really wasn't a rubric and it wasn't being used. It seems that the reading sessions have devolved over the years to: read the essay, give it a number. Low passing = 4; high failing = 3. That is at the extreme end of the holistic definitions, and you can find it defined that way.

"In assessment, holistic scoring means assigning a single score based on an overall assessment of performance rather than by scoring or analyzing dimensions individually. The product is considered to be more than the sum of its parts and so the quality of a final product or performance is evaluated.  Holistic scoring criteria might combine a number of elements on a single scale." http://www.sbctc.ctc.edu/College/_e-assessglossary.aspx

ETS has this take on the subject:

"...With this method, essays are read for the total impression they create, rather than for individual aspects. Raters award a single score for the overall quality of essays based on an integrated set of specified criteria. Criteria typically include organization, development of ideas, style, mechanics, diction, and usage."

I'm closer to that definition. It frightens me to think of a reader finishing an essay and having a gut feeling that it's a "3." It reminds me of a cattle call for dancers on a Broadway stage. "Dancer 3, 7, 9 and 12. The rest of you, thank you for coming."  What did I do wrong? What did they do right?

Back to ETS:

"Most programs have published scoring criteria that are applied consistently across scoring sessions. While many raters have vast experience in evaluating writing, they should understand that, for essays to be scored fairly and consistently, they must adjust to the overall standards set for the particular test program."

It's the naming of those criteria that seem to give some people a problem. As with taxonomy, it's training "the eye of the tagger" that seems critical.

Part of my personal reaction to all this at PCCC with the College Writing Exam is that the exam is high-stakes. If a student doesn't pass, a student doesn't graduate, and about 40% of the test takers in each test session are not passing. Those are students that our new writing center needs to work with and our staff and tutors need to know more than "3" in order to discuss with students ways to improve for their retesting.

Folksonomy, the people's taxonomy that is so prevalent on the Web through sites like delicious, Flickr and Digg, works as long as there is some agreement on the naming. If I tag something as "satire" but others tag it as video, or comedy or TV or SNL, does that make it better/broader or break the system? True taxonomies rely on agreement.  Plant taxonomy classifies one plant as Gerbera jamesonii so that the common name "African daisy" or the altered versions of the scientific name ("gerber daisy" or "gerbera daisy") all point to the same thing. It's not arbitrary at all.

I ask my students to tag useful sites for my visual design course in delicious with the unique tag of "msptc605" so that their fellow students can share bookmarks and so that the list can increase in future semesters. We agree on that tag so that all our bookmarks can be together, but we also need to have additional tags such as typography, color, usability so that the list is useful. It's not arbitrary at all.

I believe in holism for grading and scoring. I certainly wouldn't want to endorse the use of its opposite, reductionism, as a solution.

I value the diversity of opinions that come out when you have a group of a dozen teachers sit down and read essays together. But, it's hard for me to hear someone talk about a pure holistic approach and then say that an essay fails for its "terrible mechanics." Is the whole greater than the sum of its parts when it can fail for one of those parts?

Aristotle said “The whole is more than the sum of its parts” and so proposed an idea of holism. The term itself did not appear until 1926 when Jan Smuts defined it as “The tendency in nature to form wholes that are greater than the sum of the parts through creative evolution.” in his book Holism and Evolution.