Monday, June 18, 2007

Rethinking Assessment

I attended Lisa Cadora's take on assessment, which was the second plenary session on Thursday morning, right after Amber's talk on geography. Last year, I did not have the opportunity to know Lisa in real life, but her 2006 breakout session entitled Charlotte Mason Meets Veggie-Tales helped me know how she deeply thinks. I have a confession to make . . . when I first read Veggie-Tales in the title, I flippantly scoffed and bypassed her session completely. However, when I actually spent time reading the description, I realized what a gem I had missed because of my complete ignorance (and arrogance)! I ordered the CD from SoundWord. Her talk helped me figure out why I felt a vague uneasiness about the Sunday school curricula I had seen all my life and enabled me to become a better Sunday school teacher. Because of the spiritual growth her talk inspired last year, I was eager to hear more from Lisa in real life.

Lisa calls her talk, "Knowing What Knowers Know: Rethinking Assessment for Human Learners". Lisa begins dramatically with a cold reminder of our collective experience with assessment, "We are weighed, measured, and found wanting" (a paraphrase of Daniel 5:27). This painful pronouncement reflects our culture's view of assessment: we wish to weigh and measure retention of information through examinations and a battery of tests. We nearly always end up finding our students wanting. This leads us to find our teachers, schools, and curricula wanting and, in an effort to escape blame, teachers catch themselves teaching to the test. If test scores continue to drop, then we toss the curricula and replace it with the new and improved curricula on the horizon, only to find ourselves in the same situation as few years down the road.

Americans have developed a theory that we can measure knowledge and knowers through standardized testing and hammer it into a reality of producing better-informed knowers. Charlotte Mason took a different tack: she hit upon narration, her primary assessment tool, by observing how children picked up and retained knowledge. Her observations led her discard oral lessons and textbooks and to favor reading living books. Experience teaching real-live knowers ruled out testing by standards and pointed her to assessing through narration. She refined her ideas by testing them out with students, striving to secure knowledge (children in the PNEU schools). Thus, she applied the exact opposite approach by observing reality and developing a theory based upon that reality.

Lisa Cadora points out factors that affect testing are not taken into account when using standardized tests to critique students, schools, teachers, and curricula. Some students might already know the material, so the educational environment had nothing to do with learning it. Likewise, the student might have learned it through other methods (tutor, self-education, etc.). Unfortunately, the students (and/or teachers) might have cheated. Many students might have simply memorized and promptly forgotten it after the test, a phenomenon covered by Frank Smith in some of his writings. (I called it "pump and dump" during my eighteen years of formal education!)

I would like to add two more factors to Lisa's list. First, we assume that the person who cannot meet the standards cannot reach certain goals. Temple Grandin could not learn algebra for she thinks in pictures and found nothing to visualize in the traditional instruction of algebra. She was told she could not be a cattle chute designer unless she earned a degree in engineering and that required her to master her mathematical nemesis. She settled for a doctorate degree in animal science. She ended up designing cattle chutes anyway: nearly a third of all cattle chutes in the United States are hers! Her chutes are innovative because she thinks of ways to make them more humane by imagining how a cow might perceive her equipment (and, thus, remain calm). Since she can visualize equipment in her mind just by looking at a blueprint (i.e., her brain works like a CAD), she is much more effective in quickly designing new plans. By the way, she pioneered a smaller version of a cattle chute to make a squeeze machine, a device used by autistic children to calm themselves.

The second factor I would add is how inaccurately tests measure achievement of these standards. When we lived in Pennsylvania, homeschool laws required that I test Pamela because her age would have made her a third grader. I reported her as first grade for math and reading based upon her abilities. I chose the Peabody Individual Achievement Test, which requires the child to point out the answers. Due to Pamela's expressive language delays, I thought this might be the best testing format for her. Wrong! The test ends when the child misses five questions in a row. The pre-kindergarten portion of the test required more knowledge of language than math: when shown a picture of a dime, she could say it was ten cents, but could not remember the word dime! She missed five questions in a row like that, so this standardized, normed test assessed someone who could add and subtract and count coins at a pre-kindergarten level! Three months later, I had her tested on the math portion of the WIAT, where she scored at a third grade level! She must have had a lucky day for I know she really was near the end of first grade in math.

Then, Lisa transitions to our official view of knowledge, the scientification of education. At the turn of the century, assessment of education shifted from the literary to a standardized, normed known product. We started to prize certainty over subjectivism, or, to quote Esther Meeks, we began "chasing the carrot of certainty." We take knowledge from the "Knower of All Knowers", God, and chop it up into tiny pieces. The teacher, "our showman to the universe" (page 188), spoon-feeds an isolated piece of knowledge to the student. The student focuses his whole attention on that isolated morsel of knowledge. Like blinders on a horse, the teacher blocks out all other related pieces of knowledge and cuts off access to the knower of all knowers.

At this point, Lisa presents an altered view of knowledge. In this case, knowledge is in its proper context and not chopped up into tiny pieces. Knowledge in this form exerts a force on our mind. Many things affect the student, a knower: the knowledge being studied, knowledge learned previously, other students, the teacher, and the Knower of All Knowers. Likewise, many things affect the teacher, who is also a fellow knower: the knowledge being studied, knowledge learned previously, the students, and the Knower of All Knowers. This model of knowledge is much more dynamic than the formulaic "official" view.

Lisa points out a great example of pursuing knowledge for its own sake from the 2007 Scripps Spelling Bee. An ESPN reporter was interviewing contestants before the bee began. When asked what he thought of the spelling bee, Evan M. O'Dorney disappointed the reporter by gushing about math and music, instead of spelling. In one press report, Evan explained,
I like the way that math works. It's not like spelling, where you have to memorize all the exceptions to all the words. Math is orderly. It makes sense. You can always figure out whether something is true, and once you figure it out, it will always be true, if the logic is correct.
After Evan won the final round of the contest, that reporter hurried back to Evan and asked what he thought of the spelling bee now that he had won. Evan told him he was still more excited about his performance in math contests like the Sandia Go Figure Mathematical Challenge. In fact, the Associated Press article begins like this,
The winner of the spelling bee sounded as if he'd rather be at a math Olympiad. Thirteen-year-old Evan O'Dorney of Danville, Calif., breezed through the Scripps National Spelling Bee with barely a hitch Thursday night, taking the title, the trophy and the prizes in a competition that he confessed really wasn't his favorite. . . Afterward, Evan spoke more enthusiastically about attending a math camp in Nebraska this summer than about becoming the English language's top speller."

Lisa concludes that Charlotte Mason, like Evan, knew the secret behind life-long learning,
The question is not,––how much does the youth know? when he has finished his education––but how much does he care? and about how many orders of things does he care? In fact, how large is the room in which he finds his feet set? and, therefore, how full is the life he has before him? (page 170-171).

In an earlier post, I mentioned that one expects a richer vocabulary after hearing Jack Beckman speak. Well, one thing I expect from Lisa Cadora is a longer book list: her session added four more books to mine!

* Book of Learning and Forgetting by Frank Smith (Book Review by Derrel Fincher)
* Personal Knowledge by Michael Polyani
* To Know as We Are Known by Parker Palmer (Book Review by Elizabeth Leborgne)
* Longing to Know by Esther Lightcap Meek

No comments: