Testing the Truth About the SAT
This revised and updated version of David Owen’s 1985 exposé is: a) a balanced and objective analysis of the merits and limitations of the SAT as a college entrance examination; b) a one-sided diatribe that can be easily dismissed; c) a significant extension of the original volume; d) none of the above.
The obvious answer to this question makes it a poor candidate for an SAT question. However, the limited information it provides about the depth and breadth of this well-researched inquiry mirrors the insufficiency of the SAT as a test of scholastic aptitude, achievement, ability or whatever its administrators will claim it measures next.
The two prefaces and two forwards set the tone for this unapologetic attack on the Educational Testing Service — which develops, revises, and administers the SAT for the College Board — and all of its products, but especially the SAT. The book then launches into a full frontal assault on ETS, poking fun at the corporate greed, misplaced values and extremely thin scientific veneer under which lurk the bigoted measurement masterminds who perpetuate this travesty on unsuspecting youth.
Unfortunately, the verbal assault kicks into full gear before the evidentiary proceedings reach a similar level. I don’t imagine that many proponents of ETS, or of standardized testing, would have the stomach to read past the first chapter — if they made it through the prefaces and forwards.
The strength of this book comes in the middle chapters that poke substantial holes in every conceivable aspect of the development, implementation and use of the SATs to screen college applicants.
Also unlike the SATs, this book will appeal to multiple intelligences as it uses a variety of forms of evidence including statistics, stories, logical reasoning and expert opinion. It includes an excellent lay presentation on how the essence of SAT item development — statistical reliability — provides a basis for training students on how to ‘beat the exam.’
It includes a compelling story about a cheating lawsuit that leads you to believe that ETS employs defense tactics that would make J. Edgar Hoover blush. The presentation is at times uneven as exemplified by a weaker treatment of test validity (compared to reliability). Moreover, the bias of this book is so blatant that any true skeptic would be wary of the selective attention that the author likely paid to the entire field of available material.
Owen uses the analogy of a ‘kettle defense’ to describe the standard ETS response to critics. This defense starts with ETS denying that any damage has been done. They then claim that the damage was pre-existing. Finally, they seek credit for fixing the damage.
Ironically, the book presents a perverse kind of kettle offense. First, the authors establish that the SAT has no value as a college entrance examination. Then they illustrate how little it affects students’ likelihood of entering the vast majority of colleges and universities. But ultimately, they accuse the test and its creators for inflicting significant damage on millions of students and their families.
Although it overstates in the early chapters the importance of the SAT as a college entrance exam, the book does not overstate the impact of this exam on the perceptions of many college-bound students.
Marilyn Doerr deserves credit and accolades for bringing this book back to print. She also has succeeded in bringing some of the book’s arguments and statistics up-to-date without impacting the unabashed spirit of the original volume.
However, because of this minimalist revision strategy, several recent trends are excluded from consideration. For example, the SATs have come to play a prominent role in rating colleges and universities. The U.S. News and World Report annual college rankings use SATs both directly as one of several measures of selectivity, and indirectly, as one of the ‘controls’ for predicting an institution’s expected graduation rate against which the actual graduation rate is compared.
Thanks to this kind of rating mentality, there is more pressure on universities and colleges to increase the entering student SAT profile than there is to improve the student learning experience in college.
Another trend that deserved some attention is the increasing prominence of the ACT entrance exam. This SAT competitor is now running nearly even with the SAT in annual number of test-takers. The ACT claims to be more closely aligned to the high school curriculum, which relates closely to one of the recommendations that Owen makes for improving college entrance screening processes.
But Owen’s strong arguments against any form of standardized testing make me wonder whether he would view the ACT assessment and its supporting institution as a lesser evil, or just as bad.
“None of the Above” is a compelling book. It’s strong evidence and mostly well-reasoned conclusions are mitigated only moderately by its rancorous tone and uneven development. Opponents of standardized testing will applaud this tone but those who stand to gain the most from this treatise might well be turned off by the approach.
It tears down all the walls at ETS but does little to construct an alternative. It’s weakest chapter is the last one that begins with a few positive ideas but ends with recommendations that are a final attack on the SAT and ETS.
It is entertaining, if overdone, but leaves us with a significant question: How can the college application screening process be improved so as to motivate student learning in high school and improve the likelihood of student success in college?
You have twenty minutes to answer the question, please use a number 2 pencil and do not go on to the next section of the exam until you are instructed to do so.
Ready…begin.
— Dr. Victor M.H. Borden is director of information management and institutional research at Indiana University-Purdue University Indianapolis.
© Copyright 2005 by DiverseEducation.com
