FairTest - An Interview with Bob Schaeffer

The National Center for Fair and Open Testing (FairTest) promotes fair and educationally sensible assessments of students, teachers, and schools. The group works to educate people on testing tools with the goal of eliminating standardized testing and other damaging evaluation tools that are counter to high-quality education. I recently interviewed Bob Schaeffer, one of the leaders of FairTest about this group’s work.

Despite the widespread criticism of No Child Left Behind (NLCB), why is it still going strong in our schools?

If the decision were in the hands of parents or professional educators, the so-called "No Child Left Behind" law would have been substantially reformed or even repealed years ago. Public opinion polls, such as the one done annually by Gallup for the Phi Delta Kappan magazine show that there is widespread and growing criticism of NCLB for narrowing curriculum, encouraging drilling for poorly-designed tests instead of high quality teaching and learning, and assuming a "one size fits all" approach is best for all children. The closer observers are to classrooms, the more clearly they see these problems. Unfortunately, NCLB was hatched "inside the Beltway" in Washington, DC. That is where its strongest supporters, professional politicians and the lobbyists who support them, are located. They cling to the illusion that high-stakes tests are a cheap, simple way to improve schools, even when there is overwhelming evidence that this scheme does not work. Until members of the public strongly press their elected representatives that NCLB must be overhauled, the law will remain in place despite the damage it is doing in many schools.

What other ways can be assess students in any grade?

Teachers and academic researchers have identified a number of superior ways to assess student performance, including portfolios of work, projects, presentations, and even low-stakes tests whose results are used to improve learning not blame and punish. Many classrooms, including those in the Coalition for Essential Schools and the Performance Assessment Consortium, regularly demonstrate that a richer , deeper, more meaningful education can be obtained once the restraints imposed by a fixation on standardized tests and their results. Graduates of these programs go on to excel in life.

How cost effective are alternative forms of assessment compared to the standardized variety?

When high quality assessments are built into the curriculum, for example by requiring regular reviews of collections of students' work by the classroom teacher, colleagues, administrators, parents and even members of the larger community, the costs are lower than those currently incurred to purchase commercial tests, buy the materials needed to prepare students for them, and train teachers to focus on what will be on the tests as well as pay for the innumerable hours spent in administering exams and analyzing scores.

I was pleasantly surprised to see that 730 colleges and universities around the nation offer the option of not taking the ACT or SAT for admissions. What criteria would you like to see higher education use for admissions?

FairTest recently updated its list of test-optional colleges and found that there are now more than 815 accredited, bachelor-degree granting universities which do not require all or many applicants to submit ACT/SAT results before admissions decisions are made. These schools have universally found that deemphasizing test scores results in improved academic quality as well as more diversity in their student bodies. Instead of focusing on how well teenagers filled in standardized exam bubbles in three-and-a-half hours on a Saturday morning, admissions offices look at academic performance over time --in the forms of grades, class rank, and course rigor --, extracurricular activities, community service, leadership, obstacles overcome, long-term interests, and other characteristics. Considered together, these factors are better guides to predicting college success than test-taking skills ever can be.

How much money do you estimate goes to testing corporations that could go to teacher salaries and materials?

There is no accurate way to measure the size of the testing industry, in part because so many firms are involved handling different parts of the process (e.g. designing test forms, printing test forms, running scoring machines, producing test-prep materials, training teachers to coach students toward higher scores, interpreting results etc.). Indeed, one academic attempt to come up with a reasonable estimate ended up producing a book titled "The Fractured Marketplace for Standardized Testing," documenting how complicated the assessment business has become. What we do know is that the three major companies involved in the college and graduate school admissions market, the College Board, Educational Testing Service and ACT, have combined revenues of almost $2 billion per year. Add to that the hundreds of millions of grade K to 12 tests administered in public schools across the nation -- due to "No Child Left Behind" every student must take at least 17 before graduating from high school, and many states add additional standardized exam requirements -- and the industry size must be in the tens of billions of dollars annually, if not more than $100 billion. These figures only include the direct costs of testing -- coaching materials, forms, and scoring. An even greater expense is the immense amount of classroom educator time spent preparing students to take exams, administering them and analyzing their scores. Clearly a tremendous amount of money is being spent on standardized exams which would better be devoted to improving teaching and learning.

I have heard about scandals in SAT scoring. Can you tell me what you have learned?

The scores of more than 4,400 students who had taken the October 2005 SAT were misreported, sometimes by as much as 450 points on the three-part test's 600 - 2400 point scale. The College Board, which sponsors the SAT, did not admit to the mistake until March, 2006 after some admissions and scholarship awards had been made based on the erroneous scores. To this day, the College Board has not offered a credible explanation of what happened or why it took five months to report the problem to test-takers, high schools and colleges. The available evidence suggests that a number of answer sheets somehow became damp and stretched, throwing off the machines that automatically tally scores. No one knows how or where water or humidity contaminated the paper. The College Board did, however, settle a lawsuit for $3 million to avoid a lengthy investigation of the fiasco in a court of law where testimony must be truthful or risk penalties for perjury and internal company documents are subject to review by plaintiffs and outside experts. As a results, students who received faulty scores were paid at least $275 with greater compensation available to those who could show greater damages. The incident demonstrates, once again, that standardized tests are not flawless, "objective" instruments of mental measurement, as their proponents often imply.

It seems that IQ, SAT and ACT are highly fluid numbers that we put a lot of weight toward understanding a person’s intelligence. These scores can change if retested and can be prepared for by studying its contents and memorizing facts. Isn’t this the tail wagging the dog?

If tests accurately assessed enduring human traits, such as "intelligence" and "aptitude," reported scores for the same person would not change significantly from one administration to another. Yet, a number of researchers have shown that short-term, intensive training can boost results on the SAT (and other tests) by tens of points, if not hundreds. That means students whose parents can afford expensive test-prep courses have a much greater opportunity to pump up their college application credentials, further tilting the admissions playing field in their favor. Large score gains also undermine the credibility of standardized testing industry firms which claim their products measure meaningful knowledge, not test-taking skills

I’ve heard that these tests are often discriminatory against minorities and low-income groups. How so?

The test-makers themselves admit that exams such as the SAT underpredict college performance for females and students whose home language is not English. The nature of the tests, fast-paced, multiple-choice "games," often including odd language and esoteric problem types, places a premium on speed and strategic guessing. This build in a disadvantage against groups.which may either read more slowly in order to translate unfamiliar words, those inclined to linger over complicated items rather than filling in an answer and moving on, and individuals who are hesitant about offering a response unless they are certain of its accuracy. These may all be good skills in college and life, but they result in lost points on standardized exams.

It is said that Emotional Intelligence is a stronger predictor of success in college and work life. All things being equal, how can colleges and companies take that into account when looking for the most likely to succeed student or employee?

Skilled admissions professionals, particularly those at test-optional institutions, know how to evaluate an applicant from a comprehensive or "holistic" perspective. The process takes into account multiple factors, not just "the numbers" of test scores and grades, including an assessment of the student's course-taking patterns, learning growth, and academic honors balanced by other considerations such as family background, career interest, unique skills and, at some institutions, emotional intelligence..

What is the purpose of education in your view and how has standardized testing changed that?

The purpose of education has been an unresolved subject of debate at least since the beginning of recorded history. FairTest believes children should "learn how to learn," developing a range of knowledge, skills and experiences that enables them to become productive participants in 21st Century society. An over-emphasis on standardized exams, which can measure little more than factual recall, dumbs-down teaching and learning so that some schools become little more than test-preparation centers. To encourage an emphasis on higher level thinking skills, cross-content learning, and real-world competencies, politicians must be persuaded to back away from their fixation on simple-minded tests and encourage what really matters by moving toward performance-based assessments.

For more information about FairTest, see: www.fairtest.org