Rooted in Reason: Nurturing the Seeds of Liberty

Statistical Dangers in Standardized Testing by grassroothawaii
July 7, 2010, 2:41 pm
Filed under: Uncategorized

By David Jaress

Ever since the Bush administration’s push to centralize American public education through No Child Left Behind, efforts to establish federal benchmarks for education have persisted on both sides of the American political spectrum.  As Kelsey and Carson noted in their own blog posts, Hawaii’s DOE will soon adopt a standardized assessment program and curriculum along with schools in 47 of the Union’s states.  While they focused on the importance of flexibility in education and the typically lax standards used by such programs, another important factor to consider is the tendency of the standardized testing frequently used in national programs to misrepresent local academic performance.  Indeed, if past difficulties with standardized testing are any indication, centralized achievement indicators are particularly prone to deflating the performance of smaller and more ethnically diverse schools, while inflating the performance of larger, more homogenous institutions.

The cause of these discrepancies is largely a consequence of the statistical methods used in national education evaluations.  A 2002 study of No Child Left Behind’s standardized testing practices by Thomas Kaine and Douglas Staiger1 highlights many of the ways in which statistical error and misguided federal policies have made centralized education a problem rather than a solution.  In fact, Kaine and Staiger found that up 70% of the change in test scores from year to year were the result of random fluctuation and statistical error.  Rather than following the progress of the same group of students across a given period of time, different groups of students are typically compared to track the quality of education at a given grade level, leading to serious difficulties with establishing a fair statistical average of performance.  From the perspective of a centralized bureaucracy, however, such changes are interpreted as indicators of school performance, and under NCLB resulted in punitive measures and mandatory restructuring for numerous schools receiving federal funds.

This problem is particularly troubling for smaller institutions who draw their data from fewer students.  With a smaller pool of data to tap into, the probability that random fluctuations in performance will skew average results is much higher, making it more difficult to show signs of stable “progress” to a centralized organization.  Stipulations such as those in NCLB that require various ethnic subgroups to be tracked have exacerbated such issues by further warping the data pools used to gather statistics.  Indeed, Kaine and Staiger found that comparing such small subgroups actually punished ethnically diverse campuses, while favoring schools dominated by a single racial group.  Ultimately, while federal systems treat students as passive commodities that are manipulated by a factory-like education system, the truth is that no two classes of children are quite the same; until centralized programs can account for this, they will inevitably further complicate the task of fixing America’s schools.

1 6.  Thomas J. Kane and Douglas O. Staiger (2002).  “Volatility in Test Scores: Implications for Test-

Based Accountability Systems.”  Brookings Papers on Education Policy No. 5


Leave a Comment so far
Leave a comment

Leave a Reply (Comments are approved by a moderator before they are posted.)

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: