Lying with statistics - a tutorial

I confess that I dropped out of the only course I ever took on statistics because my basic mathematics was so poor that I could not keep up. So I find the following exposition extremely illuminating, and reproduce an edited version because over here we are fed such a steady ration of statistical horse-shit.

What’s the story behind the popping of the New York schools test score bubble, asks VDARE? The proximate cause was that standards were finally raised this year by New York State's Board of Regents, which is not under the control of New York City Mayor Bloomberg's control. See this NYT article

The main explanation is that the state’s tests got so easy to pass from 2006 to 2009 that whites started running out of headroom. In 2006, 77% of white students passed the math test. By 2009, that percentage had been inflated to 92% of whites. Making tests super-easy automatically narrows the racial gap as measured by the simpleminded method of subtracting the percent of blacks passing from the percent of whites passing.

In 2006, 77.2% of whites passed (blue line in the graph above) compared to 46.5% of blacks (black line), so the gap (red line) was announced as 30.7% (77.2 - 46.5 = 30.7). By 2009, 92% of whites passed versus 75% of blacks, so the gap was proclaimed to have shrunk from 31 points to 17 in just four years.

Yet, is subtracting the black passing percentage from the white passing percentage the best way to track the racial gap? Would Bloomberg and Klein get a passing grade on a math test?

If you look at failing rates rather than passing rates, and use division rather than subtraction, you’ll find the “racial ratio” (black failing rate / white failing rate) moves in the opposite direction as the percentage of whites failing falls toward zero. In 2006, 23% of white students in NYC flunked the math test compared to 53% of black students, for a black / white ratio of 2.4. By 2009, only eight percent of whites and 25% of blacks flunked, giving a black / white ratio of 3.2.

Indeed, as the percentage of whites failing approaches zero, the ratio of black to white failure rates approaches infinity. This paradox should come as a warning to us that the apparent size of racial gaps can be manipulated by clever salesmen.

In truth, using differences in percentages passing is statistical chicanery. Statisticians know that the proper way to measure differences like this is with standard deviations, not percentages.

The pseudonymous statistician La Griffe du Lion pointed out in 2001 in the above graph that the white passing percentage minus black passing percentage racial gap must be zero when either nobody passes the test or when everybody passes the test. Assuming a constant one standard deviation gap, the percentage point gap is largest (38 points) when 69% of whites pass.

So, if you start with a test on which about 69% of whites pass, you can artificially narrow the racial gap by either making the test easier or harder. It doesn’t matter!

If Mayor Bloomberg, who made his billions delivering statistics to traders, doesn’t know about standard deviations, well, Bloomberg L.P. employs hundreds of people who could have explained it to him. It is more likely that Bloomberg, a master salesman, knew that talking about gaps in percentage terms was misleading, which is why he did it.

If we make the simplifying assumption that the scores of both whites and blacks follow the normal probability distribution, we can easily plot the New York City math test passing rates in terms of standard deviations, giving us a more accurate picture. While the Bloomberg-Klein percentage method showed the racial gap narrowing almost in half from 2006 to 2009, the superior standard deviation difference (red line) shows only a slight decline through 2009. The white-black gap was 0.83 standard deviations in 2006, before falling to 0.74 in 2009, then bouncing back to 0.90 in 2010.

The Fire Department of New York tried the same tactic to avoid a disparate impact suit over the notoriously innumerate Equal Employment Opportunity Commission’s rule that blacks must score at least four fifths as well as whites in percentage terms. They pushed the white passing rate on the fireman’s hiring test to 97 percent so that the black rate hit 85 percent.

So, to sum up in my own words, the intellectual development of young blacks in New York City has not improved in the slightest and arguably has suffered (by distracting educators and policy-makers from remedial action) as a result of this exercise in statistical chicanery.

I need hardly remind British readers that the entire educational system on this side of the Atlantic has been "managed" for decades to produce exactly the same false evidence of gains for disadvantaged groups.

The main thrust of the VDARE article is to support the argument in the book Bad Students, Not Bad Schools, namely that the intelligence and motivation of the students is by far the most important factor in whether a school is “bad” or “good”. I know that any such argument can be turned to racist use - but that does not make the argument itself intrinsically racist.

IQ tests were originally devised to identify those among a given population in need of special attention in order to reach a desirable level of educational achievement. The very fact that properly coached individuals can significantly improve their IQ scores proves that it is NOT a measure of innate cognitive ability.

"Comprehensive" education denies special needs, with the result that those in most need of help get shafted, while the bright ones are bored out of their minds and turn their talents to trouble-making.  

Hat-tip to Jay.