Tuesday, February 19, 2013

Truth-O-Meter doesn't measure veracity, not even truthiness

PolitiFact Oregon attempts and fails to test the statement by an Oregon state senator that one in eight Oregon children and one in 18 adults suffer from mental illness. They falsely conclude that it is mostly true.

They start out well noting that the statistics were not based upon diagnosed cases of mental illness. They found out that the basis of the stats is that they "take census figures annually and apply a standard formula developed in the late 1990s to determine how many people have a "serious mental illness" or "serious emotional disturbances."

Census data and standard formula so if the Oregon population increases according to census data then we have more mental illness, and of course the reverse is true too. But it gets better, e.g., "the research institute [National Association of State Mental Health Program Directors Research Institute] applies a percentage to the civilian adult population -- in this case, 5.4 percent -- to derive the estimated number of people 18 years and older with a "serious mental illness."

One well ask where does the 5.4% come from? Well ask all you want, PolitiFact Oregon assumes that is correct or at least not subject to any further scrutiny and its inquiry essentially ends there. Thus using the institutes' determined percentage, made up out of whole cloth as far as anyone knows, PolitiFact Oregon applies that to the census data and voilĂ  the senator had it correct.

In fairness to the senator he had used a statistic that he should have been able to rely on because of the source's apparent credibility. However, PolitiFact Oregon just gives up any attempt to erase a skepticism it had at the beginning and finds: that "all in all, it’s hard to argue with a nationally accepted standard . . .  ." All the more true when you don't try.