Who will fact check the fact checkers?
Both Hipkins and Luxon subjected to embarrassingly inappropriate criticism from the truth police.
Public fact-checking has been around since at least the 2004 US presidential election. With the current, if selective, moral panic about misinformation, there have been various and sundry attempts by organisations in New Zealand to set themselves up as guardians of the truth. The problem is that fact checkers need to have a cast-iron discipline about what they do.
Good fact-checking is objective, meaning it is limited to the assessment of asserted fact that can be measured against verifiable data. The problem is, human nature being what it is, fact checkers very commonly venture into speculative or subjective territory.
Sometimes it’s because of an ideological thumb on the scale. Often just mission creep. To a man with a hammer…
A masterclass in what not to do
For one reason or another, Auckland University's Public Policy Institute has unfortunately fallen into this trap. It has issued a fact check of the debate between Christopher Luxon and Chris Hipkins last night. You can read a summary here.
Very quickly, you will see that it is plagued by the inappropriate application of “fact checks” to statements by both leaders.
Luxon’s claim that no fruit and vege GST savings will be passed on to customers
This claim by Luxon was rated as false, because the “Grocery Commissioner will monitor pricing to prevent this."
Lexon’s statement is predictive of future market behaviour. It's not rooted in verifiable fact but in conjecture about future actions by retailers and the Grocery Commissioner. Here, the fact-checker seems to lean towards a belief in regulatory effectiveness, which may not always hold true in reality.
Luxon’s statement has not in any meaningful sense of the word been checked against a fact.
Hipkins’ statement that National’s tax cuts would make inflation worse
This was rated as being only “half true” because it “depends on government spending."
This is also not a fact check for the exact same reason. The statement by Hipkins is speculative in nature. The fact checkers are relying on economic theory suggesting that tax cuts offset by reduced government spending will not be inflationary. But they don’t know that will happen in this case and Hipkins is at liberty to predict that it won’t.
Economic outcomes are influenced by a multitude of factors, including, but not limited to, taxation and government spending. Fact-checking this statement requires the assumption of future conditions and actions that are inherently uncertain. So what the fact checkers are doing here really amounts to meeting speculation with speculation.
Luxon’s claim that ram-raids happen twice a day
This was also rated as only half-true. The reason given was that “it’s not an official crime so it’s hard to quantify." So why are they fact checking it?
But in any event, the statement seems to ignore available data. According to this Newshub report, police figures showed an average of two ram raids a day over a six-month period. A minimally competent adult, understanding the nature of crime statistics and the use of averages in conveying such data, would reasonably infer that Luxon's statement about two ram-raids happening per day referred to an average occurrence rate..
Luxon’s claim that National’s foreign home buyers tax will raise $750 million
Fact-checking Luxon's claim that a foreign home buyers tax would bring in $750 million is inappropriate given its predictive nature. This claim, much like any prediction about future fiscal revenues, is inherently speculative and dependent on a multitude of unpredictable factors. The fact-checkers' introduction of a different estimate of $210 million doesn't serve as a valid fact-check, but rather introduces another prediction that is equally uncertain. Both estimates may fall short of, or exceed, the eventual reality, and without the ability to foresee future outcomes, fact-checkers are not in a position to definitively adjudicate the veracity of such claims. This approach risks conflating fact-checking with conjecture and introduces the potential for the fact-checkers' own ideological preferences to influence their assessments.
Hyper-literal interpretations of idiomatic English
For instance, when fact-checking the claim that "National invented 'By Māori, for Māori'", PPI marked it as false based on the fact that the phrase was used in Māori knowledge systems long before. However, in the context of political and policy discourse, Luxon could have been using the term 'invented' in a more metaphorical or idiomatic sense, implying that National popularised or brought the concept into government in some significant way
It is inappropriate for a fact checker to arbitrarily ascribe one particular meaning out of several available interpretations, and adopt it for the purpose of fact-checking. Interpretations of historical facts are, within wide parameters, not assertions of facts that can be objectively fact-checked.
In the same way, the fact-checking of Luxon's statement that "Every single health outcome has gone backwards under Labour" is another foolishly literal interpretation of spoken English. The phrase "every single" is often used in colloquial speech for emphasis rather than an absolute claim of universality. Contextually, it's plausible that Luxon was emphasizing a general perceived decline in health outcomes, rather than asserting that literally all of them have worsened.
Asleep at the switch?
At the same time, PPI made no mention of claim made by Chris Hipkins that he was responsible for banning fizzy drinks in primary schools. As Newshub outlines, there is no current required ban on fizzy drinks in schools.
The claim was a clear assertion of verifiable fact. But while the PPI were preoccupied with attempting to fact-check statements that were speculative, predictive, or subjective in nature, they seem to have just done nothing about a straightforward, factually incorrect claim.
A pretty weak effort
All in all, the fact-checking effort falls short of the rigorous standards one would expect from those who hold themselves out as epistemic guardians. Those who aspire to be seen as reliable arbiters of truth should focus on concrete assertion of facts, rather than picking winners about predictions that have yet to play out, together with a more realistic approach to the context in which statements are made.
As for today’s efforts, I award the PPI a D+.