As a teacher and language test setter, I am well aware how individual questions or whole tests can tend unwittingly to disadvantage candidates of different social, racial and educational backgrounds. In an ideal world, we would eradicate all such inequalities but, in practice, attaining such a perfect equilibrium would be impossible and would not necessarily even be desirable, were it to produce tests that everyone can pass. We should however aim to eradicate any aspects of a test that unfairly disadvantage or even deliberately exclude candidates of a certain background.
The Louisiana State Voter Registration Literacy Test of the 1960s was a test designed specifically for the purpose of disenfranchising African Americans and it provides a good guide as to what not to do when designing a test. The Jim Crow establishment in the Southern States was concerned to hold back social change and civil rights for as long as possible. Although it was no longer legal to disenfranchise citizens on the basis solely of race, it was still permissible to withhold the right to vote from those who were deemed illiterate and hence ‘mentally deficient’. The State of Louisiana thus introduced a test to be applied to all citizens who had not completed basic education, a disproportionate number of whom were, for reasons of historical discrimination, inevitably going to be black.
The test the State of Louisiana came up with is devilishly effective. It is not just tricky, it is carefully crafted, in every respect, to ensure that every candidate will fail and, furthermore, with a very low score, thereby further confirming the mental deficiency of the potential voter.
The test consists of 30 questions of a ‘logical’ nature.
At the time, it was widely believed that tests based on pure logic were less likely to discriminate and more likely to assess ‘pure’ intelligence than those based on language skills, math or other school subjects. This in itself is a highly dubious assumption and one that I return to below.
Candidates were given 10 minutes to answer all questions and would fail and thus be prevented from voting if they fell short of a score of 100%. It is clearly impossible, even for a highly trained logician hyped up on amphetamines, to accomplish such a task.
Were this not barrier enough, most of the questions are of the kind that tie your brain in knots or are virtually impossible to answer with any degree of certainty. Questions of the type “Underline every word in this sentence that contains the letter e except this one” abound.
This test is obviously as laughable as its intent is despicable, but it is the only example I know of a test specifically designed to garner a 100% failure rate and, for this reason, I recommend that test setters study it as a way of ensuring that none of their questions resemble it even in the smallest degree.
The racist electoral authorities in Louisiana were only able to slip this transparent exercise in bigotry and deceit under the eyes of higher more liberal authorities because there was a general idea at the time that supposedly logic-based intelligence quotient (IQ) testing provided a fair unbiased measure of intellectual ability. Such tests were routinely applied in the UK to children as young as eleven to determine whether they would be granted a more academic education providing later access to better paid clerical jobs or one based on more manual skills. They were also used by psychologists to classify individuals as ‘morons’ or ‘cretins’ (these now highly offensive epithets were then regarded as technical terms) and thereby deny these individuals access to certain opportunities and rights (including, in some cases, the right to procreate).
IQ tests seem fair. There is a balance between verbal, mathematical, visual-spatial and logical skills. They do not appear to depend on any prior book- or family-learnt knowledge. A whole impressive-looking baggage of science and statistics is at hand to back them up.
More critical scholars who examined IQ test results found, however, again and again, that, while such tests could not be accused of discriminating on the grounds of race or class, children who are raised in an urban environment perform significantly better on them than those from rural areas.
In an attempt to explain this anomaly, they pointed to questions (common in IQ tests) of the ‘odd-one-out’ variety.
To give a fictional but not atypical example:
Which is the odd one out? A. dog B. cat C. cow D. chicken
The ‘correct’ answer is D, because the other three animals are mammals. But you could arrive at the same conclusion by way of other forms of logic: dogs, cats and cows have four legs, all produce milk, only chickens have wings etc. etc. Even if we take knowledge of the difference between mammals and birds out of the question, it does not, on the face of it, appear to be a particularly difficult question.
Children from rural areas, however, tend to answer differently in disproportionate numbers. And the most common type of explanation given by the children themselves for opting for Cow in this fictional case, for example, was something akin to “Dogs chase cats and chickens, but they don’t chase cows.”
What is interesting here is not the fact that the rural children got the question ‘wrong’ but the way that they explain their reasoning. After all, the question is highly flawed and has many possible answers. Cow, for example, could be the right answer because cows are bigger than cats, chickens or dogs, or because cows are used to produce milk for human consumption, while dogs and chickens and cats are not. In fact there are an infinite number of possible answers and explanations.
The researchers concluded that the rural children differed in their response for two different but related reasons. First, the children brought up in the countryside noted the order of the list, while the city kids assumed it to be random. So the rural kids were thinking something like this: “OK. The dog comes first. So this must be a question about what dogs do to the other three animals. Well, I’ve seen them chase cats and chickens, but never a cow. Pronto. Obvious answer.”
The rural children visualize the order of the words into a kind of dynamic everyday narrative with which they are familiar, whilst the urban kids are happy to regard the order of words as no more significant than that in which the melon, cherry, lemon and orange fall in a fruit machine. They are used to randomness; while rural dwellers look for a fixed pattern.
The other difference concerns the way the children of different backgrounds categorize things differently. The urban children rely primarily on abstract categories learnt from books in school. Even though they may be unfamiliar with the details of Linnaean taxonomy, they are used to organizing things by counting legs or according to functional use. And even if they got the question ‘wrong’, they would be inclined to do so on the grounds of the cow being bigger—another abstract quality.
What we are witnessing here is a clash between two kinds of wisdom. Rural knowledge is practical, visual and specific. Urban knowledge is abstract, categorical and random.
The test is the big loser here, because it fails to strike a reasonable balance between these two kinds of understanding, both of which are equally valid and important. And tests fail to be fair for this kind of reason again and again and again.
*
There are all sorts of tests in life. Academic tests form a small (if privileged) subset, IQ tests (fortunately) an increasingly smaller one.
But I wonder whether the same sort of criticism might be applied to other kinds of more important tests: elections, for example, and the dark shadow of elections that is polling.
Like multiple choice IQ tests, ballot papers present voters with a randomly ordered list of candidates affiliated to various parties. It is different from an academic test in that there is no ‘right’ answer and you can change your mind the next time the test comes around. It is also different, in so far as the test works both ways. The democratic system is testing the feelings of the population and the people are testing the policies of their potential rulers.
I therefore find it somewhat disturbing that many people, in the wake of the two most controversial and unexpected national elections in UK history and other similar ones around the world, posted comments on social media to the effect that ‘voters got the election wrong’.
This presupposes that there is some ‘right’ answer and, in so far as this negates the very essence of democracy, it is a belief that serves as a gateway to various shades of ugly authoritarianism—a slippery slope towards the reintroduction of something akin to but more sophisticated than the Louisiana Voter Registration Literacy Test.
I personally find Donald Trump’s personality and policies odious, but more insidious still is the notion that by electing him president, the American people somehow ‘got the election wrong.’ I think Brexit is a tragedy for Britain, but I would not argue that, having had their say, the British people somehow ‘got the result of the referendum wrong’.
The recent general election in Britain has thrown any notion of electoral right or wrong to the wind. It is deeply unfair to argue, given the Tory Party and Teresa May’s dismal performance in recent years, that the electorate got it wrong by failing to deliver her a stomping majority to push through unfair and misguided policies. But it is equally unfair to argue that hers is not, at least for the time being, a legitimate government.
In a true democracy, the right answer is always the one that the people opt for and they always have the right to change that next time round, which will rightly come sooner rather than later, if elections throw up ambiguous results. Hung parliaments provide good ‘teachable moments’ for politicians and electors alike.
Elections enable voters not only to opt for the best answer to their problems but also, in the event of a hung parliament, to demand a better test.
Since 2015, I have been sifting through as much British election data as possible in an effort to find an explanation for the tumult of the past two years. A whole rainbow of choices is now available compared to previous elections, in which voters were confronted with a simple choice of switching directly from Labour to Conservative or vice versa, or in more highly polarized areas, shifting from either side to the middle ground of the Lib Dems.
This was predictable and pollsters, those other testers, usually ‘got the result right’. This is obviously no longer the case.
Sifting through the data, I have identified numerous complex patterns among so-called floating voters. I shall focus here on one group alone: those who voted Liberal Democrat or Labour in 2010, UKIP in 2015 and Labour in 2017. Some of these individuals have voted for three different parties, with widely diverging policies, in the space of just seven years. Are they wrong?
Obviously, I think not. They are rather like the rural children who see things not in terms of the abstract categories beloved of pollsters (Do you consider yourself basically Labour, Conservative or Lib Dem?) but instead in those of the more dynamic practical issues of who better chases whom and what this means in everyday life.
During the Blair administration, when Labour and the Tories were largely in cahoots, these voters drifted to the Lib Dems, stirred by the populist man-down-the-pub manner of Lib Dem leader, Charles Kennedy. Disappointed by the Conservative Lib Dem coalition, these voters then shifted in large numbers to UKIP, whose leader not only had an earthy man-down-the-pub manner but also appeared to explain why all three parties had failed them for so many years, albeit by unfairly pinning the blame on immigrants and the European Union. Whence Brexit. Now, in 2017, these same voters are rallying back around Jeremy Corbyn’s more progressive Labour Party and look set, in the near future, to overturn decades of Conservative or quasi-conservative rule.
Are all these people wrong? Of course not. They are simply and rightly more interested in which candidate/party seems better placed to chase off the chickens than in fitting neatly into the abstract socio-economic/fiscal-policy categories into which, according to the pollsters and the politicians, they should belong.
Never have we been further from Louisiana. Never has the test of an election got things so right.