The way we choose our sample is vitally important, and also somewhat intriguing. Here’s an example to show how it’s done. Let’s suppose voting has been completed in the Bangalore Rural constituency for the Lok Sabha elections. About 8 lakh votes have been cast. The results are to be announced 7 days later, but, before the final count, we are asked to predict the likely winner using a quick ‘day-after’ survey. What should be the minimum sample size for this survey?

First alert: In India’s first-past-the-post system, winning margins can be very small. So, while choosing the sample size, we have to decide what kind of a winning margin are we expecting? 5% more votes for the winner, or 2% or 1%? Second alert: We’re playing with probabilities, and there’s always a chance that we’ll get our prediction wrong. How often do we want to get it right? 99 times out of 100, or 95 times out of 100?

Another situation often played out in the CNN-IBN studio. When one of the two contesting alliances established a significant lead, Sardesai would ask the spokesperson of the losing alliance to accept defeat: “Come on, you must now admit that you are going to lose this election!” The response would typically be on the lines of the following: “We haven’t counted even 5% of the total vote so far! By looking at a few thousand counted votes, how is it possible to know how 10 lakh voters have really voted?”

Intuitively, it is easier to believe that the larger the fraction of the vote counted, the more reliable the prediction is likely to be. The pollster sees things differently. If it’s a straight contest with the leader maintaining a 2% lead over his rival, even a random sample of 100 will correctly predict the winner 65% of the time.

If the sample size is increased to 1000, then the prediction will have a 90% chance of being correct. And when the sample size is raised to 4161, as we explained earlier, there’s a 99% chance of success. Even a ten-fold increase in the sample size thereafter will merely increase the chance by a fraction of a fraction. So, increasing the sampling fraction makes no sense at all: it only makes the game more expensive.

The only situation when it will make more sense to increase the sample size is when the contest is very tight. In such situations, pollsters might want to work with tighter error margins. Instead of an error margin of plus or minus 2%, they could consider an error margin of plus or minus 1%. ^[5] With this tighter margin, the sample size for 99% success will go up from 4161 to 16,644. ^[6] In most cases, it simply isn’t feasible to insist on such high accuracy because of the cost factor.

It’s now easy to guess what might have happened with a large number of seats in the Bihar Assembly poll. A simple count shows that the margin of victory was less than 2% in as many as 40 seats out of 243. ^[7]

In many ways, this is the central issue in calling elections in India: how does one correctly predict the winner, and also the number of seats that each party or alliance will win?

Poll prediction in India has two stages. In the first stage, we conduct surveys to estimate the likely percentage of votes every party or alliance will get. In the second stage, we convert these vote percentages into seat estimates. If the gap in the vote percentage between the largest and the second largest party or alliance is sufficiently large, it is easy to guess who will win the seat; if the gap is smaller, then it becomes tricky.

It is practically impossible, or very expensive, to survey every constituency, especially if it is a Lok Sabha poll. ^[9] The pollster therefore has to be innovative in choosing which constituencies to survey, and, more generally, in devising a sampling scheme that most faithfully represents the trend in the state and the country. Surveys are a hard grind, and not sufficiently remunerative. ^[10] There is also irony in the fact that while election telecasts attract big sponsorship, election surveys seem to lack sufficient funding, even though they determine the quality of the prediction.

When it comes to converting a party’s vote percentage into its number of seats, all pollsters claim to have a formula. In India’s first few election forecasts in the 1980s, led by Prannoy Roy, Ashok Lahiri and others, the key ingredient was the so-called ‘Index of Opposition Unity’ (IOU). The IOU relied on the classical first-past-the-post electoral axiom: all that matters while making an election forecast is to see who’s winning the race.

And, since this is hardly ever a two-horse race, the winner doesn’t even need to poll 50% of the votes. With three serious parties in the fray, even polling 38% of the votes fetches a win in most cases.

The best way to make it harder for the leading party is for all or most of the other parties to come together. The IOU was an attempt to measure the degree of togetherness amongst the rivals; the higher the IOU, the harder it is for the first in the race to score a facile win.

The swing is often a double-edged sword. If there is a swing of +3.3% in favour of the BJP in a two-party contest, it means that the BJP has pulled away from its rival by 6.6%, which is significant.

Next, consider the scenario where BJP has suffered a 2% loss in swing, but the gain is distributed among several other parties. In this case, the BJP isn’t hurt as much.

It is easy to understand why the IOU becomes so critical in such situations. But, and here’s the rub, the exact mathematical formula connecting IOU to the seat count prediction is not easy to find. I searched through the big and small print of The Verdict by Dorab Sopariwala and Prannoy Roy, but the formula remained elusive.

Rajeeva suggests that it was likely based on simple heuristics: something like ‘if the IOU is less than 25%, give the first-placed party 75% of the seats.’ It may also have involved intelligent tweaking based on current survey data, historical data, informal feedback, expert opinion, gut feeling, and so on.

The catch is that we absolutely cannot rule out the possibility that the candidate currently trailing 49-51 could eventually win the race. There could be sampling errors. This is why we need that plus-minus 2% margin of error.

Now we must ask: What is the probability that the trailing candidate becomes the winner? The candidate is clearly a little less likely to win than the rival, so the winning chance is below 50%. But it isn’t 0%. A reasonable guess could be that there is still a winning probability of, say, 44%, or “0.44” if we want to pretend to be a probabilist of pedigree.

If we choose a sample of 4161 from each of the 543 Lok Sabha constituencies, we are already looking at an impossibly large sample size of more than 22 lakh. Do we need such a large number? Can anyone afford such a large number? If this number has to be trimmed, what might be the best way to do it?

We are curious to know how other election pollsters handle this situation, because nobody discloses such information openly. Many rely on quota sampling, an idea borrowed from market research, where you identify attributes (age, gender, caste, religion, education level) that influence the population’s voting choice, and choose a sample that best represents this population. The benefit of quota sampling is that it is quick and relatively inexpensive. But it is definitely not random sampling. Quota sampling is probably a good option if you want to decide the colour of a new toothpaste brand. It doesn’t lend itself as well to a national poll.

Telephone surveys have been popular abroad, and are increasingly gaining currency in India. These are sufficiently inexpensive and infuse at least a modicum of randomness, but we can’t be sure if such surveys touch the entire voting population. It is not hard to imagine that this method would favour rich, urban and male voters. To be sure, tweaks and correction factors are possible, ^[14] and pollsters can get quite clever at this, but the overall integrity of the process is still somewhat compromised.

How, then, does one conduct an all-India election survey with a manageable sample size and while respecting the tenets of statistical randomness? In 1997, the team from CSDS, with Yogendra Yadav at the helm, was the first to consider the idea of circular random sampling for election surveys. But they needed a proven statistics expert to confirm that the scheme will work. They called Rajeeva, then of the Indian Statistical Institute in Delhi, to ask “Scheme chalegi?” Rajeeva replied in Delhi lingo:“Daudegi!” The scheme’s got wings.

When the I.K Gujral government collapsed in November 1997, a surprise mid-term poll was announced. The CSDS team, who thought they had a couple of years to develop their election methodology, now discovered that they had only two months. “Could we take off in just two months?” they asked Rajeeva. That was the starting point of an association that would last for more than a decade.

In the CSDS circular sampling method, only every fifth Lok Sabha seat becomes part of the sample. As a result, we’re extrapolating the result for 543 seats from a sample seat size of 108. To choose these seats, we number Lok Sabha seats 1, 2, 3, and so on till 543. Then, we randomly pick a number between 1 and 543—378, for example—and then select seats with numbers 378, 383, 388 and so on. While numbering seats, we also make sure that all seats in a state are in a single bunch so that the sample size is proportional to the total number of seats: Uttar Pradesh, with 80 seats, will therefore have the most (16) seats in the sample.

We continue (constituency -> booth -> voter) with the same circular process, randomly picking polling booths within a constituency, and voters within a polling booth. With 8 booths per constituency and 25 voters per booth, we get a sample size of 25 x 8 x 108 = 21,600; with 50 voters, the sample size doubles to 43,200. It can be proved that with a sample size of 21,600, the accuracy is at least 99%. With 43,200, the accuracy is at least 99.5%.

The first circular random sampling exercise succeeded. Based on the CSDS-conducted survey and Rajeeva’s prediction of the seat count, India Today forecast 214 seats for the Atal Bihari Vajpayee-led NDA coalition a week before the first phase of the 1998 polling. After polling was completed, the NDA seat count was revised to 251 based on the ‘day-after’ poll projections. The final NDA count was 252.

Finally, it’s time to put together all the pieces of the election prediction jigsaw puzzle. Let’s look at how the result of the 2019 Lok Sabha election was predicted. We’ll illustrate with the example of Karnataka, which sends 28 MPs to the Lok Sabha. For each of these seats, we already knew the percentage of votes obtained by each of the political parties in the 2014 Lok Sabha election.

The circular sampling exercise would’ve picked up the 2019 vote percent for about a fifth of these 28 seats—5 or 6 seats. For these seats, suppose that BJP gained an average of 3.3% of the vote in comparison to 2014, and INC has likewise lost an average of 2.2%. In the pollster’s language, this will be a +3.3% swing for BJP and a -2.2% swing for INC. We now make a crucial assumption: the swings of +3.3% for BJP, and -2.2% for INC, observed in reality over only the 5 or 6 sampled seats, will reflect in the remaining seats in the state. In other words, we assume a uniform swing over all of Karnataka.

The rest of the calculation is now simple. To get the 2019 vote percentages for each of the 28 Karnataka seats, add 3.3% to the 2014 BJP vote percentage and subtract -2.2% from the 2014 INC vote percentage. Finally, use the Probabilistic Count Method to predict the party seat tally for Karnataka.

We repeat this process for every state: first calculate the 2019 vote percentages for every party, and then use the Probabilistic Count Method to predict the number of seats for every party. By adding the party seat tallies over all the states, we arrive at an all-India seat prediction.

Which works best: opinion poll, exit poll, or day-after poll?

Currently, pre-election opinion polls are not allowed once the campaigning for the first phase ends, because of the suspicion that it influences voting preferences. There is also strong evidence that unlike in the UK or US, with their numerous ‘safe’ seats, or steadfast red and blue loyalties, there is a lot of volatility in India; studies have shown that as many as 30% voters change their mind in the two-three weeks before polling day.

Opinion polls also fail to identify which of the voters in the sample will actually cast their vote. Therefore, opinion polls conducted a few weeks before the poll can, at best, indicate the mood of the nation at that point of time, but may be unable to accurately predict the final outcome.

Exit polls confirm that the voter has indeed voted, but it is hard to enforce random sampling. Additionally, respondents may be uncomfortable to talk in a crowded public space.

Day-after polls are ideal, because they can be random, and the voter is more relaxed. Asking them to cast their vote in make-believe ballot boxes also ensures more truthful responses.

What are the questions one asks in an opinion poll?

The top two questions in an opinion poll are, in order: which party will you vote for, and who is your preferred Prime Minister or Chief Minister? A lot depends on how the questions are asked. CSDS’s face-to-face meetings with randomly chosen voters have shown significantly better results than anonymous phone calls. CSDS’s method takes time and resources and that is what makes their data more expensive than others’.

Are Indian and US poll predictions different? What exactly does that Nate Silver guy do?

Nate Silver doesn’t do his own surveys. He relies on data published by others and makes appropriate adjustments based on the track record of these agencies. His ‘popular-vote-to-electoral-college-vote’ method is comparable to our ‘vote-to-seat’ model, but instead of probability computations, he uses scenario simulations. While Nate Silver was successful in 2008, 2012 and 2020, don’t forget that he failed in 2016—very likely because of overwhelming reliance on the prior classification of states as ‘red’ (Republican), ‘blue’ (Democrat) and ‘battleground’ states.

In the early years, it was not possible to physically verify if EVM counts were correct. This weakness was corrected by the introduction of the Voter Verifiable Paper Audit Trail (VVPAT) system, starting 2014. Every vote cast by every voter on the EVM could now be audited.