Every day since September, the basement of a suburban Lucknow mansion has buzzed with the voices of hundreds of young women, enquiring, cajoling, and occasionally pleading into headsets.
“Please, please,” one begs a reluctant villager down the line, “I only need five or 10 seconds.”
Crammed around tables, powered by an unending stream of tea, the women are combing lists of voters across Uttar Pradesh, an Indian state so populated it could be the sixth-largest country in the world.
As the answers flow, they carefully fill in boxes, reading “caste”, “voting pattern” or “preferred chief minister”. Upstairs, the details are fed into spreadsheets that form a database of 75 million voters. Around 45,000 calls are made each day.
This is “big data”-gathering, Indian-style, and on an Indian scale.
This past Saturday, the first of Uttar Pradesh’s 140 million voters cast their ballots in India’s most-anticipated state elections.
In the 2014 national elections the state delivered a landslide win to Narendra Modi, the Indian prime minister. How his Bharatiya Janata Party (BJP) fares over the next month’s voting is viewed as a crucial test of whether Modi still has that magnetic appeal, with almost two years until he himself faces voters again.
Along with the upstart Aam Aadmi party that now controls Delhi, Modi was among the first to employ the data-driven campaigning techniques that helped propel Barack Obama to the White House twice, and which contributed to Donald Trump’s upset win in November.
But the frenetic dialling and number-crunching underway in the Lucknow suburbs is the most sophisticated data operation yet deployed in Indian politics, says its co-founder, Adwait Vikram Singh, the deputy campaign director for the ruling Samajwadi party (SP).
“Other parties wouldn’t believe we have this,” the 29-year-old says, flicking through an app his team has distributed to SP candidates, showing voter preferences down to the level of individual booths, and along caste, gender and literacy lines.
Unlike in the US or UK, where firms such as Cambridge Analytica could rely on Facebook quizzes to create detailed “psychographic” profiles of voters, data gathering in Uttar Pradesh – one of India’s poorest states, with a reputation for lawlessness – still requires a human touch.
“There’s no way around it. Facebook, the digital ecosystem, it’s not so deeply rooted. India is a place where you have to dirty your hands and feet,” Singh says.
One problem is that phone numbers are not always active for long in poorer areas of the state. An itinerant labourer or farmhand might be charged 20 rupees for call credit. Buying an entirely new sim card, pre-loaded with call credits, could cost five rupees less.
Singh and his colleagues had to send field teams into all 403 seats in the vast state. They returned with not just detailed lists of voters, but also relationships with local “influencers”, he says. Well-connected people such as village chiefs, postal workers or teachers can help report popular sentiment – or pass on new phone numbers when someone has changed their sim card again.
Making the task harder, too, is that India makes available far less digital information about its citizens than governments or corporations in the west.
“In the US and UK, because of their credit bureaus, because of the digitalisation that happened 20 or 30 years back, they’ve figured out laws around data usage, and a lot of that data is available for you to analyse,” says Milind Chitgupakar, the chief analytics officer at Modak Analytics, a Hyderabad-based data research firm.
In 2013 the firm embarked on a larger-scale – though less granular – task than Singh’s, combining census and election rolls to create a database of 810 million Indian voters. “It was a huge challenge, because of the way language has evolved in India, and created its own phonics,” he says. “We found out the word ‘Srinivas’ has 430 different variations.
“Or take the area Najafgarh,” he says. “It’s spelt differently in NSSO [National Sample Survey Office] data, central government data, in polling-level data. If a human reads it he’ll say, yeah, it’s all the same. But to train a computer to learn that these two names are the same … A lot of the analytics we did was trying to solve these hiccups.”
The final result threw up trivia – the most common woman’s name in Uttar Pradesh is Sunita; the oldest voter is, according to official records, aged 7,982 – but also paid dividends for one major party.
“The data showed that the highest number of registered voters in India were aged 27,” Chitgupakar says. But India’s population skews much younger. “So we knew there was this huge missing gap of people not registered to vote between 18 and 27.
“There was extensive polling done and we found that this age group, because they were very aspirational, were planning to vote for a certain political leader and his party, who I can’t name,” he says, citing contractual agreements – though the profile clearly fits Modi.
“As soon as we got that information, the party launched a massive voter registration drive to bring those young people in. From opinion polls we figured out that regardless of socioeconomic status, age, even religion, the overriding factor was that this leader had a 10% greater chance of being voted for by this young group.
“In Uttar Pradesh in 2014, in that one year, there were one crore [10 million] additional voters because of this campaign, and 70% of them were below the age of of 35. That was because of data analytics,” he says.
Mining huge datasets might be the newest phenomenon in Indian politics, but for the SP, it was nearly outdone by one of the oldest.
Just as data gathering was getting underway in September, a bitter dynastic fight between the chief minister, Akhilesh Yadav, and his uncle and father, also prominent party figures, spilled out into the open. Yadav was dumped as the party’s state president, briefly losing access to party money he hoped to use to build the database.
With the system now up and running, Singh says candidates can “micro-target” messages they know appeal to young, college-graduate, Muslim women, for example, in booths that skew towards those demographics.
More detailed information, to which access is more restricted, allows Singh and his colleagues to track almost every family in around 200 seats.
He declines to discuss specifics while the campaign is ongoing, but says in theory, he could advise a candidate to call a particular, influential member of a village whose support is wavering, according to their research. “That’s an integral part of my campaign,” Singh says.
Not all party stalwarts are enamoured with the Harvard graduate’s attempt to “disrupt” their methods. “They tell me, ‘We know what our people want’,” he says.
“But there’s a reason why no government has been re-elected in Uttar Pradesh since 1984.
“You don’t know what people want. If you don’t have great feedback, it’s not going to work.”
In a state notorious for its thuggish politics, with an unprecedented number of candidates accused of crimes, gathering such sensitive information might also present a security risk. Singh maintains confidential information is never passed on to party figures, though “on occasion” phone numbers are shared.
Generally, India is “quite perilously placed in terms of privacy protection”, says Apar Gupta, a technology lawyer.
“There’s an absence of protections in law, the constitutional right to privacy is under doubt, and even an expectation of privacy from state instrumentalities is absent,” he says.
Politics along caste and religious lines is still blatantly practised in Uttar Pradesh. As data analytics pushes Indian political campaigns into the 21st century, there is also the risk that it drags divisive caste politics with it.
But Singh says he believes otherwise: that the flood of new information about voters’ incomes or literacy rates will actually push political campaigns to go “beyond caste”. In a rapidly changing India, “caste is a diminishing return”, he says.