Alan Gould was hitting a wall. It was the late 1990s, and the political advertising operative had an idea about using a relatively newfangled tool – banner ads on web sites – to promote political candidates. “It was pretty clear to me at the time that the ability to target and tailor messaging was perfect for political campaigns,” Gould recalled recently. “I did a whole presentation on the internet and the power to connect, track, do fundraising, target.”

But when Gould finished his pitches, he would be met with blank stares. “I was a very lonely pied piper,” he says.

Finally, in 1998, Gould found a political candidate who was so far behind in the polls, and so strapped for cash, that he was willing to take a risk and spend $100,000 on banner ads on the New York Times homepage. Peter Vallone, then a New York City council member challenging George Pataki for the governorship, gave Gould the green light for an ad buy that has since entered the history books as the first significant use of online advertising in a political campaign.

The ads themselves are lost to internet history – Gould believes he may have copies somewhere on floppy discs. But it’s not hard to draw a line from that moment to Robert Mueller’s 16 February indictment of the Internet Research Agency, which alleges that Russian agents carried out a conspiracy to interfere with a US presidential election, in large part by purchasing targeted Facebook ads designed to “encourage US minority groups not to vote”. Or to the news recently revealed in the Observer that 50m Facebook profiles were obtained and misused by data mining company Cambridge Analytica to target voters during the 2016 presidential election.

The Vallone ads contained rudimentary versions of many of the attributes that make digital advertising such a powerful – and terrifying – force today: the ability to target specific audiences with tailored messages, then track their reaction.

“Come November 2000, I expect the question will no longer be whether web-based political advertising works,” wrote Cyrus Krohn, then the manager of political advertising for the Microsoft Network, in a prescient 1999 column for Slate, “but whether it works too well.”

Nearly 20 years later, the world has caught up to Krohn’s concerns, with some critics making the not entirely hyperbolic argument that micro-targeted “dark advertising” on Facebook is a fundamental threat to democracy itself. Is it too late for democracy to fix itself?

Game-changing

In February, Donald Trump named Brad Parscale as his 2020 re-election campaign manager. The decision lends credence to what Parscale has been saying for the past year: that his Facebook advertising operation won Trump the election.

Parscale had been a little-known digital marketing executive working out of Texas when he was tapped to build Trump’s campaign website in 2015. Until then, digital advertising was barely a rounding error in campaign budgets. In 2008, the year Barack Obama became the first social media candidate, candidates spent just $22.25m on online political ads, according to an analysis by Borrell Associates. That number grew significantly in 2012, but the real explosion came in 2016, when campaigns pumped $1.4bn into digital ads.

US presidential campaigns are often remembered – and understood – by their advertisements. Lyndon B Johnson’s “Daisy” ad powerfully (and controversially) set the stakes of an election in a nuclear world. George HW Bush’s “Willie Horton” attack ad still epitomizes the racist dog-whistle politics of the tough-on-crime era. The message, as much as the messenger, is a key part of the debate over who is best equipped to lead the country.

But no such public debate took place around Trump’s apparently game-changing digital political advertisements before election day.

This is partly due to a loophole in the prevailing campaign finance law, which was written in 2002 and did not include internet ads in the class of regulated “electioneering communications”. But perhaps even more important is the very nature of online advertising, which is self-serve (just sign up with a credit card and go) and highly iterative.

Parscale claims he typically ran 50,000 to 60,000 variations of Facebook ads each day during the Trump campaign, all targeting different segments of the electorate. Understanding the meaning of a single one of those ads would require knowing what the ad actually said, who the campaign targeted to see that ad, and how that audience responded. Multiply that by 100 and you have a headache; by 50,000 and you’ll start to doubt your grasp on reality. Then remember that this is 50,000 a day over the course of a campaign that lasted more than a year.

“The reason I said it might work too well,” Krohn said in a recent interview with the Guardian, “is that mass marketing went away and micro-targeting – nano-targeting – came to fruition.”

Any candidate using Facebook can put a campaign message promising one thing in front of one group of voters while simultaneously running an ad with a completely opposite message in front of a different group of voters. The ads themselves are not posted anywhere for the general public to see (this is what’s known as “dark advertising”), and chances are, no one will ever be the wiser.

That undermines the very idea of a “marketplace of ideas”, says Ann Ravel, a former member of the Federal Election Commission who has long advocated stricter regulations on digital campaigning. “The way to have a robust democracy is for people to hear all these ideas and make decisions and discuss,” Ravel said. “With microtargeting, that is not happening.”

Parscale and his staff told reporters with Bloomberg that they used Facebook ads to target Hillary Clinton supporters with messages designed to make them sit the election out, including her own forays into dog-whistle politics from the 1990s, which the Trump campaign hoped would discourage black voters from turning out to the polls.

That degree of political manipulation might be unsavory, but it’s also relatively old-fashioned. One digital campaign staffer (not affiliated with the Trump campaign) compared it to Richard Nixon’s Southern Strategy, only “technologically savvy”.

But new reporting by the Observer has revealed that the data analytics team that worked for Trump, Cambridge Analytica, went far beyond Nixonian dirty tricks. The firm obtained Facebook data harvested under the auspices of an academic study, the Observer has revealed, and then used that data to target millions of US voters based on their psychological weaknesses.

“We exploited Facebook to harvest millions of people’s profiles,” whistleblower Christopher Wylie told the Observer about the data theft, “and built models to exploit what we knew about them and target their inner demons.”

European elections

Political advertising in the US is the wild west compared with other western democracies, which tend to have shorter election campaigns with strict regulations on the amount and type of spending permitted. Such rules may enhance the impact of digital advertising, which is much cheaper than television and largely unregulated.

The UK has seen a rapid shift to digital campaigning following the Conservative party’s embrace of Facebook in the 2015 general election. The Tories outspent Labour by a factor of 10 on Facebook advertisements, a decision that many political observers saw as decisive. In a country that bans political ads on television, Facebook enabled the Conservatives to reach 80.65% of users in targeted constituencies with its promoted posts and video ads, according to marketing materials created by Facebook. (At some point in the past year, the company began hiding previously produced “Success Stories” about its ability to sway election results.)

The Vote Leave campaign in the 2016 Brexit referendum went on to spend almost its entire budget on Facebook advertising, an investment that resulted in about 1bn targeted digital ads being served to voters over the course of a 10-week campaign.

Though it is impossible to parse the exact impact of Facebook advertisements amid all the other factors that shape an electoral result (including organic Facebook content), the platform is increasingly cited as a factor in the growing electoral might of far-right groups in Europe.

The radical right-wing Alternative for Germany (AfD) party reportedly worked with a US campaign consultancy and Facebook itself to target German voters susceptible to its anti-immigrant message during the 2017 election in which AfD surged in popularity to become the third-largest party in parliament.

Campaigning in Italy’s recent election, which saw the rise of anti-establishment parties, including the populist Five Star Movement and the far-right League, largely took place on social media. Facebook advertisements and targeting information gathered by Italian transparency group Openpolis found that the neo-fascist Brothers of Italy party ran a Facebook ad targeting Italian adults who are interested in the paramilitary police force, the carabinieri.

After the polls closed in Italy, the League’s Matteo Salvini shared some words of gratitude with the press: “Thank God for the internet. Thank God for social media. Thank God for Facebook.”

Targeting the midterms

While investigations into the 2016 US election and Brexit referendum continue, it’s worth remembering that more elections are fast approaching. Scores of countries will hold national elections in 2018, including Sweden, Ireland, Egypt, Mexico and Brazil.

In the US, candidates for the 435 congressional and 35 Senate seats that are up for grabs in November are already running campaigns on Facebook, and we may never know what they’re saying in those advertisements.

Take, for example, Paul Nehlen, a candidate who is running a Republican primary challenge against the speaker of the House, Paul Ryan, in Wisconsin. Nehlen is primarily known as a vehement antisemite who was once embraced by Steve Bannon and the Breitbart wing of the right, but was excommunicated after appearing on a white supremacist podcast.

According to his FEC filings, Nehlen spent at least $2,791.72 on Facebook ads in the final six months of 2017.

What did that money buy?

In the first instance, everything that any Facebook advertiser can get: access to one of the most powerful databases of personal information that has ever existed, with insights into individuals’ intimate relationships, political beliefs, consumer habits and internet browsing.

Beyond that, we don’t know. Nehlen could be using Facebook to target likely voters in his district with a message about infrastructure. Or he could have taken a list of his own core supporters (he has more than 40,000 likes on Facebook), used Facebook’s “lookalike audience” tool to find other people inclined to support his particular politics, then fed them ads designed to persuade more people to join him in hating Jews.

Last fall, after Facebook had been forced to admit that, despite its initial denials, its platform had been used by foreign agents seeking to illegally influence the election, the company announced a set of reforms designed to assuage its critics – and stave off actual, enforceable regulation.

Starting this summer, the platform has promised that every political ad will be linked back to the page that paid for it. The pages themselves will display every ad that they’re running, as well as demographic information about the audience that they are reaching, a measure that Mark Zuckerberg claimed would “bring Facebook to an even higher standard of transparency” than the law requires for television or other media.

A version of these reforms is already live in Canada, where users can see all the ads being run by a political candidate in a designated tab on their page.

But there is good reason to be skeptical.

Since 2014, Facebook has had a transparency tool for all ads served on the platform. Click on the upper right-hand corner of a Facebook ad and you’ll see an option reading “Why am I seeing this ad?” Click through and you’ll get an explanation of the characteristics that make you desirable to the advertiser.

So far so good, but a new study by computer scientists found that Facebook’s ad explanations were “often incomplete and sometimes misleading” in a way that “may allow malicious advertisers to easily obfuscate ad explanations that are discriminatory or that target privacy-sensitive attributes”.

Alan Mislove, a professor of computer science at Northeastern University and one of the study’s co-authors, said that he gave Facebook credit for having the feature at all, noting that it is one of the only examples of a company offering any kind of explanation of how an algorithm actually works. But the findings do not paint a particularly pretty picture of Facebook’s ability to self-regulate.

“They’ve built this incredibly powerful platform that allows very narrow targeting, a very powerful tool that anyone on the internet can use, so that scares me,” Mislove said. “And up until very recently, there was very little accountability. You as a malicious actor on Facebook don’t even really need to obfuscate your behavior, because the only person watching is Facebook.”

Honest Ads Act

The best hope for bringing some order to the realm of digital political ads is through updating US law for the Facebook era.

A bipartisan proposal to do just that exists. In October, Senators Amy Klobuchar, Mark Warner and John McCain introduced the Honest Ads Act, which would close the loophole that allows internet ads to avoid regulation, and also require internet platforms (ie Facebook and Google) to maintain a public file of all the political ads they run and who paid for them.

But as much as we need transparency around political ads to maintain democracy, we also need a functioning democracy to get that transparency. And it’s not clear that it’s not already too late.

“In an ideal world, with a fully functioning Congress, there would be hearings around the Honest Ads Act, and you would have Facebook and Google and Twitter and experts testify to shine a light on the nature of political advertising,” said Brendan Fischer, director of FEC reform at the Campaign Legal Center. “We’re not close to that at all.”

In the absence of a fully functioning Congress, what is to be done? Should we expect Facebook to simply stop selling political ads?

Antonio Garcia Martinez, a former product manager for Facebook who helped develop its advertising tools says that he has come to realize that political ads are simply a different beast than commercial ones, which can and should be treated differently by his former employer.

“Selling shoes needs to be different than selling politicians, even though the mechanics of it are identical,” he said. “Morally it’s different.”

Or should we pressure Facebook to stop allowing candidates with hateful or extremist views to use its tools?

“If we farm these important democratic responsibilities out to a private company, today they might be regulating antisemitism, but tomorrow they’re regulating what people can say about the Honest Ads Act,” Fischer said.

Indeed, Facebook could already be suppressing political views unfavorable to its business practices, and we would have no way of knowing. It’s possible Facebook wouldn’t even know. In response to queries about inconsistent moderation of clothing advertisements, a Facebook spokeswoman recently told the New York Times that “the company could not ask an automated system about [its] decisions”.

Frankenstein’s monster is not under any human’s control.

If this all seems positively dystopian, one person who is surprisingly sanguine is Alan Gould.

Gould left politics soon after the Vallone campaign, founded an advertising analytics firm, sold it, and is now a tech investor. He does have concerns about media literacy and Facebook’s tendency to trap people in filter bubbles, but says: “If people choose to stay in that bubble and not explore anything outside of it, that’s a statement about who they are and not about the technology.

“If you’re going to have a representative democracy, then you have to have a way to communicate with the voters and you’re going to use whatever is available, whether that’s newspapers or mail or email or Snapchat,” he said. “I don’t regret it at all.”