As technology generates new and more complex ways to collect, store and interpret personal data, politics – lacking a clear road map – is being forced to play catch up.

When a big data story breaks, like the Guardian’s revelation last week that Medicare records are for sale on the dark web, politicians scramble to default crisis management positions: the opposition voicing grave concerns, the government responding with an independent inquiry. But if Labor was in power and the Coalition was in opposition, you get the feeling the story would be playing out exactly the same way.

That’s because neither side of politics has yet established a principled approach to how to manage this burgeoning national resource.

From Medicare to our tax returns, our kids’ Naplan results and the big national data dump that is the census, records of our transactions with the government have immense value in designing better services, tempered by the risk of personal information being misused.

Government access to our data has been contentious for more than 30 years, ever since Gareth Evans suggested a national identification card to bring together tax, health and welfare details in a bid to minimise fraud.

Then, the backlash against the Australia Card was brutal, with many alarmed at the idea of trusting the state with providing a single touch-point of their personal information. Despite winning a double-dissolution election triggered by the issue, Labor abandoned the idea.

Similar strains of suspicion surfaced around last year’s census, with scepticism around the online collection of information seen as one of the factors contributing to the relatively low take-up.

There has also been a running controversy over the ability of national security agencies to access depersonalised phone and email records in order to oversee terrorism threats. Again the backlash has been vocal, with journalists in particular taking umbrage at the suggestion that their sources could be exposed.

But as this week’s Essential Report shows what is most striking about these issues of personal data is that the public is relatively comfortable with the idea that authorities can get their hands on personal records.

Trust in government to handle data is less clear cut, with issues like the Medicare hack meaning that strong, effective and clearly articulated protections will be required for the public to support more of their information being collected and curated.

But what’s most striking about these findings is that the real concerns about the use of data comes when private companies are the ones holding the information. And the real story behind those numbers is that the information the big private tech giants hold is already way beyond most people’s expectations.

Government is being asked to manage the use of this data on our behalf, to balance the benefits and risks of public and private entities holding large swathes of data, but what are the principles that should guide this trade-off?

At face value, the collective use of publicly generated data feels like an exercise in socialisation: individual bits of information are combined to create more valuable bytes, making the sum greater than the whole. Click send, comrades.

Suspicion about the collection and storage of such data seems to fit into classic Cold War anti-government dogma, where state collection of information was an industry closely tied to surveillance and repression.

But that libertarian streak has also fuelled the activities of Julian Assange and his ilk, disruptors whose stated mission is to undermine state control by exposing their secrets, even at the cost of undermining democratic elections.

And just to make the politics of data even more fraught, there are rising technology behemoths for whom the generation and exploitation of personal data is the foundation for global empires that have tapped cultural veins – while disrupting and destroying traditional industries. Are these entrepreneurs to be celebrated or challenged?

Facebook, for example, collects everything from the standard data that you input (age, sex, location, etc) to data that they extrapolate from your user profile (income, household type, sexuality, political preferences, etc). It even collects data from its new messenger application.

Meanwhile Google collects all your searches and search engine clicks, email data, map searches and location, and your total web history if the websites you visit use Google analytics (which is almost 90% of websites).

Both tech giants monetise this data by selling your data profile to companies who want to reach you, so-called micro-targeting that sells the option of an audience of one to advertisers.

Then there are the “data marketing companies” that would sell access to the search and web browsing history of a specified group of people. For example you could ask these companies for cardiac surgeons in Sydney and get back a profile of the web data of that group for the last six months. You can tell everything from where they lived, how much they earned, who they banked with and what porn they’ve watched.

While the prime minister has vociferously embraced innovation and the opposition has started calling out these companies for their tax practices, their positions on the impact of these business models is still a work in progress.

What we don’t know is where either party stands on the role of government in maximising the benefits and minimising the risks of this growing industry in public and privately held data.

Is the government our protector or the facilitator of more trade in personal data? What are the lines that holders of data shouldn’t cross? Who should police those lines? And what of the benefits from sharing information – should they generate profit or go back to the public as a dividend for our collective provision of a digital footprint?

As the technology train hurtles along, it’s incumbent on our political parties to adapt their own philosophies, platforms and policies for the way our personal data fuels the future.