There is growing awareness among internet users that our surfing habits can be tracked and used to define what adverts we see online. But what if the same thing was possible with television?

It is. Technology firm Gracenote is holding trials of a technology called "full ad replacement", which helps broadcasters deliver different ads to households with internet-connected TVs based on their demographic data and viewing habits.

"There's an opportunity to go out and look to see if there's a more appropriate advertisement for you than the one being broadcast, using the information we have about you – most of which is gathered when you set up the TV and register it," Gracenote president Stephen White told The Guardian.

"If we tie that information in with what the ad-decisioning platforms know about you – which are very rich sets of content pulled from public databases, typically 100-200 different items – the technology can decide that there's a better ad for you than the one that's about to be shown and overlay it over the broadcast."

White suggested that some brands might want to show different ads to different viewers – Toyota might show a mini-van to a family and a sports car to a young single male – or decide not to show their ad to some demographic groups, leaving that commercial break open to being "skimmed" by other brands.

Gracenote pitches technology to broadcasters

Gracenote is pitching the technology to broadcasters as "an uplift to their current sell" due to the ability to zero in on specific groups of TV viewers: "Target the world, it'll cost you X. If you want to target a specific demographic, it'll be X plus Y," said White.

"Almost 100% of the US broadcasters are working with us on this. There's universal excitement and acceptance of the idea that they can target, and that broadcasters can control that targeting," he continued.

"It's about to go into live trials in the US, and we're planning to hold trials in Europe next year, in the late spring or early summer time. We're hoping to have this in the market with OEMs [TV manufacturers] next spring."

Profiling via video fingerprinting technology

Full ad replacement partly relies on Gracenote's video fingerprinting technology, developed after it bought a startup called Bulldog United in June 2012. It can be used to monitor the shows watched by individual households in order to flesh out their profile in Gracenote's database.

"We can understand what the user is doing and start to profile them: if cartoons are on at 10am, it's probably a young child. If it's soap operas in the afternoon, a stay-at-home mom. And if it's sports all day, it's more likely to be a twentysomething guy," said White.

"Then we can start to do interactive overlays of content on top of the broadcast stream, from polling for news or political shows through to extensions of advertisements, or putting up recipes for cooking shows. Anything a broadcaster wants to do with that space, including full ad replacement."

The obvious question is: surely a large chunk of those TV viewers will be freaked out by the idea of their TV watching them as much as they watch it, and that data being shared with advertisers even on an aggregated basis?

"A lot of people might not care, but some people might. We're being very mindful of what the consumer wants: it's got to be an opt-in model, and we have to give more value to the consumer," said White.

"For example, users have to be able to manage their profiles: 'Hey, stop profiling me when my wife goes to sleep!'. We've done extensive legal work around this."

Expect more debate over the coming months as the technology gets closer to launching. A survey published in May by research firm Analysys Mason suggested that less than half of smart TV buyers have actually connected their sets to the internet.

That means tracking technology from companies like Gracenote won't work, of course, but it also suggests that if these people do eventually connect up those TVs, they may be surprised at the devices' ability to track their viewing and serve personalised ads.

Gracenote's pitch to the TV and consumer electronics industries portrays full ad replacement as a win-win-win: brands can better target their TV ads, broadcasters can charge more for this targeting, and TV manufacturers can share in the revenues. But explaining this to viewers may be a tougher, longer conversation.

In the meantime, Gracenote is continuing to work on other forms of TV technology, such as second-screen apps designed to interact with shows. The company recently worked with US network NBC on a show called Million Second Quiz, where the audience at home could play along with the contestants on-screen.

"It was treating people at home just like participants in the show, and the highest scoring folks are flown in and put on the show the next day," said White. "It's one of the first big primetime second-screen pushes in the US, and we had over 1m people using the app even before the show aired."

Gracenote is competing increasingly fiercely with rival Shazam to convince broadcasters to work with it on such second-screen apps, with both companies touting their audio fingerprinting technologies.

Gracenote has previously used watermarking technology embedded in broadcasts to help apps synchronise with TV shows, but White said the momentum has moved towards fingerprinting technology over the past year or two.

"Fingerprinting has won over watermarking at this point, which is good for us because we're one of the only companies that does it on a global scale, and on a business-to-business basis," said White.

"Shazam has taken the branded approach where they put their logo on top of your logo, and force people to use their app. We're pushing this stuff out embedded into the apps of the broadcasters and shows."

Meanwhile, Gracenote is also hoping to persuade more developers to make innovative apps and services based on television, and recently held a hack day for developers to explore new technologies to interact with television, inspired by similar events in the digital music world.

"We had voice control, motion control, our stuff... It was a fantastic hack event, with really cool stuff coming out of it," said White.

"We're trying to foster the developer innovation in the TV space, just as you've seen on the music side. We've been very aggressive in opening up our APIs, and we're starting to see a few folks follow on."