Is it really goodbye shop? This has been another terrible month for retailing. Monday’s figures show March footfall down 6% on last year, and almost 9% down in high streets. Clothing is in freefall, down 20%. My two local high streets – one rich, one poor – both look as if they’ve been hit by the plague, with naked windows pockmarked by “For Sale” signs.

Toys R Us, BHS and Maplin have gone. We hear of closures from Mothercare, Homebase, House of Fraser, even M&S. Local data shows a net 1,700 shops shut last year. Paula Nickolds of mighty John Lewis said last week that this is her “toughest time in 25 years in the industry”, adding of her own business, “we need to reinvent”.

Even allowing for variables such as the suppressant effect of snow and storms – the Beast from the East – this March, the underlying reason is no secret. Those online raiders of the lost high street, the little white vans. Peak shopping time for the average family is now 8pm on a Sunday, when they are comfortably at home. Robots will soon retrieve orders from warehouses and drones and driverless vans will bring them to our doors. Thousands of retail jobs are at risk.

If high-street footfall is shrinking by a 10th every year, they will soon simply vanish. But does it matter? These are time-honoured features of towns, but unlike deserted churches and other “community assets”, no one lists them for preservation. Gone will be their diversity of shops, cafes, pavement life and humanity. A “town” will become a zombie settlement of commercial and residential estates. The public realm will be private.

High streets have been under assault since the 1980s, when governments capitulated to the supermarket lobby and its craving to get retail out of town. Hypermarkets were the totems of the motoring age. They sprawled across the countryside and hollowed-out town centres. No one thought long-term, or else they didn’t care.

I watched King’s Lynn empty its shops on to its bypass. Chichester did likewise. Penrith in Cumbria was doughnutted by “edge of town”. Dartford in Kent imploded because of adjacent Bluewater. Bradford gave in to supermarkets so completely that its central shopping streets are largely empty.

As always, markets only speak short-term. Now out of town is feeling the pinch. In 2011 a Guardian planning survey showed a 20% surge in supermarket planning applications, 80% of them out of town, encouraged by David Cameron’s let-rip planning policy. Sites were bought, fields bulldozed, concrete laid. Tesco proposed three new stores a week. Four years later, the bubble burst. Overnight Tesco abandoned 49 projects and closed 43 stores. The damage was done. As city centres were stripped, out-of-town sites lay derelict. It was a classic planning fiasco.

The market may be the engine of capitalism, but regulation is its steering wheel. French towns protected their high streets with new infrastructure and control over uses. Tobacconists, butchers and chemists had to remain just that. In Britain the market was distorted the other way. Out-of-town was blessed by planning easements, free parking, new roads and low taxes. High streets were cursed with no parking and business rates.

Even so, much of this is fake gloom. The internet may be killing the high street. But we were once told that television would kill cinemas; cinemas would kill theatres; records would kill live music; cars would kill trains. As for books, Amazon would wipe out bookshops, and Kindle wipe out books. None of this has happened. Novelty is the curse of futurology.

The key is to look not at what is dying in the high street, but what is living. People may no longer want video rentals, travel agents, shoe shops or banks. But what has moved into the two thirds of BHS stores that have been relet? The answer is gyms, golf ranges, art galleries, market stalls and places to eat and drink. High-street shops that are now doing well are hair and beauty salons, coffee bars, fast-food outlets, health and convenience stores. Farmers’ markets are booming. As rents fall, charity shops selling “pre-loved” goods move in – and a good thing too. John Lewis offers eye tests and Pizza Express offers live music.

All of this is called the “experience economy”. The reality that retailers such as Nickolds must consider as the “reinvent” is that shopping was always a reason for congregating, somewhere to bump into old friends and new ones. It was always Scarborough Fair. Rather than wander around replica airport terminals, people are bound to buy most of their goods online. But high streets are switching to “people” services that cannot be supplied by drone or Deliveroo.

The tell-tale is bookselling. A decade ago bookshops were closing by the week. Then Waterstones was taken over and its new boss, James Daunt, fought against digital hysteria. “We were loss-making and dead in a ditch,” he said. He decided to recreate traditional bookshops in which people could dawdle, as in a library. Book sales are up, e-books have gone and Waterstones has been two years in profit. Meanwhile the great shop-slayer, Amazon, is building – you guessed it – a chain of bookshops across America.

The truth is that a screen is not a life. We do not want to sit at home all day punching glass. There are no “experiences” to be had in a bypass car park. But the key must lie in smart regulation. New high streets are emerging where developers and planners concern themselves with character, as in London’s Marylebone High Street, Manchester’s Northern Quarter or Birmingham’s Gas Street Basin.

As high streets decline in value, developers cannot be allowed to raise rents, board up shops and demand change of use. After the war, central York was written off as an urban disaster, its buildings old and outdated. No one realised that this was their salvation. To locals and tourists alike, they were attractive and friendly, and their buildings were adaptable. Petergate is thriving.

Seven years ago the retail expert Mary Portas pleaded that high streets should get wise to new markets, but said that planners had to help rather than hinder them. All they got for her pains was business rates and speculators hoping for change of use. Government has never taken high streets to its heart. The present planning minister, Sajid Javid, has a craving for urban towers and rural sprawl. He cannot complain if towns depopulate, crime rises and car-miles soar.

High streets will never sprout on bypasses. They are not just strips of property. They are social institutions, the heart and soul of urban Britain. They must be helped to stay that way.

• Simon Jenkins is a Guardian columnist