It was the most high-profile moment of New Labour’s “Cool Britannia” campaign: a celebration of a modern, outward-facing Britain with a new kind of industry, and a new kind of workforce.

“We saw it as a chance to redefine what the UK’s economic future would be about,” recalls John Newbigin, then a government special advisor on culture, of Tony Blair’s garish Downing Street reception for the great and good of the UK’s creative industries, held 20 years ago this month. “Not just factories or pinstriped bankers, but creative entrepreneurs drawn from across society.”

Ever since, the creative industries have been a favourite of the British government and media. Successive prime ministers, including Theresa May, have hosted receptions for the entertainment and fashion industries, and lauded the UK as a “cultural powerhouse” that generates jobs and wealth for the country.

“One of the UK’s little-known export successes in recent years has been creative industries policy,” says Newbigin. “Countries hire British consultants and academics to develop their own strategies for the sector.”

According to Unesco, this sector now accounts for more than $2tn worldwide, or 3% of the world’s total economy. But who, precisely, is benefiting from this growth of Britain’s creative industries? For behind the glamorous events and economic clout, there remain profound issues of inequality, exploitation and lack of opportunity.

The 2015 Oscars were notorious for the lack of ethnic diversity among nominees, and the controversy was echoed at this year’s Baftas, which revealed how top British actors were overwhelmingly drawn from wealthier homes and fee-paying schools. In the music industry, there was a high-profile spat in 2015 between Labour MP Chris Bryant and singer James Blunt, after Bryant complained the industry had become dominated by those from privileged backgrounds. Blunt, who attended Harrow School, responded by calling Bryant a “classist gimp^”.

Such flurries of indignation tend to focus exclusively on those in the public eye. People may notice that their favourite actors, comedians and singers live in Primrose Hill and have upper-class accents, without appreciating the industry and networks they operate in. Whether it is television commissioners, newspaper editors, art curators or casting agents, the people who actually produce our national culture and sell it around the world are drawn from an astonishingly small demographic pool.

As Newbigin acknowledges, “We have to accept that, despite 20 years of government attention, Britain’s creative industries still fail to reflect the country as a whole.”

A pathway for the privileged

The chronic lack of diversity, equality and social mobility throughout the creative economy is borne out by the latest research. A forthcoming academic article in the journal American Behavioral Scientist analyses official employment figures to present a bleak picture of diversity in the UK’s sector.

In particular, the data shows how much the middle classes now dominate the workforce. In publishing, for example, 63% of employees have parents from Social Economic Classification groups 1 and 2 (the two highest of eight social-economic categories, comprising those with higher managerial and professional occupations). By contrast, just 13% of those in publishing are drawn from families in SEC groups 6-8 (unskilled or unemployed).

In film and television, the figure is 54% for those with parents in SEC Groups 1-2; in music and the performing arts, it is 49%. By way of comparison, the average for the British workforce overall is 29% from SEC groups 1-2, and 35% from groups 6-8.

The figures for ethnicity and gender are similarly dismaying. Women make up 52% of the UK workforce, but no more than a third of employees in the creative industries, while ethnic minority employment in industries such as film, television and music is less than half the national average.

More surveys undertaken by creative industry bodies and sector skills councils indicate that diversity levels are significantly lower for the more senior jobs. Only 14% of UK film directors are female, while the proportion of black and minority ethnic executives in film production, statistically speaking, stood at zero in 2015 – which is why actor-turned-executive producer Idris Elba has been praised for giving young black actors a chance to work both in front of and behind the camera of the TV series Guerilla.

“The creative sector likes to portray itself as tolerant, meritocratic and diverse,” says Prof Kate Oakley, one of the report’s authors. “But this image is starkly at odds with the facts. The types of people who make so much of the culture in this country bear little resemblance to the people who consume it.”

While many of the factors stifling access to the creative industries reflect general trends – the cost and barriers to studying at university, rising housing costs and low levels of union representation – Dave O’Brien, Oakley’s co-author, argues there are also structural inequalities that are distinct to the creative sector.

The preponderance of project-based activities, with teams coming together to work on a film, advert or record, means the workforce has high levels of freelance and self-employed workers: stylists, directors, photographers, editors, session musicians and many more. While some command substantial fees and day-rates, for the majority it is a case of periodic and badly paid work – a third of all actors, for instance, work fewer than 10 weeks a year.

“Without independent wealth and personal contacts, it is difficult for many of those in the creative industries to get by,” O’Brien says. “For those without these resources, the experience of creative employment can be exploitation and anxiety, rather than freedom and artistic creativity.”

Related to this is the widespread use of unpaid interns – an uncommon practice 20 years ago, but something that has proliferated across the creative industries. For the television runner or fashion assistant, entry into these industries is not via a graduate traineeship, apprenticeship or even low-level job, but through spells of temporary – and usually unpaid – work experience.

As Lucian Evans, owner of production company Leap Films, describes: “I graduated in the 90s with a degree in film and television, then spent a year doing the Soho rite of passage: being a runner, temporary gigs, unsociable hours helping out on shoots. You could barely live on the money, but at least I could pay my rent and it got me started in the industry. It’s a lot worse now – people are expected to do repeated runner jobs and work experience placements which can go on endlessly, with no expectation of payment or progress.”

According to a recent report by the IPPR on internships in Britain, the creative industries have the highest ratio of internships to vacancies – representing 4% of all job vacancies but 11% of all internships. Within this criteria, publishing and media are particularly striking, but even libraries and museums now have a higher proportion of internships than job vacancies.

The lack of diversity in the museum workforce is also highlighted in Oakley and O’Brien’s research findings, which suggest that only 3% of employees in museums, libraries and galleries are black and minority ethnic. It seems the years of austerity have meant that commercial practices are becoming increasingly common in the publicly funded arts sector, and that exploitation of graduates is no longer the preserve of the television and fashion industries.

“It is vitally important that museums, faced with reduced funding, do not adopt working patterns based on unpaid labour,” O’Brien says. “While there may be well-qualified curators and conservationists prepared and able to work for nothing to get a chance at a paid job, this will only exacerbate the unrepresentative nature of the sector.”

Alienating audiences

Omar Kholeif, senior curator at Chicago’s Museum of Contemporary Art, sees a direct link between the makeup of those who run, and those who visit, museums and galleries.

“My own background fuels my programming,” he says. “The majority of my work has focused on the representation of artists from the Middle East and south Asia. When you only have people who are schooled in the same classical ‘art historian’ way – from the Renaissance to the contemporary, and only focusing on British or American art – you miss out.”

Kholeif, who studied in the UK and worked in galleries in Glasgow, Manchester and London before moving to Chicago, points to a recent exhibition of work by the artist Kerry James Marshall
that drew substantial African American audiences. By contrast, “When the content at the National Portrait Gallery or Tate Britain doesn’t reflect the diversity of the audience, they won’t go.”

Kholeif acknowledges that he was only able to develop his career with financial assistance from Arts Council England’s Inspire programme, and that such support is more vital than ever to maintain audience diversity. “With more cuts, you will see more exhibitions by famous white artists, who have big commercial galleries and a collector base willing to support them.”

It is this relationship, between the people who work in the creative industries and those who consume it, which makes the issue so crucial. If an independent income and good contacts are required for a career in curatorship, publishing, fashion or television, then how can the culture these sectors produce reflect the lives of ordinary people?

This point was forcibly made by the actor Riz Ahmed in a recent lecture in Parliament. Condemning the lack of ethnic diversity in film and television, Ahmed warned that those from minority backgrounds are likely to “switch off” from a popular culture that fails to represent them or provide a sense that their lives and stories are valued. He warned that “if we fail to represent, we are in danger of losing people to extremism”, as people from different communities seek fringe narratives and subcultures.

Creative solutions?

For a sector dependent on a supply of creative talent, problems relating to access and diversity threaten not just its reputation but long-term competitiveness. There are signs, however, that the issue is being recognised.

Businesses who nonchalantly advertise unpaid internships have felt the ire of social media. When design agencies and fashion houses seek out the latest graduate talent, many tutors will no longer promote work opportunities to their students unless they are fully paid – something that leading arts universities such as Goldsmiths have now adopted as a policy.

But it is also incumbent on audiences and consumers to drive change. In the US, the growth of African American audiences demanding content and characters, pioneered through such sitcoms as The Jeffersons and The Cosby Show, has led to profound shifts in popular culture. More recently, the #OscarsSoWhite campaign showed the power of audiences to lobby the film industry, while negative responses to The Great Wall and Marvel’s Iron Fist television series point towards a public willing to reject “whitewashed” productions.

Challenging industry structures and working practices largely invisible to the public is a daunting prospect and requires dedicated campaigning. In this, trade unions can still be a potent force, as shown by the industrial action at the National Gallery which, after 100 days of strikes in 2015, resulted in an agreement that protected staff conditions and secured them the London living wage.

A similar dispute between Bectu and the Picturehouse cinema chain is ongoing, with the union attracting support from high-profile figures in the film industry. The acting union Equity’s Professionally Made, Professionally Paid campaign is fighting against low pay and poor working conditions for actors, recognising that while legislation such as the national minimum wage might be in place, strong pressure is needed to enforce it and to stamp out employer malpractice.

The organisation Arts Emergency has launched an “alternative old boys network” to promote access and progress within the sector for those from all backgrounds, Arts Council England’s Creative Case for Diversity initiative is intended to embed diversity in arts organisations, and the Barbican arts centre has announced a major programme to better understand “who makes and consumes art”. Entitled Panic!, the initiative will employ sociologists to investigate artistic and audience inequalities in the creative sector, matching industry research with audience data and interviews. Accompanying the research programme will be public events, careers advice for young people and newly commissioned artworks.

In February, the culture minister, Matt Hancock, outlined his vision of the creative sector as an “incredible force for openness and social mobility”. Anyone who is a television runner, unemployed actor or fashion intern might find this hard to believe, as indeed would anyone sitting in the audience at the Royal Opera House.

“If I was doing another event in Downing Street now,” Newbigin reflects, “it wouldn’t be a party. It would be to think about making our creative workforce more inclusive, using all the talent that goes to waste in this country. I’d want to work out how to be even smarter, in a world that’s beginning to beat us at what we thought was our game.”

Tom Campbell was a cultural adviser at the Greater London Authority, where he worked on policies around employment and skills in London’s creative industries. His most recent novel, The Planner, was published in 2015.

Follow the Guardian’s Inequality Project on Twitter here, or email us at inequality.project@theguardian.com