Lynn Eaton 

Is it right to use AI to identify children at risk of harm?

Machine learning is being used to help protect children, but it raises ethical questions
  
  

Teenager Talking To Counsellor
Tech companies have been working with councils to sift through data sets to identify families in need of support. Photograph: Getty

Technology has advanced enormously in the 30 years since the introduction of the first Children Act, which shaped the UK’s system of child safeguarding. Today a computer-generated analysis – “machine learning” that produces predictive analytics – can help social workers assess the probability of a child coming on to the at-risk register. It can also help show how they might prevent that happening.

But with technological advances come dilemmas unimaginable back in 1989. Is it right for social workers to use computers to help promote the welfare of children in need? If it is right, what data should they draw on to do that?

Maris Stratulis, national director of the British Association of Social Workers England, first voiced concerns last year. She remains worried. “Machine learning in social care still raises significant issues about how we want to engage with children and families,” she says. “Reports on its use in other countries have shown mixed results including potential unethical profiling of groups of people.”

Stratulis is also concerned at the role of profit-making companies in the new techniques. “Rather than focusing on learning from machines and algorithms, let’s focus on good, relationship-based social work practice,” she says.

Machine learning is an application of artificial intelligence (AI). Computer systems enable councils to number-crunch vast amounts of data from a variety of sources, such as police records, housing benefit files, social services, education or – where it is made available – the NHS. In children’s services, a council may ask for analysis of specific risk factors which social workers would otherwise not know, such as a family getting behind on the rent, which can then be triangulated with other data such as school attendance.

“We don’t decide what databases to trawl – the client does,” says Wajid Shafiq, chief executive officer at Xantura, a company he set up 11 years ago which has recently been working with Thurrock council and Barking and Dagenham council in east London. “And the public sector is very aware of the ethical issues.”

Most councils trialling predictive analysis are using commercial organisations to set up and run the analyses. Only one, Essex, is known to be using its own purpose-built database collection. Thurrock is working with Xantura in using data analytics to help, in the words of a council spokesperson, “better identify those most in need of help and support, and to reduce the need for statutory interventions”.

Such is the sensitivity of the issue, however, that all councils dipping their toes into the machine-learning water are at pains to stress the caution they are adopting. “It is important to emphasise that data analytics systems are only part of the process,” says the Thurrock spokesperson. “Further verification and checks are carried out in line with statutory requirements prior to any intervention.”

Shafiq says the Thurrock system should be live in the next few months. He, too, emphasises that the analyses don’t tell the social workers what to do, but are merely another piece of the toolkit to aid decision-making.

Independent social worker Steve Liddicott has advised Xantura on various projects, including Thurrock. He explains that the analysis can help social workers see where a child not already on their radar might be at risk. They can then take steps – perhaps alerting the child’s school or health visitor – to prevent matters escalating.

Where a child is already known to them, the social worker can use data analysis to consider the intervention most likely to be effective for that child, based on the child’s previous behaviour. “There’s a temptation to say: ‘Doesn’t the social worker know that anyway?’” says Liddicott. “They may do, but it’s also helpful to step back and look objectively at what the picture is for this child.”

One of Essex’s projects, using in-house predictive analysis, has looked at school readiness. This has focused on Vange, a part of Basildon where one in four children has been judged by the Ofsted inspectorate to be not school-ready. The county council’s Essex Centre for Data Analytics, launched this summer after a three-year trial, analysed data for Vange based on the council’s social care reports, rates of youth offending and patterns of drug and alcohol misuse. It combined this with housing and benefits data from Basildon council and crime data from Essex police.

The results were presented to the local community and a community-led commissioning group was set up, aiming to give parents, volunteers and teachers a chance to improve outcomes for children in the area. The council also hopes it will help prevent an increased demand on services later on.

Hillingdon council, in west London, has pioneered award-winning work in the use of predictive analytics to prevent the exploitation of vulnerable children and young people. Its Axis project, launched in 2017 with support from the Office of Data Analytics and the police, analyses information from the police, social care, health, youth offending, schools, youth services, community groups and the public to identify high-risk geographical areas. This has then enabled the council to redirect intensive youth work support services to these areas and divert young people from crime or violence.

However, other machine learning pilots have proved less successful. Hackney council in east London recently abandoned its initiative in childcare because of the difficulties matching information across databases. “We were working with systems that aren’t naturally compatible with each other to collect information in different ways and in different timescales,” explains a spokesperson.

What Works for Children’s Social Care, part of the network of government-funded What Works centres, is examining the effectiveness of predictive analysis in the sector. Preliminary results from one of six councils being studied have shown that its predictions are “pretty accurate”, according to the centre’s executive director, Michael Sanders. But he is yet to be convinced that they really offer value for money. “People are spending money buying in these services even when they are short of cash,” he says. “They have been promised a silver bullet, but there are no silver bullets.”

The key issue from a purely evaluative perspective is whether machine learning will perform better than traditional analysis. Sanders believes it may in some cases, though he stresses the jury remains out, which would then trigger the further question of practitioners’ ability to understand the data without having a doctorate in statistics.

But the other issue for wider debate is whether politicians and the public are comfortable with the harvesting of personal data in this way – even if it does offer the prospect of saving a child’s life. As Sanders says: “We do need to think about the extent to which statisticians and people who are doing this are operating with genuine society-level consent.”

The view from the Information Commissioner’s Office

The Data Protection Act provides some legal safeguards on using personal data – one of which is that organisations must let people know how their details and profile are being used.

The Information Commissioner’s Office (ICO), which regulates the act, says it recognises that machine learning can present “some of the biggest risks related to the use of personal data”.

The ICO has set up Project ExplAIn with the Alan Turing Institute, which specialises in data science, to open public discussions on how such data should be shared and used across both public and commercial sectors. It plans to issue further guidance on the use of machine learning in the coming weeks.

In an interim report, earlier this year, the ICO concluded: “While there are undoubtedly benefits to this use of AI, there are also risks. Increasingly, governments and regulators are considering how to mitigate these risks. One such risk is the lack of transparency around how AI decisions are made.” LE

• This article was amended on 19 November 2019 to remove a reference to the use of machine learning in social care in New Zealand.

 

Leave a Comment

Required fields are marked *

*

*