There’s something particularly horrific about watching a person you love gradually lose all semblance of themselves.

In the psychological horror film Relic – which debuted to critical acclaim at Sundance in January and arrives on Stan this week – the Australian first-time director Natalie Erika James explores the nightmare through a metaphor which, for her, is close to home: the devastating impact Alzheimer’s disease can have on its sufferer and their loved ones.

Emily Mortimer, Bella Heathcote and Robyn Nevin star as three generations of women, reunited in their small-town home in Creswick, Victoria, after the elderly matriarch Edna (Nevin) goes missing.

When Edna mysteriously reappears with no explanation, the family confront an unknowing, ominous presence that threatens to possess their rickety house and those inside it.

“I started writing Relic when I was [in Japan] visiting my grandmother, who had Alzheimer’s,” James tells Guardian Australia from her home in Melbourne. “And on this trip, it was the first time she couldn’t remember who I was. It made me think about the ways in which, over time, her relationship with my mother and me had shifted.”

Relic, a co-production between Australia and the US, was written by James and Christian White, with Jake Gyllenhaal and the Russo brothers attached as producer and executive producers respectively. A single-location drama, it patiently stews in claustrophobia, with unnerving thrills and a surprising, harrowing intimacy.

As a self-described “bookworm”, James’s childhood was spent living a “transient life” between Japan, China and Australia, shooting school events and her own personal art films with a video camera.

But it was her summers at her grandmother’s place that made the lasting impression.

“[My grandmother] lived in this house that I had always been scared of as a child, because it was an older, more traditional-style Japanese house – and I was watching way too much Asian horror,” she recalls with a laugh.

The restraint of gothic and Asian horror films, particularly J-horror, set the tone for her debut feature. “There’s this suspension of what is real and what is supernatural. In Relic, there is a similar thread of: is there an intruder, is there someone in the house, or is there a ghost? Or is it dementia that’s causing all of this?”

In Relic, the horror is steeped within the darkened hallways and candle-lit, labyrinthine interiors of the family home, with tension culminating into heightened, grotesque territory.

James didn’t want the fear to come from traditional jump-scares; instead, she frames the forthcoming menace through “what is unseen”. This meant favouring practical effects over VFX, including the use of an animatronic puppet, and incorporating a sound and lighting design sourced primarily from within her storyworld.

The raw construction of the film, she believes, grounds Edna’s deteriorating state – even in its eventual monstrous externalisation – in the real, lived troughs of human horrors.

“The film talks specifically about Alzheimer’s, but it also applies to anyone who is experiencing loss in all its forms – in having to parent their own parents, or seeing their decline,” James explains. “It’s a really emotionally fraught situation because there’s often the sense of the grandparents needing help, but being unwilling to ask for it as well.”

These taut familial dynamics are at the heart of the narrative. “There’s a sense of simmering resentments between the three [women]: the idea of one family member’s expectations, another’s guilt and a judgement of inaction.”

Having premiered the film at the Sundance film festival to glowing reviews – Variety called it “impressively scarifying” – Relic also drew comparisons to Jennifer Kent’s The Babadook and Ari Aster’s Hereditary: all inward-looking female-led arthouse horror films encased in a domestic setting. Both films, which also premiered at Sundance, have since launched the careers of their directors; whether or not Relic will do the same is yet to be seen (IFC Midnight has acquired the film for US distribution).

James is already in the early development stages for her next feature, inspired by her short film Drum Wave – a folk horror centred on a young pianist who must confront her fear of motherhood when she marries into a remote island community.

The trend towards both critically and commercially successful mid-budget “smart horror” – films including Get Out, A Quiet Place and The Witch – has presented a new opportunity for her as a film-maker.

“It’s a really exciting time because audiences are more receptive to the idea that horror can be the perfect space to talk about social or deep, emotional issues in an accessible way,” she says. “I think we’re seeing an influx of films stand up on their artistic integrity, as well as being these thrilling genre rides.

“The meeting point between the two is my favourite type of film – and certainly the films I want to continue making.”

• Relic premieres in Australia on Stan, and in US cinemas, on Friday 10 July