Pedro Almodóvar has found a more intensely personal register than ever for his tender new movie about an ageing film director in retreat from his profession, facing ill health, depression and the decline of his powers. It’s Almodóvar’s 21st feature: maybe it could be called 20 ½.

Pain and Glory is an autumnal film in a ruminative minor key, with more pain than glory – although glory does make a late resurgence. It brings Almodóvar’s focus to death: his own and those of the people he loves, but also to the passion of film-making which may yet conquer death, or provide a way of coming to terms with it.

As ever, Almodóvar has made a film about pleasure, which is itself a pleasure: witty, intelligent and sensuous. It is about love, memory, art, mothers, lovers and most of all it is about itself, which in the hands of a lesser director would be dismayingly indulgent. But Almodóvar is a master of self-reference and intertextuality: the film within a film, the story within a story, the dream within a dream. Almodóvar operates on a kind of internal combustion engine of creativity and I felt that this movie was running so smoothly and so seductively that it could have gone on for another five hours.

Antonio Banderas steps up to the role he was born to play, though without the giveaway snowy pompadour. He is Salvador Mallo, a movie director who has not made anything for years but who has accumulated enough money to live in comfort among expensive artworks, brooding on his various ailments – headaches, backaches, a tendency to choke on any solid food – and general depression; all of these having some mysterious cause-or-effect relationship with his creative block.

A chance meeting with an old actor friend, Zulema (a cameo for Cecilia Roth) reconnects him with former star actor Alberto (Asier Etxeandia); they quarrelled decades ago while making his masterpiece. Dishevelled Alberto introduces Salvador to heroin, a dangerous new taste that reminds him of an abandoned autobiographical theatre script called Addiction, about a former lover’s drug abuse. He gives it to Alberto to perform on stage and this brings people from the past back into Salvador’s life – and his state of mind is suddenly altered. All the time there are ecstatic memories of his mother, gorgeously played by Penélope Cruz and as an older woman on the brink of death by Julieta Serrano. The older mother is sharply disapproving when her son starts asking for her own memories of her life: “I don’t want any of this in your films.”

As ever, cinematographer José Luis Alcaine and production designer Antxón Gómez give the film a wonderful richness and warmth, the colours swarming and popping. There are some big laughs, especially when Alberto and Salvador have been persuaded to do an onstage Q&A after a screening of their controversial film. Almodóvar may well have mixed feelings about the Q&A as a film tradition.

There is a masterful nimbleness in Almodóvar’s narrative style: one thing leads on to another, or back or sideways to another, or to an associated memory, or to a created fictional version. We hop back and forth between the past and the present. If there is anything about the film that left me wistfully unsatisfied, it is that this indirectness means the full-on emotional hammer blow never quite comes: the flood of tears or joy or eroticism is somehow deflected or deferred.

There is something incomplete or unfinished in this work, but perhaps this simply represents the condition of life itself. Pain and Glory leaves you with a sweet sadness, but a sharp appetite for the next film.

• Pain and Glory screened at the Cannes film festival and is due for release on 23 August in the UK and 4 October in the US.

• This article was amended on 12 November 2019 to correct the spelling of José Luis Alcaine.