It’s not been a great year for Melissa McCarthy, an actor who has enjoyed a number of standout years since she broke out in 2011’s Bridesmaids, bagging an Oscar nomination in the process. Her brand of often hilarious yet too often lazily repetitive comedy had been wearing thin already and with the one-two punch of Life of the Party and The Happytime Murders, two critically eviscerated and commercially underperforming disappointments, her luck seemed to be coming to an end. But with quite miraculous timing, she has found her way into this year’s awards conversation with her first of two more serious-minded roles.

In the charming, fact-based Can You Ever Forgive Me? the character of Lee Israel allows for McCarthy’s comic skills to shine through, exemplified in an opening scene that quite perfectly sets up her irascible, heavy-drinking antiheroine as she gets fired from a day job she hates anyway. While Israel had written a number of moderately successful biographies of historical women, the money hadn’t lasted very long and with her agent constantly brushing her off and her ex-girlfriend refusing to accept her calls, she struggled to survive. Through an unlikely set of circumstances she found herself carving out a new career as a forger of celebrity letters, enjoying both the lucrative rewards and the ego boost to her skills as a writer.

With direction from Marielle Heller, who impressed with her nuanced debut The Diary of a Teenage Girl, and a script co-written by Nicole Holofcener (who was originally attached to direct), Can You Ever Forgive Me? benefits immensely from their combined abilities. Both have incredible respect for their characters, avoiding over-embellishment, and with Israel they have (semi-)constructed another fascinating, flawed, frustrating and funny female character. As her agent (a fun cameo from Jane Curtin) explains, life would be a lot easier and work would be a lot more fruitful if she just stopped being so difficult. “You can be an asshole when you’re famous,” she tells her.

But at 51, Israel is uninterested in playing the game (or selling out like Tom Clancy) and there is an understandably exhausted arrogance from someone whose undeniable talents are not enough to warrant a steady income. For every moment that Israel deserves sympathy, there are five others when her behaviour infuriates and McCarthy is so perfect at expressing just how head-smashingly difficult it is to be around her. It is hard to imagine that Julianne Moore was originally set to inhabit the role as McCarthy plays it with such confidence that it feels written directly for her. One of the reasons she makes for such a talented comic actor is her ability to take her characters so far and with this more dramatic turn, there is a similar fearlessness. It’s one of her best performances to date and opens the door on an entirely new and hugely exciting stage of her career.

She is buddied up, reluctantly of course, with Richard E Grant’s decaying dandy and together, they share a spiky, increasingly warm, chemistry. There is an unfettered mustiness to the pair and the bookshops and bars they exist in, with Heller creating such an evocative atmosphere of early 1990s New York that a light whiff almost emanates from the screen. It is rare to see a film led by two gay characters over the age of 50 and there is a specific friendship they share; both are implicitly aware of the alternate routes they have taken in life partly as a result of their sexuality and both more explicitly aware of the loneliness that now hangs over them. There is a melancholic atmosphere to the film at large, a frankness about the impossibility of dreams and the importance of growing old without shackling one’s self to an idealised notion of what life will be.

Sometimes the shagginess of the film can make it feel a bit slight and at times it does work better as a concentrated character study, but it’s such a joy to spend this time with McCarthy, drunkenly scheming and grumbling, that it’s hard to complain.

• Can You Ever Forgive Me? is showing at the Toronto film festival and will be released in the US on 19 October and in the UK on 1 February