David Mamet arrives at his 70th birthday this week, and there couldn’t be a better moment for his classic Glengarry Glen Ross to be storming London’s West End once again: that gripping spectacle of desperate middle-aged men competing to sell real estate in a recession or get fired. They are twitching rats in a laboratory of capitalism, terrified of failure and terrified of death.

For me, the author’s birthday milestone is a time to ponder something else: a quintessentially Mamet moment in the 1991 movie Homicide, which he wrote and directed. Two cops – Bobby Gold, played by Joe Mantegna, and his partner Tim Sullivan, played by William H Macy – are working on a case with unexpected personal implications for Bobby. He appears, in the eyes of his aghast partner, to be suffering some kind of breakdown. Tim fiercely lectures him on the need to stay tough: “It’s like the old whore says, ‘Once you start coming with the customers, it’s time to quit.’”

There is no sign of Mamet showing this kind of empathy with his own customers, and no sign of him quitting. Those customers were treated to an unforgiving display of control freakery recently when he banned all post-performance Q&As, on pain of a $25,000 fine. Mamet was perhaps irritated by a concession to audience debate that encourages disruptive campus-style challenges as if in the academic classroom, that arena for painful ideological confrontation he explosively depicted in his 1992 play Oleanna, about teacher-pupil harassment. Mamet took out a cease-and-desist action against a Milwaukee theatre company who wanted to change the female character to male, in order to make it about same-sex harassment.

Yet Mamet reaches his 70s with his face on Rushmore pretty well complete. He is an American icon who has been influenced by Arthur Miller, Harold Pinter and Philip Roth and who in turn has influenced Aaron Sorkin and James Gray. He has compellingly tackled masculinity, sexuality, capitalism and Judaism. His movies are serious, but he can take on pop forms like thrillers, capers and neo-noir dramas, while films such as House of Games and The Spanish Prisoner are wonderful trompe l’oeils, essays in illusion.

His non-fiction writing is also terrifically witty, readable and smart and he is a master of the Hollywood insider genre. Above all, there is his legendary dialogue – terse, fierce, telegrammatic, often imitated and spoofed, celebrated as a unique prose poetry of disquiet and confrontation. It is a fugue of verbal violence. His characters’ riffs smack into each other like sumo wrestlers’ bellies. His 2001 movie Heist has one of the great noir lines of all time. Danny DeVito’s villain lies dying after an almighty shootout and sneeringly asks Gene Hackman: “Don’t you want to hear my last words?” Hackman replies coolly: “I just did” – and finishes him off with a final shot.

Yet Mamet does not have an endless golden touch, perhaps inevitably for someone so magnificently productive. He has had some recent Broadway flops: China Doll (2015), starring Al Pacino, received tepid reviews, The Anarchist (2012), about a Weather Underground-type protest veteran, and the political comedy November (2008) did not run long.

Furthermore, his 2011 book The Secret Knowledge: On the Dismantling of American Culture dismayed his fans by trumpeting a personal shift to the right, with climate-change scepticism and attacks on feminism, progressives and liberal academia. Christopher Hitchens called it “an extraordinarily irritating book, written by one of those people who smugly believe that, having lost their faith, they must ipso facto have found their reason”.

It is, however, possible to be sceptical about this shift. Mamet’s tough-guy aesthetic has always looked conservative – and provocations like Oleanna and Race (in which a top defence lawyer’s female African-American junior appears to leak information to the prosecutor in a racially charged rape case) were male-centred. His aggressive tendencies became weaponised in the war-on-terror era with his frankly absurd movies such as 2004’s Spartan, in which Val Kilmer’s special forces hombre interrogates a suspect in a back alley by threatening to cut out an eye, and Redbelt (2008), about Mamet’s infatuation with martial arts.

Viewed unsympathetically, even the famed Mamet dialogue is just a Tourette syndrome chorus of macho self-pity, and the Hollywood-themed works are very much of the Weinstein age, which takes a drolly forgiving view of Hollywood honchos seducing younger women, who are often shown as knowing and complicit.

Glengarry Glen Ross will perhaps go down as his single greatest work: a compelling display of male paranoia and male competition, although Blake, the guy from head office who appears briefly in act one to tell everyone that under-performers will be canned, is a character invented specifically for the film version, not the play. He is famously portrayed by a shark-like Alec Baldwin. “Coffee is for closers,” he snarls at over-the-hill salesman Jack Lemmon, who had presumed to pour himself a cup while Blake was speaking. Then he announces the bonuses: “First prize is a Cadillac Eldorado, second prize is a set of steak knives and the third prize is you’re fired.”

There is a lot of vintage Mamet dialogue, when the subject of burgling the office for privileged information is furtively discussed among the sweaty, hapless losers. But it wasn’t until I watched the film again that I realised that there is a further dimension to Mamet dialogue: absurdism and black comedy. Perhaps Mamet has been influenced, not by Pinter but Abbott and Costello and their famous “Who’s on first?” routine. Yet always there is desperation – the need to grab the brass ring.

There’s something similar in his play American Buffalo: the junk-shop owner needs to get hold of a “buffalo nickel”, which could be hugely valuable, the kind of a once-in-a-lifetime shot at riches. With it, his entire life in the crummy shop might have meant something – but if it slips through his fingers, his life could look terrifyingly meaningless.

For the almost existential maleness of Mamet, you couldn’t have a better example than his 1974 play Sexual Perversity in Chicago, whose showstopping and atypically grabby title did much to get him noticed. It is about sex, intimacy, communication and miscommunication; a rackety, juicy comedy that audiences still love. Danny and Bernie are buddies who confide in each other, and their relationship is in some way closer than Danny’s with his girlfriend, Deborah. Separate scenes show Deborah listening to her own best friend Joan confide her exasperation with hopeless males. Eventually Danny and Deborah break up; the male-male bond appears to survive where the romantic partnership doesn’t.

This play, with its language and worldview, is the polar opposite of a writer like Nora Ephron and her much more female-centric When Harry Met Sally, which it might appear to resemble superficially. Sexual Perversity has been adapted twice for cinema, both times with the title About Last Night, and both films look as if they have been forced uncomfortably into the Ephron-romcom-date-movie worldview. Mamet is more male, more cynical.

As for Hollywood, no student of Mamet should neglect the importance of his longtime producer and collaborator Art Linson, who wrote a hilarious memoir of the producing life entitled What Just Happened? – with an uproarious account of trying to get Mamet’s utterly forgotten 90s thriller The Edge, starring Anthony Hopkins and Alec Baldwin, off the ground. Linson is a very funny writer and almost a Mamet character himself. Just as desperate real estate guys sell property to the public, so desperate Hollywood producers sell hot properties to studio heads – and these movies’ putative status as artworks mean the producers invest so much of themselves in them, as well as their male professional self-esteem.

It is often about reclaiming or refusing reality, a land-grab of the mind. Wag the Dog was Mamet’s adaptation of Larry Beinhart’s novel about a movie producer and political spin doctor creating a bogus war in Albania to distract the public from a political scandal. State and Main is about an independent film crew that turns up to shoot in a remote Massachusetts town only to find that it is wholly wrong for the story. And the crew’s troubles include an ageing leading man with a taste for younger women – a part of the comedy that looks exposed in today’s Hollywood.

But these movies are not as highly charged and intensely felt as his play Speed-the-Plow, which, crucially, has a much more corporate, workplace-oriented setting. A Hollywood producer seduces a younger woman in the office (someone over whom he has power) at least partly as a result of a crude bet with another guy. But this woman turns out to be more Machiavellian and less of a victim than anyone thought.

Again, it’s the kind of satire that, at this moment, feels dangerously obtuse and dated. But it still has a good insight into the aggressive, scared mentality of the producer who fears that he will never again get another movie into cinemas. Movies are, after all, unlike real estate in that they are pure luxury items: you don’t need a film, but you have to be persuaded that you need it by people who have already frantically persuaded themselves. In 2008, it was notably revived at London’s Old Vic with Kevin Spacey.

But perhaps the most tactless, painful and radioactively brilliant work is Oleanna – a play and then a film from the Clinton 90s that can still trigger furious responses: it shows a college professor accused of sexual harassment and sexual abuse who is basically innocent. Ultimately, both man and woman are in the grip of something in the air, some tragic force of violence and despair that overwhelms everything and everyone, an imperative that drives them to open warfare.

Mamet’s new play The Penitent, about a psychiatrist who is attacked for appearing to side against an LGBT patient, is an obvious cousin to Oleanna. I’m surprised Mamet hasn’t weighed in to the trans debate, with a professor or teacher or some other flawed careworn male authority figure being fired or disgraced for appearing to victimise a trans person by failing to use gender-neutral pronouns. Perhaps, once his 70th birthday celebrations are out of the way, he will start something on this theme and prove that no one can match him for detonating an almighty rage.

• Glengarry Glen Ross is at the Playhouse, London, until 3 February. Box office: 0844-871 7627.