Terry Downes, the “Dashing, Crashing, Bashing Paddington Express” who has died aged 81, was a popular all-action fighter who became world middleweight champion for nine months in 1961-62, retiring aged only 28 in 1964, when he was beaten in a challenge for the world light heavyweight title. A headline writer’s dream with a sharp cockney wit that spawned numerous one-liners, he was beloved by boxing promoters. His aggressive style rarely produced a dull fight, and Downes also had the charisma to ensure he was a major box-office draw.

Born and brought up in London, where he boxed as a junior for the Fisher amateur boxing club, Downes completed his boxing education in the US, where he fought for the Marine corps, which he had joined after his family emigrated in 1952. Downes’s elder sister, Sylvia, had previously travelled to Baltimore to join the famed Ringling Bros circus, but her family moved to the US to be with her when she lost an arm in a road accident. She was thrown from a bus on the way to the circus, suffering injuries when she was trapped between the vehicle and a telegraph pole.

Downes’s first fights were for the YMCA, but he was persuaded to join the Marines after boxing against them and performing impressively. In four years, he lost only a handful of fights, was believed never to have been floored in 51 contests and missed out on selection to box for the US Olympic team for the 1956 games in Melbourne because selectors ruled he was ineligible through failing to satisfy a residence qualification.

Downes had been a champion in the American amateur Golden Gloves competition, and had become all-services champion, defeating Pearce Lane in the final. Lane went on to be chosen for the Olympics, representing the US in the welterweight division. Although Downes boxed in the Olympic trials and was given rave reviews in American newspapers – some of which campaigned for his selection – he missed out and reflected: “I was all right to be in the bloody Marine corps, catching bullets in the frontline, but with boxing gloves they said no, you can’t represent us.”

His Olympic ambitions thwarted, in 1956 Downes returned to the UK and soon turned professional. Managed by Sam Burns, who had made his name as the righthand man of Jack Solomons, Britain’s leading promoter after the second world war, Downes won his first two contests in the paid ranks before being matched with a young Nigerian named Richard Ihetu, dubbed Dick Tiger by promoters. Tiger had lost seven of his first 21 fights. He would ultimately become world middleweight and light heavyweight champion, but was handpicked, at that stage of his career, by matchmaker Mickey Duff, as no more than a tough fighter likely to make Downes look good.

Duff spectacularly miscalculated. Downes was battered and defeated, pulled out by his corner after just five rounds. Asked for his thoughts about the contest by reporters, Downes said: “I thought, fucking hell they’ve put me in with a giant. Then I realised I was flat on my back looking up at him. I don’t remember much after that, so I look forward to reading your reports to see what happened next.”

But the quote which went down in boxing history as one of the great one-liners came in the follow-up question, when a reporter asked the beaten Downes who he thought he would like to fight next. Quick as a flash, he retorted: “The fucker who made that fight.” Perhaps wisely, Duff was nowhere to be seen at the time.

Downes would say of his manager, Burns, that he “didn’t know a left hook from a meat hook”, which was a tongue-in-cheek appraisal of a man who was linked with several other leading British fighters including Kevin and Chris Finnegan, and Tony Sibson. But Burns served Downes well in terms of financial advice.

By 1961, Downes was established as one of the biggest names in British boxing. He had beaten John McCormack to regain his British middleweight title and was regularly packing in big crowds to Wembley’s Empire Pool. He had defeated the highly rated American Joey Giardello to set up a world title challenge against another American, Paul Pender.

However, it was outside the ring that Burns gave Downes his shrewdest guidance, with the two men taking advantage of the newly relaxed gaming laws to set up a chain of betting shops. Making the most of Downes’s name, they would eventually own 90 shops before being taken over in the early 1970s by William Hill, of which Burns would be managing director for the next decade, with Downes investing his money as a major shareholder.

The shops provided Downes with financial security for life, but he achieved his dream of winning the world title when he fought Pender for a second time at Wembley in July 1961. Their first contest, in Boston, Massachusetts, had been stopped in the seventh with Downes having suffered an injury to his nose, but Downes prevailed in London when Pender was stopped on cuts after nine rounds.

A third rubber match in Boston between the two was won on points by Penderover 15 rounds in April 1962, in what Downes would always contest was a hometown decision. Burns had said his man would need a knockout to win, and he was proved right.

Downes defeated the faded 41-year-old Sugar Ray Robinson later that year but drew the curtain on his fighting career in November 1964, when he was stopped by Willie Pastrano in a world light heavyweight title challenge at Belle Vue, Manchester. Downes had been winning before being twice sent to the canvas in the 11th round.

Resisting the temptation of a comeback, Downes said: “That was as good as I could do. If I couldn’t win that fight, it was all downhill from there.”

After retirement from boxing, Downes appeared in several films, including Roman Polanski’s Dance of the Vampires (1967), and various television series where he often played the role of thug, villain or minder and was last seen in a bit part role in The Bill, in 1990. He also owned a nightclub and car dealership.

A regular at major fights, before he was afflicted by declining health, Downes’s robust observations – often in colourful, expletive-strewn language were much loved by ringsiders. A staunch supporter of the London Ex-Boxers Association, Downes remained a respected figure, although his public appearances were limited after the onset of cancer. In 2012 he was awarded the British Empire Medal for his charity work and services to sport.

In 1958 he married Barbara Clarke. She survives him, along with their children, Terry, Paul, Richard, Wendy and Melanie, and eight grandchildren.

• Terry Downes, boxer, born 9 May 1936; died 6 October 2017