When Hidetaka Miyazaki was a child, he was a keen reader, though not a talented one. Often he’d reach passages of text he couldn’t understand, and so would allow his imagination to fill in the blanks, using the accompanying illustrations. In this way, he felt he was co-writing the fiction alongside its original author. The thrill of this process never left him – and it is very much there in his arcane and fascinating video games, the latest of which, Bloodborne, has just been released to wild acclaim.

After some cajoling, Miyazaki reveals that he grew up “tremendously poor” in the city of Shizuoka, 100 miles south-west of Tokyo. His office-worker parents couldn’t afford books or manga, so he had to borrow whatever he could find in the library. That’s why he ended up with works beyond his reading capabilities. “I found so much joy in those stories,” he says, seated in his Tokyo studio, having granted the Guardian a rare interview. “It was a rich reading experience, even if I wasn’t always reading.”

Barely known outside of the gaming world, Miyazaki’s studio – called From Software – is already responsible for two of the most revered games of the last 20 years: the dark fantasy adventure Demon’s Souls and its spiritual sequel, Dark Souls. These eccentric and demanding games take the basics of Dungeons and Dragons gameplay – combat, monsters, exploration – and place them in terrifying, intricate worlds that work like a series of fiendish clockwork traps, complete with spike pits, poison mists and falling masonry. While gingerly exploring, players must master a uniquely complex combat system to battle a menagerie of Lovecraftian beasts.

Many games require a button-bashing, hit-and-hope approach to fighting, but Miyazaki’s take a more lifelike approach: you must meticulously time your sidesteps and parries, and wait for an opening through which to slide an eager pike. And unlike most adventure games, which are overloaded with backstory and cinematic narratives, it’s never entirely clear what you’re fighting for. Miyazaki’s games may be rich in lore, but nothing is made explicit. Your character is always a nameless warrior lost in an inscrutable, archaic realm; the friendly characters you encounter speak in riddles or grunts; everything is arcane, much is hidden.

Critics describe these worlds, leaden with dread and portent, as uniquely inhospitable. Every step is dangerous, every fight potentially deadly. Indeed, players often die hundreds of times, either slaughtered by monsters, or ineptly tumbling off battlements. But players always come back. The games have sold in their millions, drawing plaudits, awards and dozens of celebrity fans: Peter Serafinowicz, who played Pete in Shaun of the Dead, loved Dark Souls so much he begged for a voicing role in its sequel.

At a time when many believe the creativity of Japan’s once-bold video game industry is in decline, Miyazaki has captivated global audiences without compromising his vision. He is as enigmatic as his creations: he refuses to appear on film and, while he often meets fans, never talks about his own life, always shifting the focus on to the games or his team. He appears younger than his 40 years, despite the grey pallor of his face – the result, perhaps, of too many days spent under the fluorescent lights of his studio, by a dirty underpass in the Shinjuku area of Tokyo. He is clearly admired by his staff, who, one of his team quietly admits, will apparently call him for personal advice while he’s travelling overseas.

Miyazaki describes himself as a difficult child. “Unlike most kids in Japan, I didn’t have a dream,” he says. “I wasn’t ambitious.” He eventually found himself aimlessly pursuing a degree in social science at the well-respected Keio University. As he approached graduation, he considered applying to a game development studio, but drifted into a job at the US IT company Oracle Corporation.

Several years later, he started thinking about video games again. He met up with some former college friends who suggested new titles to play. One was Ico, a mystical fairytale in which players assume the role of a boy who must lead a waif-like girl by the hand along a castle’s craggy ramparts, pursued by their ghoulish captors. “That game awoke me to the possibilities of the medium,” says Miyazaki. “I wanted to make one myself.”

But Miyazaki had a problem: at 29, he was too old to apply for graduate positions and too inexperienced for anything else. “Not a lot of places would take me,” he says. “From Software was one of the few.” Ema Kodaka, who edits Miyazaki’s scripts, believes this career change, which resulted in a considerable drop in pay, might be why he is held in such regard by his staff. “He is a unique talent,” she says. “In Japan, even today, people usually join a company as a graduate and stay for life. For Miyazaki to change career and, within 10 years, become company president – that’s unprecedented in Japan. It’s inspiring.”

Miyazaki started there as a coder in 2004. After working on the Armored Core robot-combat series, he heard about a game in development elsewhere in the studio. “Demon’s Souls wasn’t doing well,” he says. “The project had problems and the team had been unable to create a compelling prototype. But when I heard it was a fantasy-action role-playing game, I was excited. I figured if I could find a way to take control of the game, I could turn it into anything I wanted. Best of all, if my ideas failed, nobody would care – it was already a failure.”

Miyazaki was assigned to the game and “changed pretty much everything about it”. Demon’s Souls launched in Japan without fanfare. The game had a disastrous reception at the Tokyo Game Show a few months before its release; many players didn’t even make it past the character-creation screen. It sold around 20,000 copies in the week of its release, far fewer than the publisher, Sony, had hoped.

But then word spread. Demon’s Souls demanded that its player master its complex armour and weapons systems, or face severe punishment – a design at odds with the industry-wide trend for simplification. It took gamers a few weeks to realise that this wasn’t a typical power fantasy so much as a test of mettle and patience. It also had a unique feature: players were able to leave messages for each other in the game world, scrawled on floors and walls. They could warn of nearby hazards – or trick players into death. It was a highly prescient implementation of social mechanics into a single-player game, and one that dozens of developers have since copied. A vibrant community built up around the game, with experienced players writing guides and sharing their wisdom on forums. Within a few months, sales passed 100,000, and soon the game found a publisher in the west. Miyazaki was becoming a name.

Dark Souls followed, with a similar demand that players progress through a relentlessly hostile world. The game outsold its predecessor in just one week. Miyazaki was promoted to president and promptly began work on Bloodborne, a game of similar poise but with a different style, melding Victoriana, arcane technology and Bram Stoker-esque nightmares. Bloodborne is defined by obfuscation. It takes place in the ruined city of Yharnam, a fever-dream of spindly cathedrals set upon chinked cobblestones, recently devastated by plague. But the story is hazy. You, like young Miyazaki, must fill in the blanks with your imagination, co-authoring the narrative as you trudge the streets in ragged trenchcoat and blood-slicked boots, fending off the city’s deranged inhabitants. The game is, if anything, more demanding than its predecessors: you have no shield to cower behind. Instead, you must duck and roll away from your attackers’ lunges, engaging in primal battles between closely matched opponents, retreating to recover strength and health, before heading into the throng again, to take on ever more frightful horrors.

Miyazaki’s move from fantasy into horror with Bloodborne is interesting, but so too is the way – in an industry in which auteurs often cling to bankable franchises – he chose to develop an entirely new and unfamiliar game. It’s all the more notable considering that, even though Miyazaki is now the studio’s president, he’s still obsessively hands-on.

His position has, in fact, brought unexpected benefits. “I’ve been to places I wouldn’t have visited before,” he says. “I get to see different things, which is important. It gives me new experiences to draw from.” In researching the architecture of Bloodborne, Miyazaki and the team visited Romania and the Czech Republic, drinking in their gothic grandeur. But he’s taken inspiration from closer to home, too. “Now I’m president,” he says, “I get to meet a lot of other company presidents. They’re such weird people. I’m fascinated by them.” With a smile, he adds: “I use some of them as enemy characters in our games.”

• This article was amended on 17 April 2015. It had referred to Miyazaki taking an 80% drop in salary from his career change. This was the result of a mistranslation; the speaker did not specify a precise figure.