Throughout the course of writing Gone Girl, and then later while adapting it for film, Gillian Flynn's expectations were low. The novel, her third, was met with cautious approval by her editor, who liked it, but thought the ending was risky. There wasn't a hero; in fact, everyone in it was kind of horrible, and it wasn't like the novels she had written before. Flynn's first two books, Sharp Objects and Dark Places, were straightforward mysteries that did modestly well, so that, she says, everyone assumed this one "would build a little on that, and that would be great".

In the event, Gone Girl entered the New York Times bestseller list the week it was published, rising to No 1 the following week and selling around two million copies in the first year, beaten only by the likes of 50 Shades of Grey and The Hunger Games. The film has been released this past week and Flynn has gone through the same process of expectation management. "We all know how it goes: the little author goes to Hollywood and gets punched in the face and cries on the bus right back home." Since the director is David Fincher and the stars are Ben Affleck and Rosamund Pike, it seems likely this forecast will be overturned, too.

The day we meet, in a hotel in Chicago, Flynn is two weeks away from giving birth to her second child, has just moved house, and is engaged in the buildup to the release of Gone Girl, a confluence of stresses she is trying to see as a virtue. ("My mindset, when I get the 3am panics, is we must think of it as a big adventure, don't think of it as a crisis. A crazy moment to look back on and laugh about.") The 43-year-old's default setting is pragmatism, something one is tempted to link to her midwestern background. Flynn grew up in Kansas City, Missouri, and went to graduate school in Chicago, where she returned after years of working as a journalist in New York and LA. "It was a place to settle down. It had everything I loved about New York, but on a more chilled-out scale."

The midwestern setting of Flynn's novels is a big part of their draw, the dingy rural towns and endless gothic planes of Truman Capote's "out there" giving a sense, simultaneously, of too much and not enough space. Given that her books fall into the oft-dismissed category of genre fiction, Flynn has had extraordinary critical success with Gone Girl, which was reviewed positively in the literary press and named by the New York Times as a book of the year. Her writing is sharp, acute, with social observations and convincing relationships that outweigh the sometimes outlandish plot turns and overboiled symbolism. (In Sharp Objects the heroine Camille spent her childhood carving words into her body, before growing up to be – yes – a tortured writer.)

In Gone Girl, the sourness of the marriage between the protagonists, depicted in fights about whether the sofa he loves is, as she believes, actually hideous, is a deeply convincing account of a terrible relationship and the real reason for the book's success.

"I suppose these questions storm cloud over every marriage," muses Nick before his wife Amy disappears and he falls under suspicion. "What are you thinking? How are you feeling? Who are you? What have we done to each other? What will we do?"

"It's relatable," says Flynn. "The push and pull of a long-term relationship – the little power plays and dynamics." It is also a book about class. Nick is from a midwestern background similar to Flynn's and feels judged and condescended to by his wealthy and sophisticated New Yorker wife. After she loses her trust fund they return to the midwest to open a bar, bringing with them their big-city snobbery – hers innate, his adopted. "We thought we were being clever New Yorkers," Nick says, of their decision to call the bar The Bar, "that the name was a joke no one else would really get, not get like we did. Not meta-get." Their first customer, however, "a grey-haired woman in bifocals and a pink jogging suit," says to them: "I like the name. Like in Breakfast at Tiffany's and Audrey Hepburn's cat was named Cat." Flynn is very good at identifying these social currents.

She would never have reached this level of execution, she says, if she hadn't had 15 or so years of working in journalism and learning to scrap extraneous words. While at Entertainment Weekly, where she wrote about film before becoming the television critic, she would see her many-thousand word articles hacked down when a page lost its advertising. "I could not have written a novel if I hadn't been a journalist first, because it taught me that there's no muse that's going to come down and bestow upon you the mood to write. You just have to do it. I'm definitely not precious."

This attitude extends to how she treats her characters. One of the appeals of Flynn's writing is how willing she is to make every single person in a novel unsympathetic; dastardly, even. In Dark Places the heroine, Libby, is a bitter, cynical twentysomething, who as a child was the sole survivor of a family massacre and who regards everyone around her with ferocious disdain. A typical pen portrait of someone wandering across Libby's critical gaze: "she was somewhere north of 50, with dark, darting eyes that bulged out of a bony face" and "the shiny coating of someone on too many pills". There can, occasionally, be something flat and humourless abut Flynn's writing, the sharpness turning sour with a sense that she despises the people she's writing about. But she is never less than compelling.

Flynn says it's just a case of keeping herself interested. Her first reader is always her husband, a lawyer: they met at college but only got together after she returned to Chicago in her mid-30s. His opinion has turned around projects that were destined to fail. Libby, for example, started out in Dark Places as a nice, sunny girl with lots of friends. "Coming off Sharp Objects, I was determined not to write another dark female character. So I wrote this entire first draft in which Libby was church-going and really stable. It didn't work at all. When I gave it to my husband, I knew it was going to be bad because he was like, OK, you want to talk about Dark Places now? Shall we open a bottle?" She smiles. "That's a good husband."

Sharp Objects had also gone through several stages. Flynn started and abandoned many novels over the years before hitting on the idea of writing about a journalist covering a gruesome story in her home town. She had a vague idea that she wanted to write about female violence. "Men write about that all the time, the darkness within and what you inherit from your parents as far as your psychological wiring goes. I felt like that wasn't there as much for women, and I was very interested in it."

Her novels are so dark that readers often ask about Flynn's own background. (Her mother once stood at the back of a bookstore after a reading and made frantic gestures at Flynn when a member of the audience asked whether she came from a bad family.) In fact, her upbringing was happy and healthy. Her father taught film and theatre at junior college and her mother was a teacher; they still live in the house where Flynn and her older brother grew up. She was a little shy as a child, a big reader who loved movies as much as books and thought from an early age that she would be a writer. "Or a farmer."

Flynn did her undergraduate studies at the University of Kansas and, by her own admission, "came to New York with a bit of a chip on my shoulder. Entertainment Weekly at the time was awash with Ivy League graduates and a lot of boarding school and a lot of east coast. I felt like there was a whole language I needed to learn. It wasn't that I was being outsmarted necessarily, but I just felt different. And then I switched into my other mode which was Fuck You All."

She was also puzzled by the assumptions east coast people would make about what life was like in the middle of the country. "They would ask where I'm from, and when I  told them they'd say: 'Good for you, that you got out.' You've never been there! You've no right to say how lucky for me that I escaped!"

She would, eventually, come to believe that drawing on somewhere other writers ignored was a distinct advantage in writing fiction. "The midwest is great because it hasn't been entirely claimed. There's more room to write about it; it's harder to write about New York, because even if you've never been there, you think you know what it's like. To do it in any sort of fresh way is trickier."

The most enduring literary depiction of Flynn's part of the world is probably Truman Capote's In Cold Blood – the town she comes from is a few hours drive from Holcomb, where the killings described by Capote took place – which she read "way too young". Dark Places was a conscious homage to the book.

In those early years of writing, Flynn worked on her fiction after work and on holidays, but found herself going nowhere. Sharp Objects came out like a tedious essay. "My way of doing it was to have people talk in a room for a long, long time. They'd talk and talk and talk about female violence. And then I'd move them to a different room, or perhaps they'd go outside for a walk. And they'd still be talking. It wasn't working."

She had a lot of other work on at the time. Her job as a film writer required her to travel the world visiting sets. "I have distinct memories of writing part of it in New Zealand, where I was for Lord of the Rings. Or being on the set of Jackass: The Movie, drinking with the guys and coming back to write this really dark story."

What resuscitated a dead project in the end was reading Mystic River, Dennis Lehane's terrific police procedural set in Boston. Flynn had always loved mysteries and saw, suddenly, that that's how she should do it. "I have a distinct memory of grabbing the novel – it was floating around the office – taking it home and literally reading it into the morning until I was done. I remember thinking, if I can tie it to a mystery, that will give me the discipline and the engine to push it forward. I think it's a shame people say 'I don't read that kind of book', because something like Mystic River is tackling topics as deep as any sort of mainstream psychological realism."

Sharp Objects ended up going through several more drafts, the first of which didn't even contain the character Flynn would ultimately name as the killer ("in the first draft, the actual killer was the cheerleader"). It was published in 2006 to respectable reviews and Flynn got going on Dark Places. Then she was, unexpectedly, made redundant.

It was a difficult time, she says. Flynn was 35, "at that stage where it's tricky to know how to make yourself over again". Like most journalists, she had no other skills. "It was at the height of the recession and you were only hearing about people getting laid off. I was like: I guess I could waitress again. That's the only other job I've ever really had. And so I was about six months waiting to see if I'd get another book contract. And I had thought I was pretty good at my job. No one wants to be laid off; it feels embarrassing."

She did get another book contract, after her husband delivered his critique of Dark Places. His question to her was: "'I'm curious, do you love Libby? Do you like writing her?' And I was like, 'Ugh, I can't stand her.' You would think I knew her personally. He said he could tell I didn't like writing her or believe in her. So I trashed it all and started over again." Libby became a dark, malevolent creature and the novel took off. "Very inefficient way to write," says Flynn, "but I get there in the end."

Until the recent house move, she wrote at a rickety desk in the basement ("I hated that desk"), and constructed her day around the completion of scenes. While she's writing, Flynn says she thinks almost filmically rather than novelistically. "I see everything really clearly. That's the first thing that comes to me; the way a room looks. I see it almost the way a director would. The long shot, that sort of thing. Then I just fiddle around with it until it feels correct."

She loves working at home. "I like the cosiness of it and I'm able to compartmentalise pretty well. I like not having to put on proper clothes. I have a friend who works from home who is very disciplined and is always like: 'You have to put on a bra!' That takes away all the fun of working at home. Why would you do that to yourself?"

For a while, she tried to hit a certain word count every day, but "it led me to panic writing. Just trying to hit the word count, rather than being focused on writing anything good. So I start in the morning and at midday I take a long break and go for a long, rambling walk, so I can clear the air." She returns to work for a few hours in the afternoon.

One of the great virtues of Gone Girl is Flynn's ability to move effortlessly between Amy and Nick's stories, with each chapter alternating between their points of view. Having two unreliable narrators gives the novel a vertiginous sense of risk; there is no safe harbour for the reader. It also shows off Flynn's imaginative range, particularly in the writing of Nick, who is a thoroughly convincing, hard-done-by husband. "I've always had a lot of guy friends; more than female friends. Guys have never felt like this other thing. So when I started writing from a guy's point of view, I never thought: 'OK, well, he'd better like football. What would a guy say here?' He just needs to be a believable person. It helped giving him a biography close to mine."

Her husband, Flynn says, is tolerant of the dark moods that can descend on her when she is in the middle of a book "and writing something really dark and toxic – he doesn't take it personally". Although, she says, when people ask him if he has a favourite among her three novels, he tends to  pick Sharp Objects because he wasn't around when it was being written. "I get really tense during the first draft. Really tense. That's not great for my family, because the first draft usually takes about a year."

Writing the screenplay for Gone Girl was a much easier task, although Flynn spent the months before filming convinced she was going to get fired. Despite having written about movies for 10 years, she had never written a screenplay before. "And I was one of those really geeky kids who would read film scripts at the age of 12, before the age of the internet, hunting down screenplays and talking about them with my dad. I really wanted to do it for the fun of it, and also so that I could say: 'Well, if it doesn't turn out well you have no one to blame but yourself.'"

When Fincher signed on as director, fresh from directing The Social Network and before that, The Girl With the Dragon Tattoo, she assumed he would want to bring in his own people. To her amazement, he kept her on as principal screenwriter. "I felt like I was working with a really good editor. He never told me what to do. But he was great about making me articulate why I wanted a certain scene a certain way."

She was not precious about making changes to the story. "I've seen movies that are slavishly devoted to books but don't work, because they haven't turned it into a movie; they've turned it into a dramatisation of the different scenes." Nonetheless, her anxiety remained. Before any actors came on board it was just Flynn and her husband, reading the script to each other. "And I'd be like, I think that sounds awful, but I don't know if that is because we're such awful actors."

A week before filming began in Missouri, Fincher called her into his office. "And I thought: this is where I get fired. Finally, I get it over with. It turned out he wanted to play a song for me that he liked. I do wish I could go back and tell myself to enjoy the moment a little bit more."

Excitement about the film's release has been building steadily all year. Flynn, for her part, is happy that "people are really psyched about a grownup film". She hopes that the success of Gone Girl will open the door to genre novels being taken more seriously. She isn't as enraged about this issue as, say, Jennifer Weiner, the romantic novelist who is on a campaign to be reviewed alongside Jonathan Franzen et al, but, says: "I think they only let a few of us squeak through at a time. There's a lot of great stuff out there – mysteries, science fiction, fantasy – that is described as genre and there's still a grudgingness to accept it. A book is a book and, if you're tackling interesting issues and you're writing well and smartly, do we need to say in what form it is?"

With all the publicity surrounding the film, Gone Girl is back in the bestseller lists, which surprises Flynn. She has never, she says, bought a book on the basis of seeing a movie. Her hunch is that men are finally starting to buy the novel, thinking "it must be OK if Ben Affleck and David Fincher think so. They seem like manly men."

The baby will be two months old when the film comes out – "past that crazy first month," says Flynn, "so I want to be in the mix as much as I can." And she will continue to write in the same vein, only from now on with less scepticism from her publishers.

"If you are someone who reads books to feel like you have a friend on the page, my book is not going to be the book for you," she says. "I write for people who are readers the way I'm a reader. I don't care if I dislike a character; I care if I find them interesting or they make me laugh, or if I'm trying to figure them out. I am always more interested in that."