I suppose I should worry that the French and military codeword avant-garde still seduces me. This word, after all, can conceal so much snobisme. But then: there's no reason to dismiss something just because it's impure, and this idea of the avant-garde, in its essence, is a noble ideal. The avant-garde is wildness: a wildness of content, and a wildness of form.

And this is one reason why I harbour another complicated attraction. My idea of the avant-garde is so often Parisian.

This isn't, of course, entirely a form of romance. In the bourgeois 19th century, Paris was where the avant-garde was invented. But even then, the ideal of wildness was precarious. It was Walter Benjamin who observed how the association of art and isolation was "all the more dangerous because, as it flatters the self-esteem of the productive person, it effectively guards the interests of a social order that is hostile to him". And he sardonically quoted Marx: "The bourgeois have very good reasons for imputing supernatural creative power to labour . . ."

As for now: well, you know it. You take the Eurostar to the avant-garde and you end up shopping. The avant-garde is much more likely to be found in Hamra. Paris, I think, can still be useful – but an imaginary, historical Paris: a jukebox of wild examples.

Because you can rehearse the usual arguments about how impossible it might be for writing to be avant-garde, in this era when everything can be recuperated as orthodoxy. Or, say, you can consider the ambiguous pleasures of the trial in Paris in 1956, to determine whether to allow the republication of four novels by the Marquis de Sade. Among the witnesses for the defence were two famous literary figures: Jean Paulhan and Georges Bataille.

When the judge asked Paulhan if he didn't find Sade's dismantling of moral values dangerous, Paulhan sadly agreed. "I knew a young girl who entered a convent after reading Sade's works, and because she had read them." Was entering a convent, he was asked, such a bad result? "I note that it's a result," shrugged Paulhan.

And then Bataille came to the stand. Bataille was a man who, in his novel Story of the Eye, had imagined scenes with bull testicles, pissing, the whole shemozzle. Now, 30 years later, in a courtroom, he soberly observed that he couldn't see how Sade's works could be harmful to the public. "I have to say," he added solemnly, "that I have a lot of confidence in human nature." To which the judge replied: "I congratulate you, Monsieur. Your optimism does you honour."

I am not sure who, in this courtroom, was joking. Just as I am not sure how far – to tell an opposite story – Jean Eustache was joking when, in 1977, he made his great short movie in two parts – called Une sale histoire. In the first part, a man tells the dirty story of how he discovered a peephole in his Paris café that allowed him to see into the women's bathrooms: a peephole that doesn't allow him to observe faces, only genitals. And he asks his listening friends why we are so agile in our descriptions of a face's beauty, and yet so reticent in discussing the beauty of those organs to which the beauty of the face might lead. This film is reckless with the ordinary proprieties.

And then in the second part the script is repeated – with the actors replaced by the true protagonist of the story and his true friends. The feature film becomes a documentary. And so this miniature film moves from being playful with propriety to being playful with reality.

Which reminds me of Isidore Isou's movie Traité de bave et d'éternité, made in 1951; a film that's not so much remarkable for the ordinary love story in St Germain told by its narrator, as for the random, scratched images that Isou uses to accompany the words: making nonsense of the usual ideas of montage. But then, Isou – a Romanian and Jewish immigrant to Paris – was a great experimenter with the limits of meaning: such as his "lettrist poetry", which submitted the classics of French poetry to phonetic decomposition.

As he demonstrated in an essay on himself, Isou liked to transform, say, Mallarmé's famous line, "La chair est triste, hélas, et j'ai lu tous les livres", into nonsense: "Gdjagass, gdjagass, la berr est biste jétu toutétive . . ."

And this delirium, I think, is likeable, as well as laughable. More permanent than avant-garde delirium is avant-garde sobriety, like the moment in an essay by Michel Leiris on the poetry of Raymond Roussel, where Leiris classes Rimbaud and Roussel – these two visionary poets who tried to make language expressive of the infinite – as twin souls of disillusion: who in turn dissolved the ideas "that literature 'changes life', that its exercise transforms us into a kind of God."

The wildness of the avant-garde is never pure: this is obvious. But that's no reason to reject the category entirely.

Two years ago, the New York artist Paul Chan made a piece called Sade for Sade's Sake. The main element of this series was a video piece that Chan first showed at the 2009 Venice Biennale: an animation projection of sexually active shadows. On Chan's website that's also a store (www.nationalphilistine.com), in which he offers the random downloader a sample of projects. One of these is a related series of fonts, called Sade for Fonts Sake. I say fonts: I'm not sure this is the right word. It's more a metaphor for a font. Where usually fonts take your words and make them look different by applying new visual rules, Chan's fonts take your words and make them different by applying new semantic rules. They set them as new words entirely, all taken from some kind of schlocky porn. Like Isou's lettrisme, they're a machine for decomposition. So that, in the font Oh Monica, for example, "Adam Thirlwell" would be set like this: "I need it, teach me, yes show me, I deserve it, as you like, yes oh sir go on Jesus yes go on go on." Which is about right.

And this New York project is partly a nonsense joke, and it's partly a new way of continuing to be sarcastic about the usual 18th-century categories: such as freedom, and reason. Because the meaning of Paris, after all, like the meaning of the avant-garde, is very simple. You can set up Paris anywhere. It's there wherever the prevailing categories of representation are suspended. Paris is portable.