There’s nothing I find so cheering, these days, as the rise of the horror movie. Take its intrusion into this year’s summer blockbusters. We have the usual soulless franchises and deadly repeats – Despicable Me 4, Deadpool 3, Furiosa: A Mad Max Saga, Bad Boys: Ride or Die – and then we have a flicker of light in the dark. Turns out that audiences do want new stories, they do want new characters, and they do want inventive film-making after all. Because a genuinely imaginative – arthouse, even – movie is predicted to draw in big audiences and make a great deal of money. It has come in the form of a horror film: Longlegs.

Just released on Friday and starring Nicolas Cage as a serial killer, Longlegs has been reviewed, variously, as “the scariest film of the decade”, and “a film in which every frame is a nightmare”. But it is also starkly beautiful – starting from the opening shot, as we follow a small girl’s progress through a snowy landscape. We move through claustrophobic basements and misty woods, our eyes flicking to layers of shadow in the background, to wherever the characters have last omitted to look. The film is thick with references for film buffs; flashbacks are indicated through texture and ratio changes; there are arty bursts of absurdity.

Yet the film is also expected to gross some $20m (£15.5m) in the US on its opening weekend – an astonishing haul for an indie movie. A BBC review suggests that in future, “horror movies could become the new summer blockbusters, while superhero movies become the counter-programming alternatives”.

This is a strange turnaround indeed – as is the growing respect for the horror genre. For years, horror has languished under the idea that it is cheap, nasty and formulaic, as if the worst slasher movies stood in for the whole genre. It has been frequently snubbed by the industry. Shelley Duvall, who died last week, is recognised now for her masterful performance in The Shining – yet was at the time nominated for worst actress at the Razzies, a decision which was only retracted two years ago. The film itself received no Oscar nominations.

In fact, until a couple of years ago, it was quite routine for horror films to promote themselves to journalists and awards committees by strenuously denying they were horror films at all: instead they claimed to be “elevated horror”, “post-horror”, or “extreme drama”. Darren Aronofsky once described his film Mother!, in which a newborn child is eaten by a mob, as a “thriller” with “home invasion elements”.

But now the genre is at last on course for rehabilitation. What started in 2017, when Jordan Peele released the high-concept gothic Get Out, has continued in a stream of inventive horror films: Hereditary, Us, Nope, Lamb, M3GAN, Talk to Me, Beau Is Afraid. Accolades have mounted, and film-makers no longer have to dredge around for actors, worried the stigma will tank their careers.

As a horror fan, I find all this very heartening news. I have perhaps two friends I can reliably drag to horror films, one of whom is now the father of twins and only available for daytime showings. (Have you seen the crowd that sits down for a midday showing of a horror film? That is a more terrifying afternoon than you may have bargained for). So I like the idea that more people might be pressured into this scary genre, if only to keep up with trends. Last year, Vogue ran an article: “Am I missing out on the zeitgeist because I don’t like horror?” This is the sort of coercive atmosphere we should encourage.

But the really good news, I think, is that the rise of the horror film bucks a dismal pattern. Mainstream cinema is now choked by franchises from the likes of Marvel and DC. Films for grown-ups are increasingly packaged in the bright colours of comic books and infected with parables from the nursery: good overcomes evil, hard work pays off, friendship is nice. We get the same characters, and the same stories, in the same fish-bowl universes.

Horror, by contrast, has become ever more sophisticated, interrogating contemporary anxieties – where do evil and vice really lie? – and playing with form. Last year, Huesera: The Bone Woman drew us into the experience of postpartum psychosis; 2020’s The Invisible Man took us on an empathetic journey with a victim of domestic abuse.

How has horror squared this circle? What is its secret? How does it cram worthy messages and experimental techniques down the throats of audiences who reject that sort of thing elsewhere? The secret has been in front of us all along. Horror has always been a genre designed to hold our full attention: brains hyper-alert, scanning for danger. This is how it coped with low budgets and unknown actors: people watch anyway. And this is the sprinkling of addictive sugar that helps us take our medicine now. As long as it scares us, horror can get away with anything: complex political allegories, tricky camera-work, a plot point that turns on the inventive use of a scented candle, like in Hugh Grant’s upcoming Heretic.

I have a feeling its potential is only just starting to be explored.

• Martha Gill is an Observer columnist