"It's true. All of it," says Han Solo (Ford) to the young whippersnappers opposite him in the middle of The Force Awakens. It's a moment that marks the baton being passed from one generation to the next - a recognition of what's behind in order to move forwards.

That balance of old and new is something of a recurring theme in Hollywood this year, as franchises return, reboot and are otherwise rejigged for a fresh audience. The Force Awakens arrives in the shadow of SPECTRE, two titles that knowingly acknowledge the phantom of history. Even JJ Abrams comes to the project with an attempt to do something similar for the Starship Enterprise hanging over him. Where both failed, though, Star Wars succeeds: Episode VII is the film that SPECTRE and Star Trek Into Darkness wanted to be.

It's no mistake that original trilogy writer Lawrence Kasdan penned the script, alongside Toy Story 3's Michael Arndt and Abrams. Together, the trio pull off the balancing act that eluded James Bond and Captain Kirk before them: they manage to be self-referential, but also reverential; a mix of awareness and humour, coupled with respect and pathos.

When jokes (and there are many) are made at Star Wars' expense, they're not the meta gags of fans and writers, but are driven by the characters. At one point, former stormtrooper Finn (John Boyega) talks to Solo about how they're going to achieve a seemingly impossible mission. "We can figure it out. We'll use the Force," he suggests. "That's not how the Force works, kid," comes the deadpan reply. When Rey (Daisy Ridley) encounters the Millennium Falcon, she gasps. "It made the Kessel Run in 14 parsecs!" "12," mutters Solo.

The result is a feeling that it's not just the movie mantle being passed down, but also the mythology - and the familiar faces, from Carrie Fisher's Leia to R2-D2 and C-3PO, really are the stuff of legend. Set years after Return of the Jedi, those events have now passed into myth, details forgotten, others misinterpreted. It's telling that each character has their own understanding of what the Force is, and how it relates to them. With Vader gone and Luke Skywalker missing, there's a sea of confusion across the galaxy: good guys are rumoured to have gone bad, bad guys feel the pull of the Light Side on their conscience, and Mark Hamill has a beard. There's a balance to the Force, but not necessarily in how people perceive it.

All of that detail lingers in the background, but it's never spelled out for audiences; this is blockbusting at its smartest and most subtle. It's only natural, for example, that the plot should echo what's gone before, albeit with a twist. For the first time, we glimpse under a stormtrooper helmet. We understand in more detail how a Death Star might work. We have a father figure in the form of Solo's craggy veteran, rather than a Sith or Jedi master. We even get a funky take on a Lightsaber, complete with hand guard.

The backdrop has also evolved: we gaze, open-mouthed, at our first hand-to-hand duel in a snow-filled forest. We witness the new generation of fighter pilots - in the form of Oscar Isaac's Poe Dameron (who wins our affection with only a few minutes of screen-time). And, throughout, we see women everywhere, from officers on the bridge of battleships to leading villain Captain Phasma (Gwendoline Christie). Even our young Jakku scavenger, Rey, shrugs off the sexism of decades ago. "I can run without you holding my hand," she snaps at her male counterpart, as she dashes through the desert.

The newcomers are natural successors to their starring roles. Ridley has the same hopeful optimism, and apparent gifts, that once defined Luke Skywalker - less a "Mary Sue" and more a progressive interpretation of the genre's traditional hero archetype; Adam Driver's Kylo Ren has as much complexity and depth as his stunning hair; John Boyega, meanwhile, moves from Attack the Block to space with the confidence of an experienced Hollywood A-lister, his charisma and natural comic timing fused with a wide-eyed enthusiasm about everything around him. "Did you see that?!" he cries, after shooting a TIE fighter. You can almost hear Han shouting back from 1977: "Don't get cocky."

The key, though, is in making all that seem so effortless. Ford slips back into his roguish boots with undeniable charm, while Fisher easily brings emotion and authority to her combat general. Only a subplot involving R2-D2 feels forced and overly convenient; the rest is as sleek as Abrams' modern visuals, yet grubby as your childhood memories of George Lucas' saga. Through this battered, beaten-up universe coasts a contemporary wave of creativity - led by the gleeful rolling of the droid BB-8. (Only in Star Wars can a piece of metal steal the show without it being a bad thing.)

The result has the feel of something ancient given a rough polish; the kind of recycled vibe that has always defined Star Wars, from the rebuilt Death Star in Episode VI (how fitting that the villains should lack the imagination to come up with a different plan) to Luke's repaired speeder in Episode IV. Episode VII sits right alongside those first three films, both for pacing, plotting and sheer entertainment. It's a hard line to hit, the one between the past and the future, but The Force Awakens nails it like Luke Skywalker hitting a womb rat in a T-16 Skyhopper. Unlike Star Trek Into Darkness and SPECTRE, the only thing it leaves you wanting is more - more details about these new characters and how they fit in. More dogfights. More Lightsabers. And more BB-8. As the box office figures keep on climbing around the world, the hype is only set to do the same. The magical thing? It's true. All of it.