Steven Moffat knows time travel. Steven Moffat knows knockabout, witty romance. So many have approached his new adaptation of 2003 novel The Time Traveler’s Wife (sorry for the US spelling, but for consistency’s stake let’s stick with it) as the perfect marriage of his talents, with fans eagerly awaiting his long-gestating take on the material.

The finished product is slick, funny, clever and sometimes profound – nothing less than you’d expect from a screenwriter with Moffat’s talents. However, it’s also a series that hits some serious stumbling blocks from its first episode, struggling to balance a complex timeline and plot with some weighty topics. It’s a mixed bag, basically, that you might find yourself liking against your first instincts.

From the off, The Time Traveler’s Wife looks like a project tailor-made for Steven Moffat. Not only is it, superficially, a story all about time travel – something the Scottish screenwriter and producer made great hay out of during his years in charge of BBC sci-fi institution Doctor Who – but it’s also based on source material that Moffat knows intimately.

In fact, the DNA of Audrey Niffenegger’s hit novel is wound all the way through Moffat’s Doctor Who tenure (a fact he’s admitted). Her book tells a love story out of order – of a man named Henry (Theo James in this new adaptation) with a rare condition that has him involuntarily time travel to different points in his (or his family members’) lives.

One day, in his “present” he meets a woman called Clare (Rose Leslie) – but she’s known him her whole life, thanks to visits from his “older” self when she was a child. The rest of the book – and the series – puzzles out these meetings, their present-day courtship and their troubled future in a winding, complex timeline that you might need a few graphs to follow.

Anyone who’s seen Moffat’s work on Doctor Who will instantly recognise the parallels with his character River Song (Alex Kingston), the future wife of the time-travelling Doctor who’s horrified that her love doesn’t recognise her (as in 2008’s Silence in the Library/Forest of the Dead).

But when watching the show, it’s also clear how much of this story he also cribbed for his Amy Pond/Eleventh Doctor storyline in 2010, right down to the red-haired little girl waiting for her imaginary friend for years (and 2006’s Girl in the Fireplace, which Niffenegger included in her Time Traveller’s Wife sequel as a nod to the connection).

In other words, it’s clearly material that Moffat finds compelling to explore, and in this retelling he helps the audience see why. Really, it’s a story about nostalgia, how we change for love – and how we don’t. It’s about growing up, and wanting what you don’t have. Really, it’s about almost everything except time travel.

Mainly, though, it’s about Clare and Henry – all of them. We’re all different people throughout our lives, and in The Time Traveler’s Wife that becomes literal. Clare first experiences Henry as a figure another character calls “George Clooney” or “a catalogue model” – a slightly salt-and-peppered hunk with a sweet gravitas and firm hand. He plays checkers with her in the woods, and is essentially her childhood crush who stays handsome forever.

But later she meets his younger self – a more youthful but selfish figure with long hair – and struggles to reconcile the two versions. To become the man she fell in love with, he had to meet her. It’s made all the more challenging by the fact that the older version does sometimes still visit, much to the chagrin of his earlier counterpart (at one point he bemoans “coming in second” to himself).

HBO

If that sounds confusing, you’ve seen nothing yet – this is a series that jumps in and out of different time periods and stages in character’s lives regularly, often within the same scene. But Moffat’s firm hand keeps things clear, partially by a regular motif of Henry and Clare’s respective ages flowing onto the screen, partially with (slightly dicey) hair and make-up choices that emphasise the differences.

James and Leslie’s performances also help sell these time-travel changes. Despite some pretty iffy US accents and wigs (you do get used to both) the central duo keep this thing going through their performances, particularly James who has probably the trickiest job playing so many different versions of the same person (though later, Leslie begins to catch up).

Fundamentally, despite the difficulties of their situation you can see why Henry and Clare stay together. They have chemistry, and shared sensibilities, and a similar sense of humour. They work – even when everything in the universe is trying to pull them apart.

But that relationship is also paradoxically where the show struggles. Since the book’s release, many have commented on the slightly unsettling nature of Clare and Henry’s partnership, with the older man essentially “grooming” a six-year-old girl to become his wife. Moffat tries to tackle this criticism head-on in his adaptation, but with mixed results – and other more challenging parts of the source material also escape his grasp.

HBO

The series’ third episode – one of six made available to press from a total of eight – tackles some very serious issues, and completely fumbles them. The way the story is presented made me genuinely uncomfortable, and more generally the sexual politics of the show might be too iffy for some audience members to get on board with.

Other parts of the series also disappoint in other ways. Perhaps drawing from its status as an adaptation, a lot of the dialogue feels overwritten, especially in cloying “talking head” interviews Clare and Henry have recorded in lieu of the novel’s internal monologues. Episode 1 is particularly tricky to get on board with, though 2 is much better (and 4, featuring a farcical dinner party with two Henries, is a triumph).

The tricky thing with the Time Traveler’s Wife is that like Henry’s life, it’s a bit all over the place. Some scenes and ideas are full five stars – others are one or two. I enjoyed this show a lot, but I also cringed at it, winced at certain line readings moments before I’d laugh at a brilliant joke. It’s all peaks and troughs, highs and lows, like the turbulent relationship at its heart.