How good are AI chatbots at recommending bike routes? Reassuringly awful, it seems
First, the good news. Ignore the alarmist hype about artificial intelligence. It is not going to ‘destroy civilisation’. Climate change will get us first.
That said, AI is clearly a big deal. It will affect life profoundly, like other paradigm-shifters such as the motor car, plastic, the internet, disc brakes and the like.
AI chatbots such as Microsoft’s ChatGPT and Google’s Bard (and now, Elon Musk’s Grok) are familiar to many.
In astoundingly assured English, instantly and free, they answer questions with the slick confidence of a grifter politician.
Their immediate distillation of the millions of words on the subject they find online, into text of length and style you specify, is very impressive.
But superficially. Because, like a populist politico on TV, it’s vague, boilerplate stuff. Worst of all, chatbots make things up and most readers won’t know the truth from cobblers.
York’s city walls are a must-ride
I know about cycling in East Yorkshire, where I live. So I asked ChatGPT and Bard to write some web pages, blog posts and magazine articles about it.
The results were at best fuzzy, misleading and useless; at worst, utter nonsense and in fact worse than useless.
For example, both chatbots recommended York’s city walls as a must-ride. Nope.
They’re medieval walkways (dating back to, er, Victorian times): physically impossible to cycle on, never mind legally or practically.
Similarly, both programs suggested cycling around York city centre, not to mention that’s not allowed during the day, when ‘Footstreets’ regulations ban it.
Not that you can ride up the bolt-shearingly cobbled Shambles anyway: it’s crowded with selfie-snapper tourists.
Bard thinks you can cycle around York Museum Gardens. Oh no you can’t. ChatGPT maintains you can’t cycle over the Humber Bridge. Oh yes, you can: it’s the second-longest cyclable single-span bridge in the world.
About East Yorkshire generally, both were full of baloney. Bard picked out the Wolds Way as its top ‘cycle route’.
Um, that’s a long-distance footpath, illegal for bikes (it also listed the 140-mile Wolds Cycle Route – fair enough, although it claims it’s only 50 miles.)
ChatGPT blatantly made up a ‘Pocklington Canal Railway’ heritage line that takes bikes. No such thing exists. Bard suggests you ride to Driffield for its ‘racecourse’ (there isn’t one) and so it went on…
Amid all the random hammering, they occasionally hit the nail on the head. ChatGPT mentioned York Knavesmire’s racecourse perimeter road that forms an informal velodrome: that’s good.
It also pointed out York’s excellent Solar System bike route – but doesn’t know there are two (the other is in York Uni).
But any writer submitting stuff like this would be advised to find alternative employment, such as delivering takeaways. Which would offer more job security, at least.
Does it matter if AI gets it wrong?
Okay, okay. It’s fun, and easy, to catch out ChatGPT and Bard on one’s own specialist subject. But does it matter?
Yes it does. Because some websites have already started using AI chatbots to write their content.
Given a choice between commissioning a knowledgeable, professional writer, and a chatbot that produces crap, but instantly and free… well, we know what would usually win.
Sadly, we have the prospect of millions of web and print pages being generated this way.
Pages that are smudged, watered-down, summarised versions of existing pages.
Pages that will in turn serve as the basis for future chatbot larceny, and so propagate their reinforced untruths even more.
Maybe there’s hope. Genuine content written by actual people may become more precious.
Cycling recommendations written by humans who’ve done the routes and rides, tried the bikes and made the mistakes so you don’t have to, can rise above the auto-generated textual morass.
Well, maybe. Meanwhile, if you haven’t, play with ChatGPT, Bard and the rest. Grill them on a topic you know about. Get a feel for how they work – and how they fail.
Then you’ll be able to tell the genuine article from copy generated on the cheap – understand and adapt. That’s what I call intelligence.