Chatbots ≠ Search Engines

Here’s a mini-rant I contributed to Semafor’s tech newsletter. (Sign up here! It’s free!)

Chat bots aren’t search engines, and tech companies need to stop hinting that they are!

An Australian mayor is threatening to sue Open AI because ChatGPT apparently named him — falsely — as the guilty party in a bribery scandal.  A law professor was named — also falsely — by ChatGPT in a list of legal scholars who had sexually harassed someone, citing an also non-existent Washington Post article.

None of this should be surprising. That’s because chatbots don’t have any real ability to discern fact from fiction. They have no concept of verification. What they do is process language, and they do that very well. They’re idiot savants of language. Request lyrics to a song about nuclear physics in the style of Taylor Swift, and you’ll be astonished.

And yet there’s something seductive about a human-like interface that seems to be an oracle of information and can answer any question we pose.  Because chatbots have the same simple, single-box interface of search engines, we expect the same kind of responses from them.

We should know better, and the user interfaces should help more.

Bard, the chatbot recently released by Google, has a small disclaimer under its input bar: “Bard may display inaccurate or offensive information that doesn’t represent Google’s views.” ChatGPT has much the same thing: “ChatGPT may produce inaccurate information about people, places or facts.” And Bing Chat is a tad more coy: “Bing is powered by AI, so surprises and mistakes are possible. Make sure to check the facts, and share feedback so we can learn and improve!”

It’s good that the disclosures are there, but they’re easy to miss. And more importantly, they imply that mistakes — ”hallucinations,” as they’re called — may happen from time to time, when in fact they should say, “please don’t count on this chatbot for any facts.” These disclosures are like an AI image generator saying, “the images you see may not represent the real world.”

But until that changes, the onus is on us to keep reminding ourselves: chatbots aren’t search engines.