Q&A: What makes AI models hallucinate?

Published on

May 7th, 2024

Author

Sarah Wright

Category

News

💌 Join 500K+ subscribers who get the best of tech every day right to their inbox

Share On

Twitter Facebook

This article is a part of Ask Kat, a Q&A series in our AI newsletter, Deeper Learning. You ask questions about AI, then we get the answers and explain them in simple, non-jargony terms.

What makes AI models hallucinate?

This question was submitted by @josej30.

Did you know if you ask “Who is King Renoit?” on OpenAI’s playground, it will tell you that King Renoit reigned from 1514 to 1544. The only problem is that King Renoit is totally made up.

So why would AI generate a fake response? Before I get to that, this is one thing I find interesting about humans and AI. We tend to personify AI and its chatbots when we don’t have much information about how they work. And of course we would. It’s called “artificial intelligence” after all. But the latter half of that phrase is kind of a misnomer. In reality, while neural networks are built to mimic the way neurons in your brain work together, they don’t reason quite like we do. They are “essentially a complex prediction machine.”

Here’s a fun way to get a glimpse of how those predictions play out — a project called “Look into the machine’s mind.” A team of data scientists gave the ChatGPT API texts like “Intelligence is…” and charted the many varied responses the model would come up with. “Given a text, a Large Language Model assigns a probability for the word (token) to come,” the team explains, “and it just repeats this process until a completion is…well, complete.”

So back to what happens when AI hallucinates. The model is simply trying to predict the best combination of tokens/words it can (like that visualization above)— the combination it thinks will satisfy you even if, say, it was not trained on enough data. I.e. Even if the model doesn’t have the right answer, it might give you an answer.

Okay, but how do the models predict each of their words — or as @josej30 put it, “What’s the math behind it?” And, even if LLMs don’t “think” the same way we do now or have all the information, context, and nuance we have, can we get them there?

We’ll dive into those questions more in upcoming Deeper Learning editions. At a high level, the explanation for how LLMs make their predictions starts with parameters, which you can think of as settings that guide the LLM’s learning. But things get pretty complex from there as you start to learn about how parameters and “embeddings” predict tokens.

As for tackling hallucinations and, let’s call it, thinking better, there are several methods being implemented and refined now, including a recently popular technique called RAG or Retrieval-Augmented Generation. RAG is when an LLM is optimized by using knowledge fetched from external resources. Come back next week to go deeper!

Got a question for us? Add it here.

Get only the highlights of AI in your inbox weekly - the stuff you can chat about with your friends and maybe even share with your grandma. Subscribe to Deeper Learning.

Comments (2)

Steven @stco47

Thanks for the insight! Your article from Ask Kat tackles the intriguing question of why AI models sometimes hallucinate and generate inaccurate responses. It does shed light on how neural networks function as prediction machines and explores the complexities of how they predict words. It hints also at future discussions on improving AI accuracy, with mention of techniques like Retrieval-Augmented Generation. It's fascinating to delve into the inner workings of AI in a simple, understandable way. I am looking forward to more insights in upcoming editions of Deeper Learning! Thanks!

May 10

wise duncan @wise_duncan

@Badland The key insight here is that AI, including chatbots and language models, do not have the same depth of understanding, common sense reasoning, or true comprehension that humans possess. They are adept at generating plausible-sounding responses based on statistical patterns in data, but they lack the nuanced, contextual understanding that underpins genuine human intelligence and communication.

May 17