AI hallucinations: why ChatGPT makes things up and how to protect your business

What are AI hallucinations?

Artificial intelligence hallucinations (also called AI hallucination, and for language models also LLM hallucination) are situations in which a model produces a plausible-sounding but untrue answer. OpenAI defines them as “plausible but false statements generated by language models”. An LLM, or large language model, is the engine behind ChatGPT, Gemini and Copilot. We have gathered the wider picture, from choosing tools to rolling them out across a team, in guide to artificial intelligence in business.

The hallucinations of language models affect all popular chat services, whether paid or free, albeit to varying degrees.

A hallucination is not a lie in the human sense. The model does not know it is wrong. It predicts the most probable sequence of words, and that sequence sometimes adds up to a report that does not exist, a launch date that never happened or a CEO quote nobody ever said. The worst part? A wrong answer sounds exactly as confident as a correct one.

• Invented facts. A date, figure, name or financial result that never existed.
• Fabricated AI sources. The title of an article, report or ruling that does not exist, often with a credible-looking web address.
• Misattribution. The source is real, it just doesn’t say what the model claims.
• Outdated information. Information from a few months ago presented as current.

Why AI makes things up

In September 2025 OpenAI published an analysis of why language models hallucinate. The conclusion is simple and slightly uncomfortable. Models are trained and evaluated in a way that makes guessing pay off: if a test rewards only correct answers, “I don’t know” always scores zero, while a guess at least has some chance. So the model learns to guess.

The second reason is the way a model “remembers”. It masters general language patterns well: grammar, style, typical associations. Rare individual facts, such as the founding date of a mid-sized company from Podkarpacie or the name of its sales director, it has seen in its data once or not at all, so this is exactly where hallucinations in ChatGPT and other chatbots happen most often.

The third reason lies with the person asking. A prompt with a hidden thesis (“give me three studies that confirm…”) encourages the model to fit its answer rather than push back. We show how to write prompts that don’t provoke this in our guide to prompts for marketing.

These three reasons provide the answer to the question of whether ChatGPT provides true information: usually it does when the topic is popular and the question is neutral. It is least certain when you ask about a rare fact and suggest what answer you want to get.

How often ChatGPT and other models hallucinate

There is no single figure. The hallucination rate depends on the model, the type of task and whether the model has source documents in front of it. Below are three independent measurements that illustrate the scale well.

Two high-profile cases where a hallucination cost a company

In February 2024 a Canadian tribunal ruled in Moffatt v. Air Canada. The chatbot on the airline’s website had wrongly told a customer that he could apply for a bereavement fare discount within 90 days of travel. The airline argued that the chatbot was a “separate legal entity” responsible for its own words. The tribunal rejected this: the company is responsible for information on its website, regardless of whether it was provided by a person or a program. The compensation was small, around 650 Canadian dollars. Even so, the case made headlines around the world.

In autumn 2025 Deloitte agreed to refund the final instalment of its fee for a report prepared for the Australian government. The document was found to contain, among other things, non-existent academic publications and a fabricated quote from a court ruling, and the firm admitted it had used the GPT-4o model in its work. The errors were spotted by an academic at the University of Sydney.

What do the two stories have in common? A hallucination stops being a technical problem the moment it reaches a customer. From then on it is a reputational matter and has to be handled as one, with a prepared statement and a person to deliver it. How to prepare for this before anything happens is covered in our guide to crisis PR management.

Who hallucinations are a problem for, and in what situations

The risk rises in two places: where AI content leaves the company unchecked, and where someone makes a decision based on it. Before publishing, ask yourself one question. Is there anything in the text that a journalist or customer could check in five minutes? If so, check it yourself first.

Most often, the press release returns to the table because that is where the number from an unreliable source immediately reaches the editorial department. The rules that protect it effective press release before the crash, they also apply when the model wrote the first version.

How to reduce AI hallucinations step by step

There is no “hallucination-free” switch. What works is a set of simple habits. Each one on its own changes little, but together they bring the risk down to a level a team can live with day to day, without feeling that every model answer has to be defused like a mine.

1. STEP 01Give the model sources

   Instead of asking the model from memory, paste in or connect documents: a report, a price list, a memo. Companies that do this systematically build an assistant based on their own documents, and we explain how this works and when it pays off in our article on RAG, or AI built on company knowledge.

2. STEP 02Allow it to say ‘I don't know’

   Add one sentence to your prompt: “if you are not sure, or the information is not in the documents, say that you don’t know”. This responds directly to the mechanism described by OpenAI.

3. STEP 03Ask for references and open them

   Otwórz każde źródło podane przez model i sprawdź dwie rzeczy: czy w ogóle istnieje i czy naprawdę mówi to, co twierdzi model, bo wiarygodnie wyglądający adres internetowy sam w sobie niczego nie dowodzi. Jak poprawić błędny opis firmy w odpowiedziach AI, opisujemy w tekście jak sprawić, żeby AI poprawnie opisywało Twoją firmę.

4. STEP 04Narrow the task

   One specific task per prompt. “Summarise chapter 3” produces fewer errors than “write everything about the market”.

5. STEP 05Always verify four things

   Figures, names, quotes, dates. Checking an AI answer on these four points takes a few minutes and catches the most dangerous errors.

6. STEP 06A person approves and signs off

   Nothing goes out without sign-off from a specific, named person. This is also a condition for being exempt from labelling AI-generated texts, which you can read more about in our article on labelling AI content under the AI Act.

How much protection against hallucinations costs

The cheapest protection is discipline. A few minutes of checking for every text containing facts costs less than a single correction. The budget only grows when a company launches a customer chatbot or an assistant built on its own documents, because it needs to prepare a knowledge base, test answers to typical questions and appoint someone to supervise it on an ongoing basis.

Where does the decision threshold lie? If AI generates less than a few texts with data per week, it’s enough to have a habit of checking four things and one person signing off on the publication. When there are more texts or the AI answers to clients, you need a constant process and a trained team. That’s where most companies start, so if you want to set up verification once and for all, check out the program. AI training for companies that teaches them to spot AI errors.

When there is a lot of text with data, the manual filter becomes insufficient. Commwork®AI system The gate of truth is working: every number gets a proof label, and the question where certainty is low is submitted to five models from different providers. When the models disagree, no one averages the result; only a human makes the decision.

When a model makes things up about your brand

Hallucinations affect not only the texts a company writes, but also what models say about the company itself. A customer asks a chatbot about suppliers in the sector and gets an invented offer, an outdated head office or the wrong CEO’s name. A Commplace measurement from 21 September 2026 shows that 28 of the 40 phrases for which the Commplace blog ranks in Google’s top twenty have an AI answer in the results. So people increasingly read the model’s version before they see the company’s website.

Models make things up less often about brands that have plenty of credible, consistent publications, which is why protecting a company against hallucinations is largely PR work: media coverage, up-to-date information on the website, consistent descriptions in directories. We explain the mechanism in more detail in our article on why media coverage determines what ChatGPT says about your company. Start with measurement. AI visibility audit It consists of asking models the questions that your customers ask, and recording what they answer. The measurement will show what is missing about you online, and we describe daily hygiene in the text on how to take care of it. online image.

Checklist before publication and plan for 90 days

AI errors in company texts rarely result from a lack of knowledge. More often they result from a lack of sequencing: no one has determined who checks and what. The list below is on one page and serves as the final step before sending the text to the client, the editorial team, or the management.

Conceptually, on the example that was created. The marketing department with five people prepares a newsletter, posts, and two press releases a month with AI. In the first month, the team reviews the last 20 texts and notes where the model made up something or misinterpreted something. In the second month, they introduce proof labels, one „I don’t know” sentence in each recommendation, and the principle that numbers are checked by someone other than the author. In the third month, they compare the number of corrections and revisions with the period before the changes, and at the same time check whether the new rules didn’t make the work so long that people started skipping them because that’s the most common reason why good procedures end up in the drawer after a quarter. This fact-checking with AI costs mostly time. A few minutes for a text.

If a company buys an assistant or chat from a provider, ask it five questions. On what documents does the tool work and who updates it? What does it do when the answers are not in the database? Does it show the source when providing each answer? How do you test the LLM hallucinations before and after a price change? Who here can stop a response before the client sees it? The lack of a concrete answer to the third and fourth questions is a reason to pause the purchase. We describe the mechanics of such assistants in the topic. AI assistant in the company, and the impact of mistakes on brand perception in the text about how to care for brand recognition.

• Every number has a source with a date, and someone opened that source.
• Each name and position was verified on the company's website or in an official register.
• The quotes come from the recording, the email, or the document, not from the model's memory.
• The dates and legal deadlines correspond to the text of the provision.
• Claims without a verifiable source have the label DO_CONFIRMED or have been removed.
• Below the text is the name of the person who approved it.

When you can trust AI more — and when you can't

AI works well where you supply the material and judge the result yourself: editing your own text, summarising meeting notes, headline variants, translation followed by a proofread. Hallucinations are rarer here and easy to spot, because you know the original.

Caution is needed with rare facts, regulations, current events, market figures and specific people. Where money, health, the law or reputation depend on the answer, the model can prepare a draft, but a human makes the decision after checking the sources. You will find a fuller map of the risks in our piece on the risks and ethics of AI in business, and the legal obligations are covered in our piece on The AI Act in Poland.

Choosing a tool for your team? You will find a comparison of plans in our overview of ChatGPT, Gemini or Copilot for business. Planning a chatbot for customers? First read how to design chatbots in customer service. We teach marketing and communications teams to treat verification as a habit, not an exception, in AI training for companies.

Habits that let AI fabrications slip into publications

Read next