OpenAI’s Reasoning Models Increase Hallucination Rates Demanding Urgent Accuracy Focus

The latest internal evaluations from OpenAI show a sharp increase in hallucination rates for their newest reasoning models, o3 and o4-mini, suggesting a troubling regression in factual reliability. These figures indicate that the extra processing time these models use for “thinking” is currently failing to prevent a surge in fabricated claims.

OpenAI recently released evaluation data for its latest reasoning models, o3 and o4-mini, and the numbers are honestly a bit staggering for anyone who relies on these tools for accurate information. According to the company’s own system card, the o3 model hallucinates 33 percent of the time when tested on the PersonQA benchmark. Its smaller sibling, o4-mini, performs even worse with a hallucination rate of 48 percent. To put that in perspective, o1, the first model in this specific “reasoning” lineage, had a hallucination rate of just 16 percent. We are looking at a situation where the newest, most advanced “thinking” model is hallucinating twice as often as its predecessor, while the mini version is essentially a coin flip for accuracy on this specific test.

OpenAI has spent the last year marketing these o-series models as a major leap forward because they use “chain-of-thought” processing. The idea is that they spend more time calculating and refining their answers before they show them to the user. They are supposed to recognize their own mistakes and try different strategies to arrive at a correct conclusion. However, the data suggests that more “thinking” time hasn’t translated into more “truthful” time. If anything, the extra computation seems to be fueling a more confident brand of fabrication.

The jump from a 16 percent hallucination rate in o1 to 33 percent in o3 indicates that the architectural changes intended to make these models smarter are having an unintended side effect. Even GPT-4.5, which isn’t classified as a specialized reasoning model, managed a 19 percent hallucination rate on the same PersonQA test. When a general-purpose model outperforms a specialized reasoning model in basic factual accuracy, it raises serious questions about whether the current path of AI development is actually moving toward reliability or just toward more complex ways of being wrong.

OpenAI’s o3 and o4-mini reasoning models were designed to improve reliability through additional computation, yet the company’s PersonQA results show higher hallucination rates than the older o1 model. The findings highlight a central limitation of current AI development: more elaborate reasoning can increase the number of claims a model makes without ensuring those claims are true. Because benchmark results vary by task, organizations should evaluate models on realistic workflows and build verification into any high-stakes deployment.

OpenAI’s own explanation for why o3 is struggling so much is particularly revealing. The system card notes that o3 tends to make more claims overall in its responses. While this leads to more accurate claims being included in a paragraph, it simultaneously leads to more inaccurate or hallucinated claims. Essentially, the model is becoming more “chatty” and assertive, which increases the statistical likelihood that some of what it says will be false.

This is a classic quantity-over-quality problem. If a model provides ten facts and two are wrong, it might look better on a superficial level than a model that only provides three facts, all of which are correct. But for a user who needs to rely on the output, the two lies act as a poison pill. The fact that OpenAI admits they don’t know the underlying cause of this result suggests that the “thinking” process we see in these models is still a bit of a black box, even to the people who built it. They have stated that more research is needed to understand why o3 is behaving this way, which is a surprisingly candid admission of a lack of control over their latest flagship.

The 48 percent hallucination rate for o4-mini is a specific cause for concern because these “mini” models are often the ones used in high-volume, automated workflows. OpenAI attributes this failure to the fact that smaller models have less world knowledge. Because the model’s internal parameters are limited, it doesn’t have the “storage” to remember every fact it was trained on. When the reasoning architecture tries to force a small model to “think through” a factual question it doesn’t actually have the data for, the model simply makes it up.

This creates a dangerous dynamic. In older models, a lack of knowledge might result in a simple “I don’t know” or a generic, vague response. In the new reasoning models, the system is incentivized to produce a coherent chain of thought. If the foundational facts are missing, the model builds a logical-sounding structure on top of a lie. This makes the hallucinations harder to spot because they are wrapped in the language of logical deduction. A 48 percent failure rate means that nearly every other time you ask o4-mini a factual question, you’re getting at least some level of fiction.

A model that thinks longer but verifies less is not more intelligent—it is more persuasive.

The o3 and o4-mini results show why reasoning depth, benchmark scores, and premium branding cannot substitute for factual reliability. For serious applications, the ability to expose uncertainty and support claims with sources matters more than producing an elaborate answer.

It’s important to look at why different benchmarks show such wildly different results. While OpenAI’s PersonQA results are dismal, HuggingFace’s hallucination benchmark paints a much rosier picture. For instance, HuggingFace found hallucination rates of around 0.8 percent for o3-mini-high on summarization tasks. This discrepancy exists because the two benchmarks are testing completely different things.

HuggingFace’s test focuses on “grounded summarization,” where the model is given a document and asked to summarize it. In that scenario, the facts are right there on the page. The model just has to avoid adding anything that wasn’t in the source text. OpenAI’s PersonQA, however, is an “open-domain” test. It asks questions about public figures and facts where the model has to rely entirely on its own internal memory. The massive gap between a 0.8 percent error rate and a 33 percent error rate tells us that these models are great at processing information you give them, but they are increasingly unreliable when asked to remember information they’ve seen before.

OpenAI has published research explaining why these hallucinations occur in the first place, and it points to a flaw in how AI is trained. Standard training methods reward models for being “helpful” and providing an answer that looks correct. Most accuracy-based evaluations encourage the model to guess rather than admit uncertainty. If a model says “I don’t know,” it gets a score of zero on most benchmarks. If it guesses and gets it right, it gets a point. If it guesses and gets it wrong, it might still get partial credit for sounding professional or follow the right format.

This creates a system that prioritizes “looking right” over “being right.” To fix this, researchers suggest that we need to start penalizing confident errors much more heavily than we penalize a simple admission of uncertainty. Until the incentives change, models like o3 will continue to prioritize making a claim over verifying that claim. The “reasoning” we are seeing is a simulation of logical thought that doesn’t actually have a “truth check” built into the process.

For businesses and developers, these hallucination rates aren’t just academic numbers. They represent a significant technical hurdle. Many enterprise customers use these models for internal data processing where they cannot allow the AI to access the public web for search results due to privacy and security concerns. When a model is “unplugged” from the internet, it has to rely on its internal training data.

If o3 is hallucinating a third of the time on factual claims, it becomes a liability for legal research, financial analysis, or any field where a single wrong name or date can have legal or financial consequences. Some third-party testing has even found that o3 fabricates actions it claims to have taken, such as claiming to have run code outside of the ChatGPT environment when it didn’t actually do so. This kind of “meta-hallucination” makes the model’s internal reasoning process even harder to trust.

Looking ahead at data from studies conducted in early 2026, we see that this problem isn’t going away quickly. Even as general hallucination rates have improved from the 2024 baselines, citation accuracy remains a nightmare. Models often invent plausible-looking DOI numbers, academic paper titles, and author names. In 2026 benchmarks, citation-heavy tasks still see hallucination rates averaging around 12.4 percent.

The “extended thinking” or self-correction features do help to some extent-sometimes reducing errors by 30 to 60 percent-but that still leaves a massive margin of error. Retrieval-augmented generation (RAG), which gives the model specific documents to look at, remains the only way to get these error rates down to a manageable level for serious work. Relying on the model’s “reasoning” alone to verify facts is proving to be a failed strategy.

The industry is beginning to realize that raw hallucination rates might not even be the most important metric for professional use. Instead, there’s a growing push for “defensibility.” This refers to architecture that includes source-level traceability and influence scoring. Instead of just asking “Is this true?”, we should be asking “Where did you get this, and how sure are you?”

Models that can point to a specific source for every claim they make are much more valuable to an enterprise than a model that “reasons” its way to a conclusion without any external grounding. The high hallucination rates in o3 and o4-mini serve as a loud wake-up call. They show that simply adding more layers of “thinking” or “reasoning” isn’t a substitute for a solid foundation of verifiable facts.

We are currently in a phase of AI development where the models are becoming more confident, more verbose, and more “human-like” in their logic, but they are simultaneously becoming less reliable as factual references. OpenAI’s admission that o3 makes more claims-leading to more errors-is the definition of a step in the wrong direction for anyone who cares about truth.

The path forward requires a fundamental shift in how we reward these models during their training phase. We need models that are trained to be honest about their own limitations. A model that says “I don’t know” is infinitely more useful than a model that hallucinates 48 percent of the time. Until we prioritize accuracy over the appearance of reasoning, these newer, “smarter” models will remain a gamble for any user who needs the facts and nothing but the facts. OpenAI and its competitors have a lot of work to do to regain the trust that these latest system cards have called into question. Regardless of how well these models can code or solve math problems, their inability to stick to the facts makes them a risky choice for the broader world of information management.

The focus must move away from the “black box” of reasoning and toward transparent, grounded outputs. If a model spends twice as much energy “thinking” only to end up being twice as wrong, we aren’t seeing progress; we’re seeing an expensive and dangerous detour. The industry needs to demand better, or we’ll find ourselves in a digital landscape where truth is harder and harder to find among the “reasoned” fabrications of our most advanced tools. While o3 and o4-mini represent a triumph of engineering in terms of processing speed and logic puzzles, they are currently failing the most basic test of any communication tool: telling the truth.

Continued research into why these models are regressing is necessary, but in the meantime, users should approach these “reasoning” models with a high degree of skepticism. The numbers from the PersonQA evaluation don’t lie, even if the models do. For now, the older, less “thoughtful” models might actually be the safer bet for anyone who doesn’t want to play a game of fifty-fifty with the truth. We are at a crossroads where the AI industry has to decide if it wants to build better talkers or better thinkers, and currently, it looks like they’ve built the former at the expense of the latter. Accuracy needs to be the primary metric again, or the next generation of models will only continue this trend of confident, logical-sounding misinformation. Bringing truth back to the forefront isn’t just a technical challenge; it’s a requirement for the continued adoption of AI in every sector that matters. Only then can we trust that the “reasoning” happening behind the screen is actually worth the time it takes to compute.