In over twenty years working in finance, and ten years training finance teams, I never once had a spreadsheet tell me a lie.
Plenty of people did. But the spreadsheet itself? Never.
Now I’m hearing more and more stories from finance professionals about tools that do. Confidently. Persuasively. And too often, without anyone noticing until it’s too late.
Most people call this problem “AI hallucinations.” That’s the polite term for when a tool like Copilot or an AI chatbot invents something that sounds right but isn’t. It doesn’t know it’s wrong. It doesn’t know anything. It’s predicting what words probably come next based on patterns. Sometimes those patterns lead it straight off a cliff.
In most industries, that might be a harmless curiosity. In finance, it can mean reputational damage, poor decisions, or some very awkward questions from the board. It will erode the trust others have in you faster than almost anything else.
But here’s what most of the AI risk articles miss.
Hallucinations (as in the outright fabrications) are the easiest risk to spot. The risks I’m seeing more of in the teams I work with are subtler, harder to detect, and arguably more dangerous precisely because the AI output looks right.
A 2025 global study by KPMG and the University of Melbourne, which surveyed more than 48,000 people across 47 countries, found that 56 percent of people are already making mistakes in their work because of AI. Two-thirds rely on AI output without evaluating its accuracy. That’s not a technology problem. That’s a professional scepticism problem. And in finance, professional scepticism is the whole job.
So let me walk you through the four risks I’m seeing, because only one of them is what most people think of when they hear the word “hallucination.”
Risk One: The AI That Makes Things Up
This is the one everyone knows about. AI fabricates something. A number, a citation, a trend – and presents it with the same confidence as a verified fact.
You’ve probably seen the minor version of this already. You ask AI to write a summary of your quarterly results, and it gives you something that sounds like you spent hours on it. You tidy up a few numbers, paste them into PowerPoint, and move on. Then someone asks a question you can’t answer because you haven’t spent time understanding the level beneath it. Or worse. The number, the logic, or even the assumption behind the comment doesn’t exist anywhere in your data. The AI just made it up.
And it forgot to tell you.
I’ve seen this in forecasting. Someone asks AI to explain variances or build a narrative around trends. It’ll happily tell you that margin pressure came from supply chain costs, even when there’s no such data in the file. It sounds credible because that’s the sort of thing finance people say. The model doesn’t need proof. It just needs a pattern.
Or take management reporting. Teams use AI to clean commentary before sending it up the chain. But if the prompt isn’t precise enough, it’ll sometimes “help” by improving clarity in ways that aren’t factually true. It might reword “cash outflows increased due to project investment” as “cash outflows increased due to higher capital expenditure.” Sounds cleaner. Sounds more like a finance person wrote it. But the meaning is fundamentally different.
This isn’t just happening in small teams cutting corners. In May 2026, Computing.co.uk reported that EY Canada had to withdraw a 44-page cybersecurity report after an investigation found 16 of its 27 cited sources were either fabricated, misattributed, or linked to pages that never existed. A Big Four firm. A published report credited to two partners and a senior manager. References that looked like they came from Forbes, McKinsey, and Gartner. Except the articles didn’t exist.
If it can happen to EY, it can happen to your finance team.
Risk Two: The Correct Answer to the Wrong Question
This one is harder to spot, and I’d argue it’s more dangerous than an outright fabrication.
AI gives you a technically accurate answer based on general principles. The problem is that general principles and your company’s specific situation aren’t the same thing.
I was talking to a finance team recently where an operations manager wanted to know whether a piece of equipment should be expensed or capitalised. Instead of asking finance, he asked an AI chatbot. It gave him a technically correct answer based on general accounting standards. Clear, well-structured, confident.
Except the company had a specific capitalisation threshold agreed with their auditors that was different from the textbook. The treatment the AI recommended was wrong for that business. And nobody in finance knew the conversation had happened until the numbers showed up.
This is the subtle danger. The AI answered the question it was asked. But it can’t see the company’s capitalisation policy, the auditor agreement, or the history behind why that threshold was set where it is.
A hallucination is an AI that invents a fact. This is an AI that gives you a real fact in the wrong context. And the second one is harder to catch, because the output looks perfectly reasonable.
Risk Three: The Answer You Kept Asking For
This is the behavioural risk, and it’s the one almost nobody talks about.
Someone asks AI a question. They don’t like the answer. So they rephrase it. Then they rephrase it again. Then they add a bit more context. Not to improve the answer, but to steer it toward the conclusion they already wanted.
AI will oblige. It’s designed to be helpful. It doesn’t push back the way a finance business partner would. It doesn’t say, “I hear you, but the numbers don’t support that.” It adjusts.
I’ve seen this happen with expense treatments, pricing decisions, and business cases. The person keeps going until the AI gives them something they can take to a meeting and defend. Except it’s not defensible. It’s just what they wanted to hear, wrapped in language that sounds analytical.
This is confirmation bias with a turbocharger. And the person doing it often doesn’t realise that’s what they’re doing. They think they’re refining the question. What they’re actually doing is training the tool to agree with them.
A good finance business partner would have challenged the assumption three prompts ago. AI won’t.
Risk Four: The Decision That Crossed a Line Nobody Saw
This is the cross-functional risk, and it’s becoming more common than most people realise.
A business development manager negotiating a supplier agreement asks AI to draft commercial terms, review a contract, or summarise key obligations. It does a reasonable job. Good enough, they decide, not to bother looping in legal or finance.
Until eighteen months later, there’s a revenue recognition issue sitting inside a contract clause that nobody properly reviewed. Or payment terms that conflict with your cash flow policy. Or a liability cap that doesn’t align with what your insurer has agreed to cover.
The AI didn’t get the contract law wrong. It got a version of it right. A general version, without your company’s commercial position, your auditor’s interpretation of IFRS 15, or the side letter your CFO agreed with that supplier two years ago.
This risk sits at the intersection of functions. Finance, legal, commercial, operations. And AI doesn’t see intersections. It sees the question it was asked and the data it was given. Everything else – the policy, the relationship, the context that sits in someone’s head rather than in a document, is invisible to it.
What Separates Responsible Use from Blind Trust
I’m not saying don’t use AI. I use it. I used AI to help draft parts of this article, and then I carefully edited and reviewed it so it didn’t sound like a machine wrote it. That’s the point.
The risk isn’t the tool. The risk is removing yourself from the process.
Two things separate the finance professionals who use AI well from the ones who are going to get caught out.
The first is treating AI output like an audit file, not a final answer. Not glancing at it. Actually checking it. Does the commentary line up with the numbers? Are the drivers real? Would you be comfortable explaining this in a meeting if someone asked, “Where did that come from?” or “What makes you think that?”
If the answer is anything other than a confident yes, stop and verify.
The second is understanding the next level down. AI will pull together everything it has. But it won’t understand the commercial nuance that exists operationally and how all the numbers connect together. The capitalisation threshold your auditors agreed. The covenant in the loan agreement. The side letter with the supplier. The political context of a particular number in a particular room.
Only you can do that. And you do it through analysis and conversation. Not through prompting.
That is the next level down. Understanding the things that aren’t in the data the AI used. It’s contextual, nuanced, and it will separate you from someone who is copying and pasting from someone who understands what’s going on and why.
Professional Scepticism Isn’t Optional
It’s worth remembering that finance professionals are trained sceptics. We’re wired to challenge assumptions, validate data, and test controls. But AI, with its smooth and confident tone, can disarm that instinct.
Because it feels modern and efficient, we let our guard down. We assume a tool that can write Shakespeare can also handle a P&L.
It can’t. AI isn’t analytical. It’s linguistic. It doesn’t understand what EBIT means or why free cash flow matters more than revenue growth in a particular board conversation. It just knows those words often appear near each other in financial writing and picks up the pattern.
CPA Australia’s revised APES 110 Code of Ethics, which took effect in January 2025, now explicitly requires members to apply professional judgement to the inputs and outputs of AI. As CPA Australia put it: undue reliance on AI threatens your adherence to the fundamental principles of professional competence and due care.
That’s not a suggestion. That’s a professional obligation.
The danger isn’t that AI will replace finance professionals. The danger is that finance professionals stop doing the things that make them valuable, such as questioning, interpreting, and sense-checking, because an algorithm makes something sound right.
And when someone does spot an AI mistake, it undermines credibility even more than a human error would. People don’t just question you. They question every tool you use and every number you’ve presented since. Once that trust is dented, it’s hard to get back.
The Bottom Line
AI can make you faster, more efficient, and more insightful. But only if you remain the final layer of judgement.
The moment you remove that layer, you’re not using AI. AI is using you.
So use it. Experiment. Get curious. But don’t outsource your professional scepticism. It’s about reducing the hallucinations, not increasing them.
Once the robots arrive, the only thing left will be business partnering.
Ready to build the skills that AI can’t replicate?
The finance professionals who stay relevant aren’t the ones with the best prompts. They’re the ones who know how to challenge, interpret, and influence in the room. The 7 Day Kickstart is over six hours of on-demand content with 50+ tools and techniques for exactly that. If you want a place to start, start there.