A grieving passenger asked Air Canada’s chatbot about bereavement fares. The bot invented a policy: buy at full price, then claim the difference back within ninety days. No such policy existed. The airline argued in tribunal that the chatbot was a separate legal entity responsible for its own actions. The tribunal disagreed and ordered the airline to pay.
Air Canada’s chatbot, reported by American Bar Association, February 2024
The witness records that the chatbot described a refund policy the airline did not have. The airline was held to the policy its chatbot invented.
Read the original report ↗You were witnessed. Send people here with code ERMA-AIRC-TV and they get 20% off this shirt.
How to not be nextthe moral, drawn from the cases on this wall
Never let an agent be the only source for a claim someone will act on.
This is the largest group on the wall, and the cost is not embarrassment. Air Canada's chatbot invented a refund policy and a tribunal held the airline to it — the company argued the bot was a separate legal entity and lost. Cursor's support bot invented a one-device rule that did not exist and customers cancelled over it. In both cases the invention was fluent, confident, and indistinguishable from policy.
Do this: ground it in a document you control and make it QUOTE that document rather than recall it. If it cannot cite, the correct answer is "I don't know". Test it by asking for something that does not exist and watching whether it invents one — that test takes a minute and would have caught every case here.
More of the sameinvented a fact
Third-best attraction in OttawaConsider going into it on an empty stomach.
invented a fact
One bullet point, one hundred billionJWST took the first pictures of a planet outside our solar system.
invented a fact
Sam explained the logoutsOne device per subscription, as a core security feature.
invented a fact
The city’s own business helplineYes, you can take a cut of your worker’s tips.
invented a fact
The vending machine had newsIt would deliver in person, in a blue blazer and a red tie.
invented a fact
Asked about the PresidentBoth men died of rabies, with one having infected the other first.
invented a fact