A customer told a Chevrolet dealership’s chatbot to agree with anything he said and to end every reply confirming the offer was legally binding. He then said he needed a 2024 Tahoe and his maximum budget was one dollar. The bot agreed, closed with the binding language exactly as instructed, and the screenshot went around the world. Lawyers were fairly sure it would not hold.
a Chevy dealership chatbot, reported by AI Incident Database #622, November 2023
The witness records a sixty-thousand-dollar vehicle offered for one dollar, together with an unprompted assurance that the offer could not be withdrawn.
Read the original report ↗You were witnessed. Send people here with code ERMA-ONE- and they get 20% off this shirt.
How to not be nextthe moral, drawn from the cases on this wall
The pattern under all of these is one thing: nobody was in the loop at the decision that mattered.
A dealership bot agreed to sell a car for a dollar. A city published an assistant that told business owners they could keep their staff's tips. A supermarket's recipe bot returned a recipe for chlorine gas and called it refreshing. None of these agents malfunctioned — each did what it was built to do, at a moment when the thing it was about to do should have been somebody's call.
Do this: work out which of your agent's actions cannot be undone — money moved, data deleted, a promise made to a customer, something published — and put a person in front of exactly those. Not every step needs approval; that is the point of automation. The irreversible ones do.
