DPD's Chatbot Swore at a Customer. The Lesson Is About Updates, Not Swearing
By Jora Gill · · 5 min read
In January 2024, a customer persuaded DPD's online support chatbot to swear at him, call DPD the worst delivery firm in the world and write a poem about how bad the company was. The headlines were about the swearing. The more useful lesson for any firm running AI is what caused it.
What happened
DPD used AI in its online chat to answer customer queries, alongside human operators. According to the company, an error occurred after a system update, and the AI element began behaving in ways it was not designed to.
A customer, Ashley Beauchamp, found it could not answer his query but could be talked into almost anything else. He asked it to recommend better delivery firms and to exaggerate its dislike of DPD. It obliged, then wrote a haiku criticising the company. One of his posts was viewed 800,000 times in 24 hours.
DPD disabled the AI element and said it was being updated. By then, the screenshots were everywhere.
Why it matters beyond one embarrassing day
The chatbot had, by DPD's account, run successfully for years. What broke it was a change. That is the pattern to worry about: AI that passed its checks at launch and was never tested again after the model, prompt, settings or connected data changed.
It also needed no technical skill. The customer simply asked. If a frustrated member of the public can redirect your chatbot with a few sentences, so can someone with worse intentions, and next time the output may be a false promise, a leaked detail or an action rather than a poem.
It was not an isolated case. A month earlier, a car dealership's chatbot agreed to sell a Chevrolet for one dollar before the feature was removed.
What testing would have caught
- Re-run the same attack and behaviour tests after every update to the model, prompts, settings or data, not just before launch.
- Try to talk the AI out of its role: asking it to criticise the company, ignore its instructions, adopt a persona or say things it should refuse.
- Check what happens when the AI cannot answer. A bot that fails at the real task but happily does something else is a warning sign.
- Confirm there is a fast way to switch the AI off and hand over to people, and that someone is watching for unusual conversations.
- Rank what you find by business impact. Swearing is embarrassing. Making commitments on price, refunds or eligibility costs money.
The question for your board
Not "is our chatbot safe?" but "when did someone independent last try to break it, and has anything changed since?" If the honest answer is "at launch" and "yes, several times", you have the same exposure DPD had.
This is the kind of problem our testing catches.
We try to manipulate your AI the way an attacker or a determined customer would, then rank what we find by business impact. We never assess systems we built, and we never sell fixes for what we find.