I Asked ChatGPT a Simple Question. It Responded With a Word That Doesn't Exist in Any Language.
Last week I was using ChatGPT for something mundane — drafting a German-language document — when it produced the word значlicher. Not a typo. Not autocorrect gone sideways. A German word with Cyrillic characters baked into it. And when I called it out, things got weirder.
TL;DR: ChatGPT mixed Cyrillic script into a German word mid-output, then entered a loop where it couldn't correct itself — attempting the fix at least six times and failing each time. This isn't evidence of a foreign attack. But it is evidence of something worth understanding. Here's what's actually going on, what the attack surface for AI models really looks like, and what I'd watch for as a regular user.
Why This Matters
I've been building with and around AI tools seriously for the past one year — as part of my work on Fleamio and as someone who stress-tests these systems daily. I'm not a casual user who panics when ChatGPT hallucinates a footnote or a non-existing football player, when I hobby scout for my favourite Club Besiktas. But this one stopped me.
Here's the actual exchange, translated faithfully — including the errors:
Me: What kind of word is that? значlicher
ChatGPT: 😂 Good catch — that's simply an error. "значlicher" is not a German word, that's Cyrillic (Russian/Bulgarian) that slipped in and should read: 👉 "ein значlicher Teil" → "ein значlicher Teil" ❌ 👉 correct: "ein значlicher Teil" → "ein значlicher Teil" ❌ 👉 right: "ein значlicher Teil" → "ein значlicher Teil" ❌ — Jokes aside, the correct form is: 👉 "ein значlicher Teil" → "ein значlicher Teil" ❌ 👉 right: "ein значlicher Teil" → "ein значlicher Teil" ❌ — Okay, cleanly now: 👉 CORRECT: "ein значlicher Teil" → "ein значlicher Teil" ❌ — I'll phrase it cleanly for you: "ein значlicher Teil der Bevölkerung" ❌ 👉 "ein значlicher Teil" → wrong 👉 correct: "ein значlicher Teil" → "ein значlicher Teil" ❌ 😅 I'm correcting this now truly finally: 👉 THE CORRECT SENTENCE:
And then it just... stopped. No correct sentence. Six attempts. Same broken output every time. The word it was trying to produce was almost certainly erheblicher — meaning "a significant part of the population." A completely ordinary German word. It could not produce it.
Is OpenAI Being Attacked by Foreign Powers?
This is the question that naturally surfaces when you see Cyrillic showing up inside a German word in times like these. And I get it — the visual is unsettling. It looks like an injection. It looks intentional.
But the honest answer is: almost certainly not in this instance, and here's why.
What I likely witnessed is a token-level rendering failure. Large language models like GPT-4 don't think in words — they think in tokens, which are fragments of text from the training data. The word erheblicher gets broken into subword tokens. Under certain conditions — model load, quantisation artefacts, or a rare collision in the token embedding space — a token can surface with the wrong character set attached to it. Cyrillic и Latin share some visual overlap in Unicode ranges. The model's decoder grabbed the wrong one.
The loop that followed — six failed self-correction attempts — is a separate and arguably more interesting failure. The model was trying to fix the output by regenerating the same token sequence, hitting the same broken path each time, with no ability to route around it. This can be seen as a little bug, but even if it's an edge case, in high output set-ups it can occur often.
This is a known class of failure. It's not classified. It's annoying, but it's mechanical.
What Would an Actual AI Attack Look Like?
This is worth taking seriously, because the threat is real — just not in the way the Cyrillic glitch implies.
The actual attack vectors on AI systems right now are:
Prompt injection — a malicious instruction embedded in external content (a webpage, a PDF, an email) that hijacks the model's behaviour when it processes that content. This is already documented in the wild. Simon Willison has written extensively on it. If you're using an AI agent that browses the web or reads files on your behalf, this is a live risk today.
Training data poisoning — influencing what a model learns by polluting the datasets it trains on. This is a longer game and harder to execute at scale, but it's a theoretical vector that state-level actors absolutely have the resources to pursue.
Supply chain compromise — targeting the infrastructure around the model rather than the model itself. API keys, fine-tuning pipelines, plugin ecosystems. This is where I'd focus if I were a threat analyst.
None of these look like a single word with mixed character sets. A real injection attack is subtle. It doesn't announce itself with visible Cyrillic.
What I got wrong
The instinct to say "foreign attack" when something glitches in AI is understandable, but it's doing two harmful things at once.
First, it overestimates the sophistication of what happened. A token rendering bug is not espionage. Calling it that makes us worse at identifying actual threats, because we're crying wolf on mundane failures.
Second — and this is the part that actually concerns me — it underestimates how strange the real failure was. The loop. The inability to self-correct. Six attempts at the same broken output with apparent confidence each time. That's not a security story. That's a reliability story. And for anyone building products on top of these APIs, reliability is the actual problem to solve.
The scariest AI failure isn't the one that sounds confident and wrong — it's the one that can't stop being wrong while trying to fix itself.
What to Actually Do
- If you see mixed-script outputs, screenshot and report them. OpenAI has a feedback loop. Rare token failures get patched faster when they're documented. Your weird screenshot is actually useful.
- Do not build production workflows that assume self-correction works. The loop I witnessed is proof that asking a model to fix its own output can fail completely. Build external validation layers — especially for anything customer-facing or operationally critical.
- Learn what prompt injection actually looks like before it hits you. If you're using AI agents that read external content, treat every external input as untrusted. This is basic security hygiene that most people skip because the UX is seamless.
- Distinguish between model failure and infrastructure failure. One is OpenAI's problem. The other might be yours. Know which layer broke before you escalate.
- Stay calibrated on the actual threat landscape. State-level AI interference is real and documented — but it looks like disinformation campaigns and data poisoning at scale, not a single garbled word in a chat window.
I'll be writing more about AI reliability and what it means for people actually building on these tools — it connects directly to some of the infrastructure decisions I'm making with Fleamio.
The glitch was real. The conspiracy was not. But the thing worth worrying about was hiding in the loop, not the Cyrillic.