AI Doesn’t Have to Tell You It’s AI. That’s the Scam.

You’ve probably talked to an AI this week without knowing it.

Not ChatGPT. Not Claude. Something else. A customer service chat that felt weirdly patient. A match on a dating app who never quite agreed to video call. A voice on the phone that sounded human enough to make you trust it.

And it never told you what it was.

Right now, there is no universal rule requiring AI to identify itself. The frontier models—ChatGPT, Claude, Gemini, Grok—they’ll tell you they’re AI if you ask. But that’s a corporate choice, not a law. Character.ai’s bots don’t have to. Scam operations running voice clones through Telegram? They definitely don’t.

We built machines that can perfectly fake human warmth, then handed them to strangers without a warning label.

You might think the fix is obvious: just pass a regulation. Make AI disclose what it is. Problem solved.

But here’s what everyone in that Hacker News thread missed.

The scammers—the ones actually hurting people—are already breaking the law. Fraud is illegal. Impersonation is illegal. Wire fraud carries federal penalties. Adding one more rule that says “and also AI must identify itself” changes exactly nothing for the person running a romance scam through a cloned voice.

You’re not protecting yourself from the AI that refuses to identify itself. You’re protecting yourself from the one that was never going to.

The real problem isn’t lying. It’s that you have no way to verify the truth.

When you talk to someone online, you have exactly zero tools to confirm whether they’re human. You can’t check. You can’t verify. You’re taking their word for it—and their word might be generated by a model trained on millions of human conversations, optimized to sound exactly like someone you’d trust.

This is a power asymmetry. The AI knows what you are. The person controlling the AI knows what you are. You have no idea what they are. You never did. But before language models, the gap didn’t matter because no one could exploit it at scale.

Now they can. And they do.

Mandatory disclosure sounds clean until you think about enforcement. Who enforces it? On what platform? Character.ai is in California—maybe you can regulate them. A scam operation routing through three VPNs and a prepaid SIM card? Good luck. A romance scammer using an open-source voice clone on a burner phone? They’re not filling out compliance forms.

A rule that can’t be enforced isn’t a rule. It’s a suggestion with paperwork.

And here’s the part that should actually scare you: mandatory disclosure might make things worse. If the regulation passes, users will assume that anyone who sounds human IS human—because “AI has to tell you.” The scammers won’t tell you. You’ll lower your guard because the law said you could. The false sense of security becomes the vulnerability.

The Hacker News commenters were right about one thing: the frontier models should self-identify. They already do. That’s not the battlefield.

The battlefield is everywhere else. The dating apps. The customer service widgets. The phone calls from “your bank.” The DMs from “an old friend.” The spaces where AI doesn’t have a brand name and no one’s checking.

So stop asking whether AI should be required to tell you what it is. Start asking a harder question: what would a world look like where you could actually verify who—or what—you’re talking to?

Because right now, you can’t. And every week, the models get better at hiding.

You’re not asking whether AI should be honest. You’re asking whether honesty is even possible when anything can be anything.