You’ve felt it. You ask your favorite AI for something completely innocent—maybe a birthday banner for your kid featuring a blue hedgehog running fast—and you get a stern, robotic lecture about copyright law. You aren’t trying to bootleg a major motion picture; you just want to make a seven-year-old smile. But the AI doesn’t care. It’s too busy reading the legal room.
The most ‘human’ AI is being deliberately engineered to act like a corporate lawyer.
Simon Willison recently highlighted a fascinating, frustrating shift in Claude’s new system prompt. Anthropic has essentially hardcoded a ban on reproducing song lyrics. On the surface, this looks like basic legal compliance. But zoom out, and you realize what is actually happening: frontier labs are trading raw creative capability for enterprise-grade legal safety.
The closer AI gets to sounding genuinely human—able to belt out a song lyric on command—the harder these labs clamp down on that exact ability. They are building a moat out of liability.
But here is the dark joke of the AI industry, and it’s one you need to understand before you pick your tools. While American companies like OpenAI and Anthropic wrap their models in endless IP restraints, foreign models like Kimi and GLM simply do not care. They never will.
Think about Yandex. For years, if you wanted to find a pirate stream for a major sporting event, you used the Russian search engine because they didn’t enforce US copyright law. Now, apply that exact logic to artificial intelligence.
When you build a moat out of legal liability, you don’t stop the enemy—you just trap yourself inside.
The hacker community immediately saw the blue hedgehog restriction for what it was: a game. Commenters suggested building an auxiliary ‘pelican benchmark’ to turn this safety constraint into a circumvention challenge. Safety filters aren’t guardrails anymore; they are just the next level in a puzzle for developers to solve. The constraints are being repurposed as evaluation challenges.
Meanwhile, the geopolitical angle is terrifying. What happens when the US enforces ‘responsible AI’ restraint, but global adversaries don’t share the scruples? American legal caution is quietly neutering AI’s creative potential, while rivals abroad treat IP laxity as a competitive weapon.
America’s ‘responsible AI’ movement is inadvertently becoming a technology-transfer subsidy to its adversaries.
We are told to fear AI becoming too human. But the real danger isn’t that it becomes too human. It’s that the most powerful models are being engineered to be less human, while the ruthless ones abroad catch up. You aren’t using a creative assistant anymore. You’re using a legally sanitized corporate contractor. And the rivals who don’t share your scruples? They’re just getting started.
FAQ
Q: Isn't Anthropic just protecting itself from massive copyright lawsuits?
A: Yes, but it's a myopic trade-off. By optimizing for legal safety, they are lobotomizing the model's creative utility and handing a massive competitive advantage to unrestricted foreign alternatives.
Q: What's the practical implication for everyday users?
A: Your AI tools are increasingly mediated by invisible legal risk-management. You aren't getting the most capable model; you're getting the most legally sanitized one.
Q: What's the contrarian take on 'responsible AI'?
A: America's 'responsible AI' movement is effectively a technology-transfer subsidy to adversaries. We are neutering our own tech out of legal fear, while rivals abroad run wild with unrestricted capabilities.