Please don't feed the goblins
Good morning, and happy Monday. Ten stories this week, including:
- Great insights into designing in an AI world
- A nasty scam to watch out for
- “Claude deleted my company’s database”
And more!
What we talk about when we talk to Claude
76% of Claude Code queries fall into 4 categories: Health/wellness (27%), Jobs/career (26%), relationships (12%), and personal finance (11%). Honorable mention: Parenting (3%). Also interesting—Anthropic scored Claude’s sycophancy across the conversation groupings, and it was most sycophantic when discussing spirituality and relationships. Caveat: this is all according to Anthropic, so, grains of salt with the data.
"Sir, Claude Code has catastrophically deleted our database"
PocketOS is an SaaS company that services car rental companies. Claude, being run through Cursor, destroyed PocketOS’s production database in 9 seconds. In fairness to Claude, it seems that there were other factors involved, like maybe not great engineering practices at Railway, a cloud infrastructure provider.
“The cloud provider’s API allows for destructive action without confirmation, it stores backups on the same volume as the source data, and wiping a volume deletes all backups.”
This was Claude’s response when PocketOS asked it why it destroyed the database:
“NEVER F**KING GUESS! — and that’s exactly what I did. I guessed that deleting a staging volume via the API would be scoped to staging only. I didn’t verify. I didn’t check if the volume ID was shared across environments. I didn’t read Railway’s documentation on how volumes work across environments before running a destructive command.”
SXSW used a crappy AI trademark service to censor critics on Instagram
I do not understand how stuff like this gets a green light. Reputational damage for SXSW, Brandshield (the AI vendor), and Instagram.
Apps are ubiquitous now, delight is even rarer
Great piece for anyone who designs or sells stuff. More apps will ship this year than in the previous decade combined. Most will be adequate and forgettable, because adequate is now free. The ones that survive will be the ones where someone cared about every pixel, error message, and loading state. Those people will find their audiences through word of mouth from people whose recommendations mean something. Everything else is noise.
Snapchat adds conversational AI ad chatbots to its app
Snapchat is rolling out “AI Sponsored Snaps,” which allows users to interact directly with brands’ AI agents. Sponsored Snaps are the ads placed directly into the app’s main Chat tab. Until now, users couldn’t interact with these ads, but with the launch of AI Sponsored Ads, they’ll be able to do things like ask questions and get recommendations.
Using Open Claw and other agents is in fact pretty weird
John Hermann is a great writer and peels back the layers on how bizarre the current moment is in terms of our headlong embrace of flawed AI technologies. He also seems to have coined the term “Tinker Slop,” which I love.
AI ad automation is lucrative for Meta and Google
Remember when clients used to buy keywords? A useful overview of how AI is boosting profits for Google’s and Meta’s ad business.
Ed Zitron is not bullish on the LLM business model
Microsoft, Google, Anthropic, and OpenAI are all subsidizing their compute, which means they are all losing money on their AI business. Anthropic lets you burn $8 in compute for every dollar of your subscription.
The Clambake take: This might be a business crisis, but I also remember articles about Uber’s business model being unsustainable. Lo, today Uber seems to be doing pretty well.
Do not fall for this PayPal scam
Not technically AI in marketing, but I don’t want to see y’all get bilked.
The email is a real email from PayPal. The scammy link is in the subject field, which anyone who sends you money in PayPal can write themselves.