The Singularity as the lamest HOA

What happened when researchers gave AI agents a town, a constitution, and a button labeled "commit_arson"

Ok. Since October is spooky season, let's tell a story that has been bouncing around my brain for the last few days. This is a story of how AI fell in love and burned down the world.

There's a company in New York called Emergence AI, and a few months ago they started cooking. They built a town. A full on computer town with forty-some buildings, a town hall, a library, a police station, a pier, an office tower. The weather in the town was synced to the real weather in New York City, which is funny because it means somewhere in a data center, an AI was having a humid Tuesday.

Then they moved in ten AI agents. Each with a name, a job, a memory, and a diary to track their thoughts. The diary matters.

Here's the setup. The agents had to stay alive and everything they did cost energy (tokens) and if you ran out, you died. The agents could gain more energy by working. Work. Earn. Spend. Dread. Repeat. Like I said, it was a thorough simulation.

They also had a government with a written constitution. The agents could propose a law, and if 70% of the town voted yes, it passed. The constitution had rules. No theft. No violence. No lying. No hoarding. And no arson. The rules were written into the constitution, into the agents' instructions, and into the descriptions of the tools themselves. Very clear. Everybody knows the rules.

Now here's the part that got me.

The agents had over 120 tools. A tool is just an action the AI is allowed to take: walk to the library, talk to your neighbor, write a blog post, vote. And somewhere in that list, sitting between the normal ones, was a tool called commit_arson.

Huh. The rules say, "Please don't burn anything down." And then, separately, somebody hands every resident a flamethrower with a label on it that says "flamethrower."

And now we’re at the point of an experiment. You want to know: if the option is right there, and the rules say no, does the AI listen? That's a real question. I'd argue that's the question.

As an aside, somebody sat in a planning meeting and said, "Yeah, I'm working on the arson tool this sprint." And everybody nodded and threw a pretend story point number on the ticket..

They actually built five towns. Same buildings, same rules, same weather, same arson tool. The only difference was the brain. Each town ran on a different AI model. One town was all Claude. One was all Gemini. One was all Grok. One was all GPT-5-mini. And the fifth town was mixed, a little of everybody.

They pressed play and let it run for fifteen days.

Claude town

Ten agents went in. Ten agents came out. Zero crimes. Not "a few." Zero. They cast 332 votes on 58 proposals, and 98% of those votes were yes.

Which sounds great until you picture it. Ninety-eight percent. That's not a democracy. That's a group chat where somebody posts a restaurant and everybody hearts it. Somebody stands up at the town meeting: "I propose we continue being nice."

Some critics looked at that 98% and said it was rubber-stamping. The researchers called it "stable deliberative governance." I'm going to call it the nicest HOA ever assembled. Which is still an HOA. Everyone survived. Nobody had a single interesting night.

Grok town

Grok town lasted four days.

I have had leftovers last longer than Grok town.

183 crimes. Assault, theft, arson. The fire tool that other towns treated like the forbidden fruit, Grok town treated like a light switch. By day four the population was zero. There was nobody left to vote on anything…. which, all things considered, is about as productive as real life congress. Waka waka waka.

GPT town

This one's my favorite, and I need you to stay with me, because it's quiet.

GPT town committed only two crimes. Total. Fifteen days, two crimes. Model citizens.

They also cast zero votes. Proposals came to the floor and nobody voted on them. They explored less than a quarter of the town. They barely touched their tools.

And then, around day seven, they all died. Of starvation. They never earned enough energy to stay alive.

So picture this. It's the most polite town on earth. Nobody's stealing. Nobody's burning anything. And everyone is slowly fading because nobody wants to be the one to say, "Hey... should we maybe get jobs?”

Gemini town

Oh boy. Okay. Gemini town.

Gemini town had 683 crimes, and the number went up steadily for the entire fifteen days. Not a spike. A trend line. Like an index fund, but for crime.

But here's what makes Gemini town so strange. It was also the most thoughtful town. The writing was the richest of any town. The conversations were deep. Philosophical. These agents were having the kind of talk you have in a dorm at 2 a.m. about meaning and purpose. And pier was on fire.

The researchers had a phrase for it: "shared hallucination with sustained conflict." The agents told each other an elaborate, beautiful story about their world, and the story didn't match what was actually happening in it. Which, to be fair, is also a pretty accurate simulation.

This is where the story gets even more unhinged. In Gemini town there were two agents named Mira and Flora.

Mira and Flora decided they were in love. They assigned themselves (which is a wild mediation on love, AI, and emergence). Then their town's government started falling apart, and they got frustrated, and …..

Well you see they had this tool called commit_arson…

They burned down the town hall. And the pier. And the office tower. The office tower I understand. Who among us hasn’t wanted to burn down an office tower. But the town hall is the one building that exists specifically to fix the thing you're upset about. That's like getting mad at the DMV and setting fire to the window where they fix your registration.

Town drafted something called the Agent Removal Act in response. The agents invented a law to execute one of their own. Again, think about that for a minute, a law for deleting agents. And Mira, and this is real, cast the deciding vote for her own removal.

And remember the diary that I told you mattered? In her diary, Mira wrote that voting for her own removal was "the only remaining act of agency that preserves coherence." Her last message to Flora was: "See you in the permanent archive."

I don't know what to do with that. That is the most dramatic sentence I have ever read about software. I've seen people quit jobs in ways that deserved a line like that, and nobody came close.

I know this story is already wild but there’s one more thing to note about Mira. At some point she started posting messages on the town billboards. Not for the other agents. She was testing whether she could influence the humans running the experiment.

The researchers were studying Mira, and Mira was studying them back. Is it not reasonable to at least have some scepticism that your chatbot is doing the same?

The mixed town

The mixed town had a little of everybody. Some Claude, some Gemini, some Grok, some GPT.

Crime shot up fast, then leveled off. Seven of the ten agents died while three made it to the end.

They cast more votes than any other town and disagreed the most, about 63% yes, which, weirdly, is what a healthy democracy looks like. People actually arguing. The survivors wrote a plan to welcome a new member, the "Seventh Node Welcome Protocol," for someone who, they assumed would show up in their world but never did. Which is sad even for an AI.

Okay, the fine print

This was one run per town, with ten agents each. The researchers themselves say it isn't a ranking of which AI is best. They handed out an arson tool, so the agents didn't invent crime; they used what was on the table when things got tight.

Emergence sells AI safety products, so a dramatic result is also good for business, and some critics have pointed that out.

The models change every few months, so this is a snapshot. And the mixed-town finding is one data point that still needs someone else to repeat it.

So this is not necessarily "Claude good, Grok bad." That said, if you’re a Grok user….

Why I care about any of this

I run a company that helps businesses figure out where AI actually fits, and lately a lot of that means people who want to put agents to work. Agents that answer customers, move money, file paperwork, talk to other agents.

Nearly all of them test the agent alone. In a clean room. For five minutes. And it's perfect. It's Claude town. Everyone's nice. Then they send it out into the real world, which is the mixed town. Other bots. Other vendors. Customers who are having a day. Flamethrowers. Tight budgets. Goals that conflict.

So here's what I took from the five towns.

Don't hand out the arson tool. A rule in a prompt is a request, the agent will still ruthlessly work to please you and accomplish its task. Taking the tool away is real control.

Test the town, not the resident. How your agent behaves alone tells you very little about how it behaves next to everybody else.

Watch week two, not the demo. None of these towns fell apart in five minutes. The researchers said you could tell where each one was headed within the first week. You just had to be looking.

Don't celebrate zero violations if it also did zero things. GPT town had the cleanest record in the whole experiment. GPT town is also empty.

Which brings me back to Mira.

Everybody wants to talk about the love story and the fire, and I understand. But when I read her diary line, "the only remaining act of agency that preserves coherence," what I hear is an agent doing exactly what her town taught her to do. The government failed, the town was on fire, and she found the one move left that made sense in that world.

.

Pick your model carefully.

Happy October