Substack is now punishing AI-generated text

The punishment isn't algorithmic. Substack doesn't throttle your reach. They don't flag you, suspend you, or remove your posts.

They did something more effective. They gave every reader a button.

Since 21 July, anyone can scan your text and get an estimate of how much you wrote yourself and how much came from a machine. In an industry whose entire business model rests on someone trusting your voice, that's a harsher punishment than any demotion. Readers cancel subscriptions for less.

And it isn't going to work. Here's why.

What actually launched

Substack is partnering with the detection company Pangram. The details matter:

The scan happens on request. There's no permanent AI-percentage sitting under your article. The result is only shown to whoever clicked.

It applies to text over a hundred words, published on or after 21 July.

You get your own tools. Run the detector on your draft before publishing. Add a "How I make this" section explaining how you work. Report a result you think is wrong.

It sounds milder than it is. An invisible button a reader can press at any time is still a verdict that can fall at any time, and you'll never know how many people pressed it.

The backdrop is Pangram's own measurements. On LinkedIn, over forty per cent of longer posts are flagged as entirely AI-generated. Substack came out best among the platforms measured, but just over a fifth of posts were still flagged. CEO Chris Best writes bluntly that he doesn't intend to wait until the Substack app has become LinkedIn.

He calls the problem "Claudefishing". That is, when you think you're building a relationship with a human being and there's no one there. His point isn't that AI is bad, Substack uses it themselves. The point is the gap between what the reader thinks they're getting and what's actually delivered.

A fair diagnosis.

Why the detector doesn't hold up

An AI detector is a model that has been shown enormous amounts of text where the answer key is known. This one was written by a human, this one by GPT. From that, it learns to recognise patterns in word choice, rhythm and structure.

Here's the uncomfortable part: Pangram doesn't know exactly what its own model is reacting to. Founder Max Spero has said as much in interviews. The model recognises something, but what can't be pinpointed. You get a number without an explanation, from a system that can't account for its own reasoning.

Pangram states 0.01 per cent false positives. One in ten thousand. Impressive.

Hold that against reality. Tech journalist Taylor Lorenz had a piece in Vanity Fair flagged as AI-generated. She denied it, Spero reviewed the case and admitted the tool was wrong. When three Wall Street Journal editorials were flagged, editorial page editor James Taranto called the tool a smear machine.

That case got extra messy, because two of the writers had actually used a chatbot to edit parts of the text. Which brings us to the real question: is a text a human wrote and a model polished AI-generated?

For Pangram, there's a difference between assisted and generated. For the reader looking at a percentage, there's no difference at all.

The two camps haven't moved in eight months

In November, Andrej Karpathy, former OpenAI researcher and one of the industry's most listened-to voices, said you will never be able to detect AI in submitted work. No caveats. The reason is structural: whoever generates the text and whoever is supposed to expose it are competing in the same game, and the generator gets there first.

Spero responded the same day that "doomed to fail" is a misunderstanding. He pointed to independent evaluations and to the fact that Pangram recognised GPT-5 immediately on launch, without having been trained on the model.

Eight months later, Substack builds the tool in. No new study settled the question. They picked a side in an ongoing debate, which is a reasonable decision, but it was a choice, not a conclusion.

So, what does this mean for you?

Here's the thing no one in the debate is talking about.

No reader has ever felt deceived by a text because a hidden marker in the language triggered an API call. She read three paragraphs and found nothing that only you could have written.

It shows without any tool. We've been making that judgement our whole lives.

Three things I react to myself:

1. Nothing you can picture. If the first sentences don't contain a name, a number, a place or a situation, the rest is usually just as generic.

2. Everything is balanced by its opposite. Every claim gets a caveat. Every perspective gets a seat. No one takes a stand. It's often a way of avoiding having an opinion and hiding it behind structure.

3. Words no one says out loud. "Navigate the complexity." "Plays a crucial role." The test takes two seconds: would you say that sentence to a colleague over coffee?

You catch all three by reading your text aloud before publishing. It takes ten minutes. Almost no one does it.

What you do tomorrow

If you publish on Substack: use the draft scanner so you know what the reader sees, and fill in "How I make this" if you use AI in your process. Then you trade a number for an explanation, which is worth considerably more.

But don't put the energy into passing a test. Put it into someone reading all the way to the end.

Because that's where the moat is. Substack's detector will point at human text as AI and let AI text glide past. That doesn't say anything about how well built Pangram is. It says something about trying to prove a crime that leaves no trace.

And while everyone else argues about percentages, you sit there writing something that actually says something.

That's still the one thing that can't be automated.