Real or Fake Is the Wrong Question to Ask an AI Detector

Every AI detector I’ve tested gets sold as a lie detector, and that framing is the single biggest reason people misuse them. Pangram’s Max Spero has been making a version of this argument publicly, and after spending time with these tools for reviews on this site, I think he’s right in a way that’s inconvenient for the entire category — including his own product.

The pitch buyers hear is binary. Paste text, get a verdict, act on it. Teachers act on it by failing students. Editors act on it by killing pitches. Hiring managers act on it by dropping candidates. That workflow assumes the tool answers “real or fake,” and it doesn’t. It answers a much narrower statistical question, and the gap between those two things is where the damage happens.

What the tool is actually measuring

Spero’s framing of the detection problem is worth sitting with. Large language models keep getting better, and as they pull from a wider body of research and text, the signal detectors rely on gets harder to isolate. That’s not a temporary bug that a patch fixes. It’s the direction of travel. Every generation of model narrows the statistical distance between machine-written prose and the human-written prose it was trained on.

Spero has also said most detection tools can catch copy that’s been run through online “humanizer” services. That’s a genuinely useful data point, because humanizers are the first thing anyone reaches for when they want to beat a detector. But notice what that claim is and isn’t. Catching a laundered output is a pattern-matching win. It is not the same as certifying that a clean piece of text came from a human brain.

False positives are the whole ballgame

Pangram’s stated focus is reducing false positives, and that’s the right thing to optimize for. It’s also the thing most buyers never ask about. In my experience reviewing tools, people compare detectors on headline accuracy and ignore the error type that actually ruins someone’s week.

A false negative means a machine-written essay slips through. Annoying, low stakes, self-correcting over time. A false positive means a person who wrote their own work gets accused of cheating. There is no symmetry there. One is a miss, the other is an accusation. Any vendor that reports a single accuracy number without breaking out false positive rate is telling you what they want you to hear.

Pangram’s own comparison content, including its side-by-side against Turnitin, leans on false positive rates and ESL bias as the axes that matter. I’d normally be skeptical of a vendor choosing its own scoring criteria, but in this case the criteria are the correct ones. If you’re evaluating detectors and your spreadsheet doesn’t have a column for how often the tool flags non-native English writers, your spreadsheet is incomplete.

Detection is not fact-checking

The most useful line in Pangram’s technical material is one that runs against its own commercial interest. Their report states plainly that AI detection is not a substitute for, or a reliable tool for, proving whether text is factually true. AI does get used for disinformation and scams, but “this was probably machine-generated” and “this is false” are unrelated claims.

That distinction is being flattened everywhere. Spero has pointed out how much AI content now saturates social platforms, LinkedIn especially. The reflex is to treat a detector as a truth filter for the feed. It isn’t one. A human can write nonsense. A model can produce something accurate and well-sourced. Origin and veracity are separate questions, and any workflow that collapses them into one score is building on sand.

How I’d actually use one

Here’s where I land after testing these tools against real copy:

• Treat the score as a signal that prompts a conversation, never as evidence that ends one.
• Ask any vendor for false positive rates before you ask about overall accuracy.
• Check performance on non-native English writing specifically, because that’s where bias shows up.
• Keep human review in the loop on anything with consequences attached — grades, employment, publication.
• Never use a detector to judge whether a claim is true. That’s a different job entirely.

The uncomfortable conclusion is that the better these tools get, the more careful institutions need to be about how they deploy them. High accuracy invites automation, and automation is exactly what you don’t want when the failure mode is a false accusation against a real person.

Credit where it’s due: a detection vendor whose public message is “this is hard, we’re reducing errors, and you still need humans” is more trustworthy than one promising certainty. Certainty isn’t on the menu. What’s on the menu is a probability estimate that’s getting better while the problem underneath it gets harder. Use it accordingly.

🕒 Published: September 3, 2026