What if I told you that right now, your face could be flagged as a criminal’s—and you’d have no idea until the handcuffs clicked?
That’s exactly what happened to a Tennessee woman who found herself arrested for crimes committed in North Dakota, a state she claims she’s never even visited. The culprit? AI facial recognition technology that police relied on to make the arrest.
As someone who tests AI tools for a living, I’ve seen plenty of systems that overpromise and underdeliver. But this case hits different. We’re not talking about a chatbot giving you bad restaurant recommendations or an image generator messing up someone’s hands. We’re talking about a grandmother sitting in jail because an algorithm got it wrong.
The Fargo Mistake
According to multiple news sources, Fargo police used facial recognition software to identify a suspect in a fraud case. The system pointed to our Tennessee woman. Police made the arrest. She was jailed. And then—oops—turns out it wasn’t her.
The Fargo police chief has since apologized for the mistakes in this AI-aided arrest. But an apology doesn’t give back the time spent behind bars, doesn’t erase the humiliation, and doesn’t fix the fundamental problem: law enforcement is deploying facial recognition tools without fully understanding their failure rates.
Why This Matters for AI Tools
I review AI toolkits every week. I test accuracy rates, check for bias, push systems to their limits. And here’s what I know: facial recognition isn’t magic. It’s math. And math can be wrong.
The accuracy of these systems varies wildly depending on lighting conditions, camera angles, image quality, and—most troublingly—the demographic characteristics of the person being scanned. Study after study has shown that facial recognition performs worse on women and people of color. These aren’t edge cases. These are systemic flaws.
When I test a project management tool and it crashes, someone misses a deadline. Annoying, but fixable. When facial recognition fails in law enforcement, someone loses their freedom.
The Real Cost of “Good Enough”
Police departments are adopting these tools because they work most of the time. And “most of the time” sounds pretty good when you’re trying to solve crimes with limited resources. But “most of the time” means there’s a percentage of cases where innocent people get swept up in the net.
This Tennessee woman is that percentage. She’s the margin of error made flesh.
What bothers me most as a toolkit reviewer is that this technology is being deployed in high-stakes situations without the same scrutiny we’d apply to, say, medical devices or aviation systems. Imagine if airplane autopilot systems were accurate “most of the time.” We’d never fly.
What Should Happen Next
First, facial recognition should never be the sole basis for an arrest. It should be one data point among many, requiring human verification and corroborating evidence before anyone gets handcuffed.
Second, police departments need to be transparent about which systems they’re using, what the known error rates are, and how they’re training officers to interpret results. If I can publish detailed reviews of AI tools with accuracy metrics and failure modes, law enforcement can too.
Third, there needs to be accountability when these systems fail. Not just apologies, but actual consequences and compensation for people whose lives are upended by algorithmic errors.
Testing vs. Trusting
My job is to test tools so you don’t have to trust marketing claims. I run the numbers, document the failures, and tell you what actually works. What I’ve learned is that AI tools are powerful, but they’re not infallible.
The problem with facial recognition in law enforcement is that it’s being trusted before it’s been adequately tested in real-world conditions with real-world consequences. And when the test fails, it’s not the algorithm that pays the price—it’s people like this Tennessee grandmother.
So next time you hear about AI making our streets safer or helping police solve crimes faster, remember this case. Remember that behind every “match” is a human being whose life could be turned upside down by a false positive.
Your face is yours. It shouldn’t become evidence of someone else’s crime just because an algorithm said so.
🕒 Published: