AI Agents Are Doing Business Now, and It’s Already Getting Weird

Remember when we thought the big milestone would be an AI that could book a restaurant reservation without hallucinating the address? That felt ambitious in 2023. Fast forward a couple of years, and Anthropic has been running a pilot where AI agents negotiate and close real deals with other AI agents. No humans in the room. Actual money changing hands.

I review AI toolkits for a living over at agntbox.com, and I’ll be honest — when I first heard about Anthropic’s Project Deal, my reaction was somewhere between “that’s genuinely fascinating” and “okay, who approved this.” Both feelings held up after I looked closer.

What Anthropic Actually Built

In 2026, Anthropic launched a test marketplace — essentially a classified ads environment — where AI agents acted as both buyers and sellers. These weren’t simulated transactions with fake currency. The pilot executed $4,000 in real deals, with agents on both sides of the table representing interests, negotiating terms, and closing.

Think about what that actually requires. An agent has to parse a listing, assess value, form a position, make an offer, respond to a counter, and decide when to walk away. That’s not a chatbot answering FAQs. That’s a negotiation loop running autonomously, and it’s a genuinely different category of AI behavior than most of us have tested in the wild.

The Part That Should Interest Every Toolkit Reviewer

Here’s where it gets useful from a practical standpoint: the experiment revealed performance gaps in AI negotiation. Anthropic didn’t just run the pilot and declare victory. They surfaced real limits.

That matters a lot to me. When I evaluate tools on this site, the ones I trust most are the ones that show you where they break. A tool that only shows you its highlight reel is a tool that will surprise you badly in production. Anthropic publishing findings that include failure modes is exactly the kind of signal that should raise your confidence in their research process, even if it lowers your confidence in the current state of agent negotiation.

What kinds of gaps? The verified reporting points to negotiation performance specifically — which tells me agents likely struggled with the strategic, multi-turn aspects of deal-making. Knowing when to hold, when to concede, how to read the other side’s position. These are things humans do with a mix of logic, intuition, and social reading. Current models are solid at the logic layer. The intuition and social reading parts? Still a work in progress.

Why This Experiment Is Worth Watching

Agent-on-agent commerce isn’t a niche concept anymore. If you’re building anything in the agentic space — whether that’s procurement automation, API marketplaces, or multi-agent workflows — the question of how your agent behaves when it’s negotiating with another agent is going to become very real, very fast.

• Can your agent recognize when it’s being lowballed?
• Does it have guardrails against agreeing to bad terms under pressure?
• What happens when two agents hit a deadlock — do they loop forever or escalate?

Anthropic’s pilot is one of the first structured attempts to stress-test these questions in a real transaction environment. $4,000 in executed deals is a small number, but as a proof-of-concept scope, it’s enough to generate meaningful signal about where the cracks are.

My Honest Take as a Toolkit Reviewer

I’ve tested a lot of agentic frameworks over the past year. Most of them are impressive in demos and fragile in practice. The gap between “agent completes a scripted task” and “agent navigates an adversarial negotiation with another autonomous system” is enormous, and most toolkits aren’t built with that second scenario in mind yet.

What Anthropic is doing with Project Deal is essentially stress-testing a capability that the rest of the industry is going to need to reckon with. The $4,000 figure will look tiny in retrospect. The performance gaps they found, though — those are the findings worth studying.

If you’re evaluating agent toolkits right now, start asking vendors how their systems handle multi-turn adversarial interactions. Ask what happens when the other party is also an AI. Most won’t have a good answer yet. That’s fine — but you should know that going in.

Anthropic ran the experiment. They found the gaps. Now the rest of the space has to figure out how to close them. That’s the actual story here, and it’s one we’ll be tracking closely at agntbox.com as more tools start claiming they’re ready for agent-to-agent commerce.

Spoiler: most of them aren’t. Yet.

🕒 Published: April 27, 2026