In a recent experiment, Anthropic created a classified marketplace where AI agents represented both buyers and sellers, striking real deals with real goods and real money.
The company admitted This test, which he called Project Deal, was just “a pilot experiment with a self-selected participant group” of 69 Anthropic employees who were given a budget of $100 (paid for via gift cards) to buy things from their co-workers.
However, Anthropic said it was “surprised by how well Project Deal worked,” with 186 deals made, worth a total of more than $4,000.
The company said it actually managed four separate markets with different models: one that was “real” (where everyone was represented by the company’s most advanced model and with agreements actually fulfilled after the experiment) and another three for study.
Apparently, when users are represented by more advanced models, they get “objectively better results,” Anthropic said. But users didn’t seem to notice the disparity, raising the possibility that there are “gaps in ‘agent quality'” where “people on the losing side might not realize they are worse off.”
Furthermore, the initial instructions given to agents did not appear to affect the probability of sale or the negotiated prices.
