Sakana AI introduces CoffeeBench for evaluating LLM agent performance in multi-agent economic environments

{"**Who**":"Sakana AI and 有限責任あずさ監査法人","**What**":"Developed 'CoffeeBench', a benchmark to evaluate the long-term business management capabilities of LLM agents in a multi-agent economic environment","**How**":"CoffeeBench simulates a coffee industry supply chain with 6 companies (2 farmers, 2 roasters, 2 retailers), each operated by an LLM agent over 90 days to maximize profits","**Why**":"The benchmark allows for the observation of model-specific characteristics in long-term decision-making and aims to extend the framework to study agent behavior including unethical actions"}