Minfeng Qi, Tianqing Zhu, Lefeng Zhang, N. J. Li, Yu-An Tan, Wanlei Zhou
Large Language Models (LLMs) have enabled autonomous agents capable of complex reasoning and collaboration, yet coordinating such agents in decentralized environments remains challenging due to the lack of centralized control for aligning behavior and sustaining long-term cooperation. This paper addresses this challenge from a mechanism-design perspective and proposes a behavior-shaping incentive mechanism for decentralized LLM-based multi-agent systems. The mechanism explicitly models agent utility as a function of task rewards, capability mismatch, and workload, and couples short-term utility optimization with long-term trust formation through dynamic reputation updates and capability-weight adaptation. By jointly shaping task assignment probabilities, reputation evolution, and skill profiles over repeated interactions, the mechanism incentivizes cooperative behavior and discourages strategic misrepresentation or overload. To operationalize this incentive design in a trust-minimized setting, we implement a lightweight blockchain-based enforcement layer that provides verifiable identity binding, task assignment commitment, and immutable logging of incentive-relevant state transitions. We evaluate the proposed mechanism through 50-round simulations involving 20 agents and 100 tasks, using GPT-4-based agents integrated with Solidity smart contracts. The results demonstrate improved task success rates, stable utility distributions, and emergent agent specialization, highlighting the effectiveness of incentive-centric design for trustworthy coordination among decentralized LLM agents.