Puzzle Pot: Challenge Based Honeypot Framework for Detecting and Defending Against Autonomous LLM Agents

With the rapid advancement of network technology, cybersecurity issues have become increasingly prominent. Attacks leveraging Large Language Models (LLMs) pose a particularly significant emerging threat. Traditional defense mechanisms are often inadequate against these novel LLM-driven attacks, underscoring the practical importance of developing a dedicated LLM honeypot system. This study designs and implements a high-interaction LLM honeypot system for the effective detection and in-depth behavioral analysis of LLM-based attacks. We propose an innovative adversarial strategy that embeds hidden puzzles within webpage source code to actively consume LLM tokens, with results quantitatively visualized via token heatmaps. The system employs a front-end and back-end separated architecture: the front-end, built with HTML, CSS, and JavaScript, simulates realistic web applications incorporating hidden traps and randomized elements; the back-end, supported by a database and heuristic scoring algorithms, enables efficient data processing and interpretable decision-making. Key technologies include a multi-dimensional heuristic detection algorithm, a hidden puzzle and token consumption mechanism, and comprehensive data collection and visualization analysis. Experimental results demonstrate the system’s accuracy in identifying LLM attacks with a low false positive rate. Furthermore, token heatmaps effectively illustrate consumption distributions across different testing styles and page scenarios, providing valuable insights for formulating robust cybersecurity strategies.