科研速览 · Science Skim继续刷下去 · Keep skimming →
◆ Frontiers in artificial intelligence2026-01-01

What really happens when a dev vibes with the code? An empirical study on LLM behavioral divergence in response to expressive code comments.

Angela N Johnson

一句话结论 · In one sentence

These findings position inline comments as model-sensitive latent semantic prompts, with implications for AI-in-the-loop development and design of comment conventions for AI-assisted maintenance.

原始摘要(英文原文)· Original abstract
INTRODUCTION: We investigate how expressive inline code comments written in various developer styles, functional to progressively poetic, philosophical, and misleading, affect large language model (LLM) behavior during code optimization. METHODS: In this pilot study, we used a controlledmerge sort implementation across five stylistic variants and evaluated GPT-5 and Claude Opus 4.1 under standardized console prompts, isolating the effect of embedded comment semiotic variation. Seven expert developers (three senior, four mid-level) scored model outputs against adapted ISO/IEC 25010 criteria and novel LLM suggestibility index (LSI) framework. RESULTS: Semiotic character of comments measurably altered code quality, with consensus-score reliability ICC(2, k) = 0.65-0.81 for six of seven dimensions; single-rater Krippendorff's α = 0.232 reflects substantial interpretive variability. Claude exhibited higher interpretive sensitivity (mean behavioral divergence 4.00; SD 1.16), while GPT-5 maintained stronger architectural fidelity (mean divergence 3.58; SD 1.26). Reflective comments (philosophical, conversational) were associated with Claude's highest maintainability scores in our panel (both M = 4.00, ~8% above stock M = 3.71), while the same philosophical comments reduced GPT-5 maintainability (M = 2.86), suggesting asymmetric model responses to expressive context. CONCLUSIONS: These findings position inline comments as model-sensitive latent semantic prompts, with implications for AI-in-the-loop development and design of comment conventions for AI-assisted maintenance.
读原文 · Read the paper ↗

AI 追问PRO

登录后使用 AI 追问

讨论区

登录后参与讨论

相关论文 · Related

What really happens when a dev vibes with the code? An empirical study on LLM behavioral divergence in response to expressive code comments. — 科研速览 Science Skim