Iterative hypothesis generation for scientific discovery with Monte Carlo self-refining trees

Scientific hypothesis generation is central to discovery, enabling researchers to propose ideas, design experiments, and validate knowledge. Yet, producing hypotheses that are both novel and empirically grounded remains challenging. Traditional methods rely heavily on human intuition, while automated approaches often lack scientific rigor and meaningful validation. This work presents the Monte Carlo Self-Refine Tree (MC-NEST), a general-purpose framework that automates hypothesis generation by integrating Monte Carlo Tree Search (MCTS) with adaptive sampling and iterative self-evaluation. MC-NEST treats hypothesis generation as a structured search problem, dynamically balancing exploration and refinement through strategy selection. We evaluate MC-NEST on a benchmark spanning biomedicine, social science, and computer science. Experimental results show that MC-NEST consistently outperforms state-of-the-art prompt-based baselines across four human-annotated criteria: novelty, clarity, significance, and verifiability. Specifically, it achieves average scores of 2.65, 2.74, and 2.80 in social science, computer science, and biomedicine, respectively, outperforming baseline scores of 2.36, 2.51, and 2.52. These findings highlight MC-NEST’s ability to generate interpretable, scientifically meaningful hypotheses across domains. Furthermore, MC-NEST is designed to support human-AI collaboration through its interpretable tree structure, enabling future integration where domain experts can guide exploration and validate hypotheses transparently and reproducibly. By integrating strategic search, self-refinement, and adaptive validation, MC-NEST offers a scalable and effective path toward responsible, automated scientific discovery.