A review of automated approaches to race condition detection

Abstract Automated symbolic analysis for race condition detection is critical for identifying vulnerabilities in consumer off-the-shelf software, where source code is unavailable. However, the triple-threat of symbolic state explosion, thread interleaving complexity, and semantic information loss in binaries often renders these techniques impractical for production scale security analysis. This paper evaluates seminal race condition detection approaches over the last 2 decades against a novel taxonomy which helps determine how they address the challenges mentioned. Through a systematic review of 34 contemporary approaches, we identify a significant disconnect between tool capability and evaluation rigor. Our analysis reveals that 74% of included approaches do not verify results against a verifiable ground truth, while all of those that do, utilize insufficient sample sizes ( $$\approx 10$$ ≈ 10 binaries on average) for performance claims. Furthermore, we find that fewer than 10% of tools provide a comprehensive concurrency analysis that accounts for the semantic nuances of binary-level execution. Additionally only 2/34 propose a method for handling semantic loss in binary formats. This paper concludes by proposing a few points towards improving reporting, testing and development of binary symbolic race detectors.