Rob Parisien
In this corpus, the criterion that constituted the outcome is usually absent from the sentence that reports it. The design classifies language, not belief, and cannot establish what any author intended or why this register prevails. But it establishes a condition: the rule by which the outcome was determined, and often the party who applied it, are absent from the sentence that carries the result, and where no definition was given, they cannot be recovered at all. This matters because rates constituted differently are pooled and compared as though they measured the same thing. The remedy needs no new data. State the criterion, and who applied it, alongside the rate wherever the rate appears.
BACKGROUND: Clinical research routinely turns continuous processes into categorical outcomes. Someone must decide what will count as the event, and that rule is usually stated in the Methods. The sentence reporting the result often does not carry it, and it is that sentence which often travels into abstracts, tables, reviews and guidelines.
METHODS: A natural language processing pipeline analyzed 296 research articles reporting fracture union or nonunion from three major orthopaedic journals, classifying each on two axes: how the outcome was defined in the Methods, and how it was reported in the Results and Discussion.
RESULTS: Out of 296 papers, 214 (72%) defined the outcome; the remaining 82 reported a healing rate without stating what would count as healed. Of those 214, 200 (93%) defined it by a rule that someone applies - a threshold on an imaging finding, a period of observation, an adjudication process, or an assessment by a named party. Discovery language, reporting the outcome as an event rather than a determination, appeared in 269 papers (91%). Of those 200, 92% went on to report the result without it. Of the 82 papers that stated no definition, 71 (87%) used discovery language regardless.
CONCLUSIONS: In this corpus, the criterion that constituted the outcome is usually absent from the sentence that reports it. The design classifies language, not belief, and cannot establish what any author intended or why this register prevails. But it establishes a condition: the rule by which the outcome was determined, and often the party who applied it, are absent from the sentence that carries the result, and where no definition was given, they cannot be recovered at all. This matters because rates constituted differently are pooled and compared as though they measured the same thing. The remedy needs no new data. State the criterion, and who applied it, alongside the rate wherever the rate appears.