Gavin Heindorf, Harold A Rocha, Tingting Chen, Walter P Vispoel, Peter E Clayson
The effort-doors task was developed to advance the study of individual differences in effort-reward dynamics by eliciting neural indices across multiple stages of outcome monitoring. The task's suitability for between-person inferences depends on the psychometric reliability of the scores it produces. Generalizability theory was used to evaluate the psychometric properties of the raw event-related brain potential (ERP) scores and their subtraction-based composites that are obtained from an extended version of the effort-doors task. An equation for estimating the reliability of a difference-of-differences score was derived and applied to estimate reliability and trial-count requirements at prespecified thresholds in a sample of 160 participants. The raw scores of cue-P3, reward positivity (RewP), and feedback-P3 (fb-P3) all achieved the recommended reliability threshold for between-person analyses. However, an effect of effort was observed only for cue-P3 (i.e., cue reactivity). The stimulus-preceding negativity (SPN) showed inadequate reliability; improved data quality or additional trials may increase reliability, although the projected trial requirement should be weighed against feasibility. Most raw scores from the 120-trial task were sufficiently reliable for between-person analyses. However, SPN reliability was inadequate, effort effects were not credible for SPN, RewP, or fb-P3, and fb-P3 did not vary by feedback valence. These limitations should be considered in relation to theory and alternative paradigms. Overall, the 120-trial version of the effort-doors task can support individual-differences research focused on raw ERP indices across most stages of outcome monitoring, whereas individual differences in effort-related, valence-related, or effort-by-valence modulation should not be inferred from subtraction-based scores without further optimization and demonstrated reliability.