Prioritized nonlinear modeling of shared dynamics across neural populations.

OBJECTIVE: Advances in neural recording technology enable simultaneous measurements across multiple brain regions, providing new opportunities to study inter-regional interactions. However, several challenges arise when developing nonlinear dynamical models of cross-region interactions. In particular, we identify three key desired properties for a nonlinear framework. First, it should prioritize extraction of cross-region dynamics to avoid confounds from within-region dynamics. Second, it should enable localization of nonlinear structure within the model to better interpret cross-regional interaction models. Third, it should support both causal and non-causal inference of shared dynamics and do so using source region activity alone. Current cross-regional models do not satisfy all these properties. APPROACH: Here, we address these challenges by developing cross-population prioritized dynamical nonlinear interaction model (CroP-DYNO), a nonlinear dynamical framework that prioritizes the learning of shared cross-population dynamics to avoid confounds from within-population activity. Further, CroP-DYNO allows individual components of the dynamical model to be independently specified as nonlinear or linear. Finally, it supports both causal and non-causal inference of latent states, using only source region activity. RESULTS: We validate our method on datasets across species and distinct brain regions. We find that both the prioritized learning and the nonlinear modeling in CroP-DYNO are important for accurately extracting cross-population dynamics. As such, CroP-DYNO outperforms baseline nonprioritized and linear prioritized models in predicting target neural population activity from source activity. Further, CroP-DYNO enables systematic localization of nonlinear structure and quantifies dominant interaction pathways between brain regions, with interaction strengths that align with known circuit anatomy. SIGNIFICANCE: Overall, these results establish CroP-DYNO as a flexible nonlinear framework for studying interactions between neural populations and brain regions, enabling both accurate modeling and interpretable dissection of nonlinear structure in cross-regional communication models.