Behavioral Neuroscience; Experimental and Cognitive Psychology; Social Psychology
Abstract :
[en] Relating individual brain patterns to behaviour is fundamental in system neuroscience. Recently, the predictive modelling approach has become increasingly popular, largely due to the recent availability of large open datasets and access to computational resources. This means that we can use machine learning models and interindividual differences at the brain level represented by neuroimaging features to predict interindividual differences in behavioural measures. By doing so, we could identify biomarkers and neural correlates in a data-driven fashion. Nevertheless, this budding field of neuroimaging-based predictive modelling is facing issues that may limit its potential applications. Here we review these existing challenges, as well as those that we anticipate as the field develops. We focus on the impacts of these challenges on brain-based predictions. We suggest potential solutions to address the resolvable challenges, while keeping in mind that some general and conceptual limitations may also underlie the predictive modelling approach.
Disciplines :
Neurosciences & behavior
Author, co-author :
Wu, Jianxiao ; Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Center Jülich, Jülich, Germany. j.wu@fz-juelich.de ; Institute for Systems Neuroscience, Medical Faculty, Heinrich-Heine University Düsseldorf, Düsseldorf, Germany. j.wu@fz-juelich.de
Li, Jingwei ; Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Center Jülich, Jülich, Germany ; Institute for Systems Neuroscience, Medical Faculty, Heinrich-Heine University Düsseldorf, Düsseldorf, Germany
Eickhoff, Simon B; Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Center Jülich, Jülich, Germany ; Institute for Systems Neuroscience, Medical Faculty, Heinrich-Heine University Düsseldorf, Düsseldorf, Germany
Scheinost, Dustin ; Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT, USA ; Department of Statistics and Data Science, Yale University, New Haven, CT, USA ; Child Study Center, Yale School of Medicine, New Haven, CT, USA ; Interdepartmental Neuroscience Program, Yale School of Medicine, New Haven, CT, USA ; Department of Biomedical Engineering, Yale School of Engineering and Applied Sciences, New Haven, CT, USA
Genon, Sarah ; Université de Liège - ULiège > Département des sciences cliniques > Neuroimagerie des troubles de la mémoire et revalidation cognitive ; Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Center Jülich, Jülich, Germany. s.genon@fz-juelich.de ; Institute for Systems Neuroscience, Medical Faculty, Heinrich-Heine University Düsseldorf, Düsseldorf, Germany. s.genon@fz-juelich.de
Language :
English
Title :
The challenges and prospects of brain-based prediction of behaviour.
Sui, J., Jiang, R., Bustillo, J. & Calhoun, V. Neuroimaging-based individualized prediction of cognition and behavior for mental disorders and health: methods and promises. Biol. Psychiatry 88, 818–828 (2020). DOI: 10.1016/j.biopsych.2020.02.016
Yeung, A. W. K., More, S., Wu, J. & Eickhoff, S. B. Reporting details of neuroimaging studies on individual traits prediction: a literature survey. NeuroImage 256, 119275 (2022). DOI: 10.1016/j.neuroimage.2022.119275
Cirillo, D. & Valencia, A. Big data analytics for personalized medicine. Curr. Opin. Biotechnol. 58, 161–167 (2019). DOI: 10.1016/j.copbio.2019.03.004
Dadi, K. et al. Benchmarking functional connectome-based predictive models for resting-state fMRI. NeuroImage 192, 115–134 (2019). DOI: 10.1016/j.neuroimage.2019.02.062
Dhamala, E., Yeo, B. T. T. & Holmes, A. J. One size does not fit all: methodological considerations for brain-based predictive modelling in psychiatry. Biol. Psychiatry 93, 717–728 (2023). DOI: 10.1016/j.biopsych.2022.09.024
Rosenberg, M. D. et al. A neuromarker of sustained attention from whole-brain functional connectivity. Nat. Neurosci. 19, 165–171 (2016). DOI: 10.1038/nn.4179
Lee, M. H., Smyser, C. D. & Shimony, J. S. Resting-state fMRI: a review of methods and clinical applications. Am. J. Neuroradiol. 34, 1866–1872 (2013). DOI: 10.3174/ajnr.A3263
Finn, E. S. et al. Functional connectome fingerprinting: identifying individuals using patterns of brain connectivity. Nat. Neurosci. 18, 1664–1671 (2015). DOI: 10.1038/nn.4135
Ferguson, M. A., Anderson, J. S. & Spreng, R. N. Fluid and flexible minds: intelligence reflects synchrony in the brain’s intrinsic network architecture. Netw. Neurosci. 1, 192–207 (2017). DOI: 10.1162/NETN_a_00010
Li, J. et al. A neuromarker of individual general fluid intelligence from the white-matter functional connectome. Transl. Psychiatry 10, 147 (2020). DOI: 10.1038/s41398-020-0829-3
Kumar, S. et al. An information network flow approach for measuring functional connectivity and predicting behavior. Brain Behav. 9, e01346 (2019). DOI: 10.1002/brb3.1346
Rosenberg, M. D. et al. Functional connectivity predicts changes in attention observed across minutes, days, and months. Proc. Natl Acad. Sci. USA 117, 3797–3807 (2020). DOI: 10.1073/pnas.1912226117
Avery, E. W. et al. Distributed patterns of functional connectivity predict working memory performance in novel healthy and memory-impaired individuals. J. Cogn. Neurosci. 32, 241–255 (2020). DOI: 10.1162/jocn_a_01487
Pläschke, R. N. et al. Age differences in predicting working memory performance from network-based functional connectivity. Cortex 132, 441–459 (2020). DOI: 10.1016/j.cortex.2020.08.012
Zhang, H. et al. Do intrinsic brain functional networks predict working memory from childhood to adulthood? Hum. Brain Mapp. 41, 4574–4586 (2020). DOI: 10.1002/hbm.25143
Girault, J. B. et al. White matter connectomes at birth accurately predict cognitive abilities at age 2. NeuroImage 192, 145–155 (2019). DOI: 10.1016/j.neuroimage.2019.02.060
Jiang, R. et al. Multimodal data revealed different neurobiological correlates of intelligence between males and females. Brain Imaging Behav. 14, 1979–1993 (2020). DOI: 10.1007/s11682-019-00146-z
Rasero, J., Sentis, A. I., Yeh, F. C. & Verstynen, T. Integrating across neuroimaging modalities boosts prediction accuracy of cognitive ability. PLoS Comput. Biol. 17, e1008347 (2021). DOI: 10.1371/journal.pcbi.1008347
Wei, L. et al. Grey matter volume in the executive attention system predict individual differences in effortful control in young adults. Brain Topogr. 32, 111–117 (2019). DOI: 10.1007/s10548-018-0676-1
Kaufmann, T. et al. Task modulations and clinical manifestations in the brain functional connectome in 1615 fMRI datasets. NeuroImage 147, 243–252 (2017). DOI: 10.1016/j.neuroimage.2016.11.073
Xiao, Y. et al. Predicting visual working memory with multimodal magnetic resonance imaging. Hum. Brain Mapp. 42, 1446–1462 (2021). DOI: 10.1002/hbm.25305
Scheinost, D. et al. Ten simple rules for predictive modeling of individual differences in neuroimaging. NeuroImage 193, 35–45 (2019). DOI: 10.1016/j.neuroimage.2019.02.057
Gabrieli, J. D. E., Ghosh, S. S. & Whitfield-Gabrieli, S. Prediction as a humanitarian and pragmatic contribution from human cognitive neuroscience. Neuron 85, 11–26 (2015). DOI: 10.1016/j.neuron.2014.10.047
Pervaiz, U., Vidaurre, D., Woolrich, M. W. & Smith, S. M. Optimising network modelling methods for fMRI. NeuroImage 221, 116604 (2020). DOI: 10.1016/j.neuroimage.2020.116604
Poldrak, R. A., Huckins, G. & Varoquax, G. Establishment of best practices for evidence for prediction: a review. J. Am. Med. Assoc. Psychiatry 77, 534–540 (2020).
Sripada, C. et al. Prediction of neurocognition in youth from resting state fMRI. Mol. Psychiatry 25, 3413–3421 (2019). DOI: 10.1038/s41380-019-0481-6
He, T. et al. Deep neural networks and kernel regression achieve comparable accuracies for functional connectivity prediction of behavior and demographics. NeuroImage 206, 116276 (2020). DOI: 10.1016/j.neuroimage.2019.116276
He, L. et al. Functional connectome prediction of anxiety related to the COVID-19 pandemic. Am. J. Psychiatry 178, 530–540 (2021). DOI: 10.1176/appi.ajp.2020.20070979
Gao, S., Greene, A. S., Constable, R. T. & Scheinost, D. Combining multiple connectomes improves predictive modeling of phenotypic measures. NeuroImage 201, 116038 (2019). DOI: 10.1016/j.neuroimage.2019.116038
Zalesky, A., Fornito, A., Cocchi, L., Gollo, L. L. & Breakspear, M. Time-resolved resting-state brain networks. Proc. Natl Acad. Sci. USA 111, 10341–10346 (2014). DOI: 10.1073/pnas.1400181111
Bahg, G., Evans, D. G., Galdo, M. & Turner, B. M. Gaussian process linking functions for mind, brain, and behavior. Proc. Natl Acad. Sci. USA 117, 29398–29406 (2020). DOI: 10.1073/pnas.1912342117
Mihalik, A. et al. Canonical correlation analysis and partial least squares for identifying brain–behaviour associations: a tutorial and a comparative study. Biol. Psychiatry 7, 1055–1067 (2022).
Gal, S., Tik, N., Bernstein-Eliav, M. & Tavor, I. Predicting individual traits from unperformed tasks. NeuroImage 249, 118920 (2022). DOI: 10.1016/j.neuroimage.2022.118920
He, T. et al. Meta-matching as a simple framework to translate phenotypic predictive models from big to small data. Nat. Neurosci. 25, 795–804 (2022). DOI: 10.1038/s41593-022-01059-9
Takagi, Y., Hirayama, J. I. & Tanaka, S. C. State-unspecific patterns of whole-brain functional connectivity from resting and multiple task states predict stable individual traits. NeuroImage 201, 116036 (2019). DOI: 10.1016/j.neuroimage.2019.116036
Burr, D. A. et al. Functional connectivity predicts the dispositional use of expressive suppression but not cognitive reappraisal. Brain Behav. 10, e01493 (2020). DOI: 10.1002/brb3.1493
Jiang, R. et al. Task-induced brain connectivity promotes the detection of individual differences in brain–behavior relationships. NeuroImage 207, 116370 (2020). DOI: 10.1016/j.neuroimage.2019.116370
Ooi, L. Q. R. et al. Comparison of individualized behavioral predictions across anatomical, diffusion and functional connectivity MRI. NeuroImage 263, 119636 (2022). DOI: 10.1016/j.neuroimage.2022.119636
Dhamala, E., Jamison, K. W., Jaywant, A., Dennis, S. & Kuceyeski, A. Distinct functional and structural connections predict crystallised and fluid cognition in healthy adults. Hum. Brain Mapp. 42, 3102–3118 (2021). DOI: 10.1002/hbm.25420
Mansour, L. S., Tian, Y., Yeo, B. T. T., Cropley, V. & Zalesky, A. High-resolution connectomic fingerprints: mapping neural identity and behavior. NeuroImage 229, 117695 (2021). DOI: 10.1016/j.neuroimage.2020.117695
Pat, N. et al. Longitudinally stable, brain-based predictive models mediate the relationships between childhood cognition and socio-demographic, psychological and genetic factors. Hum. Brain Mapp. 43, 5520–5542 (2022). DOI: 10.1002/hbm.26027
Hurtz, G. M. & Donovan, J. J. Personality and job performance: the Big Five revisited. J. Appl. Psychol. 85, 869–879 (2000). DOI: 10.1037/0021-9010.85.6.869
Kane, M. J., Conway, A. R. A., Miura, T. K. & Colflesh, G. J. H. Working memory, attention control, and the n-back task: a question of construct validity. J. Exp. Psychol. 33, 615–622 (2007).
Sanchez-Cubillo, I. et al. Construct validity of the Trail Making Test: role of task-switching, working memory, inhibition/interference control, and visuomotor abilities. J. Int. Neuropsychol. Soc. 15, 438–450 (2009). DOI: 10.1017/S1355617709090626
Chen, J. et al. Shared and unique brain network features predict cognitive, personality, and mental health scores in the ABCD study. Nat. Commun. 13, 2217 (2022). DOI: 10.1038/s41467-022-29766-8
Wu, J. et al. A connectivity-based psychometric prediction framework for brain–behavior relationship studies. Cereb. Cortex 31, 3732–3751 (2021). DOI: 10.1093/cercor/bhab044
Noble, S., Scheinost, D. & Constable, R. T. A decade of test–retest reliability of functional connectivity: a systematic review and meta-analysis. NeuroImage 203, 116157 (2019). DOI: 10.1016/j.neuroimage.2019.116157
Elliott, M. L. et al. What is the test–retest reliability of common task-functional MRI measures? New empirical evidence and a meta-analysis. Psychol. Sci. 31, 792–806 (2020). DOI: 10.1177/0956797620916786
Patriat, R. et al. The effect of resting condition on resting-state fMRI reliability and consistency: a comparison between resting with eyes open, closed, and fixated. NeuroImage 78, 463–473 (2013). DOI: 10.1016/j.neuroimage.2013.04.013
Birn, R. M. et al. The effect of scan length on the reliability of resting-state fMRI connectivity estimates. NeuroImage 83, 550–558 (2013). DOI: 10.1016/j.neuroimage.2013.05.099
Bennett, C. M. & Miller, M. B. fMRI reliability: influences of task and experimental design. Cogn. Affect. Behav. Neurosci. 13, 690–702 (2013). DOI: 10.3758/s13415-013-0195-1
Cremers, H. R., Wager, T. D. & Yarkoni, T. The relation between statistical power and inference in fMRI. PLoS ONE 12, e0184923 (2017). DOI: 10.1371/journal.pone.0184923
Kharabian Masouleh, S., Eickhoff, S. B., Hoffstaedter, F. & Genon, S., Alzheimer’s Disease Neuroimaging Initiative. Empirical examination of the replicability of associations between brain structure and psychological variables. eLife 8, e43464 (2019). DOI: 10.7554/eLife.43464
Genon, S., Eickhoff, S. B. & Kahrabian, S. Linking interindividual variability in brain structure to behaviour. Nat. Rev. Neurosci. 23, 307–318 (2022). DOI: 10.1038/s41583-022-00584-7
Marek, S. et al. Reproducible brain-wide association studies require thousands of individuals. Nature 603, 654–660 (2022). DOI: 10.1038/s41586-022-04492-9
Beaty, R. E. et al. Robust prediction of individual creative ability from brain functional connectivity. Proc. Natl Acad. Sci. USA 115, 1087–1092 (2018). DOI: 10.1073/pnas.1713532115
Liu, P. et al. The functional connectome predicts feeling of stress on regular days and during the COVID-19 pandemic. Neurobiol. Stress 14, 100285 (2021). DOI: 10.1016/j.ynstr.2020.100285
Ren, Z. et al. Connectome-based predictive modeling of creativity anxiety. NeuroImage 225, 117469 (2021). DOI: 10.1016/j.neuroimage.2020.117469
Fong, A. H. C. et al. Dynamic functional connectivity during task performance and rest predicts individual differences in attention across studies. NeuroImage 188, 14–25 (2019). DOI: 10.1016/j.neuroimage.2018.11.057
Wu, J. et al. Cross-cohort replicability and generalizability of connectivity-based psychometric prediction patterns. NeuroImage 262, 119569 (2022). DOI: 10.1016/j.neuroimage.2022.119569
Tervo-Clemmens, B. et al. Reply to: Multivariate BWAS can be replicable with moderate sample sizes. Nature 615, E8–E12 (2023). DOI: 10.1038/s41586-023-05746-w
Rosenberg, M. D. & Finn, E. S. How to establish robust brain–behavior relationships without thousands of individuals. Nat. Neurosci. 25, 835–837 (2022). DOI: 10.1038/s41593-022-01110-9
Spisak, T., Bingel, U. Wager, T. Replicable multivariate BWAS with moderate sample sizes. Preprint at bioRxiv https://doi.org/10.1101/2022.06.22.497072 (2022).
Van Essen, D. C. et al. The WU-Minh Human Connectome Project: an overview. NeuroImage 80, 62–79 (2013). DOI: 10.1016/j.neuroimage.2013.05.041
Glasser, M. F. et al. The minimal preprocessing pipelines for the Human Connectome Project. NeuroImage 80, 105–124 (2013). DOI: 10.1016/j.neuroimage.2013.04.127
Harms, M. P. et al. Extending the Human Connectome Project across ages: imaging protocols for the lifespan development and aging projects. NeuroImage 183, 972–984 (2018). DOI: 10.1016/j.neuroimage.2018.09.060
Chouldechova, A., Benavides-Prado, D., Fialko, O. & Vaithianathan, R. A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. Proc. Mach. Learn. Res. 81, 134–148 (2018).
Martin, A. R. et al. Clinical use of current polygenic risk scores may exacerbate healthy disparities. Nat. Genet. 51, 584–591 (2019). DOI: 10.1038/s41588-019-0379-x
Obermeyer, Z., Powers, B., Vogeli, C. & Mullainathan, S. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 447–453 (2019). DOI: 10.1126/science.aax2342
Li, J. et al. Cross-ethnicity/race generalization failure of behavioral prediction from resting-state functional connectivity. Sci. Adv. 8, eabj1812 (2022). DOI: 10.1126/sciadv.abj1812
Greene, A. S. et al. Brain-phenotype models fail for individuals who defy sample stereotypes. Nature 609, 109–118 (2022). DOI: 10.1038/s41586-022-05118-w
Greene, A. S., Gao, S., Scheinost, D. & Constable, R. T. Task-induced brain state manipulation improves prediction of individual traits. Nat. Commun. 9, 2807 (2018). DOI: 10.1038/s41467-018-04920-3
Nostro, A. D. et al. Predicting personality from network-based resting-state functional connectivity. Brain Struct. Funct. 223, 2699–2719 (2018). DOI: 10.1007/s00429-018-1651-z
Tian, Y. & Zalesky, A. Machine learning prediction of cognition from functional connectivity: are feature weights reliable? NeuroImage 245, 118648 (2021). DOI: 10.1016/j.neuroimage.2021.118648
Haufe, S. et al. On the interpretation of weight vectors of linear models in multivariate neuroimaging. NeuroImage 87, 96–110 (2014). DOI: 10.1016/j.neuroimage.2013.10.067
Chen, J. et al. Relationship between prediction accuracy and feature importance reliability: an empirical and theoretical study. NeuroImage 274, 120115 (2023). DOI: 10.1016/j.neuroimage.2023.120115
Breiman, L. Random forests. Mach. Learn. 45, 5–32 (2001). DOI: 10.1023/A:1010933404324
Yip, S. W., Kiluk, B. & Scheinost, D. Towards addiction prediction: an overview of cross-validated predictive modeling findings and considerations for future neuroimaging research. Biol. Psychiatry Cogn. Neurosci. Neuroimaging 5, 748–758 (2020).
Jiang, R., Woo, C. W., Qi, S., Wu, J. & Sui, J. Interpreting brain biomarkers: challenges and solutions in interpreting machine learning-based predictive neuroimaging. IEEE Signal Process. Mag. 39, 107–118 (2022). DOI: 10.1109/MSP.2022.3155951
Chormai, P. et al. Machine learning of large-scale multimodal brain imaging data reveals neural correlates of hand preference. NeuroImage 262, 119534 (2022). DOI: 10.1016/j.neuroimage.2022.119534
Zou, H. & Hastie, T. Regularization and variable selection via the elastic net. J. R. Stat. Soc. B 67, 301–320 (2005). DOI: 10.1111/j.1467-9868.2005.00503.x
Lundberg, S. M. Lee, S.-I. A unified approach to interpreting model prediction. In Advances in Neural Information Processing Systems 30 (NIPS 2017) (eds Guyon, I. et al.) (Curran Associates, 2017).
Pat, N., Wang, Y., Bartonicek, A., Candia, J. & Stringaris, A. Explainable machine learning approach to predict and explain the relationship between task-based fMRI and individual differences in cognition. Cereb. Cortex 33, 2682–2703 (2023). DOI: 10.1093/cercor/bhac235
Kingma, D. P. Welling, M. Auto-encoding variational Bayes. Preprint at arXiv https://doi.org/10.48550/arXiv.1312.6114 (2013).
Goodfellow, I. J. et al. Generative adversarial networks. Commun. ACM 63, 139–144 (2020). DOI: 10.1145/3422622
van den Oord, A., Kalchbrenner, N. & Kavukcuoglu, K. Pixel recurrent neural networks. In Proceedings of the 33rd International Conference on Machine Learning (eds Balcan, M. F. & Weinberger, K. Q.) 48 1747–1756 (Proceedings of Machine Learning Research, 2016).
Fried, D. et al. Speaker-follower models for vision-and-language navigation. In Advances in Neural Information Processing Systems 31 (NeurIPS 2018) (eds Bengio, S. et al.) (Curran Associates, 2018).
Rosenblatt, M. et al. Connectome-based machine learning models are vulnerable to subtle data manipulations. Patterns (in the press).
Finlayson, S. G. et al. Adversarial attacks on medical machine learning. Science 363, 1287–1289 (2019). DOI: 10.1126/science.aaw4399
Finlayson, S. G., Chung, H. W., Kohane, I. S. Beam, A. L. Adversarial attacks against medical deep learning systems. Preprint at arXiv https://doi.org/10.48550/arXiv.1804.05296 (2019).
Dubois, J., Galdi, P., Han, Y., Paul, L. K. & Adolphs, R. Resting-state functional brain connectivity best predicts the personality dimension of openness to experience. Pers. Neurosci. 1, E6 (2018).
Jiang, R. et al. Connectome-based individualized prediction of temperament trait scores. NeuroImage 183, 366–374 (2018). DOI: 10.1016/j.neuroimage.2018.08.038
