Brain; Brain Mapping; Gray Matter/diagnostic imaging; Humans; Magnetic Resonance Imaging; Psychometrics; Young Adult
Abstract :
[en] The study of associations between inter-individual differences in brain structure and behaviour has a long history in psychology and neuroscience. Many associations between psychometric data, particularly intelligence and personality measures and local variations of brain structure have been reported. While the impact of such reported associations often goes beyond scientific communities, resonating in the public mind, their replicability is rarely evidenced. Previously, we have shown that associations between psychometric measures and estimates of grey matter volume (GMV) result in rarely replicated findings across large samples of healthy adults. However, the question remains if these observations are at least partly linked to the multidetermined nature of the variations in GMV, particularly within samples with wide age-range. Therefore, here we extended those evaluations and empirically investigated the replicability of associations of a broad range of psychometric variables and cortical thickness in a large cohort of healthy young adults. In line with our observations with GMV, our current analyses revealed low likelihood of significant associations and their rare replication across independent samples. We here discuss the implications of these findings within the context of accumulating evidence of the general poor replicability of structural-brain-behaviour associations, and more broadly of the replication crisis.
Disciplines :
Neurosciences & behavior
Author, co-author :
Kharabian Masouleh, Shahrzad; Institute of Neuroscience and Medicine (INM-7: Brain and Behaviour), Research ; Institute of Systems Neuroscience, Heinrich Heine University Düsseldorf,
Eickhoff, Simon B; Institute of Neuroscience and Medicine (INM-7: Brain and Behaviour), Research ; Institute of Systems Neuroscience, Heinrich Heine University Düsseldorf,
Maleki Balajoo, Somayeh; Institute of Neuroscience and Medicine (INM-7: Brain and Behaviour), Research ; Institute of Systems Neuroscience, Heinrich Heine University Düsseldorf,
Nicolaisen-Sobesky, Eliana; Institute of Neuroscience and Medicine (INM-7: Brain and Behaviour), Research ; Institute of Systems Neuroscience, Heinrich Heine University Düsseldorf,
Thirion, Bertrand; Inria, CEA, Université Paris-Saclay, Palaiseau, France.
Genon, Sarah ; Université de Liège - ULiège > Département des sciences cliniques > Neuroimagerie des troubles de la mémoire et revalidation cognitive ; Institute of Neuroscience and Medicine (INM-7: Brain and Behaviour), Research ; Institute of Systems Neuroscience, Heinrich Heine University Düsseldorf,
Language :
English
Title :
Empirical facts from search for replicable associations between cortical thickness and psychometric variables in healthy adults.
Winkler, A. M. et al. Cortical thickness or grey matter volume? The importance of selecting the phenotype for imaging genetics studies. Neuroimage 53, 1135–1146 (2010). DOI: 10.1016/j.neuroimage.2009.12.028
Walhovd, K. B., Fjell, A. M., Giedd, J., Dale, A. M. & Brown, T. T. Through thick and thin: A need to reconcile contradictory results on trajectories in human cortical development. Cereb. Cortex 27, 1472–1481 (2017).
Good, C. D. et al. A voxel-based morphometric study of ageing in 465 normal adult human brains. Neuroimage 14, 21–36 (2001). DOI: 10.1006/nimg.2001.0786
Maguire, E. A., Woollett, K. & Spiers, H. J. London taxi drivers and bus drivers: A structural MRI and neuropsychological analysis. Hippocampus 16, 1091–1101 (2006). DOI: 10.1002/hipo.20233
Kanai, R. & Rees, G. The structural basis of inter-individual differences in human behaviour and cognition. Nat. Rev. Neurosci. 12, 231–242 (2011). DOI: 10.1038/nrn3000
Kanai, R., Bahrami, B., Roylance, R. & Rees, G. Online social network size is reflected in human brain structure. Proc. R. Soc. B Biol. Sci. 279, 1327–1334 (2012). DOI: 10.1098/rspb.2011.1959
Boekel, W. et al. A purely confirmatory replication study of structural brain-behavior correlations. Cortex 66, 115–133 (2015). DOI: 10.1016/j.cortex.2014.11.019
Genon, S. et al. Searching for behavior relating to grey matter volume in a-priori defined right dorsal premotor regions: Lessons learned. Neuroimage 157, 144–156 (2017). DOI: 10.1016/j.neuroimage.2017.05.053
Kharabian Masouleh, S., Eickhoff, S. B., Hoffstaedter, F. & Genon, S. Empirical examination of the replicability of associations between brain structure and psychological variables. Elife 8, e43464 (2019). DOI: 10.7554/eLife.43464
Winkler, A. M. et al. Joint analysis of cortical area and thickness as a replacement for the analysis of the volume of the cerebral cortex. Cereb. Cortex 28, 738–749 (2018). DOI: 10.1093/cercor/bhx308
Ashburner, J. Computational anatomy with the SPM software. Magn. Reson. Imaging 27, 1163–1174. 10.1016/j.mri.2009.01.006 (2009). DOI: 10.1016/j.mri.2009.01.006
Choi, Y. Y. et al. Multiple bases of human intelligence revealed by cortical thickness and neural activation. J. Neurosci. 28, 10323–10329 (2008). DOI: 10.1523/JNEUROSCI.3259-08.2008
Karama, S. et al. Positive association between cognitive ability and cortical thickness in a representative US sample of healthy 6 to 18 year-olds. Intelligence 37, 145–155 (2009). DOI: 10.1016/j.intell.2008.09.006
Menary, K. et al. Associations between cortical thickness and general intelligence in children, adolescents and young adults. Intelligence 41, 597–606 (2013). DOI: 10.1016/j.intell.2013.07.010
Schmitt, J. E. et al. The dynamic associations between cortical thickness and general intelligence are genetically mediated. Cereb. Cortex 10.1093/cercor/bhz007 (2019). DOI: 10.1093/cercor/bhz007
Hyatt, C. S. et al. Personality traits share overlapping neuroanatomical correlates with internalizing and externalizing psychopathology. J. Abnorm. Psychol. 128, 1–11 (2019). DOI: 10.1037/abn0000391
Owens, M. M. et al. Cortical morphometry of the five-factor model of personality: Findings from the human connectome project full sample. Soc. Cogn. Affect. Neurosci. 14, 381–395 (2019). DOI: 10.1093/scan/nsz017
Riccelli, R., Toschi, N., Nigro, S., Terracciano, A. & Passamonti, L. Surface-based morphometry reveals the neuroanatomical basis of the five-factor model of personality. Soc. Cogn. Affect. Neurosci. 12, 671–684 (2017).
Kühn, S., Schubert, F. & Gallinat, J. Structural correlates of trait anxiety: Reduced thickness in medial orbitofrontal cortex accompanied by volume increase in nucleus accumbens. J. Affect. Disord. 134, 315–319 (2011). DOI: 10.1016/j.jad.2011.06.003
Schilling, C. et al. Cortical thickness correlates with impulsiveness in healthy adults. Neuroimage 59, 824–830 (2012). DOI: 10.1016/j.neuroimage.2011.07.058
Foster, N. E. V. & Zatorre, R. J. Cortical structure predicts success in performing musical transformation judgments. Neuroimage 53, 26–36 (2010). DOI: 10.1016/j.neuroimage.2010.06.042
Rice, K. & Redcay, E. Spontaneous mentalizing captures variability in the cortical thickness of social brain regions. Soc. Cogn. Affect. Neurosci. 10, 327–334 (2015). DOI: 10.1093/scan/nsu081
Deary, I. J., Penke, L. & Johnson, W. The neuroscience of human intelligence differences. Nat. Rev. Neurosci. 11, 201–211 (2010). DOI: 10.1038/nrn2793
Heaton, R. K. et al. Reliability and validity of composite scores from the NIH toolbox cognition battery in adults. J. Int. Neuropsychol. Soc. 20, 588–598 (2014). DOI: 10.1017/S1355617714000241
Weintraub, S. et al. The cognition battery of the NIH toolbox for assessment of neurological and behavioral function: Validation in an adult sample. J. Int. Neuropsychol. Soc. 20, 567–578 (2014). DOI: 10.1017/S1355617714000320
Cox, S. R., Ritchie, S. J., Fawns-Ritchie, C., Tucker-Drob, E. M. & Deary, I. J. Structural brain imaging correlates of general intelligence in UK Biobank. Intelligence 76, 101376 (2019). DOI: 10.1016/j.intell.2019.101376
Haier, R. J. et al. Gray matter and intelligence factors: Is there a neuro-g?. Intelligence 37, 136–144 (2009). DOI: 10.1016/j.intell.2008.10.011
Fjell, A. M. et al. Accelerating cortical thinning: Unique to dementia or universal in aging?. Cereb. Cortex 24, 919–934 (2014). DOI: 10.1093/cercor/bhs379
Barch, D. M. et al. Function in the human connectome: Task-fMRI and individual differences in behavior. Neuroimage 80, 169–189 (2013). DOI: 10.1016/j.neuroimage.2013.05.033
Hedge, C., Powell, G. & Sumner, P. The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behav. Res. Methods 50, 1166–1186 (2018). DOI: 10.3758/s13428-017-0935-1
Dwan, K., Gamble, C., Williamson, P. R. & Kirkham, J. J. Systematic review of the empirical evidence of study publication bias and outcome reporting bias—An updated review. PLoS ONE 8, e66844. 10.1371/journal.pone.0066844 (2013). DOI: 10.1371/journal.pone.0066844
Nissen, S. B., Magidson, T., Gross, K. & Bergstrom, C. T. Publication bias and the canonization of false facts. Elife 5, 1–19 (2016). DOI: 10.7554/eLife.21451
Franco, A., Malhotra, N. & Simonovits, G. Publication bias in the social sciences: Unlocking the file drawer. Science 1979(345), 1502–1505 (2014). DOI: 10.1126/science.1255484
Desikan, R. S. et al. An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage 31, 968–980 (2006). DOI: 10.1016/j.neuroimage.2006.01.021
Eickhoff, S. B., Yeo, B. T. T. & Genon, S. Imaging-based parcellations of the human brain. Nat. Rev. Neurosci. 19, 672–686. 10.1038/s41583-018-0071-7 (2018). DOI: 10.1038/s41583-018-0071-7
Savic, I. & Arver, S. Sex differences in cortical thickness and their possible genetic and sex hormonal underpinnings. Cereb. Cortex 24, 3246–3257 (2014). DOI: 10.1093/cercor/bht180
Eklund, A., Nichols, T. E. & Knutsson, H. Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates. Proc. Natl. Acad. Sci. U.S.A. 113, 7900–7905 (2016). DOI: 10.1073/pnas.1602413113
Noble, S., Scheinost, D. & Constable, R. T. Cluster failure or power failure? Evaluating sensitivity in cluster-level inference. Neuroimage 209, 116468. 10.1016/J.NEUROIMAGE.2019.116468 (2019). DOI: 10.1016/J.NEUROIMAGE.2019.116468
Cremers, H. R., Wager, T. D. & Yarkoni, T. The relation between statistical power and inference in fMRI. PLoS ONE 12, 1–20 (2017). DOI: 10.1371/journal.pone.0184923
Kharabian Masouleh, S. et al. Influence of processing pipeline on cortical thickness measurement. Cereb. Cortex 30, 5014–5027 (2020). DOI: 10.1093/cercor/bhaa097
Natu, V. S. et al. Apparent thinning of human visual cortex during childhood is associated with myelination. Proc. Natl. Acad. Sci. U.S.A. 116, 20750–20759 (2019). DOI: 10.1073/pnas.1904931116
Vul, E., Harris, C., Winkielman, P. & Pashler, H. Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition. Perspect. Psychol. Sci. 4, 274–290 (2009). DOI: 10.1111/j.1745-6924.2009.01125.x
Lindsay, D. S. Replication in psychological science. Psychol. Sci. 26, 1827–1832 (2015). DOI: 10.1177/0956797615616374
Open Science Collaboration, O. S. Estimating the reproducibility of psychological science. Science 349, aac4716 (2015). DOI: 10.1126/science.aac4716
Button, K. S. et al. Power failure: Why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci. 14, 365–376 (2013). DOI: 10.1038/nrn3475
Grabitz, C. R. et al. Logical and methodological issues affecting genetic studies of humans reported in top neuroscience journals. J. Cogn. Neurosci. 30, 25–41 (2018). DOI: 10.1162/jocn_a_01192
Poldrack, R. A. et al. Scanning the horizon: Towards transparent and reproducible neuroimaging research. Nat. Rev. Neurosci. 18, 115–126 (2017). DOI: 10.1038/nrn.2016.167
Munafò, M. R. et al. A manifesto for reproducible science. Nat. Hum. Behav. 1, 1–9 (2017). DOI: 10.1038/s41562-016-0021
Wagenmakers, E. J., Wetzels, R., Borsboom, D., van der Maas, H. L. J. & Kievit, R. A. An agenda for purely confirmatory research. Perspect. Psychol. Sci. 7, 632–638 (2012). DOI: 10.1177/1745691612463078
van Essen, D. C. et al. The WU-Minn human connectome project: An overview. Neuroimage 80, 62–79 (2013). DOI: 10.1016/j.neuroimage.2013.05.041
Miller, K. L. et al. Multimodal population brain imaging in the UK Biobank prospective epidemiological study. Nat. Neurosci. 19, 1523–1536 (2016). DOI: 10.1038/nn.4393
Gelman, A. & Loken, E. The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time. Psychol. Bull. 140, 1272–1280 (2014). DOI: 10.1037/a0037714
Nosek, B. A. et al. Promoting an open research culture. Science 348, 1422–1425. 10.1126/science.aab2374 (2015). DOI: 10.1126/science.aab2374
Chambers, C. D., Dienes, Z., McIntosh, R. D., Rotshtein, P. & Willmes, K. Registered reports: Realigning incentives in scientific publishing. Cortex 66, 1–2. 10.1016/j.cortex.2015.03.022 (2015). DOI: 10.1016/j.cortex.2015.03.022
Foster, E. D. & Deardorff, A. Open science framework (OSF). J. Med. Libr. Assoc. 105, 203 (2017). DOI: 10.5195/jmla.2017.88
Gorgolewski, K. J. et al. NeuroVault.org: A web-based repository for collecting and sharing unthresholded statistical maps of the human brain. Front. Neuroinform. 9, 8 (2015). DOI: 10.3389/fninf.2015.00008
Marcus, D. S. et al. Informatics and data mining tools and strategies for the human connectome project. Front. Neuroinform. 5, 4 (2011). DOI: 10.3389/fninf.2011.00004
Glasser, M. F. et al. The minimal preprocessing pipelines for the human connectome project. Neuroimage 80, 105–124 (2013). DOI: 10.1016/j.neuroimage.2013.04.127
Gur, R. C. et al. A cognitive neuroscience-based computerized battery for efficient measurement of individual differences: Standardization and initial construct validation. J. Neurosci. Methods 187, 254–262 (2010). DOI: 10.1016/j.jneumeth.2009.11.017
Estle, S. J., Green, L., Myerson, J. & Holt, D. D. Differential effects of amount on temporal and probability discounting of gains and losses. Mem. Cogn. 34, 914–928 (2006). DOI: 10.3758/BF03193437
Myerson, J., Green, L. & Warusawitharana, M. Area under the curve as a measure of discounting. J. Exp. Anal. Behav. 76, 235–243 (2001). DOI: 10.1901/jeab.2001.76-235
Akshoomoff, N. et al. NIH toolbox cognition battery (CB): Composite scores of crystallized, fluid, and overall cognition. Monogr. Soc. Res. Child Dev. 78, 119–132 (2013). DOI: 10.1111/mono.12038
Smith, R., Keramatian, K. & Christoff, K. Localizing the rostrolateral prefrontal cortex at the individual level. Neuroimage 36, 1387–1396 (2007). DOI: 10.1016/j.neuroimage.2007.04.032
Fischl, B., Sereno, M. I., Tootell, R. B. H. & Dale, A. M. High-resolution intersubject averaging and a coordinate system for the cortical surface. Hum. Brain Mapp. 8, 272–284 (1999). DOI: 10.1002/(SICI)1097-0193(1999)8:4<272::AID-HBM10>3.0.CO;2-4
Gaser, C. & Dahnke, R. CAT—A computational anatomy toolbox for the analysis of structural MRI data. HBM Conf. 32, 7743 (2016).
Martínez, K. et al. Reproducibility of brain-cognition relationships using three cortical surface-based protocols: An exhaustive analysis based on cortical thickness. Hum. Brain Mapp. 36, 3227–3245 (2015). DOI: 10.1002/hbm.22843
Smith, S. M. & Nichols, T. E. Threshold-free cluster enhancement: Addressing problems of smoothing, threshold dependence and localisation in cluster inference. Neuroimage 44, 83–98 (2009). DOI: 10.1016/j.neuroimage.2008.03.061
Alberton, B. A. V., Nichols, T. E., Gamba, H. R. & Winkler, A. M. Multiple testing correction over contrasts for brain imaging. Neuroimage 216, 116760 (2020). DOI: 10.1016/j.neuroimage.2020.116760
Jeffreys, H. Theory of probability (Oxford University Press, Oxford, 1961).
