APECS; climate science; early career researchers; education; group review; IPCC; peer review; Global and Planetary Change; Environmental Science (miscellaneous); Pollution; Atmospheric Science; Management, Monitoring, Policy and Law
Abstract :
[en] The participation of a diverse –in terms of geography, discipline and gender– group of Early Career Researchers (ECRs) in the peer review process can help alleviate the workload of senior researchers and counteract the perceptual biases that the latter tend to show. Moreover, ECRs can benefit from developing skills that are often not included in educational programs. From 2018 to 2021, the Association of Polar Early Career Scientists, in collaboration with other associations, organized six group reviews of the Intergovernmental Panel on Climate Change (IPCC) reports by a total of more than 600 ECRs from over 70 different countries. This study aims to evaluate this group review in terms of its contribution to the production of scientific knowledge, and as a career development opportunity for ECRs. The data analyzed consists of application forms, review comments, and feedback surveys that were collected during each review process. The results of this study show that, overall, the group reviews were a success in terms of the experience of ECRs and their contribution to the peer review of the IPCC reports. Most survey respondents considered the general organization of the group reviews satisfactory and expressed interest in participating in future group reviews. However, most participants did not engage in discussions with their peers, which constitutes a missed opportunity to engage in active learning and the shared production of knowledge. ECRs made a significant contribution to the review of the IPCC reports by producing an average of 2,422 ± 532 comments per group review, 36% of which were substantive. PhD students were shown to be as proficient reviewers as postdoctoral researchers and faculty reviewers. More importantly, the diversity of reviewers in terms of geography and discipline, together with the fact that they are ECRs, can help produce more balanced scientific reports since they bring new perspectives, thus counteracting the biases that senior researchers have. These group reviews could be improved by providing more comprehensive training and facilitating communication among reviewers so that they can engage in meaningful exchanges. We conclude that the IPCC should formalize the inclusion of ECRs in future reviews of the IPCC reports.
Disciplines :
Environmental sciences & ecology
Author, co-author :
Moreno-Ibáñez, Marta; Cooperative Institute for Research in Environmental Sciences, University of Colorado Boulder, Boulder, United States ; National Snow and Ice Data Center, University of Colorado Boulder, Boulder, United States
Casado, Mathieu; Laboratoire des Sciences du Climat et de l’Environnement, UMR 8212 CEA-CNRS-UVSQ, Université Paris-Saclay, IPSL, Gif-sur-Yvette, France
Gremion, Gwenaëlle; Institut des Sciences de la Mer (ISMER), Université du Québec à Rimouski (UQAR), Rimouski, Canada
Rabanal, Valentina; National Meteorological Service (Argentina), Buenos Aires, Argentina ; Young Earth System Scientists (YESS), Buenos Aires City, Argentina
Adojoh, Onema; Natural Sciences - Geology, Northwest Missouri State University, Maryville, United States ; Earth, Environmental, and Planetary Sciences, Case Western Reserve University, Cleveland, United States
Anoruo, Chukwuma; Department of Physics and Astronomy, University of Nigeria, Nsukka, Nigeria
Arshad, Adnan; State Key Laboratory of Grasslands & Agroecosystem, Lanzhou University, Lanzhou, China ; PODA-Pakistan, Islamabad, Pakistan
Bahar, Faten Attig; Tunisia Polytechnic School, University of Carthage, Tunisia ; Institut des Géosciences de l’Environnement, CNRS, UGA/Grenoble-INP/IRD/INRAE, France
Bello, Cinthya; Carrera de Biología Marina, Universidad Científica del Sur, Lima, Peru
Bergstedt, Helena; B.Geos, Korneuburg, Austria
Caccavo, Jilda Alicia; Laboratoire des Sciences du Climat et de l’Environnement, UMR 8212 CEA-CNRS-UVSQ, Université Paris-Saclay, IPSL, Gif-sur-Yvette, France ; Laboratoire d'Océanographie et du Climat Expérimentations et Approches Numériques, LOCEAN/IPSL, UPMC-CNRS-IRD-MNHN, Sorbonne Université, Paris, France
Champollion, Nicolas; Institut des Géosciences de l'Environnement, CNRS, UGA / Grenoble, INP/IRD/INRAE, Saint-Martin d'Hères, France
Choy, Emily S.; Department of Biology, McMaster University, Hamilton, Canada
De Los Ríos, María Fernanda; Department of Science, Universidad Nacional Agraria La Molina, Lima, Peru
Detlef, Henrieka; Department of Geoscience, Aarhus University, Aarhus, Denmark
Dey, Rahul; National Centre for Polar and Ocean Research, Ministry of Earth Sciences, India
Guímaro, Hugo R.; Marine and Environmental Sciences Centre, Department of Life Sciences, University of Coimbra, Coimbra, Portugal ; British Antarctic Survey, Natural Environment Research Council, Cambridge, United Kingdom
Hansen, Christel; Department of Geography, Geoinformatics and Meteorology, University of Pretoria, Pretoria, South Africa
Hare, Vincent; Stable Light Isotope Laboratory, Department of Archaeology, University of Cape Town, Cape Town, South Africa ; Node for Isotope Biogeochemistry, South African Biogeochemistry Research Infrastructure Platform, Cape Town, South Africa
Höfer, Juan; Escuela de Ciencias del Mar, Pontificia Universidad Catolica de Valparaiso, Valparaíso, Chile
Jabir, Thajudeen; Arctic Biogeochemistry and Ecology, National Centre for Polar and Ocean Research, Ministry of Earth Sciences, India
Jain, Shipra; Institute for Risk and Disaster Reduction, University College London, London, United Kingdom
Jawak, Shridhar; Atmosphere and Climate (ATMOS), The Climate and Environmental Research Institute (NILU), Kjeller, Norway ; Remote Sensing Services, Svalbard Integrated Arctic Earth Observing System (SIOS), Longyearbyen, Norway ; Department of Civil Engineering, Manipal Academy of Higher Education (MAHE), Manipal, India
Latonin, Mikhail; Climate of High Northern Latitudes Research Group, Nansen International Environmental and Remote Sensing Centre, Russian Federation ; Institute of Earth Sciences, Saint Petersburg State University, Saint Petersburg, Russian Federation
Martin, Joseph; Meteorology and Oceanography Centre (Pacific), Royal Canadian Navy, Esquimalt, Canada ; Climate Lab, School of Earth and Ocean Sciences, University of Victoria, Canada
Fredy Mojica, Jhon; University of Miami, Cooperative Institute for Marine and Atmospheric Studies (CIMAS), National Oceanic and Atmospheric Administration, AOML, Miami, United States ; CIMAS, RSMAS, University of Miami, Miami, United States
O’Hara, Ryan; Department of Computer Science, Harvey Mudd College, Claremont, United States
Prasath, R. Arun; Observational Seismology Group, Institute of Seismological Research, Gandhinagar, India ; Seismology and Geosciences Division, Ministry of Earth Sciences, New Delhi, India
Alves, Eduardo Queiroz; Radiocarbon Laboratory (LAC-UFF), Department of Geochemistry, Universidade Federal Fluminense, Niterói, Brazil
Raez-Villanueva, Sergio; Princess Margaret Cancer Centre, Drug Development Program - Phase 1, University Health Network, Toronto, Canada
Rosenbaum, Paul; Department of Business Studies, Uppsala University, Uppsala, Sweden ; Department of Management and Organization, Stockholm School of Economics, Stockholm, Sweden
Ruiz-Pereira, Sebastián; Departamento de Ingeniería Hidráulica y Ambiental, Pontificia Universidad Católica de Chile, Santiago, Chile ; PermaChile Network, Santiago, Chile
Savaglia, Valentina ; Université de Liège - ULiège > Département des sciences de la vie > Physiologie et génétique bactériennes ; Applied Functional Ecology, Department of Plant Biology and Ecology, Universidad de Sevilla, Sevilla, Spain ; Protistology and Aquatic Ecology, Department of Biology, Universiteit Gent, Ghent, Belgium
van Soest, Maud; Catchment to Coast, Soil and Land Use, UK Centre for Ecology and Hydrology, Bangor, United Kingdom
Vural, Deniz; Permafrost Section, Alfred Wegener Institute Helmholtz Centre for Polar and Marine Research, Potsdam, Germany ; Institute of Geosciences, University of Potsdam, Potsdam, Germany
The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. Publication of this article was funded by the University of Colorado Boulder Libraries through an institutional agreement with Frontiers. CB was supported by Universidad Cient\u00EDfica del Sur (Resoluci\u00F3n Directoral No. 008-DGIDI-CIENTIFICA-2024).
Aczel B. Szaszi B. Holcombe A. O. (2021). A billion-dollar donation: estimating the cost of researchers' time spent on peer review. Res. Integrity Peer Rev. 6:14. doi: 10.1186/s41073-021-00118-2, PMID: 34776003
APECS (2020). Call for reviewers of the second order draft of the Working Group II contribution to the IPCC Sixth Assessment Report (AR6) [Online]. Available at: https://www.apecs.is/news/apecs-news/4146-call-for-reviewers-of-the-second-order-draft-of-the-working-group-ii-contribution-to-the-ipcc-sixth-assessment-report-ar6-2.html [Accessed January 6, 2024].
Bordoloi M. Biswas S. K. (2023). Sentiment analysis: a survey on design framework, applications and future scopes. Artif. Intell. Rev. 56, 12505–12560. doi: 10.1007/s10462-023-10442-2, PMID: 37362892
Braun V. Clarke V. (2006). Using thematic analysis in psychology. Qual. Res. Psychol. 3, 77–101. doi: 10.1191/1478088706qp063oa
Cantor M. Gero S. (2015). The missing metric: quantifying contributions of reviewers. R. Soc. Open Sci. 2:140540. doi: 10.1098/rsos.140540, PMID: 26064609
Casado M. (2018). Engage more early-career scientists as peer reviewers. Nature 560:307. doi: 10.1038/d41586-018-05956-7, PMID: 30108357
Casado M. Gremion G. Rosenbaum P. Caccavo J. A. Aho K. Champollion N. et al. (2020). The benefits to climate science of including early-career scientists as reviewers. Geosci. Commun. 3, 89–97. doi: 10.5194/gc-3-89-2020
Castelló M. Kobayashi S. McGinn M. Pechar H. Vekkaila J. Wisker G. (2015). Researcher identity in transition: signals to identify and manage spheres of activity in a risk-career. Frontline Learn. Res. 3, 39–54. doi: 10.14786/flr.v3i3.149
Castree N. Bellamy R. Osaka S. (2021). The future of global environmental assessments: making a case for fundamental change. Anthropocene Rev. 8, 56–82. doi: 10.1177/2053019620971664
Dance A. (2022). Why early-career researchers should step up to the peer-review plate. Nature 602, 169–171. doi: 10.1038/d41586-022-00216-1
De Vries D. R. Marschall E. A. Stein R. A. (2009). Exploring the peer review process: what is it, does it work, and can it be improved? Fisheries 34, 270–279. doi: 10.1577/1548-8446-34.6.270
Ford J. D. Cameron L. Rubis J. Maillet M. Nakashima D. Willox A. C. et al. (2016). Including indigenous knowledge and experience in IPCC assessment reports. Nat. Clim. Chang. 6, 349–353. doi: 10.1038/nclimate2954
Gao J. Yin Y. Myers K. R. Lakhani K. R. Wang D. (2021). Potentially long-lasting effects of the pandemic on scientists. Nat. Commun. 12:6188. doi: 10.1038/s41467-021-26428-z, PMID: 34702862
Gremion G. Casado M. Aho K. Caccavo J. A. Champollion N. Choy E. et al. (2021). “What peer-review experiences can offer to early career scientists and to the scientific community,” in Proceedings of the 5th International Young Earth Scientists (YES) Congress “Rocking Earth’s Future”, 5th International Young Earth Scientists (YES) Congress “Rocking Earth’s Future”. eds. Rose T. de Gelder G. Fernández-Blanco D. Sieber M. (Berlin, Germany), 144–148.
Gropp R. E. Glisson S. Gallo S. Thompson L. (2017). Peer Review: A System under Stress. Bioscience 67, 407–410. doi: 10.1093/biosci/bix034
Gulizia C. Langendijk G. Huang-Lachmann J.-T. de Amorim Borges P. Flach R. Githaiga C. et al. (2020). Towards a more integrated role for early career researchers in the IPCC process. Clim. Chang. 159, 75–85. doi: 10.1007/s10584-019-02604-5
Harrop C. Bal V. Carpenter K. Halladay A. (2021). A lost generation? The impact of the COVID-19 pandemic on early career ASD researchers. Autism Res. 14, 1078–1087. doi: 10.1002/aur.2503, PMID: 33759380
Heggeness M. L. Gunsalus K. T. W. Pacas J. McDowell G. (2017). The new face of US science. Nature 541, 21–23. doi: 10.1038/541021a
Hindshaw R. S. Mariash H. Vick-Majors T. J. Thornton A. E. Pope A. Zaika Y. et al. (2018). A decade of shaping the futures of polar early career researchers: a legacy of the international polar year. Polar Rec. 54, 312–323. doi: 10.1017/S0032247418000591
IPCC (2013). “Appendix a: procedures for the preparation, review, acceptance, adoption, approval and publication of IPCC reports” in Principles governing IPCC work (Georgia: Batumi). Available at: https://archive.ipcc.ch/pdf/ipcc-principles/ipcc-principles-appendix-a-final.pdf
IPCC (2020). IPCC extends review of Working Group I Sixth Assessment Report Second Order Draft [Online]. Available at: https://www.ipcc.ch/2020/04/07/extends-review-wgiar6sod/ [Accessed 6 January, 2024].
Jana S. (2019). A history and development of peer-review process. Ann. Libr. Inf. Stud. 66, 152–162. doi: 10.56042/alis.v66i4.26964
Jawak S. D. Andersen B. N. Pohjola V. A. Godøy Ø. Hübner C. Jennings I. et al. (2021). SIOS’s earth observation (EO), remote sensing (RS), and operational activities in response to COVID-19. Remote Sens. 13:712. doi: 10.3390/rs13040712
Kerig P. K. (2021). Why participate in peer review? J. Trauma. Stress. 34, 5–8. doi: 10.1002/jts.22647
Kovanis M. Porcher R. Ravaud P. Trinquart L. (2016). The global burden of journal peer review in the biomedical literature: strong imbalance in the collective Enterprise. PLoS One 11:e0166387. doi: 10.1371/journal.pone.0166387, PMID: 27832157
Lee C. J. Sugimoto C. R. Zhang G. Cronin B. (2013). Bias in peer review. J. Am. Soc. Inf. Sci. Technol. 64, 2–17. doi: 10.1002/asi.22784
López-Vergès S. Urbani B. Fernández Rivas D. Kaur-Ghumaan S. Coussens A. K. Moronta-Barrios F. et al. (2021). Mitigating losses: how scientific organisations can help address the impact of the COVID-19 pandemic on early-career researchers. Human. Soc. Sci. Commun. 8:284. doi: 10.1057/s41599-021-00944-1, PMID: 34901880
Matsui A. Chen E. Wang Y. Ferrara E. (2021). The impact of peer review on the contribution potential of scientific papers. PeerJ 9:e11999. doi: 10.7717/peerj.11999, PMID: 34616596
McNair R. Le Phuong H. A. Cseri L. Szekely G. (2019). Peer review of manuscripts: a valuable yet neglected educational tool for early-career researchers. Educ. Res. Int. 2019, 1–9. doi: 10.1155/2019/1359362
Min H.-T. (2005). Training students to become successful peer reviewers. System 33, 293–308. doi: 10.1016/j.system.2004.11.003
Moraru A. Quaglia F. C. Kim M. López-Quirós A. Huynh H. M. (2024). Empowering early career polar researchers in a changing climate: challenges and solutions. PLOS Climate 3:e0000332. doi: 10.1371/journal.pclm.0000332
Myers K. R. Tham W. Y. Yin Y. Cohodes N. Thursby J. G. Thursby M. C. et al. (2020). Unequal effects of the COVID-19 pandemic on scientists. Nat. Hum. Behav. 4, 880–883. doi: 10.1038/s41562-020-0921-y, PMID: 32669671
Nowell L. S. Norris J. M. White D. E. Moules N. J. (2017). Thematic analysis: striving to meet the trustworthiness criteria. Int J Qual Methods 16, 160940691773384–160940691773313. doi: 10.1177/1609406917733847
O’Connell C. McKinnon M. (2021). Perceptions of barriers to career progression for academic women in STEM. Societies 11:27. doi: 10.3390/soc11020027
Palutikof J. P. Boulter S. L. Field C. B. Mach K. J. Manning M. R. Mastrandrea M. D. et al. (2023). Enhancing the review process in global environmental assessments: the case of the IPCC. Environ. Sci. Pol. 139, 118–129. doi: 10.1016/j.envsci.2022.10.012
Pang B. Lee L. (2008). Opinion mining and sentiment analysis. Found. Trends Inf. Retr. 2, 1–135. doi: 10.1561/1500000011
Pico T. Bierman P. Doyle K. Richardson S. (2020). First authorship gender gap in the geosciences. Earth Space Sci. 7:e2020EA001203. doi: 10.1029/2020EA001203
Reynolds J. A. Thompson R. J. J. (2017). Want to improve undergraduate thesis writing? Engage students and their faculty readers in scientific peer review. Life Sci. Educ. 10, 209–215. doi: 10.1187/cbe.10-10-0127, PMID: 21633069
Rodríguez-Bravo B. Nicholas D. Herman E. Boukacem-Zeghmouri C. Watkinson A. Xu J. et al. (2017). Peer review: the experience and views of early career researchers. Learn. Publish. 30, 269–277. doi: 10.1002/leap.1111
Sharma S. Verhagen A. Elkins M. Brismée J. M. Fulk G. D. Taradaj J. et al. (2023). Research from low-income and middle-income countries will benefit Global Health and the physiotherapy profession, but it requires support. Int. J. Sports Phys. Ther. 18:83948. doi: 10.26603/001c.83948, PMID: 37881775
Smith R. (2006). Peer review: a flawed process at the heart of science and journals. J. R. Soc. Med. 99, 178–182. doi: 10.1177/014107680609900414, PMID: 16574968
Smith M. K. Wood W. B. Adams W. K. Wieman C. Knight J. K. Guild N. et al. (2009). Why peer discussion improves student performance on in-class concept questions. Science 323, 122–124. doi: 10.1126/science.1165919, PMID: 19119232
Solomon S. Alley R. Gregory J. Lemke P. Manning M. (2008). A closer look at the IPCC report. Science 319, 409–410. doi: 10.1126/science.319.5862.409c, PMID: 18218879
van der Veer L. Visser H. Petersen A. Janssen P. (2014). Innovating the IPCC review process—the potential of young talent. Clim. Chang. 125, 137–148. doi: 10.1007/s10584-014-1145-9
Vasileiadou E. Heimeriks G. Petersen A. C. (2011). Exploring the impact of the IPCC assessment reports on science. Environ. Sci. Pol. 14, 1052–1061. doi: 10.1016/j.envsci.2011.07.002
Vuong Q.-H. Le T.-T. La V.-P. Nguyen H. T. T. Ho M.-T. Van Khuc Q. et al. (2022). Covid-19 vaccines production and societal immunization under the serendipity-mindsponge-3D knowledge management theory and conceptual framework. Human. Soc. Sci. Commun. 9:22. doi: 10.1057/s41599-022-01034-6
Walker T. R. (2018). Help graduate students to become good peer reviewers. Nature 561:177. doi: 10.1038/d41586-018-06632-6, PMID: 30209376
Williams W. M. Ceci S. J. (2012). When scientists choose motherhood: a single factor goes a long way in explaining the dearth of women in math-intensive fields How can we address it? Am. Sci. 100, 138–145. doi: 10.1511/2012.95.138, PMID: 24596430
