Excellence; Lock-ins; Open science; Pathways of change; Research culture; Research evaluation; Library and Information Sciences
Abstract :
[en] Processes of research evaluation are coming under increasing scrutiny, with detractors arguing that they have adverse effects on research quality, and that they support a research culture of competition to the detriment of collaboration. Based on three personal perspectives, we consider how current systems of research evaluation lock early career researchers and their supervisors into practices that are deemed necessary to progress academic careers within the current evaluation frameworks. We reflect on the main areas in which changes would enable better research practices to evolve; many align with open science. In particular, we suggest a systemic approach to research evaluation, taking into account its connections to the mechanisms of financial support for the institutions of research and higher education in the broader landscape. We call for more dialogue in the academic world around these issues and believe that empowering early career researchers is key to improving research quality.
Disciplines :
Arts & humanities: Multidisciplinary, general & others
Author, co-author :
De Herde, Véronique ; Université de Liège - ULiège > HEC Liège : UER > UER Management : Entrepreneuriat social et durable ; UCLouvain, Belgium
Björnmalm, Mattias ; Research and Innovation CESAER, Belgium
Susi, Toma ; Faculty of Physics, University of Vienna, Austria
Language :
English
Title :
Game over: Empower early career researchers to improve research quality
American Society for Cell Biology, “San Francisco Declaration on Research Assessment,” 2013, https://sfdora.org/read/ (accessed 22 April 2021); James Wilsdon et al., “The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management,” 2015, DOI: https://doi.org/10.13140/RG.2.1.4929.1363 (accessed 22 April 2021);
Diana Hicks et al., “Bibliometrics: The Leiden Manifesto for Research Metrics,” Nature 520, no. 7548 (April 2015): 429–31, DOI: https://doi.org/10.1038/520429a (accessed 22 April 2021);
David Moher et al., “Assessing Scientists for Hiring, Promotion, and Tenure,” PLOS Biology 16, no. 3 (March 29, 2018): e2004089, DOI: https://doi.org/10.1371/journal.pbio.2004089 (accessed 22 April 2021);
Bregt Saenen and Lidia Borrell-Damián, “Reflections on University Research Assessment: key concepts, issues and actors,” EUA Briefing, April 2019, https://eua.eu/resources/publications/825:reflections-on-university-research-assessment-key-concepts,-issues-and-actors.html (accessed 22 April 2021);
David Moher et al., “The Hong Kong Principles for Assessing Researchers: Fostering Research Integrity,” PLOS Biology 18, no. 7 (July 16, 2020): e3000737, DOI: https://doi.org/10.1371/journal.pbio.3000737 (accessed 22 April 2021).
Andrew D. Higginson and Marcus R. Munafò, “Current Incentives for Scientists Lead to Underpowered Studies with Erroneous Conclusions,” PLOS Biology 14, no. 11 (November 10, 2016): e2000995, DOI: https://doi.org/10.1371/journal.pbio.2000995 (accessed 22 April 2021);
Moher et al., “Assessing Scientists for Hiring, Promotion, and Tenure.”
Colin A. Chapman et al., “Games Academics Play and Their Consequences: How Authorship, h -Index and Journal Impact Factors Are Shaping the Future of Academia,” Proceedings of the Royal Society B: Biological Sciences 286, no. 1916 (December 4, 2019): 20192047, DOI: https://doi.org/10.1098/rspb.2019.2047 (accessed 22 April 2021).
“The Cost of Salami Slicing,” Nature Materials 4, no. 1 (January 2005): 1–1, DOI: https://doi.org/10.1038/nmat1305 (accessed 22 April 2021);
Vikas Menon and Aparna Muraleedharan, “Salami Slicing of Data Sets: What the Young Researcher Needs to Know,” Indian Journal of Psychological Medicine 38, no. 6 (November 2016): 577–78, DOI: https://doi.org/10.4103/0253-7176.194906 (accessed 22 April 2021).
Melissa S. Anderson et al., “The Perverse Effects of Competition on Scientists’ Work and Relationships,” Science and Engineering Ethics 13, no. 4 (December 2007): 437–61, DOI: https://doi.org/10.1007/s11948-007-9042-5 (accessed 22 April 2021);
Kelly L. Wester, John T. Willse, and Mark S. Davis, “Psychological Climate, Stress, and Research Integrity Among Research Counselor Educators: A Preliminary Study,” Counselor Education and Supervision 50, no. 1 (September 2010): 39–55, DOI: https://doi.org/10.1002/j.1556-6978.2010.tb00107.x (accessed 22 April 2021);
Joeri K. Tijdink, Reinout Verbeke, and Yvo M. Smulders, “Publication Pressure and Scientific Misconduct in Medical Scientists,” Journal of Empirical Research on Human Research Ethics 9, no. 5 (December 2014): 64–71, DOI: https://doi.org/10.1177/1556264614552421 (accessed 22 April 2021).
Chapman et al., “Games Academics Play and Their Consequences”; Lennart Stoy, Rita Morais, and Lidia Borrell-Damián, “Decrypting the Big Deal Landscape. Follow-up of the 2019 EUA Big Deals Survey Report,” (European University Association, October 2019), https://eua.eu/downloads/publications/2019%20big%20deals%20report.pdf (accessed 22 April 2021);
Martin Paul Eve, “The Problems of Unit Costs Per Article,” 2019, https://eve.gd/2019/09/19/the-problems-of-unit-costs-per-article/ (accessed 22 April 2021).
VSNU, NFU, KNAW, NWO, and ZonMw, “Room for Everyone’s Talent. Towards a New Balance in the Recognition and Rewards of Academics,” 2019, https://www.vsnu.nl/recognitionandrewards/wp-content/uploads/2019/11/Position-paper-Room-for-everyone’s-talent.pdf (accessed 22 April
2021);
“A kinder research culture is possible,” Nature 574, no. 7776 (October 3, 2019): 5–6, DOI: https://doi.org/10.1038/d41586-019-02951-4 (accessed 22 April 2021);
Bregt Saenen et al., “Research Assessment in the Transition to Open Science. 2019 EUA Open Science and Access Survey Results,” (European University Association, October 2019), https://eua.eu/downloads/publications/research%20assessment%20in%20the%20transition%20to%20open%20science.pdf (accessed 26 April 2021);
Noémie Aubert Bonn and Wim Pinxten, “Rethinking Success, Integrity, and Culture in Research (Part 1) – A Multi-Actor Qualitative Study on Success in Science,” Research Integrity and Peer Review 6, 1 (2021), DOI: https://doi.org/10.1186/s41073-020-00104-0 (accessed 30 April 2021);
Mattias Björnmalm et al., “Advancing Research Data Management in Universities of Science and Technology,” February 13, 2020, DOI: https://doi.org/10.5281/ZENODO.3665372 (accessed 26 April 2021).
David Nicholas et al., “How is open access publishing going down with early career researchers? An international, multi-disciplinary study,” El Profesional de La Información, 29, no. 6 (November 25, 2020), DOI: https://doi.org/10.3145/epi.2020.nov.14 (accessed 26 April 2021).
Mattias Björnmalm, Matthew Faria, and Frank Caruso, “Advancing Research Using Action Cameras,” Chemistry of Materials 28, no. 23 (December 13, 2016): 8441–42, DOI: https://doi.org/10.1021/acs.chemmater.6b04639 (accessed 26 April 2021);
Mattias Björnmalm and Frank Caruso, “Robust Chemistry: The Importanc of Data and Methods Sharing,” Angewandte Chemie International Edition 57, no. 5 (January 26, 2018): 1122–23, DOI: https://doi.org/10.1002/anie.201710493 (accessed 26 April 2021).
Mattias Björnmalm, Matthew Faria, and Frank Caruso, “Increasing the Impact of Materials in and beyond Bio-Nano Science,” Journal of the American Chemical Society 138, no. 41 (October 19, 2016): 13449–56, DOI: https://doi.org/10.1021/jacs.6b08673 (accessed 26 April 2021).
Mattias Björnmalm et al., “Bridging Bio–Nano Science and Cancer Nanomedicine,” ACS Nano 11, no. 10 (October 24, 2017): 9594–9613, DOI: https://doi.org/10.1021/acsnano.7b04855 (accessed 26 April 2021).
Matthew Faria et al., “Minimum Information Reporting in Bio–Nano Experimental Literature,” Nature Nanotechnology 13, no. 9 (September 2018): 777–85, DOI: https://doi.org/10.1038/s41565-018-0246-4 (accessed 26 April 2021).
Rik Van de Walle, “Reform Is Overdue,” Research Europe, September 17, 2020. https://www.cesaer.org/content/5-operations/2020/20200917-reform-overdue-rvdw.pdf (accessed 30 April 2021).
“Excellence R Us: University Research and the Fetishisation of Excellence,” n.d., https://hcommons.org/?get_group_doc=1001240/1507389514-ExcellenceRUs.pdf (accessed 26 April 2021).
Elizabeth Gadd, “University Rankings Need a Rethink,” Nature 587, no. 7835 (November 24, 2020): 523–523, DOI: https://doi.org/10.1038/d41586-020-03312-2 (accessed 26 April 2021).
CESAER, “Sustainable Funding for Universities of the Future,” March 16, 2020, https://doi.org/10.5281/ZENODO.3712083 (accessed 26 April 2021).
Jean Lebel and Robert McLean, “A Better Measure of Research from the Global South,” Nature 559, no. 7712 (July 2018): 23–26, DOI: https://doi.org/10.1038/d41586-018-05581-4 (accessed 26 April 2021).
Bregt Saenen et al., “Reimagining Academic Career Assessment: Stories of Innovation and Change,” (European University Association, January 2021), https://eua.eu/downloads/publications/eua-dora-sparc_case%20study%20report.pdf (accessed 26 April 2021).
Stephen Curry et al., “The Changing Role of Funders in Responsible Research Assessment: Progress, Obstacles and the Way Ahead,” RoRI Working Paper No. 3, November 2020, https://rori.figshare.com/articles/report/The_changing_role_of_funders_in_responsible_research_assessment_progress_obstacles_and_the_way_ahead/13227914 (accessed 26 April 2021).
