[en] In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence generating process (EGP). We claim that EGP variation across researchers adds uncertainty: Non-standard errors
NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for better reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.
Disciplines :
Special economic topics (health, labor, transportation...)
Author, co-author :
Menkveld, Albert J.; Vrije Universiteit Amsterdam
Dreber, Anna; Stockholm School of Economics - Department of Economics
Holzmeister, Felix; University of Innsbruck - Department of Economics
Huber, Juergen; University of Innsbruck
Johannesson, Magnus; Stockholm School of Economics - Department of Economics
Kirchler, Michael; University of Innsbruck
Razen, Michael; University of Innsbruck
Weitzel, Utz; VU University Amsterdam
Abad, David; Universidad de Alicante
Abudy, Menachem (Meni); Bar-Ilan University - Graduate School of Business Administration
Adrian, Tobias; International Monetary Fund (IMF)
Ait-Sahalia, Yacine; Princeton University - Department of Economics
Akmansoy, Olivier; CNRS
Alcock, Jamie; University of Oxford
Alexeev, Vitali; University of Technology Sydney
Aloosh, Arash; Neoma Business School
Amato, Livia; University of Chicago - Booth School of Business
Amaya, Diego; Wilfrid Laurier University
Angel, James; Georgetown University - Department of Finance
Bach, Amadeus; University of Mannheim
Baidoo, Edwin; Tennessee Technological University
Bakalli, Gaetan; Auburn University
Barbon, Andrea; University of St. Gallen
Bashchenko, Oksana; Swiss Finance Institute - HEC Lausanne
Bindra, Parampreet Christopher; University of Innsbruck
Bjonnes, Geir Hoidal; BI Norwegian Business School
Black, Jeff; University of Memphis
Black, Bernard S.; Northwestern University
Bohorquez, Santiago; Universidad EAFIT
Bondarenko, Oleg; University of Illinois at Chicago - Department of Finance
Bos, Charles S.; VU University Amsterdam
Bosch-Rosa, Ciril; Technische Universität Berlin
Bouri, Elie; Lebanese American University
Brownlees, Christian T.; Universitat Pompeu Fabra - Faculty of Economic and Business Sciences
Calamia, Anna; Toulouse Business School - TBS Education
Cao, Viet Nga; Monash University
Capelle-Blancard, Gunther; Université Paris I Panthéon-Sorbonne - Centre d'Economie de la Sorbonne (CES) ; PSB Paris School of Business
Capera, Laura; Vrije Universiteit Amsterdam
Caporin, Massimiliano; University of Padua - Department of Statistical Sciences
Carrion, Allen; University of Memphis - Fogelman College of Business and Economics
Caskurlu, Tolga; University of Amsterdam Business School
Chakrabarty, Bidisha; Saint Louis University - Richard A. Chaifetz School of Business
Chernov, Mikhail; UCLA Anderson
Cheung, William M.; Waseda University
Chincarini, Ludwig B.; University of San Francisco
Chordia, Tarun; Emory University - Department of Finance
Chow, Sheung Chi; Australian National University (ANU)
Clapham, Benjamin; Goethe University Frankfurt Faculty of Economics and Business Administration
Colliard, Jean-Edouard; HEC Paris - Finance Department
Comerton-Forde, Carole; University of Melbourne - Department of Finance
Curran, Edward; Macquarie University - Faculty of Business and Economics
Dao, Thong; Nottingham Trent University
Davies, Ryan J.; Babson College - Finance Division
Blasis, Riccardo De; Lum University Giuseppe Degennaro
Nard, Gianluca De; University of Zurich - Department of Banking and Finance
Declerck, Fany; Universite de Toulouse 1 Capitole
Deev, Oleg; Masaryk University
Degryse, Hans; KU Leuven - Faculty of Business and Economics (FEB)
Deku, Solomon; Nottingham Trent University - Nottingham Business School
Desagre, Christophe; Catholic University of Louvain (UCL) - Louvain Finance (LFIN) ; ICHEC Brussels Management School
Dijk, Mathijs A. Van; Erasmus University Rotterdam (EUR)
Dim, Chukwuma; Frankfurt School of Finance & Management
Dimpfl, Thomas; University of Hohenheim
Dong, Yun Jiang; Queen's University (Canada), Queen's School of Business, Students ; Queen’s University
Drummond, Philip; Monash University
Dudda, Tom; Technische Universität Dresden
Dumitrescu, Ariadna; ESADE Business School
Dyakov, Teodor; EDHEC Business School
Dyhrberg, Anne Haubo; The University of Sydney - Discipline of Finance
Dzieliński, Michał; Stockholm Business School, Stockholm University
Eksi, Asli; Salisbury University - Perdue School of Business
Kalak, Izidin El; Cardiff Business School
Ellen, Saskia ter; Norges Bank
Eugster, Nicolas; University of Queensland - Business School
Evans, Martin D.D.; Georgetown University - Department of Economics
Farrell, Michael; University of Virginia - Darden School of Business
Félez-Viñas, Ester; University of Technology Sydney
Ferrara, Gerardo; Bank of England
FERROUHI, El Mehdi; Ibn Tofail University
Flori, Andrea; Politecnico di Milano
Fluharty-Jaidee, Jonathan
Foley, Sean; Macquarie University
Fong, Kingsley Y. L.; University of New South Wales - School of Banking and Finance
Foucault, Thierry; HEC Paris - Finance Department
Franus, Tatiana; City, University of London - Bayes Business School
Franzoni, Francesco A.; Universita della Svizzera italiana (USI Lugano) ; USI Lugano ; Centre for Economic Policy Research (CEPR) ; Swiss Finance Institute
Frijns, Bart; Open University of the Netherlands - School of Management
Frömmel, Michael; Ghent University - Department of Financial Economics
Fu, Servanna; University of Essex
Füllbrunn, Sascha; Radboud University Nijmegen - Institute for Management Research
Gan, Baoqing; University of Technology Sydney
Gehrig, Thomas; University of Vienna
Gerritsen, Dirk; Utrecht University - School of Economics
Gil-Bazo, Javier; Universitat Pompeu Fabra ; Barcelona School of Economics
Glosten, Lawrence R.; Columbia University
Gomez, Thomas; Utrecht University
Gorbenko, Arseny; Monash University - Department of Banking and Finance
Güçbilmez, Ufuk; University of Glasgow - Adam Smith Business School
Grammig, Joachim; University of Tuebingen
Gregoire, Vincent; HEC Montreal - Department of Finance
Hagströmer, Björn; Stockholm University - Stockholm Business School
Hambuckers, Julien ; Université de Liège - ULiège > HEC Liège : UER > UER Finance et Droit : Finance de Marché
Hapnes, Erik; Aalto University
Harris, Jeffrey H.; American University - Department of Finance and Real Estate
Harris, Lawrence; University of Southern California - Marshall School of Business - Finance and Business Economics Department
Hartmann, Simon; Vienna University of Economics and Business
Hasse, Jean-Baptiste; Aix-Marseille University - Aix-Marseille School of Economics
Hautsch, Nikolaus; University of Vienna - Department of Statistics and Operations Research
He, Xuezhong; Xi'an Jiaotong-Liverpool University (XJTLU) ; University of Technology Sydney (UTS) - Finance Discipline Group, Business School
Heath, Davidson; University of Utah - David Eccles School of Business
Hediger, Simon; University of Zurich - Department of Banking and Finance
Hendershott, Terrence; University of California, Berkeley - Haas School of Business
Hibbert, Ann Marie; West Virginia University - Department of Finance
Hjalmarsson, Erik; University of Gothenburg - Centre for Finance
Hoelscher, Seth A.; Missouri State University - College of Business
Hoffmann, Peter; European Central Bank (ECB)
Holden, Craig W.; Indiana University - Kelley School of Business - Department of Finance
Horenstein, Alex R.; University of Miami - School of Business Administration - Department of Economics
Huang, Wenqian; Bank for International Settlements
Huang, Da; University of Utah - David Eccles School of Business
Hurlin, Christophe; University of Orleans
Ivashchenko, Alexey; Vrije Universiteit Amsterdam
Iyer, Subramanian R.; University of New Mexico
Jahanshahloo, Hossein; Cardiff University
Jalkh, Naji; Saint Joseph University
Jones, Charles M.; Columbia University
Jurkatis, Simon; Bank of England
Jylha, Petri; Aalto University
Kaeck, Andreas; University of Sussex
Kaiser, Gabriel; Universite du Luxembourg
Karam, Arzé; Durham University
Karmaziene, Egle; Vrije Universiteit Amsterdam
Kassner, Bernhard; Ludwig-Maximilians-Universität München
Kaustia, Markku; Aalto University
Kazak, Ekaterina; University of Manchester
Kearney, Fearghal; Queen's University Belfast - Queen's Management School
Kervel, Vincent van; Pontifical Catholic University of Chile
Khan, Saad; Queen's University - Smith School of Business ; HEC Montreal
Khomyn, Marta; University of Technology Sydney (UTS)
Klein, Tony; Queen's University Belfast - Queen's Management School
Klein, Olga; University of Warwick - Warwick Business School
Klos, Alexander; University of Kiel - Institute for Quantitative Business and Economics Research (QBER)
Koetter, Michael; Halle Institute for Economic Research
Krahnen, Jan Pieter; Goethe University Frankfurt
Kolokolov, Aleksey; University of Manchester - Manchester Business School
Korajczyk, Robert A.
Kozhan, Roman; University of Warwick - Warwick Business School
Kwan, Amy; University of Sydney Business School
Lajaunie, Quentin; Université Paris Dauphine
Lam, FY Eric C; Hong Kong Institute for Monetary and Financial Research
Lambert, Marie ; Université de Liège - ULiège > HEC Liège : UER > UER Finance et Droit : Analyse financière et finance d'entr.
Langlois, Hugues; HEC Paris - Finance Department
Lausen, Jens; Goethe University Frankfurt - Faculty of Economics and Business Administration
Lauter, Tobias; Leibniz University Hannover
Leippold, Markus; University of Zurich ; Swiss Finance Institute
Levin, Vladimir; Universite du Luxembourg
Li, Yijie; S&P Global Ratings
Li, (Michael) Hui; La Trobe University
Liew, Chee Yoong; UCSI University, Malaysia
Lindner, Thomas; Vienna University of Economics and Business
Linton, Oliver B.; University of Cambridge
Liu, Jiacheng; Purdue University
Liu, Anqi; The University of Sydney
Llorente, Guillermo; Universidad Autonoma de Madrid
Lof, Matthijs; Aalto University
Lohr, Ariel; Arizona State University (ASU) - Finance Department
Longstaff, Francis A.; University of California, Los Angeles (UCLA) - Finance Area
Lopez-Lira, Alejandro; University of Florida - Department of Finance, Insurance and Real Estate
Mankad, Shawn; Cornell University
Mano, Nicola; Swiss Finance Institute - USI Lugano
Marchal, Alexis; EPFL
Martineau, Charles; University of Toronto - Rotman School of Management and UTSC Management
Mazzola, Francesco; Erasmus University Rotterdam (EUR)
Meloso, Debrah; Toulouse Business School - TBS Education
Mihet, Roxana; Swiss Finance Institute - HEC Lausanne
Mohan, Vijay; RMIT University
Moinas, Sophie; Universite de Toulouse 1 Capitole
Moore, David; Loyola Marymount University
Mu, Liangyi; Queen's University Belfast ; University of Manchester
Muravyev, Dmitriy; Michigan State University - Department of Finance ; Canadian Derivatives Institute
Murphy, Dermot; University of Illinois at Chicago
Neszveda, Gabor; John von Neumann University
Neumeier, Christian; Justus Liebig University Giessen
Nielsson, Ulf; Copenhagen Business School
Nimalendran, Mahendrarajah; University of Florida - Department of Finance, Insurance and Real Estate
Nolte, Sven; Radboud University
Norden, Lars L.; Stockholm University - Stockholm Business School
O'Neill, Peter; Financial Conduct Authority
Obaid, Khaled; California State University-East Bay
Ødegaard, Bernt Arne; University of Stavanger
Östberg, Per; University of Zurich - Department of Banking and Finance ; Swiss Finance Institute
Painter, Marcus; Saint Louis University - Department of Finance
Palan, Stefan; University of Graz
Palit, Imon; RMIT University - Blockchain Innovation Hub
Park, Andreas; University of Toronto at Mississauga
Pascual, Roberto; Universidad de las Islas Baleares
Pasquariello, Paolo; University of Michigan, Stephen M. Ross School of Business
Pastor, Lubos; University of Chicago - Booth School of Business
Patel, Vinay; University of Technology Sydney (UTS)
Patton, Andrew J.; Duke University - Department of Economics
Pearson, Neil D.; University of Illinois at Urbana-Champaign - Department of Finance
Pelizzon, Loriana; Goethe University Frankfurt - Faculty of Economics and Business Administration
Pelster, Matthias; Paderborn University
Pérignon, Christophe; HEC Paris - Finance Department
Pfiffer, Cameron; University of Oregon - Department of Finance
Philip, Richard; University of Sydney Business School
Plíhal, Tomáš; Masaryk University - Faculty of Economics and Administration
Prakash, Puneet; Missouri State University
Press, Oliver-Alexander; Copenhagen Business School
Prodromou, Tina; The University of Wollongong
Putniņš, Tālis J.; University of Technology Sydney (UTS) ; Stockholm School of Economics, Riga
Raizada, Gaurav; Indian Institute of Management, Ahmedabad
Rakowski, David A.; University of Texas at Arlington
Ranaldo, Angelo; University of St. Gallen
Regis, Luca; University of Turin
Reitz, Stefan; University of Kiel
Renault, Thomas; Université Paris I Panthéon-Sorbonne - Centre d'Economie de la Sorbonne (CES)
Renjie, Rex Wang; Vrije Universiteit Amsterdam
Renò, Roberto; University of Verona - Department of Economics
Riddiough, Steven; University of Toronto
Rinne, Kalle; Universite du Luxembourg - Department of Finance
Rintamäki, Paul; Aalto University
Riordan, Ryan; Queen's University - Smith School of Business
Rittmannsberger, Thomas; University of Innsbruck
Rodríguez-Longarela, Iñaki; Stockholm University - Stockholm Business School
Rösch, Dominik; State University of New York at Buffalo - School of Management
Rognone, Lavinia; University of Manchester - Alliance Manchester Business School
Roseman, Brian; Oklahoma State University - Department of Finance
Rosu, Ioanid; HEC Paris - Finance Department
Roy, Saurabh; University of Massachusetts Amherst
Rudolf, Nicolas; University of Lausanne
Rush, Stephen; Bowling Green State University - Department of Finance ; Vietnam National University - Ho Chi Minh City (VNU-HCM) - University of Economics and Law
Rzayev, Khaladdin; University of Edinburgh ; Koc University
Rzeźnik, Aleksandra; York University
Sanford, Anthony; University of Maryland
Sankaran, Harikumar; New Mexico State University
Sarkar, Asani; Federal Reserve Bank of New York
Sarno, Lucio; University of Cambridge - Judge Business School ; Centre for Economic Policy Research (CEPR)
Scaillet, O.; Swiss Finance Institute - University of Geneva
Scharnowski, Stefan; University of Mannheim
Schenk-Hoppé, Klaus Reiner; University of Manchester - Department of Economics
Schertler, Andrea; University of Graz
Schneider, Michael; Deutsche Bundesbank
Schroeder, Florian; Macquarie University
Schürhoff, Norman; Swiss Finance Institute - HEC Lausanne
Schuster, Philipp; University of Stuttgart
Schwarz, Marco A.; Heinrich Heine University Dusseldorf - Duesseldorf Institute for Competition Economics (DICE)
Seasholes, Mark S.; Arizona State University (ASU)
Seeger, Norman; Vrije Universiteit Amsterdam
Shachar, Or; Federal Reserve Bank of New York
Shkilko, Andriy; Wilfrid Laurier University - Lazaridis School of Business and Economics
Shui, Jessica; Federal Housing Finance Agency
Sikic, Mario; University of Zurich
Simion, Giorgia; Vienna University of Economics and Business
Smales, Lee A.; University of Western Australia
Söderlind, Paul; University of St. Gallen
Sojli, Elvira; UNSW Australia Business School, School of Banking and Finance
Sokolov, Konstantin; University of Memphis - Fogelman College of Business and Economics
Spokeviciute, Laima; Cardiff University
Stefanova, Denitsa; Universite du Luxembourg
Subrahmanyam, Marti G.; New York University (NYU) - Leonard N. Stern School of Business
Neusüss, Sebastian; Aalto University
Szaszi, Barnabas; Eötvös Loránd University
Talavera, Oleksandr; University of Birmingham
Tang, Yuehua; University of Florida - Department of Finance
Taylor, Nicholas; University of Bristol - School of Economics, Finance and Management
Tham, Wing Wah; University of New South Wales (UNSW)
Theissen, Erik; University of Mannheim - Finance Area
Thimme, Julian; Karlsruhe Institute of Technology
Tonks, Ian; University of Bristol
Tran, Hai; Loyola Marymount University - Department of Finance
Trapin, Luca; University of Bologna
Trolle, Anders B.; Copenhagen Business School
Valente, Giorgio; Hong Kong Institute for Monetary and Financial Research (HKIMR)
Ness, Robert A. Van; University of Mississippi - Department of Finance
Vasquez, Aurelio; Instituto Tecnológico Autónomo de México (ITAM) - Department of Business Administration
Verousis, Thanos; University of Essex
Verwijmeren, Patrick; Erasmus University Rotterdam (EUR)
Vilhelmsson, Anders; Lund University - Department of Economics
Vilkov, Grigory; Frankfurt School of Finance & Management
Vladimirov, Vladimir; University of Amsterdam Business School
Vogel, Sebastian; Erasmus University Rotterdam (EUR)
Voigt, Stefan; University of Copenhagen ; Danish Finance Institute
Wagner, Wolf; Erasmus University Rotterdam (EUR)
Walther, Thomas; Utrecht University - School of Economics
Weiss, Patrick; Vienna Graduate School of Finance (VGSF)
Wel, Michel van der; Erasmus University Rotterdam
Werner, Ingrid M.; The Ohio State University - Fisher College of Business
Westerholm, P. Joakim; University of Sydney Business School
Westheide, Christian; University of Vienna - Department of Finance
Wipplinger, Evert; Vrije Universiteit Amsterdam, School of Business and Economics
Wolf, Michael; University of Zurich - Department of Economics
Wolff, Christian C. P.; University of Luxembourg ; Centre for Economic Policy Research (CEPR)
Wolk, Leonard; Vrije Universiteit Amsterdam
Wong, Wing Keung; Auburn University
Wrampelmeyer, Jan; Vrije Universiteit Amsterdam, School of Business and Economics
Xia, Shuo; Halle Institute for Economic Research
Xiu, Dacheng; University of Chicago - Booth School of Business
Xu, Ke; University of Victoria
Xu, Caihong; Stockholm University - Stockholm Business School
Yadav, Pradeep K.; University of Oklahoma Price College of Business
Yagüe, José; University of Murcia
Yan, Cheng; University of Essex - Essex Business School
Yang, Antti; Erasmus University Rotterdam
Yoo, Woongsun; Central Michigan University
Yu, Wenjia; Aalto University
Yu, Shihao; Vrije Universiteit Amsterdam
Yueshen, Bart Z.; INSEAD - Finance
Yuferova, Darya; Norwegian School of Economics (NHH) - Department of Finance
Zamojski, Marcin; University of Gothenburg, Centre for Finance
Zareei, Abalfazl; Stockholm University
Zeisberger, Stefan; Radboud University, Institute for Management Research ; University of Zurich, Department of Banking and Finance
Zhang, S. Sarah; University of Manchester - Alliance Manchester Business School
Zhang, Xiaoyu; Vrije Universiteit Amsterdam
Zhong, Zhuo; University of Melbourne - Department of Finance
Zhou, Z. Ivy; University of Wollongong - School of Accounting, Economics & Finance
Zhou, Chen; Erasmus University Rotterdam (EUR)
Zhu, Xingyu Sonya; Bank for International Settlements (BIS)
Zoican, Marius; University of Toronto at Mississauga - Department of Management ; University of Toronto - Rotman School of Management
Zwinkels, Remco C. J.; VU University Amsterdam - Department of Finance and Financial Sector Management
Chen, Jian; Queen's University
Duevski, Teodor; HEC Paris
Gao, Ge; University of Birmingham
Gemayel, Roland; King's College London
Gilder, Dudley; Cardiff University
Kuhle, Paul; Universidad Autonoma de Madrid
Pagnotta, Emiliano; Singapore Management University
Pelli, Michele; University of Zurich - Department of Banking and Finance ; Swiss Finance Institute
Sönksen, Jantje; University of Tübingen
Zhang, Lu; University of Luxembourg
Ilczuk, Konrad
Bogoev, Dimitar; EDF Energy, United Kingdom
Qian, Ya; Aalto University
Wika, Hans C.; Norges Bank
Yu, Yihe; University at Buffalo, SUNY
Zhao, Lu; Southwestern University of Finance and Economics (SWUFE)
Mi, Michael; The University of Sydney
Bao, Li; University of Toulouse Capitole
Vaduva, Andreea; UC3M
Prokopczuk, Prokopczuk; Leibniz University Hannover
Avetikian, Alejandro; Pontifical Catholic University of Chile
Wu, Zhen-Xing; Zhongnan University of Economics and Law - School of Finance
Dare, Wale ; Université de Liège - ULiège > HEC Liège : UER > UER Finance et Droit : Analyse financière et finance d'entr.
