Best-worst scaling identified adequate statistical methods and literature search as the most important items of AMSTAR2 (A measurement tool to assess systematic reviews)

Objective: To assess the relative importance of A MeaSurement Tool to Assess systematic Reviews 2 (AMSTAR2) items.

Study design and setting: A best-worst scaling object case was conducted among a sample of experts in the field of systematic reviews (SRs) and meta-analyses (MAs). Respondents were asked in a series of 15 choice tasks to choose the most and the least important item from a set of four items from the master list, which included the 16 AMSTAR2 items. Hierarchical Bayes analysis was used to generate the relative importance score for each item.

Results: The most important items highlighted by our 242 experts to conduct overview of reviews and critically assess SRs/MAs were the appropriateness of statistical analyses and adequacy of the literature search, followed by items regarding the assessment of risk of bias, the research protocol, and the assessment of heterogeneity (relative importance score >6.5). Items related to funding sources and the assessment of study selection and data extraction in duplicate were rated as least important.

Conclusion: Although all AMSTAR2 items can be considered as important, our results highlighted the importance of keeping the two items (the appropriateness of statistical analyses and the adequacy of the literature search) among the critical items proposed by AMSTAR2 to critically appraise SRs/MAs.