Critical Appraisal of Systematic Reviews With Costs and Cost-Effectiveness Outcomes: An ISPOR Good Practices Task Force Report

Abstract

A systematic review (SR) can provide rigorous and complete evidence to support decision makers who consider both the effectiveness and cost-effectiveness of health interventions. A dramatic increase in published health economic (HE) studies, more specifically cost and cost-effectiveness studies, has resulted in the consequent proliferation of systematic reviews with cost and cost-effectiveness outcomes (SR-CCEO).First, such reviews help to indentify strenghts and weaknesses in HE studies, modelling methodologies, and data for modelling inputs. Second, SR-CCEOs may be informative for decisionmakers in resource allocation decisions for health interventions, especially in countries with limited capacity for health technology assessment (HTA). For the purpose of this article, cost studies are defined as studies analyzing the costs of healthcare interventions, includingcost descriptions and cost-of-illness (economic burden of disease) studies. By cost-effectivenessstudies we mean full economic evaluations, including cost-minimization, cost-effectiveness analysis, cost-utility analysis, cost-benefit analysis, and cost-consequence analysis. Sometimes cost studies might be based on an explicit comparison of alternatives.However, it is challenging to appropriately interpret SR-CCEOs owing to their heterogeneity in applied methods and reporting, and furthermore, owing to variability in clinical and health settings in the original studies they include. Methodologic guidance and checklists that improve the quality of SRs on clinical evidence or decrease risk of bias in their interpretation or synthesis have limited applicability for SR-CCEOs. There is little specific methodologicguidance for SR-CCEOs.Although Chapter 20 of the Cochrane Handbook for Systematic Reviews of Interventions of the Cochrane Collaboration 12 and 3 articles related to informing clinical practice guidelines provide guidance, their recommendations do not focus on evaluating the quality of conduct or the risk of bias in SR-CCEOs. A critical analysis of guidelines on conducting and reporting SR-CCEOs identified multiple disagreements in these recommendations, suggesting that a standardized approach to conducting SR-CCEOs is needed.Making universal recommendations for SR-CCEOs is difficult because they differ in several important aspects, in particular, with regard to their search and inclusion criteria, such as the types of studies included (trial or model-based, cost, or cost-effectiveness), or in reporting solely economic characteristics or economic data alongside clinical outcomes. They also have different objectives (eg, to assess variability in outcomes and synthesize the findings) to identify the evidence gaps, or to assess the methods used.Overall, SR-CCEO reliability and usefulness will improve with good practice guidance for SR-CCEOs with different objectives. Thus, ISPOR (The Professional Society for Health Economics and Outcomes Research) established a global, multistakeholder, multidisciplinary expert task force to address this need (Appendix 1 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.01.002). Although general recommendations on conducting SR-CCEOs are provided, the main goal is guidance on critical appraisal of SR-CCEOs regarding their quality and risk of bias. This report, which includes the ISPOR Criteria for Cost(-Effectiveness) Review Outcomes (CiCERO) Checklist, will assist researchers, producers of health technologies, and evidence users (decision makers/commissioners). The task force categorized the recommendations according to the 6 stages of conducting an SR-CCEO (Table 1).