Study-design selection criteria in systematic reviews of effectiveness of health systems interventions and reforms: A meta-review

Rockers, Peter C.; Feigl, Andrea B.; Røttingen, John-Arne; Fretheim, Atle; de Ferranti, David; Lavis, John N.; Melberg, Hans Olav; Bärnighausen, Till

doi:10.1016/j.healthpol.2011.12.007

« Previous Next »Health Policy
Volume 104, Issue 3 , Pages 206-214, March 2012

Study-design selection criteria in systematic reviews of effectiveness of health systems interventions and reforms: A meta-review

Peter C. Rockers
- Department of Global Health and Population, Harvard School of Public Health, Boston, MA, USA
Andrea B. Feigl
- Department of Global Health and Population, Harvard School of Public Health, Boston, MA, USA
John-Arne Røttingen
- Norwegian Knowledge Centre for the Health Services, Oslo, Norway
- Institute of Health and Society, Department of Health Management and Health Economics, University of Oslo, Norway
Atle Fretheim
- Norwegian Knowledge Centre for the Health Services, Oslo, Norway
- Institute of Health and Society, Department of Community Medicine, University of Oslo, Norway
David de Ferranti
- Results for Development Institute, Washington, DC, USA
John N. Lavis
- McMaster Health Forum, Centre for Health Economics and Policy Analysis, Department of Clinical Epidemiology and Biostatistics, and Department of Political Science, McMaster University, Hamilton, ON, Canada
Hans Olav Melberg
- Institute of Health and Society, Department of Health Management and Health Economics, University of Oslo, Norway
Till Bärnighausen
- Department of Global Health and Population, Harvard School of Public Health, Boston, MA, USA
- Africa Centre for Health and Population Studies, University of KwaZulu-Natal, Mtubatuba, South Africa
- Corresponding author at: Department of Global Health and Population, Harvard School of Public Health, 665 Huntington Avenue, 02115 Boston, MA, USA. Tel.: +1 615 379 0372.

Received 18 September 2011; received in revised form 28 November 2011; accepted 15 December 2011. published online 13 February 2012.

Abstract

At present, there exists no widely agreed upon set of study-design selection criteria for systematic reviews of health systems research, except for those proposed by the Cochrane Collaboration's Effective Practice and Organisation of Care (EPOC) review group (which comprises randomized controlled trials, controlled clinical trials, controlled before-after studies, and interrupted time series). We conducted a meta-review of the study-design selection criteria used in systematic reviews available in the McMaster University's Health Systems Evidence or the EPOC database. Of 414 systematic reviews, 13% did not indicate any study-design selection criteria. Of the 359 studies that described such criteria, 50% limited their synthesis to controlled trials and 68% to some or all of the designs defined by the EPOC criteria. Seven out of eight reviews identified at least one controlled trial that was relevant for the review topic. Seven percent of the reviews included either no or only one relevant primary study. Our meta-review reveals reviewers’ preferences for restricting synthesis to controlled experiments or study designs that comply with the EPOC criteria. We discuss the advantages and disadvantages of the current practices regarding study-design selection in systematic reviews of health systems research as well as alternative approaches.

Keywords: Health systems research, Systematic reviews, Study design, Selection criteria