Exploratory studies to decide whether and how to proceed with full scale evaluations of public health interventions: A systematic review of guidance

Abstract

Background: Evaluations of complex interventions in public health are frequently undermined by problems that can be identified before the effectiveness study stage. Exploratory studies, often termed pilot and feasibility studies, are a key step in assessing the feasibility and value of progressing to an effectiveness study. Such studies can provide vital information to support more robust evaluations, thereby reducing costs and minimising potential harms of the intervention. This systematic review forms the first phase of a wider project to address the need for stand-alone guidance for public health researchers on designing and conducting exploratory studies. The review objectives were to identify and examine existing recommendations concerning: when such studies should be undertaken, questions they should answer, suitable methods, criteria for deciding whether to progress to an effectiveness study, and appropriate reporting. Methods: We searched for published and unpublished guidance reported between January 2000 and November 2016 via bibliographic databases, websites, citation tracking and expert recommendations. Included papers were thematically synthesized. Results: The search retrieved 4,095 unique records. 30 papers were included, representing 25 unique sources of guidance/recommendations. Eight themes were identified: pre-requisites for conducting an exploratory study, nomenclature, guidance for intervention assessment, guidance surrounding any future evaluation study design, flexible versus fixed design, progression criteria to a future evaluation study, stakeholder involvement and reporting of exploratory studies. Exploratory studies were described as being concerned with the intervention content, the future evaluation design, or both. However, the nomenclature and endorsed methods underpinning these aims were inconsistent across papers. There was little guidance on what should precede or follow an exploratory study and decision making surrounding this. Conclusions: Existing recommendations are inconsistent concerning the aims, designs and conduct of exploratory studies and guidance is lacking on the evidence needed to inform when to proceed to an effectiveness study.