Critical Review of Evidence-Based Program Repositories/Registers for Behavioral Health Treatment

Sponsor:  U.S. Department of Health and Human Services | National Institutes of Health
Principal Investigator:  Dr. Stephen Magura
Co-Principal Investigator(s):  Dr. Daniela Schroeter, Dr. Chris Coryn
July 1, 2011 – June 30, 2013

 Abstract

Identifying “what works” in the behavioral health disciplines is crucial for making well-informed defensible decisions when selecting and implementing behavioral health-related interventions. As the need for relevant, accessible, and systematically derived information that supports policy decisions has increased, so has the demand for comprehensive sources of information about effective programs and intervention modalities. To this end, a number of “evidence based” repositories and registers have been developed by various federal and state organizations, NGO’s, and universities.

Unfortunately, the proliferation of potentially useful repositories/registers has presented policy-makers and practitioners with ambiguous, inconsistent, and incomplete recommendations for putative evidence-based interventions, since these databases often have dramatically different purposes, criteria for inclusion, definitions of acceptable “evidence,” and standards for designating interventions as “effective.” In order to provide policy-makers and practitioners with information on how best to use evidence-based repositories/registers in the decision-making process, this study comprehensively and critically examines the strengths, weaknesses, and limitations of these evidence-based repositories/registers.

Project Aims

1. Compile a comprehensive list of evidence-based program repositories/registers (EBPRS) for behavioral health-related interventions.

2. Classify those EBPRS according to their purposes, methodologies, acceptable types and standards of evidence, and other factors that are used to include and certify “effective” interventions.

3. Determine the practical consequences of using different types and standards of evidence for including and certifying interventions as effective.

Potential study outcomes

This study seeks to accomplish four main goals:

1) Identify the varying definitions of program efficacy, effectiveness, and readiness for dissemination used in evidence based practice.

2) Gain understanding of assumptions about what constitutes sufficient empirical evidence related to “what works” in the behavioral health disciplines.

3) Understand the criteria used to assess the credibility of the evaluation or research studies that support inclusion in evidence based repositories.

4) Understand and decompose the standards used to judge the absolute or relative effectiveness of the constituent interventions and modalities of intervention contained within evidence based repositories.

Our progress and lessons learned so far

Currently, we have completed the development of the coding structure for repositories/registers.  The study team has also developed an integrated search and coding system using relational database techniques that will support both quantitative and qualitative comparisons between evidence based repositories/registers (EBPR), and comparison and deconstruction of the criteria and standards used by these EBPRs to evaluate potential evidence based interventions. Additionally, we have worked with our expert advisory panel to closely examine our coding structure, methodology, and areas for improvement. One important addition to the study that emerged out of the expert advisory panel is an increased focus on intended users and how they are served by the repositories/registers.