Introduction
This Prevention Services Clearinghouse Handbook of Standards and Procedures, Version 2.0 (Handbook Version 2.0) provides a detailed description of the standards used to identify and review programs and services for the Prevention Services Clearinghouse and the procedures followed by the Prevention Services Clearinghouse staff.
Purpose of the Title IV-E Prevention Services Clearinghouse
The Title IV-E Prevention Services Clearinghouse (hereafter referred to as the Prevention Services Clearinghouse) was established in 2019 by the Administration for Children and Families (ACF) within the U.S. Department of Health and Human Services (HHS) to systematically review existing research on programs and services intended to provide enhanced support to children and families and prevent foster care placements.
The Prevention Services Clearinghouse, developed in accordance with the Family First Prevention Services Act (FFPSA) of 2018, as codified in Title IV-E of the Social Security Act, rates programs and services as promising, supported, and well-supported practices. These practices include mental health and substance use prevention and treatment programs and services, in-home parent skill-based programs and services, as well as kinship navigator programs. The Prevention Services Clearinghouse was developed to be an objective, rigorous, and transparent source of information on evidence-based programs and services that may be eligible for funding under Title IV-E of the Social Security Act as amended by the FFPSA. The Prevention Services Clearinghouse uses a systematic review process implemented by trained reviewers using consistent, transparent standards and procedures (see Exhibit 1 below).
The Prevention Services Clearinghouse Review Process
The Prevention Services Clearinghouse systematic review process, described in detail in the chapters that follow and shown in Exhibit 1, includes the following steps:
- Identify programs and services. Candidate programs and services relevant to the mission of the Prevention Services Clearinghouse are identified using an inclusive process that invites recommendations from any member of the public, including state and local government administrators and tribes, to ensure broad coverage across program or service areas and populations served (Chapter 1).
- Prioritize and select programs and services. Candidate programs and services are prioritized from those identified using criteria that include evidence of eligibility and being in active use, recommendations received, child welfare relevance, population(s) served (e.g., intending to serve underserved communities or tribal groups and nations; evidence of culturally adapted or grounded program or service content), previous evaluations and studies, implementation supports, and coverage across program or service areas. After prioritization, programs or services are selected and added to the working list of programs and services planned for review (Chapter 2).
- Literature search. Prevention Services Clearinghouse staff conduct comprehensive literature searches to locate available and relevant research on programs and services selected for review. In addition to an electronic bibliographic database search, this process includes scans of other clearinghouses and of program or service websites, review of grey literature sources, and review of submissions of research from the public (e.g., citation lists of published research or notification of individual studies published) (Chapter 3).
- Study eligibility screening and prioritization. Studies identified in the literature searches are screened against the study eligibility criteria. Studies determined to be eligible for review are considered against study prioritization criteria to determine the order and depth of their review, which include: whether study samples are from underserved communities and their child welfare relevance; study design, sample size, statistical power, outcome domains, and preregistration; and duration of effects examined (Chapter 4).
- Evidence review using the design and execution standards. All prioritized studies are reviewed by trained reviewers using the Prevention Services Clearinghouse design and execution standards. Additional studies may be reviewed using the design and execution standards, as indicated, according to study prioritization and risk of harm procedures. One of three ratings is assigned to studies reviewed using the design and execution standards: high, moderate, or low support of causal evidence (Chapter 5). Study authors may be queried to request information deemed necessary to assign a rating.
- Record and characterize impact estimates. To inform program or service ratings of well-supported, supported, or promising, the Prevention Services Clearinghouse characterizes impact estimates from high and moderate-rated contrasts as favorable, sustained favorable, unfavorable, or no effect. Characterization of impact estimates as favorable, sustained favorable, no effect, or unfavorable is based on both their direction and statistical significance (Chapter 6).
- Program or service ratings. Studies that are rated as high or moderate support of causal evidence are considered in assigning each program or service one of four ratings: well-supported, supported, promising, or does not currently meet criteria (Chapter 7). These ratings take into consideration the characterization of the impacts as favorable, sustained favorable, unfavorable, or no effect and any evidence of risk of harm.
Chapter 1. Identify Program and Services
The Prevention Services Clearinghouse identifies programs and services using an inclusive process that relies on the following sources:
- Recommendations submitted to the Prevention Services Clearinghouse from public calls. At least annually, the Prevention Services Clearinghouse issues a public call for recommendations of programs and services. These calls are an opportunity for any member of the public, including state, tribal, or local government administrators, to recommend new programs and services for systematic review as the field continues to evolve. The Prevention Services Clearinghouse announces public calls on its website and email list.
- Recommendations submitted to the Prevention Services Clearinghouse via email at any time from any member of the public and other key advisors, including federal partners, and state, tribal, and local administrators.
The public call materials and FAQ page on the Clearinghouse website provide detailed information on how to recommend a program or service for review as well as suggested information to include with a program or service recommendation – including how recommendations are responsive to program or service prioritization criteria (see Chapter 2) and any relevant study citations (see Chapter 3).
Prevention Services Clearinghouse staff log all program and service recommendations and respond to submitters notifying them that their recommendations have been received. The Prevention Services Clearinghouse retains all submissions for consideration for future review cycles. This includes recommendations from a 2018 Federal Register Notice (FRN) (83 FR 29122) in addition to all public call and ad hoc submissions.
Chapter 2. Prioritize and Select Programs and Services
Programs and services identified via the procedures described in the previous chapter are then prioritized for review using the Program or Service Prioritization Criteria. Prioritization includes an initial assessment of program or service eligibility using the Program or Service Eligibility Criteria specified below. Informed by the prioritization process, programs and services are then selected for review, and a final eligibility determination is made. Clearinghouse prioritization and selection processes are informed by expert engagement, including individuals with lived expertise. This chapter also details procedures used when there are multiple manuals or versions of a program or service for the purposes of prioritization and selection of programs and services for review.