Introduction
The Child Abuse Prevention and Treatment Act (CAPTA), first authorized in 1974 and reauthorized regularly since then, requires the examination of a wide range of topics related to the incidence of child abuse and neglect with the aim of informing efforts to better protect children from maltreatment and improve the well-being of victims of maltreatment. One promising approach to addressing these topics is to connect administrative records—such as those from child welfare, health, social services, education, public safety, and other agencies. Connecting data may help improve the quality, usefulness, interoperability, and availability of child maltreatment data.
The State Child Welfare Data Linkages (SCW) Descriptive Study aims to provide novel information regarding connected (linked or integrated) state data that may be leveraged to improve the ongoing and accurate surveillance of child maltreatment incidence and related risk.
The study examined the extent to which child welfare agencies in 50 states, Puerto Rico, and Washington, DC (hereafter collectively referred to as states) connected administrative data on child maltreatment to other data sources and aimed to learn more about agency practices related to sharing and connecting data.
The SCW Descriptive Study explored the following research questions:
What are the characteristics of states’ data systems of record?
- What states have connected data and how are those data used?
- What data sources are being connected, and what are the characteristics of the connected data?
- How do states link, manage, and govern their connected data?
- What are states’ plans for connected data and how are they building capacity?
Study Design
We designed three data collection activities to address the study’s research questions. The population of interest was state child welfare directors and child welfare technical staff from 52 states and territories with specific knowledge of the state “system of record” and connected data. We use “system of record” to refer to the state system of record for child maltreatment reporting to the National Child Abuse and Neglect Data System. We anticipated that the information needed to address the study’s research questions would come from multiple staff across varying roles within child welfare agencies. The various data collection instruments were designed to build upon one another and to tailor data collection efforts to different respondents. Exhibit 2 provides an overall picture of the target sample and respondents for each data collection activity. In this section, we discuss our overall approach to data collection, pretesting, and research approval.
Data collection approach
Our approach to instrument development aimed to maximize clarity and reduce burden for states. Across instruments, we avoided using child welfare terminology in the instruments unless it had been federally defined, and federally defined terms were accompanied by their definitions. For example, the instruments supplied definitions of “child welfare contributing agencies” when we asked how they provide data to the child welfare agency. We also used plain language to ask about data sets that could relate to child welfare by describing data contents instead of data set names, which could vary across states. For example, the survey referred to “birth records” rather than “vital statistics” or “vital records.” We designed the instruments to have minimal overlap by deciding what topics and level of detail would be covered by each instrument. Last, we elected for the surveys to be web surveys so they could be shared with and completed by multiple state staff. States were encouraged to share web links to the surveys with other designees. The team programmed surveys in a platform called Confirmit. Surveys were tested for accessibility on computers, tablets, and smartphones, and they were tested for survey specifications such as skip logic.
The Initial Survey of state child welfare directors (Initial Survey) set the foundation for future data collection activities, including identifying the best respondents for each state and gathering background information on how the state’s child welfare agencies are organized, the child welfare system of record, data management, and known data linkages. This instrument gathered information about the state structure and underlying data sets, so we could use relevant and accurate wording for subsequent survey items and apply our definition of connected data consistently across states. The instrument gathered key contextual information to determine the wording of subsequent items, including how the child welfare agencies are organized (stand-alone or part of a larger public agency); who delivers the services (local staff, state child welfare staff, or staff at other non-state agencies known as child welfare contributing agencies); and who manages and enters the data (state or county). We also collected contact information for other state contacts for connected data and used this to determine the sample of respondents for the subsequent data collection activities. The Initial Survey was sent to 52 state child welfare directors.
Pre-testing instruments
The SCW Descriptive Study team engaged with a range of experts, including current and former state child welfare agency staff, federal staff, and researchers, to review and pre-test the data collection instruments. This engagement helped shape the instruments to gather systematic information from state child welfare agencies, which vary substantially in their operation and data systems. Expert feedback on the instruments increased the clarity and relevance of questions to maximize responses. Draft instruments were reviewed by the project team and project officers in the Office of Planning, Research, and Evaluation, as well as Children’s Bureau staff, other senior researchers at Mathematica, and external consultants.
Once draft instruments had been developed, we asked for volunteers from the Casey Family Programs Data Leaders group to pre-test the instruments. The Data Leaders group is organized by Casey Family Programs and includes child welfare staff from state agencies with leadership roles related to research and evaluation, data analysis, federal reporting, quality assurance, and continuous quality improvement. Nonstate partners also participate in the Data Leaders group, including people who work with state members on research topics and former state staff with data expertise. Pre-testers identified ways that their states’ contexts could affect responses to questions about connecting state child welfare data with data from other agencies and organizations.