Evaluations

Evaluations are crucial to studying workforce programs. The Heldrich Center conducts a variety of types of evaluations: impact evaluations, implementation evaluations, and randomized control trials.

Impact evaluations measure change due to an intervention. For example, the Heldrich Center will be measuring the impact of three programs in the recruitment and retention of certified home health aides (CHHAs). The demand for CHHAs is high, but the challenges to meet the demand include low wages and poor quality of work. The Heldrich Center will recruit new CHHAs in New Jersey and train them through the implementation of three training programs. Center researchers will analyze the outcomes and effectiveness (completion rate, wages, and retention) as a result of implementing these programs.

Implementation evaluations study the process of a particular intervention. Center researchers evaluate whether the intervention is working as intended throughout the process. A current project is the evaluation of New Jersey pilot programs that will facilitate people with disabilities into working in competitive integrated employment. To study the implementation, the Center will start with a baseline survey and interviews with the participants. Partway through the implementation process, Center researchers will interview participants about their progress. Finally, at the end of the program intervention, additional interviews will be conducted and a report produced on the process of organizational change.

Other evaluations conducted by the Heldrich Center include studies of Workforce Innovation and Opportunity Act (WIOA) Title I services during Program Years (PY) 2020 and 2021, defined as the periods of July 1, 2020 to June 30, 2021 and July 1, 2021 to June 30, 2022, respectively. The PY 2020 study encompassed all three WIOA Title I main populations — adult, dislocated worker, and youth services — and examined both career and training services. The goal of that study was to examine WIOA Title I service delivery to understand how the New Jersey Department of Labor and Workforce Development (NJDOL) and local areas adapted services to virtual, remote, or hybrid formats to accommodate the influx of job seekers and the public health directives at that time, and how virtual and remote services affected overall WIOA Title I services in PY 2020. The objectives of the PY 2021 evaluation were to assess the nature, extent, and scope of transportation services provided to Title I and Title IV customers; examine service delivery strategies currently implemented in New Jersey’s southern/rural counties; identify transportation services that warrant further evaluation, expansion, and/or replication; and recommend modifications to address transportation barriers that WIOA customers, particularly those with disabilities, face in New Jersey.

Finally, randomized control trials and experiments randomly assign study participants to different groups as a way to compare and assess the effects of educational programs and social interventions. An example of a randomized control trial is the Center’s current evaluation of New Jersey’s Reemployment Services and Eligibility Assessment program. Funded by NJDOL, the program is designed to help dislocated workers receiving unemployment compensation to effectively return to work. In cooperation with NJDOL, the Heldrich Center is conducting a randomized controlled trial of a text message intervention providing program participants with motivational messages. The evaluation will assess the effect of text notifications with motivational messages on job seekers’ participation rates in the program, their duration of unemployment, and wages after reemployment. The research team is currently in the field, receiving weekly data files from NJDOL, randomizing participants, and sending text messages to those in the treatment group.

Looking for assistance with an evaluation or a randomized control trial?

Reach out to us at hcwd@ejb.rutgers.edu to see how we can help.