Assessing Vocational Rehabilitation Agency Capacity to Engage in Evidence-Based Decision Making

Background: The Evidence-based Policy-Making Act of 2018 requires that Federal agencies use their data to develop statistical evidence to support policy and programmatic decisions. Objective: This study assessed Vocational Rehabilitation agency capacity to effectively use their data to inform evidence-based decision-making. Methods: The Capacity Survey assessed agency capacity in data management, data visualization and statistical analysis. The survey asked for details about (1) the availability of relevant software programs (e.g., SPSS for statistical analysis), and (2) staff expertise to use that software. Results: Results pointed to capacity gaps that would significantly hinder most agencies’ application of even a simplified return on investment model. When examining statistical capacity, 60% of agencies responded “not applicable – staff do not have competence in the listed software packages (including SPSS, SAS, R, Python, Stata, and other)” and 71% of respondents said they lacked internal statistical capacity to analyze data using any of the listed programs. Conclusions: Results suggested that many state VR agencies lack internal capacity to meet requirements outlined in the Evidence-Based Policy-Making Act of 2018 or regulations related to reporting and performance accountability requirements. Potential solutions to overcome capacity deficits include expanding internal capacity, expanding agency/consultant partnerships, and building cross-agency collaborations.