Dissertation Defenses

Doctoral Dissertation Announcement

---

Candidate: Jafar Al-Momani

Degree of: Doctor of Philosophy

Department: Educational Leadership, Research and Technology

Title: Metaevaluation Case Study of Four Evaluations of OSHA Voluntary Protection Programs

Committee:
Dr. Liliana Rodriguez-Campos, Chair
Dr. Nancy Barnes Mansberger
Dr. Sue Poppnik


Date: Friday, July 6, 2007 10:00 a.m. – 1:00 p.m.
3208 Sangren Hall

Abstract:
The purpose of this study was to investigate the controversy about shifts in the U.S. Administration’s policy related to the Labor Department (OSHA) shift from enforcement to partnerships and voluntary protection programs (VPP). A metaevaluation of four OSHA VPP evaluation reports was conducted and included the investigation of objectives, methodologies, strengths, weaknesses, and areas for improvement. Conducting a crosswalk of the 1994 Joint Committee evaluation standards (JCS) and the Government Accountability Office Standards (GAO) standards was an opportunity to provide additional validity to the evaluation standards, evaluation reports findings and conclusions, and to highlight the importance of the shared elements between JCS and GAO. Applying these standards in the metaevalautions helped to select the better fit standards for evaluating specific programs like safety programs.
Findings of this study showed that JCS and GAO had a good number of shared elements. Metaevaluations revealed information about the usability of GAO standards for program evaluation, and supported the endeavors to link auditing and metaevaluation. Findings highlighted the applicability and usefulness of metaevaluation methodology to other disciplines like safety. Crosswalk of the two standards was a useful approach to improve and validate evaluations and standards. Subjectivity of evaluator cannot be eliminated, but it can be reduced by increasing the evaluator’s competencies. JCS showed a better fit to safety-specific programs like OSHA VPP. Human subjects-related requirements, diversity of values, cultural differences, and attention to non-English speaking stakeholders were not clearly addressed by evaluators. In addition to the relative subjectivity of evaluators, limitations to this study included lack of accessibility to reports’ evaluators and the lack of rubrics to guide rating of reports.