Jeffrey D. Greene
Doctor of Philosophy
Department: Educational Studies
Title: A Test of Factor Analysis as a Validation Procedure
for Physician Certification Examinations
Dr. Mary Anne Bunda, Chair
Dr. James Sanders
Dr. Diane Hamilton
Date: Monday, October 14, 2002, 1:00 p.m. - 3:00 p.m.
Two physician certification
examinations from different medical specialties were investigated. The
purpose of the study was twofold: 1) to determine the similarities between
the factor structure of the examinations and their respective tables
of specifications; and, 2) to demonstrate the relative efficacy of factor
analysis in differentiating the structures between two related but dissimilar
domains of information.
Specialty A is a homogenous discipline focused on a relatively narrow
concentration of organs, body systems and anatomy. This examination
contained 309 items. There were 845 cases available for analysis. Specialty
B is a heterogeneous area of specialty concerned with numerous areas
of anatomy and physiology. The Specialty B examination contained 336
items and was completed by 1460 examinees.
The table of specifications for Specialty A called for six dimensions
of content arrayed in relatively large areas ranging from 10% to 25%
of the total examination length. The two largest areas in Specialty
A accounted for half of the content. Conversely, Specialty B contained
22 areas ranging from 1% to 11% of the total content. Its two largest
areas represented only 22% of the total examination content.
A principal components analysis with varimax rotation was conducted
on both examinations. The results of the study showed that neither obtained
structure revealed any dimensions that approximated the elements of
the respective tables of specifications. In both examinations the number
of viable obtained factors was less than the number of factors expected
by the researcher and less than the number derived by the Minimum Average
Partial procedure. The two viable factors derived from the homogenous
discipline (Specialty A), diagnostic skills and treatment choices, accounted
for about 5% of the variance. Analysis of the heterogeneous specialty
(Specialty B) returned three viable factors (diagnosis and treatment,
internal medicine, and symptom recognition), explaining about 9% of
the variance. Because of the percentage of variance explained falls
below a reasonable threshold, these results are considered to be inconclusive.
It was not possible to make a definitive statement about the comparative
structure of the two disciplines.
List of Archives: