Dissertation Defenses

Doctoral Dissertation Announcement

---

Candidate: Dawn M. Mackety

Degree of: Doctor of Philosophy

Department: Evaluation, Measurement, and Research

Title: Mail and Web Surveys: A Comparison of Demographic Characteristics and Response Quality when Respondents Self-Select the Survey Administration Mode

Committee:
Dr. E. Brooks Applegate, Chair
Dr. David J. Hartmann
Dr. Paula D. Kohler

Date: Thursday, July 12, 2007 12:00 p.m. – 2:00 p.m.
2107 Sangren Hall

Abstract:
The purpose of this study was to use a non-experimental, quantitative design to compare mail and web surveys with survey mode self-selection at two data collection waves. Research questions examined differences and predictabilities among demographics (gender, ethnicity, age, and professional employment) and response quality (pronoun use, item non-response, response extremity, yea-saying, item completion errors, response length, response equivalency, anecdotal comments, and multiple response use) by survey mode and response wave. Analyses were conducted using chi-squares, ANOVAs, t-tests, and binary logistic regressions.
A questionnaire in mail and web formats containing 48 forced-choice and open-ended items was administered to a non-random sample of Illinois public school guidance counselors (n = 2,880). After four reminders, the adjusted response rate was 30.56% (n = 880); 64.32% (n = 566) by mail and 35.68% (n = 314) by web; 77.73% (n = 684) during wave one and 22.27% (n = 196) during wave two. Respondents were 75% female, 86% white with a mean age of 48 years and a mean of 19 years of professional employment.
Results revealed that mail respondents were older and had more years of professional employment than web respondents, item non-response was greater in web than in mail surveys, and response length was greater in web than in mail surveys at wave one. Age, response length, gender, and yea-saying had significant partial effects in predicting survey mode. Regarding response wave, demographics and response quality variables were neither different nor predictive.
Findings suggest that researchers need to consider the potential effects of demographic distributions in the target population when designing mail and web surveys. Mail and web surveys must also be carefully constructed to overcome potential response quality differences while maximizing the advantages of each. The low overall adjusted response rate and non-randomized design limit generalizability to the larger group of all school counselors in the population. This study, however, provides practical and timely insight regarding the use of mail and web surveys with mode self-selection among those who responded, and offers much potential for future research.