Evaluation Myths, Misconceptions, and Mistakes




April 17, 2012

Dr. Lori Wingate—Assistant Director, The Evaluation Center, WMU

In this Eval Café session, Lori Wingate will engage participants in a discussion of some common misunderstandings when it comes to program evaluation language and practice.  In a profession where virtually anyone can put “evaluator” on their business card, it is immensely important that we protect the integrity of the terminology we use and be conscientious about how our individual practices collectively influence the field.  We will look at some widely repeated evaluation mistakes and missteps, consider the causes and consequences of these problems, and generate ideas about we can each work to set the record straight and keep in that way. Participants are invited to bring their favorite examples of evaluation myths, misconceptions, and mistakes to enrich and broaden the discussion.

A Collaborative Partnership to Define and Measure Empowering Practice within a Domestic Violence Shelter Program



April 3, 2012

Dr. Cris Sullivan—Professor, Ecological and Community Psychology and Coordinator, Violence Against Women Research and Outreach Initiative, MSU

The goal of empowerment-based programs is to help clients increase their personal, interpersonal, and political power. This talk describes a collaborative partnership with a domestic violence shelter program interested in evaluating how well they integrated the empowerment model into day-to-day service provision and whether their approach to empowerment-based service delivery was contributing to the intended “empowered outcomes” for the women with whom they work. In the presentation, I will describe how we jointly defined empowerment within this setting, how the empowerment-based practices and intended empowered outcomes were measured, and how the process has impacted the work of the advocates.


Pass the Aspirin: When Projects Become Headaches

March 27, 2012

Dr. Mary Anne Sydlik—Director, SAMPI, WMU,

Dr. Bob Ruhf—Senior Research Associate, SAMPI, WMU, and

Kristin Everet—Senior Research Assistant, SAMPI, WMU

Science and Mathematics Program Improvement (SAMPI) at Western Michigan University currently has a number of evaluation projects, seven projects out for review, and six in the early stages of development with potential clients.Members of the SAMPI evaluation team will address challenges that can arise 1) during the pre-submission proposal/project development phase; 2) while trying to coordinate evaluation and project activities with another organization; and 3) when the clients’ expectations change mid-course in ways that exceed the evaluation budget, the evaluator’s time and energy, and cost-overruns threaten to shut down the evaluation before it can be completed.

This event was not video tapped.


“What is in Question?”



March 20, 2012

Lee Balcom—Interdisciplinary Evaluation Doctoral Student, WMU

Framing the evaluation question precedes all other design elements in an evaluation. It is also a key factor distinguishing the inquiry process of evaluation from that of research. However, the literature gives little mention to the particulars of choosing an evaluation question. This presentation identifies theoretical perspectives and technical implications involved in framing the evaluation question as well as a tool for selecting questions deliberately.



Utilizing the PMBOK and PRINCE2 for Project Evaluation Management

No video available

March 13, 2012

Dr. Willis H. Thomas, Ph.D., CPT, PMP—Alumnus, Interdisciplinary Ph.D. in Evaluation, WMU

The Project Management Body of Knowledge (PMBOK) and Projects in Controlled Environment (PRINCE2) are two well-recognized global standards for project management. Millions of practitioners are utilizing these standards to support project evaluations. Evaluators are engaged in all types of projects and have to choose from a multitude of models, methods and approaches. This presentation discusses managing evaluation projects using both of these standards.


The Value of Voice: Gaining Access to Marginalized Populations

View the slides

No video available

February 22, 2012

Dr. Karen Vocke—Associate Professor, Department of English, WMU, and

Dr. Brooks Applegate—Professor, Educational Leadership, Research, and Technology and Director, Evaluation, Measurement, and Research, WMU

This presentation focuses on issues related to evaluating marginalized populations. For example, known challenges include access and the complicated issues of ethical representation necessary for authentic evaluation. However, critical examination shows when the evaluation process includes members of marginalized populations, results are more tangible, valid, and generalizable, with increased participation of the sample under study. Because the needs of marginalized populations are nuanced and diverse, evaluators must carefully consider the procedures and analyses involving the evaluation participants, especially the need for authentic, and not token, participation. This demonstration offers a protocol for access, collaboration, and evaluation for working with marginalized subpopulations in the K-12 setting, namely children and families of migrant farm workers and students with disabilities. Strategies will be presented for planning evaluations, accessing populations, developing survey instruments, developing a collaborative team, data collection strategies, and data analysis.

Rawlsian Political Analysis and its Links with Evaluation


February 14, 2012

Dr. Paul Clements—Professor,  Political Science and Director, MIDA Program, WMU

My forthcoming book, Rawlsian Political Analysis: Rethinking the Microfoundations of Social Science, offers a new approach to social analysis as an improvement on neoclassical economics and rational choice theory. While these approaches are based on the assumption of rational utility maximization, I follow Rawls and Kant and assume that choice is based on independent reasonable and rational capacities. I will begin this presentation by laying out the book’s principal arguments. Then I will discuss two of the book’s applications of the proposed approach that directly involve evaluation: analyses of the Grameen Bank of Bangladesh and of the ethics and politics of climate change.

Writing Your HSIRB Protocol and the HSIRB Review Process

February 7, 2011

Julia Mays—Research Compliance Coordinator, WMU

To assist you with the HSIRB protocol and review process the Research Compliance Coordinator will discuss general HSIRB processes, levels of review, how to submit, and provide examples of research to help define “human subject research” under the Federal definition. Topics will include common errors that slow down the review process from submission to approval. If you are preparing to write a HSIRB protocol please consider attending as we will discuss applications, protocol outlines, and consent document requirements.

To access forms referenced in the presentation, please visit: http://www.wmich.edu/research/forms/complianceforms.html

Veterans: Understanding and Serving this Unique and Growing Population on Campus


February 1, 2012

Tracey Moon—Director, Office of Military and Veterans Affairs , WMU

When evaluating veterans, it’s essential to be aware of the unique attributes of this population.  This workshop will cover background information on veterans, including their strengths and struggles, and suggestions for conducting an effective data collection process.  Community resources to access participants, questions to ask or avoid, and other considerations to be mindful of when interviewing both male and female service members will be discussed, as well. Participants will have a better understanding of military and veteran perspectives and will have specifics to make evaluating this diverse group more effective.

Student Veteran Facts Handout
Women Veterans Handout
Serving Student Veterans Article
OEF/OIF VA Battle Creek, MI
Volunteers of America Vet. Services Brochure
Vet. Affairs County Offices MI
OEF/OIF Vet. Affairs Newsletter


What Does it Take for an Outsider to Evaluate in Cross Cultural Contexts: What About the Cultural Nuances & Subtleties in Language Dialects?




January 25, 2012

Dr. Tererai Trent—Founder and Principle Evaluator, Tinogona Evaluation and Research Consulting

Debate continues on the value of evaluations performed by outsider evaluators in cross cultural settings. One view maintains that it does not matter as long as the individual is a professional evaluator, and if cultural competence is an issue, a local translator can be brought to the process. Others strongly believe the nuances in local cultures and often the subtleties in the language dialects are too often lost in translation to the outsider evaluator due to language barriers. Through a review of the literature and experience, the question shifts from ‘who has the right to evaluate’ to ‘what does it take for an outsider to evaluate in cross cultural contexts’? To gain insight into the question of credibility and validity of whether an outsider can possess the cultural competence required in a cross cultural context, my observations are based on Betty LaDuke, a renowned artist, who has gained international reputation for her murals, paintings, and sketches.