Evaluation for Learning

News for an Evaluating Community Spring 2001

INTRODUCTION TO PROGRAM OUTCOMES

Would you like board members to understand the basics of outcome-based evaluation? Have new staff joined your organization since you began tracking and measuring outcomes? If so, offer to sign them up for Measuring Program Outcomes, a thorough, hands-on workshop that covers terminology, measurement, analysis and reporting. The next sequence of sessions will meet May 16, 23, and 30, 8:30 a.m. - 12:30 p.m. in the Greater Kalamazoo United Way Board Room (709 S. Westnedge Avenue). Cost is $15/person; any community member is welcome. Register by May 10, 2001. For a registration form, call Elaine Griffin at 616/343-2524.

EVALUATION STANDARDS

How do we know if evaluation is done well? Who determines whether evaluation is acceptable? This dilemma faced workers thirty years ago when evaluation was in its infant stages. To address this issue, sixteen professional associations collaborated with evaluators and users of evaluation to develop standards for program evaluation. The second edition of these standards appeared in 1994.

There is general agreement that the characteristics of good evaluation practice are that evaluation is (1) useful, (2) feasible, (3) proper/ethical, and (4) accurate. The published standards elaborate on each of these four characteristics.

The following questionnaire was developed in an evaluation class at Western Michigan University to guide a discussion of evaluation procedures based on the Standards. Once strengths and weaknesses of evaluation are identified using the questionnaire, changes directed toward improving evaluation can be discussed.

We hope you find this useful for improving your evaluation practices.


EVALUATION ACCEPTABILITY QUESTIONNAIRE

 Directions: Circle the response for each statement about the evaluation that best reflects your feelings about the evaluation.

SA=Strongly agree  A=Agree  N=No feelings  D=Disagree  SD=Strongly disagree

  1. You were involved.
  2. The evaluator was qualified.
  3. The evaluation was too narrow.
  4. The basis for making judgements was clear.
  5. The report(s) were clear.
  6. Reports were received on time.
  7. The evaluation had an impact.
  8. The procedures were workable.
  9. Cooperation was maintained.
  10. It was worth it.
  11. Participants were served by this evaluation.
  12. Obligations of all parties were clear.
  13. Participants' rights were protected.
  14. People were treated well.
  15. Program strengths and weaknesses were described.
  16. Findings were accessible to you.
  17. There was no conflict of interest.
  18. Evaluation funds were spent wisely.
  19. The program was accurately described.
  20. The context of the program was taken into account.
  21. The evaluation procedures were clear.
  22. It was clear where the information came from.
  23. The evaluation was valid.
  24. The evaluation was accurate.
  25. Evaluation data were handled properly.
  26. Evaluation data were analyzed properly.
  27. The conclusions were justified.
  28. The evaluation was biased.
SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD

SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD

SA   A   N   D   SD
SA   A   N   D   SD

SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD
SA   A   N   D   SD
COMMENTS: Please add any comments that you may have about the evaluation.

REFERENCE
Joint Committee on Standards for Evaluation (1994).
The Program Evaluation Standards. Thousand Oaks, CA: Sage.


TIPS & TOOLS


The Michigan Association for Evaluation MAE strives to improve and increase the capacity for evaluation in Michigan. MAE features an annual conference with a nationally known evaluator, and regional events for evaluators, funders, and users of evaluation. The May 23-24, 2001 conference features Dr. Heather Weiss of the Harvard Family Research Projects. For information see "What's New" at www.MAEEVAL.org.



Asking Good
Questions





Collecting and
Sharing Useful
Information




Using the
Information for:

  • Improvement
  • Accountability





Continuing
to Repeat
the Cycle







Evaluation for
Learning is:
  • Everyone's
    Responsibility
  • Continually asking good
    questions, getting
    answers, and taking
    action based on those
    answers
  • Integrated into the
    day-to-day operations
    of the organization
  • A developmental
    process
  • Collaborative and
    dependent on
    information sharing
  • Time well spent
  • Going to ensure the
    organization's health
    and viability in the long
    run in a changing
    environment








Evaluation for Learning
 newsletter is a product of the
Greater Kalamazoo Evaluation
Project with support from the
Greater Kalamazoo United Way,
the Irving S. Gilmore Foundation,
and the Kalamazoo Foundation

THE GREATER KALAMAZOO EVALUATION PROJECT PUBLISHES
THIS NEWSLETTER TO FOSTER EVALUATION
THROUGHOUT THE KALAMAZOO AREA.

Send your ideas
and examples to: 		Denise Hartsough
Greater Kalamazoo United Way
709 S. Westnedge Avenue
Kalamazoo, MI 49007-5099 		PH: (616) 343-2524
FAX: (616) 344-7250
GREATER KALAMAZOO EVALUATION PROJECT
c/o Greater Kalamazoo United Way
709 S. Westnedge Avenue
Kalamazoo, MI 49007-5099

[Evaluation for Learning Home]     [Top of Page]