MTS SUMMER INSTITUTE

Program Evaluation Course

1999(dls)

DRAFT

(Updated 5/3/99)

MTS Institute 99; Program Evaluation Course: SCHEDULE-WEEK 1

Week 1	Monday 6/7/99	Tuesday 6/8/99	Wednesday 6/9/99	Thursday 6/10/99	Friday 6/11/99
9:00-10:00	•Overview of the course (dls) •Introductions •Pre-self-rating (lw) •Matrix sample exam (lw) •Overview of The Program Evaluation Standards (dls) •Break	•Administration of matrix sample exam and presentation/discussion of Day 1 results (lw) •Review and discussion of Day 1 experiences (dls) •Participants' briefing on metaevaluation (general concept, formative metaevaluation, summative metaevaluation) •Discussion (dls) •Break	•Administration of matrix sample exam and presentation/discussion of Day 2 results (lw) •Review and discussion of Day 2 experiences (dls) •Participants' briefing on the CIPP Evaluation Model •Discussion (dls) •Break	•Administration of matrix sample exam and presentation/discussion of Day 3 results (lw) •Review and discussion of Day 3 experiences (dls) •Presentation on evaluation of mathematics projects (ag, el)	•Administration of matrix sample exam and presentation/discussion of Day 4 results (lw) •Review and discussion of Day 4 experiences (dls) •Participants' briefing on the Project Evaluation Handbook •Reaction, elaboration, and discussion (ag, el) •Discussion (ag, el) •Break
10:00-11:00	•Continued presentation of The Standards (dls) •Discussion of The Standards •Preparation for simulation exercise (lw) •Break	•Participants' briefing on standards readings (ERS Standards, AEA Principles, Joint Committee Personnel Evaluation Standards) •Discussion (dls) •Break	•Participants' briefing on Scriven's approach •Discussion (dls) •Break	•Presentation on evaluation of science education projects (ag, el) •Discussion (ag, el) •Break	•Participants' briefing on the Mixed Methods Evaluation Handbook •Reaction, elaboration, and discussion (ag, el) •Break
11:00-12:00	•Simulation applying The Standards (lw)	•DLS presentation on Analysis of 22 Alternative Evaluation Approaches (Questions/Method-oriented models, Improvement/Accounta-bility-oriented models, Social mission/Advocacy-oriented models) •Discussion of alternative approaches (dls) •Reading assignment: CIPP Eval. Model paper; Scriven self-instructional guide; and Stake self-instructional guide.	•Participants' briefing on Stake's responsive evaluation •Discussion (dls) •Reading assignment (to be selected by Gullickson, Lento, and Miron) •Break	•Presentation on evaluation of charter schools (gm) •Reading assignment: User-Friendly Handbooks (1 and 2)	•Continued discussion of both handbooks (ag, el) •Reading assignment (to be selected by McConney)
1:00-4:00	•Continuation of simulation exercise •Assignments for briefing sheets and individual class reports (dls) •Reprise: What did we learn? •Reading assignment: Metaevaluation; Standards; Anal. of 22 Eval. Approaches				•Presentation and discussion of the week's experience with the matrix sample exercise (lw) (approx. 2:00 pm)

MTS Institute 99; Program Evaluation Course: SCHEDULE-WEEK 2

Week 2 Monday 6/14/99 Tuesday 6/15/99 Wednesday 6/16/99 Thursday 6/17/99 Friday 6/18/99

8:00-

9:00

•New participants' self-assessment (lw)

9:00-10:00

•Introductions

•Overview of the week's learning plan

•Work sample method (am)

•Break

•Review and discussion of Day 6 experiences (dls)

•How to evaluate standards-based reforms in math (am)

•Break

•Review and discussion of Day 7 experiences (dls)

•How to evaluate Internet instruction and other forms of distance education-continued (am)

•Break

•Review and discussion of Day 8 experiences (dls)

•Contracting for evaluations (jh)

•Break

•Review and discussion of Day 9 experiences (dls)

•Effectively addressing multiple audiences (jh)

•Break

10:00-11:00

•Continued work on work sample methodology (am)

•Break

• How to evaluate standards-based reform programs in mathematics-continued (am)

•Break

•How to evaluate systemic reform programs in science (am)

•Break

•Budgeting for evaluations (jh)

•Break

•Managing evaluations (jh)

•Avoiding political pitfalls (jh)

•Break

11:00-12:00

•Continued work on work sample methodology (am)

•Reading assignment as appropriate

•How to evaluate Internet instruction and other forms of distance education (am)

•Reading assignment as appropriate

•How to evaluate systemic reform programs in science-continued (am)

•Reading assignment (to be selected by Horn)

•Staffing evaluations (jh)

•Establishing and working with advisory groups (jh)

•Reading assignment as appropriate

•How to design and conduct evaluation workshops (jh)

•Wrap-up discussion (dls)

1:00-4:00

•Party at Gullicksons

•Discussion of Intern Program (jh)

•Case study of CORE PLUS (D. Carlisle)

MTS Institute 99; Program Evaluation Course: SCHEDULE-WEEK 3

Week 3	Monday 6/21/99	Tuesday 6/22/99	Wednesday 6/23/99	Thursday 6/24/99	Friday 6/25/99
9:00-10:00	•Overview of the week (dls) •Writing evaluation sections of grant proposals (jh) •Break	•Review and discussion of Day 11 experiences (dls) •Using qualitative techniques (observations, interviews, focus groups) (pa) •Break	VISIT TO KELLOGG FOUNDATION	•Review and discussion of Day 13 experiences (dls) •Use of student evaluation portfolios (gr)	•Review and discussion of Day 14 experiences (dls) •Evaluation of technology (ag) •Break
10:00-11:00	•Classroom assessment techniques (pa) •Break	•Using quantitative techniques (rating scales, performance assessments, standardized tests, surveys) (pa) •Break	VISIT TO KELLOGG FOUNDATION	•How to create an assessment instrument (gr) •Break	•Evaluation of instructional materials (ag) •Break
11:00- 12:00	•More classroom assessment techniques (pa) •Developing evaluation questions (pa) •Determining data sources and appropriate data collection approaches (pa) •Reading assignment as appropriate	•Constructing outcome measures (pa) •When and how to use experimental designs (pa) •Reading assignment (to be selected by Airasian)	VISIT TO KELLOGG FOUNDATION	•Evaluation of ATE projects (gr) •Reading assignment as appropriate •Directions for afternoon party at Lake Michigan	•Evaluation of nonprofits (js) •Wrap-up (dls)
1:00-4:00			•Review and discussion of Day 12 experiences (dls) •Reporting findings (pa) •Writing good reports (pa)	•Party at South Haven	•Complete posttest (lw) •Concluding evaluation session (dh)

Week 2	Monday 6/14/99	Tuesday 6/15/99	Wednesday 6/16/99	Thursday 6/17/99	Friday 6/18/99
8:00- 9:00	•New participants' self-assessment (lw)
9:00-10:00	•Introductions •Overview of the week's learning plan •Work sample method (am) •Break	•Review and discussion of Day 6 experiences (dls) •How to evaluate standards-based reforms in math (am) •Break	•Review and discussion of Day 7 experiences (dls) •How to evaluate Internet instruction and other forms of distance education-continued (am) •Break	•Review and discussion of Day 8 experiences (dls) •Contracting for evaluations (jh) •Break	•Review and discussion of Day 9 experiences (dls) •Effectively addressing multiple audiences (jh) •Break
10:00-11:00	•Continued work on work sample methodology (am) •Break	• How to evaluate standards-based reform programs in mathematics-continued (am) •Break	•How to evaluate systemic reform programs in science (am) •Break	•Budgeting for evaluations (jh) •Break	•Managing evaluations (jh) •Avoiding political pitfalls (jh) •Break
11:00-12:00	•Continued work on work sample methodology (am) •Reading assignment as appropriate	•How to evaluate Internet instruction and other forms of distance education (am) •Reading assignment as appropriate	•How to evaluate systemic reform programs in science-continued (am) •Reading assignment (to be selected by Horn)	•Staffing evaluations (jh) •Establishing and working with advisory groups (jh) •Reading assignment as appropriate	•How to design and conduct evaluation workshops (jh) •Wrap-up discussion (dls)
1:00-4:00				•Party at Gullicksons	•Discussion of Intern Program (jh) •Case study of CORE PLUS (D. Carlisle)