Summer Evaluation Institute 2003
Hands-On Projects for Participants
We are continuing to fine-tune the summer institute. Based upon evaluation information, we are preparing more extensive and focused opportunities for you to become actively involved with authentic evaluation projects. One of the general themes identified by the external evaluator over the past four years of the institute is that “participants tend to enjoy hands-on activities.” The evaluator notes in the summary of open-ended responses to evaluation surveys from the 2002 institute that the participants “appreciate hands-on/group activities.”
Information about evaluation projects and the opportunity to work on individual and group activities have always been part of the institute. We hope to provide even more for the 2003 institute while still providing opportunity to learn about the theory and methods of evaluation. Following are two opportunities you will have to apply your evaluation skills. The first will be done during the three weeks, with some preparation before the start of the institute. The second is optional and provided for you as a chance to discuss with your peers an evaluation project in which you are engaged.
1. Team Evaluation Projects
We have solicited a list of authentic evaluation projects to which you can apply your evaluation skills as part of a team. We would like you to review the options and select and rank four or more. Based upon your choices, we will assign teams of two or three to the selected projects. The respective contact persons will then provide additional information about the project for your preparation before the institute. With the counsel of a Center staff member and in conjunction with the contact person, teams will work on the project during the first two weeks of the institute and deliver a report to the institute during the third week. Other institute participants, presenters, and Center staff will also provide consultation as requested.
The last page of this document lists the projects with blanks to rank your choices. Please fill in the last page and return it to the MTS Project. Directions are on the last page.
2. Individual Evaluation Projects
Many of you have ongoing evaluation projects about which you may have questions. The institute is an opportunity to seek input from other participants and from Center staff. To facilitate feedback for those interested (this is an optional activity), we have arranged two opportunities. First, we will help you arrange times when you can meet individually with Center staff for discussion of evaluation questions. Second, we will schedule a time when you can present a brief summary of your project to the institute participants and ask for their input on some aspect of the evaluation. Again, both these opportunities are optional and are in addition to the Team Evaluation Projects. If you are interested in scheduling time to present to institute participants, please provide the information requested on the last page.
Project Options for Selections by 2003 Summer Institute Participants
Study the list of possible projects and choose at least four that are most interesting to you. On the last page of this document, rank the four projects in order with 1 being your highest preference. If you want to rank more than four, please do.
Project #1
Title: Academic Advising
Project Description: The College of Engineering and Applied Sciences (CEAS) maintains a central advising office staffed by two full-time staff members and part-time student helpers to coordinate academic advising activities. The Advising Office maintains all student records, organizes advising workshops for the academic advisors, and provides up-to-date information regarding General Education and other policies to the academic advisors who are faculty members. Students make appointments and meet with their academic advisors at the central advising office.
Desired Tasks:
Contact Person: Sandra Blanchard, Director of Academic Advising (387-4037)
Title: Life-Long Learning
Project Description: An accreditation criterion for engineering and engineering technology programs is the programs must demonstrate that their graduates “recognize the need for and be able to engage in life-long learning.” After a review of literature on assessment of life-long learning, it was decided to use the “behavior check list” approach to assess life-long learning. The goal is to develop a group of questions that can be used in a first-year course that is required for all CEAS students as a pre-survey. The questions will be repeated in a senior and alumni survey to assess the impact of the programs on student behaviors regarding the recognition for and the ability to engage in life-long learning.
Tasks Desired:
Contact Person: Dr. Edmund Tsang, Associate Dean for Undergraduate Programs & Assessment (387-4038)
Title: On-line Assessment System with Intelligent Support (OASIS)
Project Description: The purpose of the On-line Assessment System with Intelligent Support (OASIS) is to support faculty in the use of assessment to enhance student learning. OASIS consists of a library of annotated assessment instruments and a heuristic tool that asks the user a series of questions regarding the assignment to be assessed and recommends the best assessment tool(s). Presently, OASIS is in the early stage of implementation, focusing on testing the heuristic and collecting annotating instruments to assess communication and teamwork skills.
Tasks Desired:
Contact Person: Edmund Tsang, Associate Dean for Undergraduate Programs & Assessment (387-4038)
Project #4
Title: Mobile Robotic Summer Institute
Project Description: This is a one-week residential summer institute in July for high-school juniors funded by the Michigan Department of Education. The goal is to stimulate the interest of students and to provide information about electrical and computer engineering through the activities of designing and building a mobile robot.
Task Desired:
Contact Person: Dr. Frank Severance, Associate Professor of Electrical and Computer Engineering (387-4068)
Title: Design Technology Summer Institute
Project Description: This is a two-week residential summer institute in July for high-school juniors funded by the Michigan Department of Education. The goal is to stimulate the interest of students and to provide information about engineering and engineering technology through the activities of designing and building an engineering artifact.
Task Desired:
Contact Person: Dr. Steve Butt, Associate Professor of Industrial and Manufacturing Engineering (387-3746)
Title: Summer Institute Long-Term Impact Study
Project Description: This summer will be the eighth summer evaluation institute. Considerable evaluation data has been collected during and after earlier institutes. In considering the long-term impact of the study, additional evaluation information may be available or obtained to provide insight into the effectiveness of the project. For example, at the 2002 AEA conference, nearly 20 percent of past participants were involved in the program in some direct fashion.
Tasks Desired:
Contact Person: Dale Farland, MTS Project Manager, The Evaluation Center (387-5895)
Title: Review of Project MTS Support Services
Project Description: A key component of the MTS Project has been the provision of evaluation resources via the Center Web site. Evaluations of the support services portion of the Web site have been conducted. The evaluation community does make use of the Web site. However, evidence also exists that the site could be modified and enhanced to better serve evaluators.
Tasks Desired:
Contact Person: Dale Farland, MTS Project Manager, The Evaluation Center (387-5895)
Title: Assessing the Value Added by NSF’s ATE Program: Business & Industry Perspectives
Project Description: This study addresses whether or not the National Science Foundation Advanced Technological Education (ATE) program adds value to the technician workforce serving business and industry. Added value is operationally defined in terms of two elements. First, more technicians are produced. Second, these technicians are better prepared.
The study is currently underway. A general design for the study has been prepared, site visit procedures have been prepared, and during April and May site visit teams will be gathering information from business and industry representatives from several cities. Most of the data gathered will be interview data, qualitative in nature, along with artifacts gathered from the various sites.
Tasks desired:
a. Critique the work plan–provide a contingency analysis regarding whether the work plan can meet the objectives set forward for this study.
b. Conduct a congruence analysis of work conducted to assess the correspondence between work conducted and the work plan specified
Contact Person: Arlen Gullickson, Project ATE Director, The Evaluation Center (387‑5895)
Project #9
Title: Secondary Analysis of the ATE annual Survey
Project Description: This study is an opportunity to review and analyze data gathered from this current year and previous years of our study of the National Science Foundation Advanced Technological Education (ATE) program. Survey data will be available from annual surveys across four years. Data are stored as SPSS files and annual reports are available for each of the individual years. This study presents an opportunity to look more closely at individual factors of interest to institute participants and to consider findings and trends across years. No comprehensive study of findings across years has yet been done. The survey form is available for review at the ATE Web site (http://ate.wmich.edu), as are previous annual reports.
Tasks desired:
Contact Person: Arlen Gullickson, Project ATE Director, The Evaluation Center (387‑5895)
Summer Institute 2003: Pick Your Project
Please return your choices by email by April 2, 2003,to Dale Farland (dale.farland@wmich.edu).
Rank at least four of the projects that you would choose for applying evaluation skills during the institute. Indicate your first choice by 1, second choice by 2, etc. We will assign you to a team and project, based upon your choices. Only one team will be assigned to a project. Teams will have two or three participants. Unless the worst case happens (all 15 choose the same 4 projects), you will be assigned to one of your choices and the highest possible to optimize the overall choices.
_____ Project 1 Academic Advising
_____ Project 3 On-line Assessment System with Intelligent Support (OASIS)
_____ Project 4 Mobile Robotic Summer Institute
_____ Project 5 Design Technology Summer Institute
_____ Project 6 Summer Institute Long-Term Impact Study
_____ Project 7 Review of Project MTS Support Services
_____ Project 8 Assessing the Value Added by NSF’s ATE Program: Business & Industry Perspectives
_____ Project 9 Secondary Analysis of the ATE annual Survey
If you would like us to reserve time in the schedule for you to give a short presentation to the institute about an evaluation project on which you are working, please provide a brief description. The presentations must be brief, allowing time for you to pose the question to which you would like input from institute participants and others. Assume approximately 15 minutes total for your presentation and the discussion.
If you would like to take advantage of this opportunity, briefly describe your project and the general question on which you will ask for help.
_____ Yes, I would like to present and ask for input about my project.
Description: