• Insider/Outsider Roles
  Confusion exists about insider/outsider roles, that is, the roles of neighborhood residents and outside experts. Imposition of academic models on local programs yields poor results. Researchers cannot successfully mandate local solutions. But Schorr cautions, many people have "overlearned" that lesson and embraced the opposite extreme, regarding all outside information and mandates as undesirable. That leaves local people starting from scratch each time, in each community. For example, Proposition 10 channeled $700 million in cigarette taxes to early childhood efforts in California. The State offered no programmatic guidance to organizations receiving the funding, only directions about how to plan and who should be at the table! That left organizations struggling over which agency obtained how much of the money, rather than talking about the best ways to achieve the stated outcomes.

• Biomedical Approach to Evaluation
  Schorr points out that certification of "best practices" customarily occurs when evaluators use the biomedical research model. This model includes random assignment to control and experimental groups to test for a causal relationship between one factor and one or more outcomes. Schorr's own work demonstrates that most exemplary programs are multi-faceted, complex and thus do not lend themselves to evaluation using the biomedical model. She argues that instead of seeking to determine which precise factor caused which precise effect, evaluators looking for best practices should seek to verify the overall results of complex interventions.

  Evaluation for Learning

  Schorr maintains that we can balance insider/outsider roles and generate a large body of best practice knowledge by adopting a different approach to evaluation. She advocates evaluation that combines theory, logic and evidence with insight, creativity and judgement. Such evaluation exhibits the following four critical elements:
  

  1. It systematically utilizes multiple ways of knowing.
  2. Its conclusions are based on systematic accumulation of knowledge and judgement from a variety of perspectives.
  3. It builds on current efforts to identify best practices by identifying interim outcomes, or pathways to longer-term, larger-scale outcomes.
  4. It offers easy-to-use, interactive methods that rapidly capture feedback and make it available in a timely manner.W
  With the flow of results from this type of evaluation, the knowledge base will evolve constantly, helping us learn about implementation, programs, and policies. No experimenter or central authority proscribes the solution; instead, local implementers plan variations on best practices, choosing and flexibly adapting pathways to good results.
  
  
  

  TIPS & TOOLS "Our State contract requires us to track the number of children who avoid out-of-home placements, which is a longer-term outcome objective of our mentoring program. When we began to monitor development of appropriate communication (an initial outcome), we discovered that our youth and mentors could benefit from additional training in assertiveness. We are now providing that training and will check to see what impact it has on mentees' communication skills." -- Terri Wood B., Youth Companion Program Coordinator, Bethany Christian Services

  
  
  

  NOTEWORTHY RESOURCES

  Child Trends (2000). Building a Better System of Child and Family Indicators.

  http://www.childtrends.org/PDF/bettersys.pdf

  Child Trends also offers an inventory of over 90 child, youth, and family indicator-based projects.

  http://www.childtrends.org/r_inves.cfm

  Center for the Study and Prevention of Violence (1999). Blueprints for Violence Prevention.

  http://www.colorado.edu/cspv

  Minnesota Department of Human Services (1999). Kids Killing Kids: A Thoughtful Response. St. Paul, MN: Author.

  (For a copy, call 651/215-6019.)

  
  

THE GREATER KALAMAZOO EVALUATION PROJECT PUBLISHES
THIS NEWSLETTER TO FOSTER EVALUATION
THROUGHOUT THE KALAMAZOO AREA.