Deliberative Democratic Evaluation

DELIBERATIVE DEMOCRATIC EVALUATION CHECKLIST
pdf

Ernest R. House and Kenneth R. Howe

October 2000

The purpose of this checklist is to guide evaluations from a deliberative democratic perspective. Such evaluation incorporates democratic processes within the evaluation to secure better conclusions. The aspiration is to construct valid conclusions where there are conflicting views. The approach extends impartiality by including relevant interests, values, and views so that conclusions can be unbiased in value as well as factual aspects. Relevant value positions are included, but are subject to criticism the way other findings are. Not all value claims are equally defensible. The evaluator is still responsible for unbiased data collection, analysis, and arriving at sound conclusions. The guiding principles are inclusion, dialogue, and deliberation, which work in tandem with the professional canons of research validity.

Principle 1: Inclusion

The evaluation study should consider the interests, values, and views of major stakeholders involved in the program or policy under review. This does not mean that every interest, value, or view need be given equal weight, only that all relevant ones should be considered in the design and conduct of the evaluation.

Principle 2: Dialogue

The evaluation study should encourage extensive dialogue with stakeholder groups and sometimes dialogue among stakeholders. The aspiration is to prevent misunderstanding of interests, values, and views. However, the evaluator is under no obligation to accept views at face value. Nor does understanding entail agreement. The evaluator is responsible for structuring the dialogue.

Principle 3: Deliberation

The evaluation study should provide for extensive deliberation in arriving at conclusions. The aspiration is to draw well-considered conclusions. Sometimes stakeholders might participate in the deliberations to discover their true interests. The evaluator is responsible for structuring the deliberation and for the validity of the conclusions.

These three principles might be implemented by addressing specific questions. The questions may overlap each other, as might dialogue and deliberation processes. For example, some procedures that encourage dialogue might also promote deliberation.

1. INCLUSION

a. Whose interests are represented in the evaluation?

	Specify the interests involved in the program and evaluation.
	Identify relevant interests from the history of the program.
	Consider important interests that emerge from the cultural context.

b. Are all major stakeholders represented?

	Identify those interests not represented.
	Seek ways of representing missing views.
	Look for hidden commitments.

c. Should some stakeholders be excluded?

	Review the reasons for excluding some stakeholders.
	Consider if representatives represent their groups accurately.
	Clarify the evaluator's role in structuring the evaluation.

2. DIALOGUE

a. Do power imbalances distort or impede dialogue and deliberation?

	Examine the situation from the participants' points of view.
	Consider whether participants will be forthcoming under the circumstances.
	Consider whether some will exercise too much influence.

b. Are there procedures to control power imbalances?

	Do not take sides with factions.
	Partition vociferous factions, if necessary.
	Balance excessive self-interests.

c. In what ways do stakeholders participate?

	Secure commitments to rules and procedures in advance.
	Structure the exchanges carefully around specific issues.
	Structure forums suited to participant characteristics.

d. How authentic is the participation?

	Do not organize merely symbolic interactions.
	Address the concerns put forth.
	Secure the views of all stakeholders.

e. How involved is the interaction?

	Balance depth with breadth in participation.
	Encourage receptivity to other views.
	Insist on civil discourse.

3. DELIBERATION

a. Is there reflective deliberation?

	Organize resources for deliberation.
	Clarify the roles of participants.
	Have experts play critical roles where relevant.

b. How extensive is the deliberation?

	Review the main criteria.
	Account for all the information.
	Introduce important issues neglected by stakeholders.

c. How well considered is the deliberation?

	Fit all the data together coherently.
	Consider likely possibilities and reduce to best.
	Draw the best conclusions for this context.

This checklist is being provided as a free service to the user. The provider of the checklist has not modified or adapted the checklist to fit the specific needs of the user and the user is executing his or her own discretion and judgment in using the checklist. The provider of the checklist makes no representations or warranties that this checklist is fit for the particular purpose contemplated by user and specifically disclaims any such warranties or representations.