Helpful Evaluation Resources
A number of useful research tools and guides to program evaluation are available for reference purposes. Some are tailored specifically to the mediation context, and others are more general.
I. Mediation Program Evaluation Guides
Although none of the following guides focus specifically on the higher education context, they may still be of considerable value to individuals designing evaluations of mediation programs.
A Self-Evaluation Manual for Community-Based Mediation Projects: Tools for Monitoring and Recording Data, by T. Roberts. Vancouver, Canada: University of Victoria Institute for Dispute Resolution, 1993. This example-filled manual comes out of an evaluation, conducted in 1990-1991, of two community-based mediation organizations in British Columbia. It provides a variety of useful ideas, and sample data monitoring and gathering instruments. Available from: http://www.dispute.resolution.uvic.ca or http://www.nicr.ca/
Evaluating Agency Alternative Dispute Resolution Programs: A Users' Guide to Data Collection and Use, by E. S. Rolph and E. Moller. Santa Monica, California: RAND Institute for Civil Justice, 1995. The Administrative Conference of the United States asked the Institute for Civil Justice to prepare a manual and develop prototype data collection instruments to assist those with responsibility for evaluating federal agency alternative dispute resolution programs. The manual discusses issues in designing evaluations, lays out approaches to data collection, provides sample data analysis plans, and includes a number of prototype data collection instruments. Available from: http://www.rand.org/
CADRE Resource Guide: Using Evaluation Data to Improve Quality, by Timothy Hedeen. This resource guide presents an overview of mediation evaluation, with a specific focus on how to understand and make use of evaluation findings. It is designed to help program managers understand evaluation processes and outcomes, rather than serving as a how-to manual on evaluation. CADRE, The National Center on Dispute Resolution, is a project funded by the United States Department of Education, Office of Special Education Programs. They provide technical assistance to state departments of education on implementation of the mediation requirements under IDEA '97. CADRE also supports parents, educators and administrators to benefit from the full continuum of dispute resolution options. Currently under review, available in 2002 from CADRE, P.O. Box 51360, Eugene, OR 97405-0906; or from: http://www.directionservice.org/cadre/
Evaluating Alternative Dispute Resolution Programs, by Lee Scharf. This practical 19-page document is Chapter 8 of the Federal Alternative Dispute Resolution Program Manager's Resource Manual. It is available as a PDF file from the Interagency Alternative Dispute Resolution Working Group (IADRWG) home page, as a separate chapter or as part of the entire Resource Manual. The Working Group was established to coordinate, promote, and facilitate the effective use of dispute resolution processes within Federal agencies as mandated by the Administrative Dispute Resolution Act of 1996 and White House Presidential Memorandum. Available from: http://www.usdoj.gov/adr/
Program Evaluation Kit: Victim Offender Mediation Programs, by Mark Umbreit. Minneapolis, MN: Minnesota Citizens Council on Crime and Justice, 1992. This guide includes brief introductory text explaining the importance of evaluation, and then provides a sample questionnaire designed to be administered by program staff without need for professional evaluators to assist them. Available for download as a PDF file from the Center for Restorative Justice and Peacemaking from: http://2ssw.che.umn.edu/rjp/Resources/Documents/tumb92a.PDF
Evaluating Your Conflict Resolution Education Program: A Guide for Educators and Evaluators, by Tricia Jones and Dan Kmitta, October 2001.
This Evaluation Guide, developed by the Ohio Commission on Dispute Resolution and the Ohio Department of Education focuses on methods that can be used by school conflict resolution program grantees to evaluate program effectiveness and to assess program impact at the building level. The authors prepared the manual as a workbook so that it should be easy to use. Throughout the beginning parts of the manual they have included some worksheets to help you identify the program goals and evaluation goals you want to emphasize. When they provide copies of questionnaires and interview questions they have presented them so you can simply copy the forms from the book and use them in your school. Available for online reading or download at http://www.state.oh.us/cdr/schools/evaluatingcrep.htm
Indiana Conflict Resolution Institute Searchable Bibliographic Database. This online resource is not about how to evaluate mediation services, but instead provides information on existing evaluation reports and articles. The database seeks to provide a comprehensive list of empirical field studies and program evaluations on conflict resolution. The citation search will examine each record in the bibliography database, and select records according to keywords entered in one or more of the following fields: author, title, source, publisher, and/or category. If article summaries are available, links to the summaries will be displayed in the citation results. While it is constantly under construction, the database includes evaluations of conflict resolution including juvenile court, international, employment, environmental, and labor relations programs. The list of evaluations continues to grow and should not be considered exhaustive. Available from: http://www.spea.indiana.edu/icri/datalist.htm
II. General Program Evaluation Resources on the Web
Action Evaluation Project: http://www.aepro.org This site provides information on a participatory assessment process known as action-evaluation. The process entails collaboratively articulating goals and objectives among the groups involved in a conflict intervention, including those funding it, those organizing and convening the intervention, and the participants themselves. The action-evaluator collects this information from the groups and summarizes it with the help of a computerized database (available over the Web browser) designed to systematize the process and organize the data. This goal articulation takes place at the outset of an intervention, allowing the action-evaluator to track how goals of various stakeholders evolve and use these goals as a basis for both designing the intervention and evaluating it along the way and at its conclusion. An article of special interest from this site is Action Evaluation in the Theory and Practice of Conflict Resolution by Marc Ross, Ph. D. and William R. Kenan, Jr. This article examines Action Evaluation as a theory of practice, considering its conceptual strengths and examining specific issues of its implementation.
Bureau of Justice Assistance Evaluation Web Site: http://www.bja.evaluationwebsite.org The BJA maintains an extensive Web site designed to provide a variety of resources for evaluating primarily criminal justice programs, but much of the information is applicable to other contexts as well. The site includes the Electronic Roadmap for Evaluation, which provides instructional materials to assist in planning, designing, and conducting evaluations of programs, and a section on evaluation resources. It also contains a bibliography of evaluation materials organized by specific evaluation topics.
Centers for Disease Control Evaluation Working Group Web Site: http://www.cdc.gov/eval/resources.htm This site provides an extensive list of categorized online resources related to program evaluation. You will find lots of good information here. Resources are divided into the following groups:
* Ethics, Principles, and Standards
* Organizations, Societies, Foundations, Associations
* Journals and On-Line Publications
* Step-by-Step Manuals
* Logic Model Resources
* Planning and Performance Improvement Tools
* Reports and Publications
Collaborative, Participatory, and Empowerment Evaluation Website: http://www.stanford.edu/~davidf/empowermentevaluation.html This website, developed by David Fetterman, author of the widely-used book Empowerment Evaluation (see below for full reference) provides summary links to a very nice collection of user friendly tools supporting program evaluation. These include (often free) software programs and online interactive evaluation development guides.
III. Useful Books on Evaluation Research
Fetterman, D. M., and others (Eds.). (1996). Empowerment Evaluation: Knowledge and Tools for Self-Assessment and Accountability. Thousand Oaks, California: Sage.
Patton, M. Q. (1990). Qualitative Evaluation and Research Methods. Thousand Oaks, California: Sage.
Patton, M. Q. (1996). Utilization-Focused Evaluation. Thousand Oaks, California: Sage.
Rossi, P., and Freeman, H. (1993). Evaluation: A Systematic Approach. Thousand Oaks, California: Sage.