Evaluation Strategies and Considerations

When designing an evaluation, it is important to determine what the program hopes to learn. Process evaluation questions are useful to understand how the program was implemented. They can also be used to identify program successes and challenges. Outcome evaluation questions can be used to figure out whether the program successfully accomplished its goals and measure the short- and long-term effects of the program on participants. Program planners who are developing evaluation questions should consider what data are available, what information would be useful to program stakeholders, and how the evaluation findings will be used.

Examples of process evaluation questions include:

What did the organization accomplish during the reporting period?
- How many people were served?
- How many sessions were completed?
- What services were delivered?
How did community members, participants, and staff perceive the program?
How well did documentation systems capture program and participant data?
What were the barriers and challenges that affected program implementation?
Who facilitated the implementation of the program?
What program activities were not completed and why? If activities changed, why did they change?
What are key lessons learned?

Examples of outcome evaluation questions include:

What were the outcomes of the program?
- Did the program improve participants' ability to manage their health condition?
- Did the program increase participants' likelihood to make lifestyle changes?
- Did the program increase participants' compliance with their medications?
- Did the program connect participants with a primary care physician/medical home?
- Did the program improve participant quality of life?
Did the program result in cost savings to the healthcare system?
- What aspects of the program were most cost-effective or least cost-effective?
Which outcomes are important to the community?

Evaluation Considerations

Given the diversity of community health worker (CHW) programs, there is no one-size-fits-all evaluation approach. Key considerations for evaluating CHW programs include:

Staffing – CHW programs may hire an external evaluator or assign an internal staff member to serve as an evaluator. CHWs may be assigned to collect program evaluation data or administer surveys. CHWs may need additional training to perform these tasks.
Cost-Effectiveness and Return on Investment – It can be important to measure costs associated with CHW programs. The Community Preventive Services Task Force (CPSTF) has determined CHW interventions can be cost-effective for a variety of conditions including cancer screening, diabetes prevention and management, and preventing cardiovascular disease. More rigorous program evaluations can help the program demonstrate its return on investment (ROI). Programs can use cost and ROI information to demonstrate the program's effectiveness to community partners, some of whom may be willing to invest resources in program sustainability. In recent years, research has indicated rural CHW programs can be cost-effective and succeed in accomplishing their goals.
Existing Gaps in Evidence – The CPSTF identified several evidence gaps — areas where information is lacking — including interventions with a longer follow-up period (greater than 12 months), interventions among diverse population subgroups, the impact of large-scale interventions with more than 500 people, and the effectiveness of different service delivery modes (face-to-face, telephone, groups) on outcomes.

Resources to Learn More

Best Practices for Implementing and Evaluating Community Health Worker Programs in Health Care Settings
Document
Provides guidance for healthcare organizations interested in implementing or expanding the CHW model in their system. Addresses gaps identified in the CHW professional literature and assists healthcare professionals and others with designing and implementing evidence-based programs.
Author(s): Kapheim, M.G. & Campbell, J.
Organization(s): Sinai Urban Health Institute
Date: 1/2014

Community Health Worker Evaluation Tool Kit
Document
Offers guidance on program evaluation for CHW programs. Designed to make evaluation less complicated and appropriate for organizations with limited evaluation expertise.
Organization(s): BetterEvaluation
Date: 2000

Evaluation Toolkit for Promotor(a) de Salud Programs
Document
Provides guidance, tools, and information about conducting evaluation in resource-limited settings, as is often the case for promotor(a) de salud and other community health worker programs. Designed for use by project managers or other staff with limited evaluation expertise.
Organization(s): MHP Salud
Date: 4/2016