Evaluation Strategies and Considerations for Health Equity Programs and Initiatives

Each health equity program or initiative has a unique focus, scope, process, and intended outcomes. While there is no one-size-fits-all evaluation approach that will work for all health equity programs, there are common strategies and factors to consider when designing and implementing evaluations for rural health equity programs.

Rural communities developing programs to advance health equity may also benefit from evaluation strategies outlined in the following topic-specific toolkits that focus on issues related to health equity and the social determinants of health:

Engage in Thoughtful Evaluation Planning

As some health equity outcomes can take time to observe, it is important to develop an evaluation plan that includes short-term, intermediate, and longer-term outcomes. The evaluation plan should clearly communicate evaluation goals, connect the evaluation to the underlying program logic or conceptual model, and identify plans for acting upon evaluation findings once they are available. Connecting evaluation outcomes to sustainability goals at the outset of the program can ensure that programs collect data to make the business case for continued investment. Some communities may choose to use an existing framework to guide evaluation planning, such as a health equity framework or a social determinants of health framework. The Rural Community Health Toolkit includes additional information about evaluation planning.

The evaluation plan will depend on the type of program and its goals: for example, a focus on one or more social determinants of health or an emphasis on policy change. The evaluation plan may evolve over time to incorporate changes in activities and evolving community preferences and needs identified through continuous community engagement.

The Centers for Disease Control and Prevention developed a guide for integrating health equity into evaluation that includes questions for reflection to guide planning efforts. The questions are designed to facilitate the integration of health equity in the different stages of evaluation, including planning, data collection, analysis, and sharing results. The planning stage should also include an assessment of readiness to evaluate the program and a plan for continued monitoring and evaluation to support rapid and constant learning and quality improvement. Communities may also integrate evaluation measures into existing data collection efforts or performance measurement systems.

Lift Community Voices through Community-Based Participatory Research Approaches

Through community-led and community-based participatory research approaches, rural communities can support diverse community organizations to lead research on their own terms. Successful research in rural communities is contingent on building trust and relationships with community members and the organizations that represent them. This is particularly critical in communities that have been exploited by researchers in the past, such as American Indian and Alaska Native communities. Since some researchers have acted unethically and caused harm to these communities, teams conducting research must maximize the benefits for participating communities and cultivate and uphold values of trust, authenticity, clear communication, and respect.

Evaluators — who may or may not be a part of the community — should use a shared leadership approach to research, working closely with community members and organizations to define success, meaningful outcomes, and evaluation methods and priorities. Community-based participatory research approaches can benefit the evaluation team and the community and enable the evaluation team to improve their understanding of historical and cultural factors that affect the communities with which they work.

Community members should be involved in every phase of the evaluation cycle, from identifying questions and indicators of interest to validating and communicating evaluation findings. It is particularly important for community members to help interpret, validate, and approve findings before they are disseminated or published elsewhere.

Questions to consider when collaborating with communities to conduct research include:

Who has ownership of and can access the data?
Who should be involved in measure selection and revision, analysis, and deciding how data will be used?
What are opportunities for community members to build capacity in evaluation, such as by serving as paid members of the research team?
How is the evaluation team sharing the results and findings back with the community? How will the community be involved in interpreting and framing these results?
How will the evaluation benefit community members?

Evaluation data and preliminary and final findings should be available for community members, and the evaluation team should work with partners to identify rules governing the ownership of evaluation data. Rural communities should know that determining data ownership from the outset of a research project is critical when working with tribal communities that have their own processes and principles for governing data use and ownership. It is also important to compensate community members and evaluation respondents for their time participating in evaluation activities and sharing their expertise and wisdom.

Use Culturally Responsive and Appropriate Approaches

Evaluation approaches should be responsive, appropriate, and relevant to the culture of the populations of focus. Evaluators should engage diverse perspectives and be mindful of cultural values in all aspects of the evaluation. Throughout the evaluation process, the evaluation team should also examine and acknowledge how power dynamics; cultural contexts; and their own backgrounds, identities, biases, and experiences impact their work and interactions with community members. Data collection approaches and instruments should also be appropriate, accessible, and relevant for the cultural context. For example, evaluators should consider language needs, literacy levels, broadband access, relevant metrics, and the use of appropriate and inclusive language. These strategies can help support the accuracy, relevance, credibility, and validity of evaluation findings.

Examine Potential Differential Impacts on Populations

Policy and programmatic changes can affect communities in different ways. Rural communities may consider a Health Equity Impact Assessment to assess potential unintended health impacts of health equity interventions. In addition to analyzing evaluation data at the overall population level, communities should disaggregate data for population subgroups. Some interventions may improve health at the community level but worsen or have no effect on health disparities or issues for specific groups within the community. A Health Equity Data Analysis can help identify groups in the community and root causes of health inequities to focus on during the evaluation.

Anticipate Challenges and Plan for Overcoming Them

Rural communities have many strengths that facilitate evaluation and research. However, challenges may arise. Communities can improve their evaluation efforts by anticipating potential challenges and making a plan for overcoming them. Potential challenges may include turnover of evaluation team members, particularly over longer periods of time as communities evaluate longer-term outcomes; small numbers of evaluation respondents in rural communities, particularly for population subgroups; data quality issues or a lack of available data; compressed funding opportunity timelines that do not allow for assessment of longer-term outcomes; difficulty measuring some health equity outcomes; and issues with the interoperability of different data sources and systems.

Resources to Learn More

ATE Evaluation Resource Hub
Website
Provides a variety of documents, toolkits, and webinars offering guidance when developing an evaluation plan. Covers data collection, report writing, and finding and selecting an evaluator. Includes resources on culture, diversity, and equity in the evaluation process.
Organization(s): Western Michigan University, EvaluATE

County Health Rankings Model
Website
Presents a model of community health highlighting how policies and health factors influence health outcomes. Includes measures to help communities have a better understanding of the current state of health within their population and how it will impact their future health.
Organization(s): University of Wisconsin Population Health Institute, County Health Rankings & Roadmaps

Equity-Focused and Gender-Responsive Evaluations
Website
An electronic course examining the opportunities and challenges and the related methodologies when evaluating equity-focused and gender-responsive programs and policies.
Organization(s): EvalPartners

How to Embed a Racial and Ethnic Equity Perspective in Research: Practical Guidance for the Research Process
Document
Provides tools and resources for using a racial and ethnic perspective in research.
Author(s): Andrews, K., Parekh, J., & Peckoo, S.
Organization(s): Child Trends
Date: 8/2019

Measurement of Health Disparities, Health Inequities, and Social Determinants of Health to Support the Advancement of Health Equity
Document
Discusses recommended practices for measuring and monitoring health disparities, inequities, and social determinants of health to advance health equity.
Author(s): Penman-Aguilar, A., Talih, M., Huang, D., et al.
Citation: Journal of Public Health Management and Practice, 22, Suppl 1, S33-S42
Date: 2016

Racial Equity Tools: Evaluate
Website
Provides information and resources on how to evaluate programs to achieve racial equity. Topics include preparing for evaluation, defining evaluation concepts, designing the evaluation, collecting and analyzing data, communicating findings, and reflecting.
Organization(s): Racial Equity Tools

We All Count
Website
Provides resources to further advance equity in data science developed from a variety of projects, including independent research, on-the-ground data capture, partnering with experts, and consults with communities. Includes a Data Equity Framework to help identify equity issues and weaknesses.
Organization(s): We All Count