Considerations for Health Equity Evaluation Measures

A strong evaluation can help to:

Build the case for investing in health equity programs and initiatives
Assess the progress and outcomes of health equity initiatives
Inform decision-making
Support continuous program improvement

It is important that communities carefully select health equity evaluation measures to ensure they align with the program model and community members' goals and expectations. Communities and organizations may experience barriers to data collection and evaluation for health equity programs, such as limited resources and capacity for evaluation and challenges inherent in connecting specific programs and policies to changes in health outcomes. It is also important for programs to share progress throughout the evaluation period to keep program staff apprised of any potential midcourse corrections and provide accountability to the community.

Many of the considerations and indicators for evaluating rural social determinants of health programs can also apply to rural health equity programs. Additional information on identifying strategies and measures for gathering appropriate data and evidence can be found in the Rural Community Health Toolkit.

Consider Short-Term, Intermediate, and Longer-Term Measures

Some health outcomes of health equity programs might take years to achieve. Barriers to measuring longer-term health outcomes include expedited program timelines imposed by funding organizations and other external factors. Communities should plan to measure short-term and intermediate outcomes in addition to long-term outcomes. Using a range of measures can help demonstrate the progress made toward goals over time, provide data needed to improve or adapt program strategies and activities, and support longer-term strategic planning efforts. When possible, communities should establish a baseline for key measures before the program begins to assess changes over time.

Collect Process and Outcome Measures

Combining process and outcome measures can provide a more comprehensive picture of the program's operations and influence on health equity. Process measures can help evaluate how a program performs and achieves its outcomes. Rural communities can also use process measures to assess program planning, engagement of investors and partners, resources needed for the program, and other aspects of program implementation. Outcome measures assess the program's influence on its intended goals, the community, and the population(s) of focus.

Vary Data Types and Sources

Communities should consider using both qualitative and quantitative data when evaluating health equity programs and initiatives. Qualitative data can help evaluators investigate community members' experiences and perspectives and provide depth and context for quantitative data. Qualitative data can also be used in techniques like digital storytelling, which help lift and amplify the voices of community members. Evaluation methods might include surveys, focus groups, interviews, and secondary analysis of other data sources, such as the Census. Using data from different sectors (for example, those outside of the health field) can also help to develop a stronger understanding of program impact.

Collect and Analyze Social and Demographic Data

Evaluators and implementation partners should collect social and demographic data to assess health equity program outcomes across population groups in the community. Analyzing these data is key to understanding how the program affected people with differing levels of power and privilege. Rural communities should assess whether the program reached its intended populations and contributed to unintended consequences among different groups. Examples of social and demographic data to collect include age, race, ethnicity, sex assigned at birth, gender identity, sexual orientation, language, income, educational level, country of birth, occupation, religion, disability status, veteran status, and ZIP code. These factors should be analyzed on an ongoing basis. As mentioned in the Evaluation Strategies and Considerations section, measures and data collection approaches should be culturally responsive and appropriate for the groups participating in evaluation activities. Rural communities may also consider exploring how evaluation data differs based on multiple demographic characteristics: for example, analyzing data by income level within race/ethnicity groups. Reporting data by specific population groups helps advance understanding of equitable impacts of policies and programs.

Be Mindful of the Level of Measurement

To the extent possible, communities should consider collecting and analyzing evaluation data at multiple levels of measurement, including at the individual, neighborhood, community, and county levels. Multiple levels of measurement can provide a more complete picture of the health equity issues being studied: for example, by collecting data on both individual health outcomes and community-level partnerships, policies, or environmental factors. The availability of comparison data can also inform decisions about the level of measurement and the selection of measures.

Resources to Learn More

Addressing Health Equity in Public Health Practice: Frameworks, Promising Strategies, and Measurement Considerations
Document
Discusses considerations, strategies, opportunities, and challenges for addressing and measuring health equity in public health practice.
Author(s): Liburd, L. C., Hall, J. E., Mpofu, J. J., et al.
Citation: Annual Review of Public Health, 41, 417-432
Date: 4/2020

Addressing Health Equity Through Action on the Social Determinants of Health: A Global Review of Policy Outcome Evaluation Methods
Document
Highlights promising approaches and methods in conducting health equity evaluations of public policy focused on the social determinants of health and discusses the limitations of the current methods used.
Author(s): Lee, J., Schram, A., Riley, E., et al.
Citation: International Journal of Health Policy and Management, 7(7), 581-592
Date: 7/2018

Create the Data Infrastructure to Improve Health Equity
Document
Describes strategies to improve health equity through a data collection and stratification plan that includes staff training, identifying inequities, priority setting, and drivers of change. Discusses race, ethnicity, and language (REaL) data, or stratified data focused on race and ethnicity and which may also include sexual orientation and gender identity and/or other factors related to inequities in healthcare.
Organization(s): Institute for Healthcare Improvement
Date: 11/2019

Measuring Sex, Gender Identity, and Sexual Orientation
Document
Provides recommendations for measuring sexual orientation, sex, and gender identity and for identifying sexual minority populations.
Organization(s): National Academies of Sciences, Engineering, and Medicine
Date: 2022

More Rigorous Data Needed to Determine Success of SDOH Intervention Programs
Document
Summarizes the key issues and potential challenges drawn from a webinar discussing the evaluation of social determinants of health programs.
Author(s): Sabharwal, R. & Tudor, C.
Organization(s): AcademyHealth
Date: 5/2021