Evaluation Strategies and Considerations for Rural Chronic Disease Management Programs
Rural communities can implement different types of evaluations to measure the progress and successes of chronic disease management programs. When designing an evaluation, programs should consider the intervention goals, evaluation purpose, and available resources for evaluation. The Rural Community Health Toolkit provides additional information about evaluating a program.
Choosing Evaluation Questions
A key component of designing an evaluation is determining what questions the evaluation should answer. There are two types of questions to consider when evaluating quality of care: process and outcome. Process evaluation questions focus on clinical activities: whether the program is operating as planned and why. Outcome evaluation questions focus on the desired health result and provide information on how well the program achieved its intended short- and long-term goals: health outcomes, hospitalization rates, behavior changes in patients, and improvement in patients' quality of life.
Examples of Process Evaluation Questions
- What chronic disease management services are being delivered and to whom?
- What percentage of patients are being screened for chronic diseases?
- What percentage of patients are receiving referrals to other services?
- How many people are attending chronic disease management classes?
Examples of Outcome Evaluation Questions
- Has the program demonstrated measurable improvements in its intended health outcomes, such as reduced hypertension?
- Did participants experience reductions in hospital readmission rates?
- Did participants experience a reduction in the average length of hospital stays?
- Did participants experience reductions in emergency department visits?
Choosing Appropriate Methods
Rural communities often use multiple approaches to gather data that demonstrate the impact and value of the program. Mixed methods refers to an approach to collect and analyze quantitative and qualitative data.
Examples of quantitative methods include:
- Using a questionnaire to measure pre/post knowledge gains after a chronic disease management class
- Analyzing clinical data to identify trends in clinical outcomes (for example, rates of hypertension, A1C control)
- Mapping rates of chronic disease in a community to illustrate changes over time
- Conducting an economic assessment to analyze costs associated with reduced hospitalizations and emergency department visits among program participants
Examples of qualitative methods include:
- Holding focus groups to assess the experience of participating in a chronic disease program
- Interviewing community health workers to understand facilitators and barriers to improving chronic disease management among the patient population
- Conducting site visits to ask questions and observe how a program works
Challenges with Evaluating Rural Chronic Disease Management Programs
Multiple factors contribute to an individual's ability to manage their chronic disease. These may include environmental factors such as the availability of food stores and walking paths, social and economic factors like educational and employment opportunities, and having strong social networks, among others. Rural communities should assess the community context that affects chronic disease management programs, including community characteristics and barriers to program participation.
Patients enrolled in a rural chronic disease management program may be receiving care for multiple chronic conditions and receiving different types of services. It may be difficult to attribute outcomes to a particular program. The Centers for Medicare & Medicaid Services (CMS) provide some considerations for measuring quality of care for multiple chronic conditions.
Some chronic disease management programs may include self-management activities to support patient behavior change. Behavior change occurs in stages, and individuals may experience setbacks before experiencing long-term, lasting changes. It can be challenging to evaluate behavior change due to its long-term, cyclical nature. Because evaluations typically occur at a specific point in time, it may be appropriate to track short-term or intermediate outcomes (for example, changes in diet or exercise) rather than long-term outcomes.
For an overview of the key challenges related to program evaluation, see Evaluation Challenges for Community Health Programs in the Rural Community Health Toolkit.
Resources to Learn More
Cardiovascular Disease Data, Tools, and Evaluation Resources: Evaluation Basics Guide
Website
Provides tools and resources to assist the work of states, tribal organizations, public health
agencies, and communities in their development and evaluation of programs focused on management of
hypertension, and heart disease and stroke prevention.
Organization(s): Centers for Disease Control and Prevention