Evaluation Measures and Data Collection Tools for Rural Mental Health Programs

Rural mental health programs must be flexible when implementing treatment plans because of the different mental health needs of individuals seeking professional help. Providing individualized care can make standardization and measurement of mental health programs difficult. It can also be challenging to attribute changes in mental health status directly to a program or intervention. For these reasons, rural mental health programs should focus on identifying attainable, validated measures that help assess progress toward achieving program goals.

Examples of Process Measures

Number of mental health trainings offered
Number of counseling sessions held
Number of patients provided a referral for mental health services
Number of informational resources that are translated into additional languages for program participants
Number of patients seen by a provider within one week of intake
Number of patients receiving care in their native language

Examples of Outcome Measures

Rate of mental health-related emergency department visits
Improvements in knowledge or awareness of mental health conditions
Proportion of program participants reporting a decrease in negative mental health symptoms

Data Collection Tools

The data collection tool(s) a program uses will depend on the evaluation questions the program is trying to answer.

Some mental health programs may choose to assess fidelity or how well the program adheres to a particular intervention or evidence-based practice. Evaluation tools can also assess the quality of a mental health service or system and identify quality improvement efforts. For example, the School Health Assessment and Performance Evaluation System offers measures to assess the quality of school mental health systems at the school, district, or state level.

Administrative systems and electronic health records (EHRs) can be used to collect and maintain data on process measures such as number of patient visits, documented referrals, and screening results. Intake forms can collect baseline data on participant demographics and service needs. Clinical assessment tools are validated instruments that can be used to assess a patient's mental health symptoms and severity.

Surveys or interviews can be used to collect data on patient satisfaction or knowledge. Focus groups can also be useful for gathering input from providers or program participants.

Existing secondary data sources, like the Behavioral Risk Factor Surveillance System (BRFSS), can be used to assess baseline mental health data at the state and local level, but variations in the data over time may not be directly attributable to a particular program or intervention.

Partners may also contribute evaluation data, such as patient-level outcomes data or recommendations to improve program implementation. For example, local healthcare systems can provide data on mental health-related emergency department visits and hospitalizations. Law enforcement agencies can collect and share data from crisis response activations.

All evaluation data gathered should be assessed and interpreted critically to ensure the conclusions drawn are appropriate based on the data. For example, an observed increase in the number of calls to a crisis response hotline could be interpreted as an increase in mental health concerns or as a result of successful promotion and increased awareness of the program.

Resources to Learn More

Evaluating Mental Health Promotion Programs: A Supplement to The Best Practice Guidelines Series
Document
Discusses considerations for evaluating mental health programs while providing examples of process and outcome indicators, and data sources for different types of programs.
Organization(s): Centre for Addiction and Mental Health
Date: 2018

Mental Health Quality Measures: Child Health Care Quality Toolbox
Website
Offers quality improvement measurement tools and resources for evaluating mental health programs for children.
Organization(s): Agency for Healthcare Research and Quality