Evaluation Measures and Tools for Health Literacy Programs

Rural health literacy programs should identify measures that map to the program evaluation questions. There are several resources programs can use when identifying or developing measures.

When choosing an appropriate scale, programs should consider program goals, target population, and other specific contexts such as:

Is the program addressing specific health literacy concepts: communication (speaking/listening), application, conceptual knowledge, information seeking, media literacy, or numeracy?
Is the program addressing a specific disease state, such as cancer, oral health, diabetes, or general health promotion?
Is the program designed for a specific age range, including children, adolescents, adults, or older adults?
How long will the scale take respondents to complete?
How is the scale administered: face-to-face, paper and pencil, computer-based, or mailed?
What is the language of the validated scale?

Personal Health Literacy Measures and Tools

The Boston University Health Literacy Tool Shed is a resource that describes health literacy validated scales to measure personal health literacy. These scales are tailored for specific populations and disease states and could be adapted for use in many different communities. The website gives information on peer-reviewed, validated, published measures tested for reliability with at least 100 participants.

Examples of evaluation tools in the Health Literacy Tool Shed include:

Brief Health Literacy Screener – a 3-item scale for assessing information-seeking behaviors among adults 18-64 years old.
Rapid Assessment of Adult Literacy in Medicine (REALM) – a 66-item scale, which has also been adapted for short form (7-item), for teens, for disease states, and so on, to assess familiarity with relevant words.
Test of Functional Health Literacy in Adults (S-TOFHLA) – a 67-item scale assessing reading comprehension and numeracy of adults. This is also adapted to be a shorter form scale with 40 items.

The Agency for Healthcare Research and Quality (AHRQ) Health Literacy Measurement Tools were developed to measure different aspects of health literacy, such as examining medical terminology for reading comprehension. The AHRQ resources assist with improving health literacy among those who speak English and Spanish.

Examples of evaluation tools highlighted on the AHRQ Health Literacy tool website include:

Short Assessment of Health Literacy–Spanish and English (SAHL-S&E) – an 18-item scale assessing comprehension and pronunciation (decoding) of health-related terms in English and Spanish.
Short Assessment of Health Literacy for Spanish Adults (SAHLSA-50) – a 50-item scale assessing a Spanish-speaking adult's ability to read and understand common medical terms.

Organizational Health Literacy Measures and Tools

The Health Literacy Environment of Hospitals and Health Centers (HLE2) assessment tool allows organizations, healthcare systems, and hospitals to rate health literacy specific factors within an organization. The tool helps measure organizational policies, practices, healthcare system navigation, culture and language, and communication materials (print, forms, websites, and patient portals). The resulting information will help assess health literacy levels and establish administration and organizational priorities and action steps.

AHRQ's Consensus Organizational Health Literacy Quality Improvement Measures identifies 22 measures for assessing organizational health literacy. The measures were developed for hospitals and are grouped in the following domains:

Organizational Structure, Policy, and Leadership
Communication
Ease of Navigation
Patient Engagement and Self-Management Support
Measures that Cut Across the Four Domains Above

Quantitative surveys can be used as an evaluation tool to include various types of health literacy outcomes, targeting multiple audiences. For example, the National Assessment of Adult Literacy (NAAL) is a nationally representative survey of American adults aged 16 or older that includes an assessment of adults' ability to use literacy skills with health-related materials and forms. These types of questions can be used by programs looking to improve individuals' health literacy levels. Adapting items from national datasets can be beneficial to organizations interested in examining validated scales to use for measuring health literacy.

Similarly, there are assessments such as the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Surveys that include questions focused on measuring providers' activities and interactions with patients to improve health literacy. There are many CAHPS surveys that measure patient experiences with different types of providers, health plans, facilities, and experiences with condition-specific care. The CAHPS Health Literacy Item Sets ask patients to comment on their providers' efforts to foster and improve health literacy. Topics addressed by the health literacy items in this survey include questions about communication with providers, patient self-management of health conditions, medications, tests and results, and medical forms.

The CDC Clear Communication Index Score Sheet is a tool that can be used for evaluating whether organizational health literacy program materials are health literate. The index has 4 introductory questions and 20 scored items. Organizations can use the tool to score printed materials, messages, and scripts to assess whether the main message is understandable and actionable for the population of interest. The Index helps evaluate the language and terms used as well as the overall design.

The Patient Education Materials Assessment Tool (PEMAT) is a systematic tool to evaluate and compare the understandability and actionability of print and audiovisual patient education materials. Materials are scored on six topic areas:

Overall content
Word choice and style
Use of numbers
Organization of material
Layout and design of content
Use of visual aids

The response options for the scale are agree (1) or disagree (0). Based on the total points and total possible points, a percentage is calculated, which indicates whether the materials are easily understandable by the population of interest and easily actionable.

Resources to Learn More

PA-HLC Health Literate Organizational Assessment
Tool
An assessment tool to assist when evaluating health literacy in an organization. Offers a method for scoring before and after implementation of health literacy activities in 3 broad categories: leadership, integration, and operations; preparation and communication; and information and navigation.
Organization(s): Pennsylvania Health Literacy Coalition
Date: 2018

The State of the Science of Health Literacy Measurement
Document
An overview and history of health literacy (HL) measurement at the individual level. Focuses on the association between HL definitions and measurement, methods to develop and identify stronger measures, characteristics of participants in HL studies, and considerations when administering HL measures. Discusses a strategy for advancing scientific study of HL measurement.
Author(s): Nguyen, T., Paasche-Orlow, M., & McCormack, L.
Citation: Information Services & Use, 37(2), 189-203
Date: 6/2021