Measuring Whether an Organization’s Trauma-Informed Care Efforts Are Working
As the movement toward trauma-informed care (TIC) continues to grow and evolve, organizations, systems, governing bodies, and funders are understandably asking, "How can we tell if we are making progress?" Many of us have experienced these powerful “aha” moments of TIC:
- Trauma is pervasive and explains so many negative health outcomes
- You can’t just treat symptoms, you need to look at the root cause
- Ask “What happened to you?” rather than “What’s wrong with you?”
- Harm and distrust results from relationships, so healing happens through corrective relationships
- You can’t work with trauma survivors without being impacted yourself
While TIC just makes sense, there is little hard evidence that it leads to positive outcomes. Is this because we don’t yet have the tools to measure those changes? Have we not done large enough or sophisticated enough studies? Are TIC interventions not yet as impactful as they need to be? Is TIC change not as impactful as we think it is? Or, as some scholars have suggested, are we not always asking the right questions?
In the health sector, the holy grail of outcome evaluation is a favorable change in patient outcomes: Is the user of the service better off? This can also be applied to client, consumer, and student outcomes:
- Do heavy users of health-oriented consumer services have fewer health care visits?
- Are more clients discharged to a lower level of care?
- Do more students graduate high school and not drop out?
However, as noted in a brief, Measuring the Impact of Trauma-Informed Primary Care: Are We Missing the Forest for the Trees?, issued by the Center for Health Care Strategies (CHCS), “enthusiasm for developing and implementing TIC has far outpaced the field’s ability to measure the impact of such programs.” Of the few studies that do exist, changes in patient outcomes have been limited to narrow sectors and are not necessarily generalizable to other sectors, such as primary care, corrections, or schools. CHCS led the national initiative Advancing Trauma-Informed Care supported by the Robert Wood Johnson Foundation.
Patient outcomes are of course important. We need to continue to routinely collect them and study whether they are linked to organizations' demonstrating trauma-informed care. However, is this the only way of judging the success or failure or trauma-informed care?
The CHCS article argues that it is not. The issue brief, while mainly looking at primary care, poses this question relevant to all sectors: If the outcomes are only such things as patient engagement and satisfaction or workforce turnover, is it still worth doing? Stated another way, “by focusing on the trees (i.e., patient outcomes) when evaluating TIC programs, are we missing critical parts of the forest?”
The intervention discussed in the CHCS brief—a large-scale initiative at Montefiore Medical Group to train 1,000 staff on trauma-informed practices as well as implement universal adverse childhood experiences (ACEs) screening at its 20 primary care clinics—sought to emphasize more proximal (shorter term) outcomes of their interventions rather than distal (longer term) patient outcomes (i.e., symptom level, service use). So, for example, they measured staff knowledge, attitudes, and skills resulting from workforce TIC training. To evaluate their TIC workplace wellness intervention, they measured variables such as workforce turnover, employee burnout, and compassion fatigue. They also measured practice delivery and patient engagement and satisfaction.
Like Montefiore, other organizations are asking how to evaluate meaningful change resulting from their TIC efforts other than (or in addition to) patient outcomes. They ask themselves whether these outcomes are still worth examining. Also, like Montefiore, they are concluding that the answer is a resounding “yes.”
This is the reason that many organizations have been so enthusiastic about using the Attitudes Related to Trauma-Informed Care (ARTIC) Scale. A psychometrically validated tool, the ARTIC Scale measures the more proximal (shorter term) variable of professional attitudes toward TIC that can result from a wide variety of TIC system interventions, such as workforce training, TIC supervision, or trauma screening. Even a brief intervention such as a two or three day trauma training has been shown to significantly increase ARTIC scores from pre- to post- training as well as at 3-month follow up. Tracking ARTIC scores over time in an organization or school provide important data about whether the system is becoming more trauma-informed.
While we do not yet have evidence that higher ARTIC scores result in positive client outcomes, there is anecdotal, “practice-based evidence” that organizations with higher scores have experienced both improved service quality and client engagement. The theory here (that of course needs to be tested) is that staff attitudes more favorable toward TIC can drive more trauma-sensitive behavior with clients and that this will improve client outcomes. But, short of this result, there is still great value in charting the advancement in trauma-informed system change along the way.
Reference:
German, M., Crawford, D.E., Dumpert, K. (2020, February). Measuring the Impact of Trauma-Informed Primary Care: Are We Missing the Forest for the Trees? (Issue Brief). Trenton, NJ: Center for Health Care Strategies.
Learn more about the ARTIC Scale at articscale.org
