Principle 2. The level of each item translates into levels of action. The measure must be non-arbitrary; every rating has an immediate meaning and the meaning relevant to future action.
by: Dr. John S. Lyons
Click HERE to return to the home page for a full look into this series
Concept: Blanton and Jaccard (2006) criticized psychometric measures as arbitrary.[1] In other words, the numbers do not necessarily have any immediate meaning. This is a major limitation to practical applications of measures. For example, it is impossible to know the meaning of a difference of a 60 versus a 50 or a 17 versus a 13 on a psychometric measure. Norms do not fully solve the problem because they simply convert these arbitrary metrics into percentile ranks which still do not easily translate into use in the field. What are the implications of the difference between and 80 and a 70 on an IQ test? Plus, establishing and maintaining norms is difficult and expensive, and in some cases may even be impossible, as norms require the accurate measurement of a population. Since Communimetric measures are designed to communicate, meaning is fundamental, so arbitrary measurement does not work from this perspective.
The first stage of helping is to understand the person’s (or family’s) circumstances. Before initiating help this type of discovery phase is critical. Oftentimes this phase is called assessment. However, it may also be related to access and engagement. The output of the process of understanding should be a prioritization based on what can be done to HELP. Given this recognition of the fundamental structure of helping, Communimetric measures are designed to translate the understanding of the person into a plan to help. The structure of a Communimetric measure is designed so that it fits in the space between the output of the discovery process and the input of the planning process.
There are many possible action structures for a communimetric measure. The simplest is a two level model:
- Act
- Don’t act
While this two option model is the essence of helping, such a stark distinction can be unsatisfying to professionals and the people they serve who may wish for a more nuanced approach. For this reason, the most commonly used action format is the following:
- Do not act
- Watch/prevent/assess
- Act
- Act immediately or intensively
This structure both establishes clear action levels and creates an ordinal scale of measurement that reflects escalating action. These more nuanced scales are useful for program and system level applications of the Communimetric approach.
Many measures also have strength or asset or skill type items that are positive in nature and are used differently in planning processes. The standard action levels for these items are:
- Strength/skill fully developed (mastery) and may be central to the planning process
- Strength/skill immediately useful
- Possible strength/skill identified but must be developed
- No strength/skill evident
While the action structures described above are the mostly commonly used among existing Communimetric approaches, they should not be considered exhaustive. There are actually an enormous variety of different types of action frameworks that might be relevant in different circumstances.
Background: I first proposed the action levels as an alternative way to think about the Likert type ratings in the original Severity of Psychiatric Illness (SPI) and Childhood Severity of Psychiatric Illness (CSPI). These measures had four level items that were anchored with an essential structure of:
- No evidence
- Mild
- Moderate
- Severe
However, since these tools were often used in chart reviews it became difficult for reviewers to come up with consensus understanding of these levels. To assist in chart review studies, I began training reviewers to consider whether something needed to be done in order to distinguish the mild from the moderate level. To better distinguish the moderate from the severe level, I suggested people consider whether or not the need was dangerous or disabling.
In the conversion of the CSPI to the CANS in 1999, we met with parents and professionals in Allegheny County, Pennsylvania. At this two day meeting, the original Child and Adolescent Needs and Strengths (CANS) tool was created. Walking back from lunch on the first day with the lead family representative, Julie Hladio, I was struck by something she said in passing. From her perspective she said that the family participants thought the action level way of thinking about the item structure was the single most important aspect of how we were talking about developing the CANS. She told me that parents and family members were sick and tired of undergoing assessment processes where in the end they had no idea what they were supposed to be working on next with their child. This conversation was the primary impetus for the action levels moving to the forefront of the item design. Now the action levels are the cornerstone of the Communimetric theory of measurement.
Proof of Concept: The use of the action levels — in combination with individual item reliability — has created analytic options that are simply unavailable with other measurement approaches. Communimetric measures can be effectively mapped into sophisticated treatment planning approaches. Also, algorithms can be developed to support decision making that divides items into actionable (2 or 3) versus not actionable (1 or 0). At the time of this writing there are now five independent research groups who have demonstrated the reliability and validity of this approach to decision support using the CANS (for example, see Chor, et al., 2012[2]; Israel, et al., 2015[3]). This scaling approach also fits very well into machine learning analytics that allow branching logic with decision breaks at meaningful levels of each items[4].
Despite the item level design and the action level structure, it also has proved possible to create traditional psychometric scales using Communimetric items.[5] These scales can be useful for research and evaluation purposes and systems level outcomes monitoring.
In sum, the action level structure of a Communimetric measure creates a meaningful approach at the person level to effectively support planning and level of care decisions, but simultaneously provides useful information in support of more sophisticated analytics.
FOR A FULL LOOK INTO THIS SERIES, CLICK HERE TO RETURN TO THE HOME PAGE.
If you are interested in the articles cited in this post and others in the series, contact support@TCOMTraining.com.
—
