Principle 1. Item Selection


Principle 1. Each item is selected because it is relevant to the decision-making process. Only relevant information is included.

by: Dr. John S. Lyons

Concept:    A communimetric measure is designed to represent a minimum standard of understanding of a person in their circumstances.   The individual items in this approach are designed to stand alone as independent pieces of information to communicate this understanding.   Therefore, the items are designed to be reliable and valid on their own. For many of the commonly used versions of this approach there is a core set of standard items.  For example, there is a core set of 50 items for the CANS that is recommended for most versions. Additional items are used when relevant to support decision making in different situations.   Items are selected in an effort to define a ‘minimum standard of understanding.’  In other words, items should reflect information relevant to:

  1. Contextualizing a person’s circumstances
  2. Reflecting a key target for an intervention
  3. Reflecting a key outcome of an intervention

Background:   This concept comes from the work of Ken Howard’s group, who could be credited with developing the very first outcomes management approach, eventually called Compass[1].   Previously we approached measurement from a traditional psychometric perspective:  We developed a symptom list to have a measure of symptoms; then we developed a functioning list to create a measure of function, and so on.   Once created, these measures were required to be unchanging.   Further, there was always pressure to make the list of items shorter.

In my book ‘The Measurement and Management of Clinical Outcomes’[2], I stated that clinicians could not be expected to spend more than five minutes completing a measure or they simply wouldn’t do it.    Working from this perspective, it became clear that our current design approach guaranteed that the measurement process was independent of the clinical process.   It was always an add-on.   This lack of integration between the actual work and the documentation of the work added operational burden to clinicians without providing them a perceived value.   At the same time I started working in the Department of Medicine at Northwestern University and became familiar with other communimetric measures.    In these approaches, single items were considered scales in and of themselves.     Those combined experiences led to the idea that modularized measures with different items for different applications might be more acceptable to people who complete them.

Proof of Concept:    There are a number of published studies (and more unpublished) that demonstrate that the individual items of a Communimetric measure are in fact reliable and valid on their own to better support the decision-making process.   The first reliability study was published in 2002 by Anderson, et al.[3]  These findings have been replicated in other countries with different Communimetric measures (e.g. Singapore and Europe).

In addition, the item level information has allowed these measures to be scored flexibly to allow a variety of applications that are simply not feasible with a traditional measure that results in a single score.   For instance, treatment planning applications allow for individual item needs to be allocated as Background Needs (not addressed in the plan but exists to provide context), Treatment Target Needs (i.e., causes of the individual’s current challenges) and Anticipated Outcomes (i.e., effects of the identified causes).   This scoring approach makes Communimetric tools like the CANS and ANSA valuable to clinicians for use in generating personal theories of change.   Alternatively, if the decision is about intensity of care or placement, flexible item scoring can allow for the creation of algorithms to support decision making.  These complex metrics involve patterns of actionable needs that have been shown to support effective decision making (e.g. Chor, et al., 2014).[4]   Finally, it is also possible to generate scale scores that are used for change analyses in a manner identical to applications of psychometric measures (e.g, Lyons, 2009).[5]    Thus the item level construction of a Communimetric measure allows maximum flexibility for using the information in a variety of ways to support different decisions in complex systems.

 

FOR A FULL LOOK INTO THIS SERIES, CLICK HERE TO RETURN TO THE HOME PAGE.

[1] Sperry, L., Brill, P. L., Howard, K. I., & Grissom, G. R. (2013). Treatment Outcomes in Psychotherapy and Psychiatric Interventions (Vol. 6). NewYork: Brunner/Mazel.
[2] Lyons, J. S., Howard, K. I., O’Mahoney, M. T., & Lish, J. D. (1997). The Measurement & management of clinical outcomes in mental health. New York: J. Wiley.
[3] Anderson, R.L., Lyons, J.S., Giles, D.M., Price, J.A., & Estes, G.  (2002). Examining the reliability of the Child and Adolescent Needs and Strengths-Mental Health (CANS-MH) Scale from two perspectives: A comparison of clinician and researcher ratings.  Journal of Child and Family Studies, 12, 279-289.
[4] Chor, K. H. B., McClelland, G. M., Weiner, D. A., Jordan, N. & Lyons, J. S. (2014). Out-of-home placement decision-making and outcomes in child welfare: a longitudinal study. Administration and Policy in Mental Health, https://doi.org/ 10.1007/s10488-014-0545-5
[5] Lyons, J.S. (2009).  Communimetrics:  A theory of measurement for human service enterprises.  New York:  Springer.
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s