The Value of Vignettes

Mention a case vignette to anyone who has gone through the certification process for the CANS, ANSA, CAT/CSPI or FAST and you will likely hear groans, and see eye rolls. Very few people are big fans of testing and vignettes are never as clear or easy as a trainee wishes. So, if test vignettes bring up such emotions, why do we still use them for testing?

The Nature of Vignettes

Vignettes are one page summaries that are based on actual cases, created by clinicians who are pursuing trainer certification. These cases can feel one dimensional, and seem to be “missing” important information that you “need” in order to be able to rate the items accurately. But vignettes do have all the information needed to identify needs and strengths and apply the action levels. Nothing provided on delusions, hallucinations, or bizarre thoughts? Rate a “0” on Psychosis; if it’s not mentioned, it’s not a need. No mention of whether the caregiver has family, friends or community connections to help with caregiving? Rate the caregiver a “0” on Social Resources. No evidence about the child’s outlook on life? Rate the child a “3” on Optimism for no evidence of a strength in this area.

The Purpose of Testing

Communimetric tools like the CANS and others noted above are meant to communicate action levels for needs and strengths. We ask: What are the needs/strengths of the individual/family and what needs to be done about them? In order to be confident that we are assessing risk level accurately and because we depend on the accurate output of the assessment to help us make decisions throughout our work with individuals and families, we need to make sure that we can use this tool reliably.

What is reliability? In this case, it means that when two people get information about an individual, they can agree that it either requires or does not require action. While two people may not agree on severity – one might think we must take immediate action while the second thinks action is needed. We refer to reliability as ratings being in the “ballpark,” because the closer you are to the preferred score (the “gold” score for each item in the test vignette), the more reliable you are. You become less reliable when you see evidence of a problem where no evidence exists (i.e., read into it), or when you miss evidence of a problem when evidence is present.

Testing is required in order to ensure that users of the tools are able to identify needs and strengths and use the action levels appropriately. Testing also allows us to confirm that the trainee has become familiar with the six key principles of Communimetric tools and to ensure that they are being applied when rating items. If you think of a Communimetric tool as a ‘common language,’ the vignette allows us to test on people’s mastery of the vocabulary. Remember, this is NOT a test of clinical skills.

Succeeding in “Vignette World”

Vignettes can be challenging, but if we understand that they help us build Communimetric skills, and learn not to “read in” to the vignette then we can accept them as a way to show reliable use of the tool.

Successful certification means mastering the test vignette both practically and emotionally. Use our test-taking tips document which we developed after finding patterns in testing failure. Embrace the concrete nature of the vignette and only rate what is there. Focus on what action is needed (or what strengths can be used to address needs). Vignettes remind us that the work begins with a conversation with the individual and other involved family members to determine where they are now, where they hope to be, and how we can help them get there.

A full discussion of the reliability of the CANS (and other tools) can be found in “Communimetrics: A Communication Theory of Measurement in Human Service Settings” written by Dr. John Lyons. Additional independent research and reviews have been conducted on the reliability and validity of the CANS and other Communimetric tools in this executive summary.