We talked a while back (OK, OK, it was 12 years ago – Post: 2013) about the necessary/sufficient construct: an elegant mathematical principle that can bring some order to a muddled world or a muddled head:
The ‘necessary but not sufficient’ construct, in which we distinguish between conditions that are surely required, yet not by themselves enough, to bring about a desired outcome. How much argumentation about social problems and solutions could be avoided by adopting this simple phrase and, by extension, admitting that complex problems often do not have single solutions?
I heard another elegant distinction this week from iHeart Radio’s doctor: statistical significance versus clinical relevance. He was talking about how a treatment might have a statistically significant beneficial effect (meaning, not likely due to chance), while not having any clinical significance (meaning, not helpful to real patients in the real world) due to several possible factors:
- The effect is too small to matter.
- The side effects are too many, too unpleasant, or too dangerous.
- The treatment is too hard to apply at scale.
- The cost is too high.
I’m thinking that extremely calorie-restricted diets might fall into this category: effective for weight loss, sure, but not sustainable by most people except for short periods, and not healthy for long periods. It’s also easy to imagine a hideously pricey new drug that would extend life for an unpredictable third of all cancer patients by, say, one month, but that would also cause strong nausea in most cases. Would we really rush to prescribe it for everyone? For anyone?
Statistical significance. Clinical relevance. Both matter: the former because we want to know when we’ve truly found a cause-&-effect situation rather than just an association; the latter because we often don’t want to tinker around the margins of a problem, and we never want to saddle someone with side effects that are worse than the disease or condition being treated.
Here’s the other thing. Sometimes, I think, a treatment can be clinically relevant but not statistically significant. I have two prescribed drugs in my cupboard for what are called off-label uses. I take these to be uses that deliver good results for some people, but not for enough people to be statistically proven effective–or effective enough–over the whole population.
This is life, innit, not just medicine?
In every field of endeavour, in everything humans try, we get some duds: Some things don’t work.
We get some disappointments: Some things work but have enough built-in problems that we can’t/don’t execute them.
We get some mysteries: Some things work for some people, but not for all (or, even, not for many), and we don’t know why and why not.
We get some miracles: A few wonderful things work reliably for most people, or at least for a predictable subset of people, without causing new problems.
And while we’re appreciating the miracles, we can take a minute to think about what category each of our non-miracles falls into. To convert our duds, disappointments, and mysteries into miracles, do we need to work on statistical significance or on clinical relevance?
I think we work on clinical relevance because that is the only method open to us that can build up statistical significance. For example, I was reviewing the monograph I wrote on our middle son’s recovery from schizophrenia and on his relapse and second recovery using my Focused Listening music therapy. Students visiting our stroke-victim son in hospital notice his headphones and ask about the same therapy, which is being used in a different application. I can produce some limited statistics about the application of the therapy to mental illnesses, but I have seen it applied only three times to brain injuries that are not ear-related. However, if I were to revise the monograph, I could add not only those three stories but some others where the problem being solved might not have been ear-brain related but ear-vagus-nerve related: Hashimoto’s thyroiditis, a brain infection, and a traumatic injury to the jaw that caused psychosis, apparently because it harmed the hearing mechanism. Each of those applications of Focused Listening were clinically significant but they don’t add up to impressive statistics — unless you redefine the categories. The music therapy has a neurological impact, would be one way of talking about it. It has not been tested widely enough yet to accumulate certain kinds of statistics. On the other hand, the neurological paradigm that explains the value of the therapy also explains a myriad of symptoms that nurses, paramedics, PSWs, other hospital technicians, and doctors encounter constantly. Knowing how ear function affects brain function could have a huge effect on treatments and care. I have opportunities for sharing and teaching almost daily, but no statistics on how much of that teaching is changing practice.
Laurna – Your Focused Listening is an excellent example of a treatment where clinical relevance far outstrips statistical significance. I would guess that many (if not most) new paradigms must go through this stage – painfully slow to take hold, but with lots of potential once the anecdotal evidences is sufficient to command attention.