“If you can’t measure it you can’t manage it.”
How many times have we heard this said – and accepted it as a truism? At last week’s IHI International Forum in Glasgow, Don Berwick asserted, with his characteristic mix of eloquence, humility, humor, and passion, that “all measurements are wrong”.
In measuring we create a construct that’s a faulty representative of what matters. How many dinners we share is a mere shadow of the quality of a relationship or ultimately of the love that might exist between the diners.
Dr Berwick’s arguments are prompted by the excess of measurement currently experienced in the US, the UK, and other developed health care economics. Thousands of quality metrics are mandated by regulators, government, payers and others. The cost of measurement has soared.
Berwick describes this phenomenon of excessive measurement as absurd and destructive. In excess, measurement murders spirit, and dumbs down meaningful improvement. As more measurements appear the field engages in counter-measures; then counter-counter measures; this absorbs billions of dollars.
Many of our goals in health care improvement are not accessible by a straight shot metric. In particular, single measurements of exacting accuracy are not useful. Our goals require continual adjustment. Multiple simpler measurements over time are sufficient; when we consult the weather report we do not expect or need a temperature recording of 21.35C.
All measurement is waste if it is not embedded in care, and instrumental to care, from the viewpoint of the patient. A classic example: the NHS mandate of a 4-hour maximum time to patient disposition in A&E. Measurements typically show a spike at 3h 59 mins. Does this reflect what we really want? In patients with sickle cell crisis, or crushing chest pain, is 4 hours really the target? Conversely what if we wait just 1 hour more in order for a condition to resolve or a test result to become available for the patient not to need admission? What does this game do to the sense of purpose for staff – how does it distort the meaning in their world?
- Handle measurement with great care; it is a beast to be tamed.
- Put measurement on a diet; a large health care organisation may need only 15 high level measures not 150.
- Measurement must be tamed by conversation and reflection. Put measurement to use for us not the other way around. Conversation first, last and always.
- The shorthand PDCA (C for check) was changed to PDSA (S for study) for a reason. Berwick argues we should study meaningful difference through reflection and learning not ranking and judgment. Study is a wide array of approaches, not only quantitative. We can manage without quantitative measures – numbers – we do it all the time. We should be the masters not servants of numbers and we should measure just enough to get good enough answers.
- Learn Statistical Process Control. Understand variation. Avoid “traffic light” systems of control.
What about SA?
Our situation is not necessarily the same. The burden of measurement here is relatively small.
By comparison our hospitals have few mandated or published measures for accountability or judgment.
But we also do relatively little to study, and to engage in appropriate conversation and reflection that enables improvement.
Private funders “profile” hospitals and doctors using large amounts of data that are the byproduct of payment, but little or no clinical measurement is required from them. The District Health Information System processes large amounts of data but high quality data is seldom available to the front line for clinical improvement.
Front line clinicians cannot afford to sit back and let ill-informed officials and bureaucrats impose unnecessary, and expensive measurement processes but equally cannot manage without any measurement. We cannot run our organisations without meaningful performance data, focused on what really matters to our patients. Recent large scale, costly enquiry into parts of our health care system, including its quality, have not yet produce constructive quality measurement recommendations; clinical leaders have an opportunity to lead.
We need to avoid mistakes made elsewhere. We should develop a better understanding of what our minimum measurement sets might be, while devolving other measurement to the front line, targeting improvement and learning, without waste, tailored to our specific problems and our goals.