Paul Cockwell, KQuIP Co-Chair
The major variation in clinical outcomes and patient reported experience between and within renal services is a stark reminder that the system is the major factor in the quality of patient experience and in ensuring that each patient has the opportunity for their best outcome.
A fundamental goal of quality improvement (QI) is to improve experience and outcomes for patients at a system level. The statement is easy, but the challenges are profound; QI is a heavily squeezed middle of professional activity in healthcare. Rather than embedding in practice it is often seen as a separate activity that requires its ‘own time’ to develop and deliver. Many clinical staff are not involved in QI in a way that genuinely delivers improvement, and some are not persuaded that involvement in QI is part of their job.
Let me give you two simple examples of this QI gap
The national CKD audit showed major shortfalls in albuminuria testing. As albuminuria is the cornerstone of clinical management pathways in CKD, defining treatment choice, blood pressure targets, and referral, huge numbers of patients have suboptimal management. And how much QI is going on in this area? Not a lot, I suspect.
And what of PIVOTAL? a UK based multi-centre trial of high dose iv iron vs reactive dosing. This showed that less patients who received proactive high dose iv iron reached a primary end point of a composite of death from any cause, nonfatal myocardial infarction, nonfatal stroke, and hospitalization for heart failure compared to patients who received reactive low-dose iv iron. There was a 23% lower ESA requirement in the high dose group. How many dialysis unit are using high dose iv iron almost 1-year after PIVOTAL has been published?
These are two of many major improvement opportunities. The challenge of albuminuria is defining the best QI approach(es) which may vary by location and then using it: that is implementation science research and generalisability. For PIVOTAL it is establishing a simple system for change in a clinical service.
So those of us who work directly with patients must accept that we have a major opportunity to improve the system and therefore patient experience and outcomes by developing our QI skills and embedding them in routine clinical practice. Through involvement in QI there is also a strong secondary benefit to us as individuals through developing a broader set of skills that can sustain our careers, using tools that we can apply effectively to future problems and new roles.
We must also state and restate the academic basis for QI
Eyebrows may rise here, as ‘academic’ and QI is widely considered an oxymoron, with QI described variably as ‘common sense’ ‘boring’ ‘evidence free’. Pick an area for improvement, use some fashionable language, set up a project team, and off you go. Baloney words and no methodology. Why evidence seek when there is no basis for evidence?
But there is a strong methodological science that is well established. The challenge is that this science is often not applied. Consequently, QI projects are often not credibly designed to demonstrate impact. Core questions are often not incorporated: for example; what is being measured? how is that measurement being reported? and what are the parameters within that measurement that demonstrate benefit or no benefit? Failing to define measurement a priori and to collect and analyse data with time is the rock upon which many QI projects are broken. This is despite the science of measurement in QI being long established.
However, the use of measurement in QI is different to the use of measurement in clinical and laboratory research. Standard tests for statistics are usually cross sectional and will describe uncertainty through a p-value or a Hazard ratio. This may help to describe an experiment or a clinical trial, but it does not help to describe change with time in a dynamic setting – which is a requirement in QI, where measurement needs to factor for the natural variation in a system that occurs with time. This natural variation also known as regression to the mean. This rule states (and I paraphrase) that in a complex system dependent on many variables, by chance extreme outcomes tend to be followed by more moderate ones. We all recognise this…” we haven’t had a case of anti-GBM disease for 3 years; and then we have had two cases this month”.
All measurements are made up by two parts: true and random error
Therefore, when we make a change in a system, such as increasing theatre capacity for vascular access, the first few measurements after the change should be treated with scepticism. For example, if the scores in the preceding months were on the low end for performance, then a natural variation upwards would have been expected at some stage. We should not extrapolate from small numbers over short periods.
There is a standard measurement tool for QI that addresses this challenge by providing a dynamic measurement that incorporates upper and lower limits for natural variation and therefore can identify a true effect. It is called statistical process control (SPC): this technique arose from the motor car industry in the 1920s and was first published on in 1933, 25 years before Kaplan and Meier described their ubiquitous estimate of survival function. Statistical process control measures change with time to define true changes and can be used with smaller data sets to detect changes and trends as early as possible. Repeated measures produce variation, if over time processes are stable then that variation will change around a mean level. In SPC we can assess true change and the impact of the changes that effect true change, by changes in the mean level with time.
The control chart is the key tool of SPC
It includes a centre line, typically the mean, the upper control line (UCL), and lower control line (LCL). If data falls outside the control lines or displays abnormal patterns, then this indicates special cause variation: this refers to variation causing non-random distribution of events caused by a specific factor (e.g. increase in prevalent fistula rates due to increasing theatre capacity for vascular access capacity). A control chart will often use three standard deviations (SD) for UCL and LCL. At this level the chance of it occurring randomly is low, so you probably need to consider anything outside the control lines as a special variation (true change).
In addition to one point outside the UCL or LCL other markers of true change can be used; e.g. two of three successive points more than 2SD from the mean on the same side of the centre line; four of five successive points more than 1SD from the mean on the same side of the centre line; eight successive points on the same side of the centre line; six successive points increasing or decreasing (a trend). The collecting and graphing of data with time is powerful and has the potential to engage the clinical team in measurement and interpretation, encouraging increased rigour in incorporating measurement into practice.
A further benefits of control charts is that they do not require as much data as traditional statistical analysis, which relies on large aggregated data sets. The UCL and LCL can often be calculated within 30 data points, and then each new data point can be judged for special cause variation (statistical significance).
And from an academic perspective the use of control charts is a rich area; with variations required dependent on distribution of the data, the statistics used, and a range of other choices required dependent on the need of the project.
In QEH Birmingham we have a regular CKD service meeting where we look in detail at data from the service. We introduce change in service through this as it directs us to where we need to improve the service. What is the incidence of uncontrolled starts on dialysis? When are patients being supported to prepare for kidney care? What about our AVF waiting times? We use spreadsheet generated data and then show the results as a graph. We project and assess and comment: what does ‘not as well last month’ ‘a blip here’ ‘a bad quarter there’ mean? Is this variance or a genuine trend in data? Of course, a lightbulb moment, this can be addressed through SCP. An essential tool with a strong academic basis that can be used in clinical practice.