A few weeks ago, Dr. Danielle Ofri, published her personal view in the NEJM of receiving individual feedback on how patients attain certain performance targets. On his personal blog, Dr. Kent Bottles wrote a rebuff, where he suggested that Dr Ofri was implying that because she and other doctors cared about their patients, these scorecards were irrelevant. My reply stated that Dr. Ofri was not against feedback per se, but that she believed it should be at the level of the institution. Kent's thoughts are now reposted on the Health Care Blog where it has created much debate. I’m sure that Dr. Ofri doesn’t really need defending but here is my take on what she has written.
She starts by pointing out the silliness of treating success in reaching targets as a binary outcome and particularly mentions blood pressure control in diabetes. This is a good point. Treating hypertension in diabetes is about treating a risk factor in a population; one can never know the benefit for any individual patient. The original UKPDS study which looked at the impact of ‘tight BP control in diabetes’ compared targets of 150/85 (tight control) vs 180/105 (less tight control). The tight control group managed a mean BP of 144/82 whilst the less tight control group averaged 154/87. The tight control group had reduced all cause mortality and also lower rates of nephropathy and stroke. However, the tight control group were given a regimen including an ACE-inhibitor (a treatment that we now know to decrease all cause mortality in diabetic patients, and to be protective of kidneys) while the less tight control group were to be deliberately not given this. So we don't really know how much of the benefit was down to the actual blood pressure attained, or the treatment used.
But back to Dr. Ofri's point; looking at how many patients achieve a target might, but doesn't necessarily, tell you about how the overall BP in the population has changed. And that is what matters. An aggregate measure of how much change has been produced in the BP of all patients might be a better way of describing how well BP is managed in any practice. She could spend all her time trying to get the patients with a BP of 145/85 down to 130/80 to meet the target, but completely ignore the patients with much higher blood pressures because they will be so much harder to get to the target. I doubt that Dr. Ofri would ever be so cynical as to take this approach because as she says most doctors have the good of their patients at heart, and are not just trying to make a fast buck as quickly as they can. I work in the UK, so I can't say if Dr. Ofri is seeing her colleagues through rose-tinted glasses. She afterall has chosen to work in Bellevue Hospital, the oldest public hospital in the US, where 80% of patients come from under-served poulations.
But back to Dr. Ofri's point; looking at how many patients achieve a target might, but doesn't necessarily, tell you about how the overall BP in the population has changed. And that is what matters. An aggregate measure of how much change has been produced in the BP of all patients might be a better way of describing how well BP is managed in any practice. She could spend all her time trying to get the patients with a BP of 145/85 down to 130/80 to meet the target, but completely ignore the patients with much higher blood pressures because they will be so much harder to get to the target. I doubt that Dr. Ofri would ever be so cynical as to take this approach because as she says most doctors have the good of their patients at heart, and are not just trying to make a fast buck as quickly as they can. I work in the UK, so I can't say if Dr. Ofri is seeing her colleagues through rose-tinted glasses. She afterall has chosen to work in Bellevue Hospital, the oldest public hospital in the US, where 80% of patients come from under-served poulations.
Some of the discussion has been around what are meaningful endpoints for quality measures. We might presume that reducing blood pressure is always good, but it seems to be more complicated than that. Atenolol, which was one of the agents used in the tight control group in the UKPDS trial above, does decrease blood pressure but not overall mortality so it isn't a sensible choice for first-line treatment. And in diabetes achieving even lower blood pressures through aiming for a target of 120/80, is associated with more side-effects from medication but no benefits for patients. Dr. Ofri's failure to get her patients' blood pressures below 130/80may be a good thing for some of them.
But her main complaint about these scorecards for individual doctors is not about the choice of targets, although I hope I have helped you to understand what she meant when she said it is easy to pick fault with them. No her main issue is that the scorecards place responsibility with individual doctors for the outcomes of their patients. She doesn't have a problem with tracking outcomes, and says she would be keen to see how her institution compared to others. She cites a systematic review which shows that this kind of feedback may work at the level of the organisation but not at the level of the individual. She also cites an opinion piece in JAMA where the authors suggest there may be unintended consequences to publicly sharing feedback on the performance of individual doctors, and organisations. These consequences may include "causing physicians to avoid sick patients in an attempt to improve their quality ranking, encouraging physicians to achieve "target rates" for health care interventions even when it may be inappropriate among some patients, and discounting patient preferences and clinical judgment."
Could that be true? Only this evening I came across the following tweet
Pay-for-performance quality measures will result in docs firing noncompliant patients. I know I don't want a dipshit diabetic to sink me.
The Suburban Doctor
BurbDoc
The Suburban DoctorBurbDoc
In the UK, we have already introduced pay-for-performance in primary care. When this was introduced in a new GP contract in 2003, it was lauded by Paul Shekelle, as "the boldest such proposal on this scale ever attempted anywhere in the world". But he was also worried about unintended consequences. One was that areas which were not assessed in the performance measures would suffer. This is hard to assess, and new areas have been added in each annual review in any case, but it is thought that there has been no impact. The other was that the relationship between doctor and patient would change, with loss of some of the holism thought to define UK general practice. We can gain some insights in to how that might have been realised through the ethnographic work of Checkland et al. who documented the changes that the contract brought to two UK general practices. There was an increased focus on recording 'hard' biomedical data over 'soft' patient-centred data. But the staff involved did not see any change in their practice. It is well worth reading this and their other work to gain insights in to some of the impacts that pay for performance may be having in the UK.
Primary care doctors in the UK can not choose their patients. If their 'list is open' (they think that they have spare capacity) then they must take any patient that wants to join. So there is no risk that an individual patient may not receive care because of pay-for-performance. However, unlike in the US, patients may be excluded from denominators if they are having 'maximally tolerated treatment'. This may reduce some of the frustrations that doctors in the US feel about such performance measures.
Secondary care in the UK does not have pay-for-perfomance, or even publicly accessible feedback on performance, but evidence exists that there is a tension between protocol-driven care and tailoring care to the needs and preferences of patients. Sanders et al. have done some excellent work describing how this plays out in specialist heart failure clinics.
Getting back to Dr Ofri's concerns, it is worth noting that the feedback on achievement of targets in UK general practice is at the level of the practice, not the individual doctor (although there is still a size-able number of single-handed GP practices). How does this feedback play out in real life? Here is the prevalence data of disease areas covered by the contract for my own practice in South Wales. The practice is in a deprived area, so unsurprisingly the prevalence of diabetes is 34% higher than the UK average, and the prevalence of hypertension is 32% higher. In our patients with chronic disease the prevalence of smoking is 26% compared to a UK average of 22%. This next link shows how well we compare to other practices in the area in meeting the target of having BP readings of less than 145/85 in our patients with diabetes. Despite the higher than average burden of disease in our practice, we have managed this in 75% of our patients. However, this puts us only on the 25th centile for performance within the area. Through a process of internal peer review we try to figure out how we can improve our success in these targets. We are continually reviewing our recall systems for patients, and how we can share work within the practice team.
Dr. Ofri does not say that doctors should not be subject to performance measures just because they are good people. Instead she points out that the measures should be sensible, and that they should probably be applied at the level of the institution and take in to account wider systemic issues, for example availability of cheap medications. She has drawn attention to the complexity of such an apparently simple process.
Finally, I would ask you to watch this short video of Dr. Julian Tudor Hart, a doctor who inspires many in primary care with his research and work in South Wales. He has demonstrated what can be achieved when caring for underserved populations, and the mindset and caring attitude required.
EDIT: 24/8/10 I came across this YouTube "Can we tell physicians apart without better scorecards?" I find it interesting because it starts with talking about feedback from patients about empowerment. shared decision making etc. Next, the comment is made that if this could be done by email it would significantly reduce costs, however no-one has the email addresses of patients. Lastly, the point is made that although institutional/system measures 'should' be the way to address quality improvement in real life it doesn't work that way, and individual physicians seems to be a key determinant themselves. However, this fits with the notion of process measures- not the outcome measures described by Ofri. The debate continues!
