Number Needed to Educate (NNE)

As someone who teaches both general clinical dentistry and, increasingly, evidence-based dentistry to postgraduates and undergraduates, I have been interested in how we actually measure the effectiveness of our teaching. But I have recently become very interested in how we as clinicians are encouraged to move toward more evidence-based practice and how as postgraduate educators we facilitate that in the people who attend our courses.

There's a group within the Cochrane Collaboration called the Effective Practice and Organisation of Care Group (EPOC) and these people have produced a number of fascinating reviews about how effective (or ineffective) different interventions are in producing a change in professional practice of healthcare workers. Here's a biased sample that have interested me:

• Audit and feedback: effects on professional practice and patient outcomes

• Continuing education meetings and workshops for health professionals

• Educational outreach visits to change health care professional care for patients

• Effectiveness of the use of local opinion leaders to promote evidence-based practice and improving patient outcomes

• Tailored interventions to overcome identified barriers to change effects on professional practice and health care outcomes

Now, the meta-analyses that have been conducted in these reviews result in the authors giving a summary risk difference. This is the absolute difference in the risk of a professional changing behaviour when they are subjected to, say, a CPD course compared to the risk of them changing their behaviour if, say, they didn't attend the CPD course. Granted, it's a bit odd to describe the chance of converting to a positive behaviour as a risk but that's the way it is. 

So what's interesting to me is that in the systematic review on educational meetings above there was a risk difference of 6% overall of a positive change in practice if people attended educational meetings. But when they looked at studies that tried to change complex behaviour this came down to 0%.

Now I've been thinking that if we were to treat risk difference (which is absolute) like absolute risk reduction we'd be able to determine a 'Number Needed to Educate' (NNE) that would result in a single change in practice.

You may be familiar with NNT (number needed to treat), which I have blogged about previously. Basically it describes for interventions the number of patients we would need to treat with a given treatment before we'd expect one positive result. The lower the NNT the more effective the treatment. 

"17 people sit for a whole meeting about a best practice and yet only one of them changes their behaviour."

The NNT is determined by dividing 1 by the absolute risk reduction. So if the absolute risk reduction is 0.1 (or 10%) then we'd need to treat 1/0.1=10 patients to see a benefit.

So could we start to do the same with different ways of changing our behaviour to one that is more evidence-based? If educational meetings have a risk difference of 6% overall, then the NNE (Number Needed to Educate) would be 1/0.06=17 (rounded to nearest whole number). So, get this, 17 professionals would have to sit through an educational meeting for one of them to change their practice to what is considered best practice. That's astounding. 17 people sit for a whole meeting about a best practice and yet only one of them changes their behaviour.

This has totally changed the way I see the group of professionals who attend my EBD courses. I have a 1 in 17 chance if I simply stick to a single educational meeting of changing their behaviour or it could be worse, since the risk difference was 0 for complex behaviours (i.e. NNE=1/0=infinity - I'd have to educate all the world's dentists in EBD and still I wouldn't expect one to change their behaviour...)

I think it's time to change the way we think about what we aim to do as educators. Whether we're teaching periodontics, endodontics, restorative dentistry - whatever - we need to look at those EPOC reviews and think about how best we facilitate changes in our own and others' behaviour so that as a professional group our practice becomes more evidence-based. It may be that we have to think out of the box and to engage other practices such as finding 'opinion leaders', developing outreach programmes, and encouraging the use of audit whilst providing feedback. 

Anything to make the chances of success better than 1 in 17.

Steroid cover for patients on long term steroids

A student asked me recently how I actually use evidence in my clinical practice. I think this is a good question as there's no point in teaching what EBD is but not showing how to apply it.

In several of my blogs I raise queries that arise typically from discussions with students. These often alert me to gaps in my knowledge of the current best evidence - even if I can regurgitate something I was once taught / read that sounds plausible.

Well this afternoon I had a moment of uncertainty regarding what we were taught to call 'steroid cover'. That is, when I was an undergraduate I was taught that patients on long term steroids may require a supplementary dose around the time of a stressful surgical event. I don't actually know what is taught to the undergraduates today but there was certainly a gap in my 'up-to-date' knowledge that I needed to sort out.

The patient I saw this afternoon is a not a well lady and is on a fairly high dose of steroids so I thought I had better just check that I wasn't about to send her into adrenal insufficiency and shock as a result of me taking out a tooth.

So I went to the TRIP database and plugged in 'steroids dental treatment' and this lead me to this:

And clicking through led me to a TRIP answer here.

But further down the TRIP page was this:

 Now this looked helpful but was way out of date. If something is 8 years old there's a possibility some new evidence has come to light since. However, if one clicks through to Pubmed it often shows similar articles on the right hand side. So this is what I saw:

That link on the side was for a 2008 systematic review. Still not ideal time wise but 4 years younger...However, on the right of this page there was a link to a more recent Cochrane review:

You can click through to the Cochrane review here.

I generally prefer to read Cochrane reviews because they are on the whole better done and a lot stricter on the trials they allow into the review. I therefore feel more confident with their conclusions.

The first review above had included randomised controlled trials (RCTs) and cohort studies whereas the Cochrane review only allowed RCTs. Basically, though, they both concluded there wasn't much evidence to support the use of steroid cover. However, the Cochrane review is cautious because in the 2 RCTs it included there were only 37 patients and so the evidence is not strong. The non-Cochrane review had over 300 patients but a large proportion were patients followed up after not giving them steroid cover - none of whom developed an adrenal crisis.

So to answer the student's question about how I use EBD I have in about 5-10 minutes established that the evidence is weak on the use of supplementary corticosteroids and so rather than use a poorly-substantiated intervention (extra steroids) I will take the tooth out with her having taken just her normal steroid dose. 

Now maybe all you students are saying "yeah - that's what they taught us in the Human, Health and Disease course", but this illustrates that like me you will find yourself in the future unsure whether what you were taught as undergraduates is still relevant. Don't be ashamed to ask the questions and admit to up-to-date knowledge gaps.