Skepteopathy- a reprise

Warning: this post is not a mature and reasoned contribution to intellectual discourse.  This is deliberately stirring the pot.  Sorry, I couldn’t help myself.

Anyway, take this quiz.  One of these quotes is genuine, from a genuine source.  The other isn’t.  Guess which is which.

I cannot tell you how many e-mails I have received over the years from people with all sorts of ailments and pains saying that they found relief, or just a better life, through our homeopathy service.  Some were initially skeptical, but we pulled them over.  We even get the occasional e-mail from “mainstream doctors” who claim that we pulled them over from the dark side.

from a homeopathy service?  Or,

I cannot tell you how many e-mails I have received over the years from people all along the skeptical spectrum saying that they found the movement, or their inner skeptic, through the SGU. Some were on the fence, but we pulled them over. We even get the occasional e-mail from “true-believers” who claim that we pulled them over from the dark side.

from a skeptical science site.

Ok, who answered ‘the second’?  You win a feeling of smug satisfaction.

The quote is from Dr Steve Novella, founder of the site ‘Science Based Medicine‘.  I’ve earlier commented on discussions between that mob and Healthcare etc.  Dr Novella clearly feels he and the other like-minded bloggers achieve a far greater public good than the mere personal gains of the discussion itself.  I clearly disagree.

And let me tell you if I could have made these comments on Dr Novella’s or the SBM site I would have.  Neither I nor IT-savvy mate have been able to register on numerous attempts and when I emailed the listed Help address I received a courteous reply that my query was forwarded to the webmaster.

And nothing since.

Anorak EBM

Awesome lecture from John Worrall, a philosophy of science dude at the London School of Economics.  He was influenced early on by lectures from Popper and then studied under Lakatos.  Give it a listen.

While listening I realised I need to be clear about my position regarding ‘evidence-based medicine’, not just snipe and harangue.  I advocate two main positions-

1. The hierarchy of study designs displayed in this blog’s header is invalid and its dissemination impedes the best practice of clinical medicine.  It should be done away with.

2. The term ‘evidence-based-medicine’ is a tautological and meaningless slogan; the main purpose of its use by those in the movement is to invoke a tone of scientific authority which the movement’s methods don’t warrant.  It should also be done away with.

Prof Worrall in particular questions whether the process of randomisation in itself should connote methodological supremacy to the RCT.  His comments about the movement’s objections to criticism, of ‘we never said that’ and ‘straw man, straw man’ resonate for me from what I’ve seen and read.  If the movement’s disciples chose to eschew the hierarchy and slogan described above from now on I’d be more convinced that they really have moved on from the dogmatism which did dominate the movement’s early history and which still influence some.

Rally for evidence

These are from the recent ‘Rally to restore sanity’ run by 2 comedians in the US.  I have no idea how the first placard fits in but it’s funny.  The second doesn’t really belong on this blog but is also funny.

Woo dares wins

A very ‘lively’ discussion is occurring between contributors of the blogging behemoth Science Based Medicine and Healthcare etc.  I’ve been unable to register on SBM and this comment was too long for Healthcare etc so this is the comment I would have posted-

This is my contribution, I hope you don’t find it too inflammatory.  I note that one of the more enlightened SBM cheerleaders on Dr Novella’s latest SBM post chose to demonstrate their non-arrogant non-paternalism by referring to Dr Zilberberg as a ‘dipsh*t’, so I should be on safe ground.

The SBM site won’t let me register, I’ve got an IT-savvy mate on the case but he’s had no luck, if I can crack it I’ll post this there too.

I had a bad feeling when this discussion began, when Dr Novella posted a response to Dr Zilberberg’s post.  The risk was that only a ‘meta-discussion’ would ensue ie. a discussion disputing what had been discussed, how it had been discussed and how that reflected on the character of those discussing.  This has been only partly true and kudos to Drs Novella and Zilberberg for their good faith in trying to understand each other.

I don’t agree however that hotheaded arguing is always so bad; if progress is to be made in learning from each other’s positions I think taking a Popperian approach is more important than whether discussion is civil or intemperate.  By which I mean trying to discuss concrete (falsifiable) points to arrive at a clear conclusion instead of quibbling over generalisations.  For example the statement which was discussed, ‘absence of evidence is not evidence of absence’ may apply to a specific concrete instance such as a particular study not finding a new anticonvulsant to be effective due to small numbers/bias.  The same statement may be used when discussing a different specific concrete instance such as 20 specific studies of homeopathy of which one shows benefit by p-value, with a different slant.  Specific discussions about either application might be informative.  I’m sorry but I’m not convinced that any abstraction beyond such specifics (short of a textbook on the philosophy of science) is informative, and I personally haven’t found the significance of this well-worn statement informatively addressed by either side in this discussion.  This extends to other ‘points’ of discussion; there seems to have been a lot of talk about logic and illogic about positions which are not matters of logic.  What I’m saying is it doesn’t matter how polite you are if you don’t stoop to discuss specifics.

Following from this I would bet my dear departed grandmother that if you saw Dr Novella, Orac, Dr Zilberberg or myself in clinical practice seeing real patients in the real world you would be hard pressed to say which was the arrogant paternalist and which the woo-collaborator.  I bet across the board the practices would be competent and compassionate with a barely distinguishable difference in approach in the minority of cases where the patient raised CAM as an issue.

I don’t now plan to participate in idea-sharing in this discussion, I’ll explain why not in due course.  I myself have an interest in individuals’ motives, I don’t know why, I just do.  What follows may offend some but as above that’s ok, if you would like to insult me please come visit my blog, I could use the company.

On this note let’s turn to the SBM website.  I was delighted when I discovered this gem, when I first came across Dr Attwood’s awesome series on the p-value fallacy and Bayesian analysis.  I made the site my homepage and visited daily.  I thought it would be a site discussing scientific debate and controversy within mainstream medicine; there’s plenty enough of that.  Maybe the contributors would discuss, as Dr Attwood did, specific instances where the EBM hierarchy of study designs may inhibit critical thinking, a hobby horse issue of mine.

Instead I quickly became bored.  Bored to tears.  As it turned out SBM was actually mainly an anti-woo site.  I’ve conducted an unscientific study of posts on SBM from September 1^st to yesterday and I found of 41 posts (not including site business and humour), 24 posts (59%) were primarily anti-woo, 6 (15%) had a significant anti-woo element and 11 (27%) were not about woo at all.  The non-woo posts were typically exceptional and very interesting, such as this and this.

To say I find the anti-woo posts boring is not to say that they’re unimportant or reflect poorly on their authors, it is just to say that I find them boring.  I have no interest in CAM as it seems to me that deep knowledge about it and its invidious advocates is of little real help in the clinical practice of medicine.  All this CAM talk doesn’t make one much of a better doctor.

Allow me to now go a bit arrogant and paternalistic. I know a bit about vaccines and vaccine refusal as well as whacky treatments for autism.  I note that there’s now a pediatrician contributor on SBM and that he has authored many posts about vaccination (although the majority of posts about vaccination are not by him).  For everyone else participating in this discussion I can safely say I’ve had more clinical experience dealing with vaccine-preventable disease and vaccine refusal in real patients than you’ve had hot breakfasts.  I’ve seen babies die of pertussis, children maimed by pneumococcal meningitis, and more varicella than you could poke a stick at.  (As an aside, the varicella and HPV vaccines are simply not as important as other vaccines and to imply otherwise is silly.)

What this clinical experience has taught me is the limited value or counter-productive effect of strong and emotive statements about CAM, and yes of arrogant paternalism.  I’ve learnt the hard way what is the right approach to difficult CAM families where the stake is a child’s welfare not just blog point-scoring, and that approach is humble and inquisitive.  I find these sorts of parents quite annoying but remind myself that the child is my patient, they are not.  The humble inquisitive approach leads to real improved outcomes for real children.  Bear with me here, I’m NOT accusing Dr Novella or anyone else of having the bedside manner of a paternalistic ass; I will return to this point.

Back to SBM.  It seems to me that the word ‘science’ unavoidably contains a value statement, whether that is an appeal to authority or a statement about effective methods of reasoning.  So it seems to me that those involved in a site with that name, which is actually primarily an anti-woo advocacy site, and not for example a site dedicated to hot controversies of the day within mainstream medicine, should not feel too victimised when their recruitment of the word ‘science’ within anti-woo discussions is considered by others to be largely an appeal to authority, and the site’s name largely a jab at CAM advocates (of whom I am no friend or ally).

All of which gets me thinking about motives.  The statements that I object to most strongly in this discussion have been appeals to ‘ethical’ behaviour, to not acquiescing in the evil of CAM.  When I hear ethics invoked in this way I reach for the lube.  I simply don’t believe that these strong anti-CAM opinions have done a utilitarian good for society in this regard, though all the routine workaday medical care delivered by compassionate doctors for centuries, of which the authors are part, has.  Here I am unscientifically generalising from my clinical experience as outlined above.  I believe that it is most likely that the SBM site has had no overall remedial effect on the harms from CAM and it is quite possible that it has had a negative effect through alienating half-interested observers and galvanising CAM loons. I reject the assertion that the SBM site achieves a consumer protection mission.  If you would like to shut me up you could point me to evidence of, for example, the increased vaccination rates attributable to these public opinion pieces or the imminent demise of the NNCAM.

This is not to criticise the act of blogging itself (I try not to be that hypocritical).  It provides all sorts of good and is a lot of fun.  In this discussion however I think it is fair to say that everyone’s real motives, including mine, have been the personal gains from the intellectual interest of the discussion itself and the personal gains of expressing one’s views to a sympathetic or antagonistic audience, not ensuring the best outcomes for our patients.  For this reason I find appeals to ethics in bad faith and, ironically, the most paternalistic and arrogant statements in the discussion.  If we all wanted to act as ethically as possible we would stop wasting our time with this ceaseless gasbagging and devote all our waking hours to even more mainstream medical research than we currently undertake, a choice which would indisputably provide a greater public good.  In the meantime everyone should just chill out.

I find this humerus #2

Damn interns look younger every year.

Here’s what we think we know today

Interesting piece describing complexities in interpretation of RCTs.  Anaesthetist dude in Ontario has developed an ‘Absolute Fragility Index’ to describe how vulnerable the finding of an RCT would be to change if just a few of its patient’s outcomes changed.  Smaller trials would of course be more vulnerable.  It would surely relate to the statistical power of the trial but I’m afraid I won’t do the maths for you.

The same guy looked at highly cited medical articles of the past and found that 16% were later contradicted and 16% later found to have exaggerated treatment effects.  These trials were small, demonstrating that small trials are not only vulnerable to type 2/false negative errors (no great surprise but worth describing).

Separately this reminds me of an intensivist who early in my training told me that ‘about 50% of the clinical decisions we routinely make now will be proved as wrong during your career, of which many will be proved as actually harmful’ (remembering that only some, not all, decisions can currently conceivably be validated by scientific trials).  Nothing I’ve seen since then has convinced me otherwise.

I think all studies should include the statistical power of the analysis, or if this proves a good alternative the AFI.

Some patients are created more equal than others

Very interesting post from Healthcare etc here

Good comment on the fact that trials can show a drug may (or may not) on average have an effect on a group of patients, but this may mean that they actually have a strong effect on some patients and no effect on others.  I couldn’t see a link to the original news item referred to so here it is.

False science

Great post from Beyond Anomie here

I like to tell anyone who’ll listen why it’s bad that most medical studies don’t report their statistical power, paired with the incorrect assumption that ‘absence of evidence of effect is evidence of absence of effect’.  As outlined here someone has gone and looked at a range of studies and confirmed how low statistical power typically is.

Hot off the press

This came out a couple of weeks ago from some of EBM/Cochrane Collaboration’s most inveterate foot soldiers.  I believe Glasziou’s in Australia now, he seems to have a good brain from what I’ve seen of his work.  They note a skyrocketing number of RCTs and reviews but don’t feel this is all a good thing-

• Although trials, reviews, and health technology assessments have undoubtedly had major impacts, the staple of medical literature synthesis remains the non-systematic narrative review. Only a small minority of trial reports are being analysed in up-to-date systematic reviews. Given the constraints, Archie Cochrane’s vision will not be achieved without some serious changes in course.
• To meet the needs of patients, clinicians, and policymakers, unnecessary trials need to be reduced, and systematic reviews need to be prioritised. Streamlining and innovation in methods of systematic reviewing are necessary to enable valid answers to be found for most patient questions. Finally, clinicians and patients require open access to these important resources.

Unsurprisingly, I agree with a lot of this comment from Richard Smith, former editor of BMJ-

1. Many people reading their article might reach the conclusion that Archie Cochrane’s dream is unachievable and ought perhaps be replaced by something less Utopian.

2. Have the authors noticed something of a swing away from evidence based practice? I have–despite being an enthusiast since the very beginning. Many people, including many doctors, most policy makers, and even some statisticians, have come to see it as too reductionist, unhelpful in answering the complex questions of a world dominated by people with polypathology.

3. Their tendency to be sceptical about the inclusion of non-randomised trials and other evidence in systematic reviews puts them at risk of becoming pure but irrelevant.

4. They need a more sophisticated prescription for achieving their goal. In their next article they should analyse the incentives, many of them perverse, that leads to our present circumstances and propose ways of changing the incentives.

I thought this was pretty mild but maybe it was more provocative than I realised.  This is the reply from lead author Bastian-

A less utopian version of Cochrane’s challenge is unnecessary: rather only a precise reading of it. We need to concentrate on trials that are “relevant” [1]. The information needs of clinicians and patients will not be well served by removing the floodgates and admitting non-randomised studies of questionable relevance, any more than they are being well served by the plethora of randomised studies of questionable relevance.

Randomised and non-randomised studies frequently give different answers to the same question, and choices are required based on logic. It is a choice whether or not to recommend that postmenopausal women should take hormones to prevent cardiovascular disease, but the choice has mortal consequences.

We suggest that practice should be informed by non-randomised evidence in three main situations: (i) when estimates of the effects of an intervention are so large that it is inconceivable that they reflect bias [2]; (ii) when trials are of insufficient size to address questions about important possible uncommon adverse effects [3]; and (iii) that it really is inconceivable that randomised trials would ever be feasible, for example, to address uncertainties about how to treat white phosphorous burns sustained during an intensive bombing campaign.

As illustrated by numerous examples, acquiescing in lack of randomised trials when controlled experiments are, in principle, feasible, has resulted in unacceptable suffering and death. Cochrane’s suggestion offers protection against error – but also a vital and increasingly important triage principle for coping with information. The current bias and waste in the medical research enterprise has multiple layers [4]. But understanding causes and cures of our ignorance despite a flood of information is still worth more attention and effort than it currently gets. Like Canute, we are well aware that we cannot stop the tide. Getting more selective, though, and finding new ways to cooperate, might help us stay afloat.

Geesh!  I mean could you get any more moralistic, righteous, smarmy and hysterical, all at the same time?  The language is telling, we ‘acquiesce’ in ‘suffering and death’ if we don’t accept the EBM Gospel, complicit in the ‘mortal consequences’ of choices not ‘based on [EBM] logic’.  This is the language of a social movement, not science.  And following this dogmatic version of EBM with such fundamentalist rigidity will more likely lead to suffering and death than the humble curiosity which characterises genuine scientific inquiry.

When I warn newcomers of the perils of this sort of EBM they often reply ‘but that’s not really EBM, that’s not how we’re taught that EBM is supposed to be used’.  It seems to me that while social structures have reason, as they do, to reward  this strange and harmful sort of pamphleteering, it will find a voice.

I find this humerus

Needless to say I was shown this by an ortho reg.