Friday, 9 January 2015

Credentialing: shifting more costs onto the trainee


Shape of Training has many disastrous regressive elements, not least the introduction of 'credentialing'.  Note first of all that the GMC defines a credential as a 'formal accreditation of attainment of competences'.  Therefore the direction of travel pushed by the government via the GMC and Shape is yet more competency based methods in training.  As if we haven't seen the harms of an over reliance on this deeply flawed reductionist approach in terms of encouraging minimum standards and demotivating trainees....

Look at the extract in the Figure above, it demonstrates concisely just where the cost of credentialing is going to be dumped, it will be the Royal Colleges initially and then, yep you've guessed it, the College will then pass this extra cost onto the trainee.  As if medical training wasn't expensive enough already with student debts, GMC fees, College exams and fees, Training body fees (JRCPTB/ISCP etc), now the government is dumping yet another cost onto the trainee.  It's just as well we don't have multiple recruitment crises at the moment, oh wait a minute....

This excellent piece in the NEJM summarises the lack of benefit of this kind of training bureaucracy and the huge costs involved in maintaining it.  This article relates to 'meeting maintenance of certification (MOC) ' in the US but there are stark parallels with both revalidation and credentialing. 

Much like the Shape of Training's recommendations, there is zero credible high quality evidence to justify the introduction of 'credentialing'.  Arguably revalidation and credentialing are of little, if not zero, benefit to both doctors and patients, while they introduce extremely costly self serving bureaucracies which can also do a lot of harm in a number of ways.  There are many more potential harms to credentialing, including fact that it introduces more employer control over access to training and the quality of training.  The lack of any proven benefit combined with the fact that it appears designed to dump yet more costs onto the trainee make it something that should be firmly rejected by the profession and any reasonable doctor.

Thursday, 8 January 2015

Doctors - email your trainee group NOW!


The BMA have produced an excellent statement on the 'Shape of Training' review and a large number of trainee groups have rightly signed up to reject the review in its current dangerous form.  The groups which have signed up are:

Association of British Neurologists Trainees
Association of Clinical Pathologists trainees group
Association of Otolaryngologists in Training
Association of Palliative Medicine trainees group
Association of Surgeons in Training
National Dermatology Trainee Committee
British Junior Cardiologists Association
British Medical Association Junior Doctors Committee
British Orthopaedic Trainees Association
British Society of Gastroenterology Trainees' Committee
Oncology Registrar Forum
Royal College of Obstetricians and Gynaecologists Trainee Committee
Royal College of Ophthalmologists Ophthalmic Trainees Group
Royal College of Radiologists Junior Radiologists Forum
Young Diabetologists' and Endocrinologists' Forum


There are a number of notable groups who have not signed up despite a total lack of interest in consulting their members on this potentially destructive review.  I have collated a list of email addresses for those of you who belong to the following groups, please email them and ask them why on earth they have no supported the BMA's statement:

Medics (Royal College Physicians) Trainees.Committee@rcplondon.ac.uk
RCSEd traineescommittee@rcsed.ac.uk
Emergency medicine EMTA@collemergencymed.ac.uk
Pharmacuetical medicine TraineeSubcommittee@fpm.org.uk
Anaesthetics trainee@rcoa.ac.uk
GPs aitcommittee@rcgp.org.uk
Paediatrics and Child Health training.enquiries@rcph.ac.uk
Psychiatrists ptc@rcpsych.ac.uk

Massive questions remain unanswered on the real motives of Shape......


The problems with the Shape of Training review of postgraduate medical training become more and more obvious by the day.  Today the BMA has released a joint statement on Shape calling for the powers that be to reject the review in its current form:

"We therefore reject the current direction of travel for Shape of Training"

The bottom line remains that the public have never been adequately informed or engaged in the review, and they should have been.  The most important thing is maintaining high quality patient care, for a number of reasons Shape threatens to undermine quality in the name of creating a compliant and flexible workforce.  This appears fundamentally driven by the interests of employers and not the public.

The evidence review was flawed, the review's recommendations are regressive and potentially extremely dangerous from a patient safety perspective, trainees have not been involved, in fact the profession has not been adequately consulted or involved.  The GMC refused my Freedom of Information request pertaining to the documentation of secret meetings involving Ministers and the review's Chair, as did the Information Commissioner.  I have been fortunate enough to recently be vindicated in court and the GMC were therefore ordered to release the withheld documents to me. 

The sad thing in all this is that who is looking out for transparency and the public interests?  Just who is everyone serving in all this?  My personal battle is simply to ensure that all the information reaches doctors and the public, they both deserve to be fully informed of the direction of travel that is being proposed and those running the review should listen to their opinion, not steam roller these potentially dangerous recommendations through against the public interest.

Wednesday, 7 January 2015

Solidarity - Charlie Hebdo


May those who were murdered in cold blood today in Paris rest in peace.

I hope some good can result from today's appalling crimes, in that decent rational people can unite against the misguided nonsense that motivates the misguided fools who commit acts such as this in the name of their make believe 'Gods'.

Tuesday, 6 January 2015

The 'Dean review' of medical training



I have commissioned myself to review UK postgraduate medical training.  The pre-review steps are complete, I am indeed a fit and proper person, I have 5 GCSEs, I have no conflicts of interest that I am aware of, plus I am genuinely independent of government.  Frankly who could compete with that?  The terms of reference have been decided by myself, these being to 'review medical training in any way that I see fit', excellent, now let us begin.

The first step of the review is complete, this being the evidence review.  I conducted a systematic review using the latest 2015 PRISMA guidelines.  The search terms were 'UK medical training'.  I excluded all studies that did not exactly predict how future patients would need to be treated, hence this search yielded zero studies.  I have therefore concluded that there is an inadequate evidence base upon which to determine any exact strategies for medical training reform.  This means that I shall be 'first, doing significant harm' by ensuring that my plans will be major structural reforms that will tear up everything we have in place and start from a lovely blank slate.

Now for the second step, the consultation process.  This is now complete, I have tweeted for input and there were zero responses.  I have asked the cat, who did not respond with words, I have also asked my wife who is currently watching Eastenders and not interested.  Therefore my own personal opinion shall drive the review's direction, excellent work.

Now for the final 'Dean report', this is now complete.  I have deliberately kept things simple, firstly the principle of the review is 'to create a world class training system which catalyses commissioning based patient safety augmentation collaborative excellence brilliant wonderfulness'.  There is no way anyone could disagree with that principle, but perhaps I need to mention pathways and flows?  There's no time for consideration or reflection, let's stick with that simple principle.

Finally the recommendations, these have yet to be decided and really are they that important?  Surely like in MMC and the Shape of Training it is accepting the principles that is key?  If one can accept the noble principles then surely any recommendations following will be good news for everyone, by everyone I mean myself that is.  Anyway I think that's sensible, let's leave out the recommendations, I'll simply decide those behind closed doors once you have accepted the principles.   You can trust me on this, look at the review's methodology thus far, it's rigorous beyond the Cochrane collaborative.  I may or may not have had secret meetings with those secret important people who commissioned the 'Dean review' in the first place.

Great, now that's all sorted, training's going to be so much better, I am also particularly looking forward to my knighthood, Sir Dean, that really has a good ring to it.......

Monday, 5 January 2015

Inaccuracy and Obfuscation - the facts thus far

 
I do not have all the answers, as I do not have all the facts and I do not have the knowledge of a legal expert.  However this piece is simply designed to summarise some of the key facts relating to the Meirion Thomas saga, so that you can use this to make up your own minds, and realistically we will all need more information before we can do this with any degree of certainty.

Firstly I wrote some recent pieces criticising the coverage of events by the Daily Mail and the Spectator.  The chief reason for criticising their coverage was that they told only one side of the story and ignored some key aspects of it.  Both the Mail and the Spectator ignored that Mr Meirion Thomas’ arguments contained factual inaccuracies, were not solidly based on evidence and were widely dismantled by more reasoned objective work elsewhere by the likes of the RCS and Margaret McCartney.  It was also strange not to mention the legal element to this affair when they were overtly criticising the Royal Marsden’s actions.  Therefore my chief arguments were not about the rights and wrongs of Mr Meirion Thomas’s or the Marsden’s actions, it was with the biased and stilted coverage by the Mail and the Spectator.
Secondly to my limited understanding of the law in this area, it is worth remembering throughout that I am not giving an opinion about whether I think the law is morally right, or whether anyone’s actions are morally right or wrong, I am just describing the legal aspects of this case according to the limited facts we have available.  It is worth remembering that we do not know the exact terms of Mr Meirion Thomas’s contract, we do not know what has gone on behind closed doors in discussions between him and his employer, and we do not know what the exact outcome of these discussions have been.  Let us just note what the Royal Marsden have stated:

“Following two articles that appeared in the Daily Mail on Thursday 20 November and in the Sunday Mirror on Sunday 23 November by Professor Meirion Thomas, The Royal Marsden NHS Foundation Trust would like to make it clear that Professor Thomas’ views are entirely his own and in no way reflect the opinion of those working at The Royal Marsden.

“Professor Thomas has expressed his personal views about GPs and health tourism in the Daily Mail and Sunday Mirror respectively without consulting staff at The Royal Marsden. We do not share, condone or support the views he has expressed. It is important to state that all individuals are entitled to express a view. They are not however entitled to speak on behalf of an organisation without that organisation’s consent. Professor Thomas did not seek or receive this organisation’s consent to either article.

“The Royal Marsden is very proud of its good working relationships with our GP colleagues, particularly through our GP engagement programme and referral process, and we value their experience, knowledge and the excellent contribution they make within the NHS.”

The difficulty in interpreting any of this is that we are not privy to the full information; hence some speculation is simply not sensible or wise.  It is worth remembering the facts about what was written in that the Mail article named the author as a ‘Royal Marsden consultant surgeon’ and it was never stated this articles were his personal view.  Some would argue that it should be assumed that the articles are his personal views, there is a great difficulty with this argument as he does mention his specific employer and job title, he talks about work relating to this specific employer (Lansley opinion) and he may have used a photograph on Trust premises, and he does this without stating that the articles are a personal view.  I will freely admit this is not black and white, but to suggest that it is clear these articles are his personal view is absurd and in denial of the facts.  The facts significantly close down the distance between Mr Meirion Thomas’ personal opinion and his employer, the Royal Marsden, and from the Marsden’s statement it appears that this is a significant factor in their actions thus far.  It is therefore likely that Mr Meirion Thomas' articles breached Trust media policy on more than one occasion and the terms of his contract with the Trust.
The legal arguments about reputational harm are likely relevant.  If employees have been judged to have caused significant harm to their employer’s reputation by their actions then this would be another avenue by which the Royal Marsden may choose to act.  There is certainly a reasonable case for arguing that Mr Meirion Thomas’ articles causes reputational harm given their disparaging nature.  Certainly Mr Meirion Thomas does not meet the legal definition of a whisteblower and would not have a credible case if he used a public interest defence to justify his articles.  However from what has been stated it appears likely that the Marsden felt Mr Meirion Thomas had breached his contract of employment by ‘speaking on behalf of an organisation without that organisation’s consent’.  It remains unknown which exact details in the articles led the Marsden to this opinion regarding speaking ‘on behalf’ of an organisation. We also do not know precisely what action has been taken by the Royal Marsden, the talk of confidentiality or ‘gagging’ clauses may or may not be true.
In purely legal terms given the available information it appears that the Royal Marsden have acted properly and to suggest otherwise would appear to be rather evidence free.  It is also worth noting that I have not yet given my opinion on whether I think the Marsden’s actions are morally right or the most effective course of action, this has not been my motive, I am simply trying to give my readers as much objective information as possible with which to make up their own minds.  Perhaps I shall do this at some point, but at the moment I do not want to confuse these simple arguments, I want more information before coming to any definitive moral judgements on this complex affair.  If people have an issue with the Marsden’s actions then they should either point to specific legal arguments or direct their arguments towards the legal system, and not the Royal Marsden.

Sunday, 4 January 2015

Suspicious omissions from the Mail and Spectator


It is interesting that the Spectator and the Daily Mail have both sided in rather stilted fashion with Mr Meirion Thomas, the surgeon who wrote articles for both of these publications, strange no other publication has joined in the rose tinted defence, I cannot imagine why, I really can't.  Anyway sticking to the basic facts, the Spectator and Daily Mail both have experienced Editorial teams.  Yet despite this in the Mail Mr Meirion Thomas was described as 'of the Royal Marsden Hospital in  London' and in the Spectator 'a consultant surgeon with the NHS'.  Interestingly in one of the Mail pieces a photo of Mr Meirion Thomas was used and it looks suspiciously like he was on NHS property at the time.  The following chunk has been taken from one Mail article demonstrated that Mr Meirion Thomas also clearly wrote about matters relating to his NHS role:

"Because I am a senior consultant in an NHS hospital, Mr Lansley sent me an early outline of his plan for reorganisation and asked for my advice.
I told him: ‘GPs are the problem in the health service, not the solution.’
He did not ask my advice again, but instead pressed ahead with his misguided scheme."

What is the importance of these details you ask?  Well they are vital when it comes to employment law.  Here is a chunk of media policy from a standard NHS Trust:

"If you are approached to speak to the media in another capacity (i.e. for a professional body, trades union or charity) you may do so. However, if this occurs in a way that will link you to the Trust, e.g. using the Trust address, mentioning your role at the Trust, or filming in your department for a back-drop, we would ask that you contact the communications office so that the Trust is aware of the media interest."

Notably Mr Meirion Thomas was not writing on behalf of another body such as a union, he was writing as an individual, however this was never made clear by either the Mail or the Spectator, a fact I find rather strange given their editorial experience.  In fact the Spectator went as far as saying in their recent one sided defence 'he insisted on being described simply as an NHS surgeon', this is suggestive that the Spectator may well have advised him of the negative implications in naming the NHS, I also wonder what he was advised by the Mail.

There is a lot of case law relating to employee being disciplined for speaking out against their employers.  However there are certain situations when it can be justified.  For example it is fine if you are speaking for a professional body or union, Mr Meirion Thomas was not.  It is also wise to make it clear one is speaking as an individual and not name your employer, Mr Meirion Thomas did not do this.  It is also unwise to use a photograph taken on your employer's premises, Mr Meirion Thomas may have been unwise on this.  If one is a whistleblower one can justify speaking one, sadly Mr Meirion Thomas comes a country mile short of meeting any legal definition of a whistleblower.

When one effectively speaks under an employer's umbrella and risks harm to that employer's reputation by making ill informed and disparaging comments, then one has a recipe for being on extremely dubious legal ground.  In this situation one's employer has a very strong case in bring disciplinary action against the employee, just as appears the case with Mr Meirion Thomas. 

In summary the fact that the Mail and Spectator have totally ignored the key issue of employment law is highly suspicious, they are no idiots when it comes to legality, meaning the likely explanation involves Mr Meirion Thomas being a victim of media exploitation.  Note that a story carries far more weight as 'surgeon X from hospital X' or an 'NHS' surgeon than a 'surgeon writing as an individual without his employer's approval'.  The Mail and Spectator had much to gain, Mr Meirion Thomas had much to lose, I wonder.

Overall the talk of 'gagging' and 'free speech' by the likes of the Mail and Spectator is both disingenuous and highly obfuscatory.  It appears highly likely that the Royal Marsden have acted entirely properly and have simply acted to protect their reputation by disciplining an individual who threatened to undermine that reputation by disparaging his medical colleagues.  The talk of abuse, GMC referral and the rest is all a neat distraction tactic, there are idiots on both sides throwing around needless abusive insults.  The bottom line here is that employment law is the key and it dictates that employers have every right to act to protect their reputations in cases like this.

Saturday, 3 January 2015

Guy Adams of the Mail responds


This is how the noble journalist Guy Adams responded when I made a perfectly reasonable comment on his ill informed piece on Mr Meirion Thomas.  I would like to thank him for his kind response and also for blocking me, as in doing so he has made my point perfectly for me.

There is one rule for the Mail and one rule for those of us who deign to politely criticise the Mail.  The Mail's criticism is all reasonable and justified by definition alone, however anyone criticising the Mail is written off as a 'troll' engaging in 'ad hominem abuse'.  Nice.  Anyone paying money to read this paper should really think twice, can you trust people who behave like this?

The total ignorance and hypocrisy of the Dail Mail



The Daily Mail has a long history of supporting some rather malevolent causes.  Today their journalistic standards continue within the gutter of hypocritical bile that the seem to fondly dwell within (note this link will not give the Daily Mail any hits if you are concerned about this).

The Mail's story today presents one stilted side of a story, and uses quotes taken without permission and totally out of context from Twitter and a private members forum.  I am not sure how the Mail accessed the comments on this private forum, certainly someone has breached the website's terms and conditions.  There are a number of other problems with the Mail's article and I shall attempt to summarise them below.

Firstly like the Spectator piece defending Mr Meirion Thomas, there is no mention of employment law.  It is interesting to consider that the Mail editorial team allowed Mr Meirion Thomas to be described as working for the 'Royal Marsden', making it clear he was employed by a particular NHS Trust.  It is standard NHS media policy for any doctor who writes as such a Trust employee should run the articles past the Trust media team, it seems clear that Mr Meirion Thomas didn't do this, therefore playing him in a very dubious place in terms of his contract of employment and the law.

Secondly the Mail makes no mention the the baseless evidence free slurs that Mr Meirion Thomas launched at female doctors and General Practitioners in recent articles.  The Mail simply describes his article as a 'comment article criticising...', if the Mail is going to use this angle then many of the quotes taken without permission from Twitter and Doctors.net should simply be called 'social media comments criticising' not 'slurs' and 'ad hominem attacks'.  

Not only does the Mail lazily apply its dubious form of logic in a totally selective fashion, but it then deigns to read into what a random bunch of social media posts mean as an 'overall message', truly bizarre given that the Mail has just cherry picked the most juicy and ignored the bulk of reasonable opinion that exposed Meirion Thomas' articles as evidence free nonsense.  

Certainly I do not defend all the criticism of Mr Meirion Thomas, some of it has been below the belt, however it is also worth remembering that much in his flawed articles was below the belt.  The Mail's utter hypocrisy is staggering, one one hand criticism of Mr Meirion Thomas' writing is lazily labelled 'aggressive' and 'ad hominem', while the evidence free slurs within the original articles is totally ignored and erroneously spun as justified 'criticism'.  Essentially the Mail has trawled the social media and private members fora, and the worst 'ad hominem abuse' it can find is the word 't***', that says it all really.  

In summary the Mail has done a very good job in humiliating itself with the woeful journalism contained within the aforementioned piece and I would like to congratulate them for this spectacular own goal.  I have not abused Mr Meirion Thomas myself, I simply think his evidence free smears against various professionals were unhelpful, harmful and highly ignorant.  I disagree with the referral to the GMC but those who did refer him had a right given that GMC guidance did appear to be breached.  However it is clear that Mr Meirion Thomas has put himself onto some very dodgy ground in terms of employment law, I can't imagine why both the Mail and Spectator have totally ignored this key issue.  Maybe the Mail will now selectively quote my blog or tweets, then accuse me of 'abusing' people, who knows, what I do know is that I will not be silenced by the Mail's biased hypocritical gutter journalism.  

Thursday, 1 January 2015

'Prof' Meirion Thomas and the 'Spectator'


The 'Prof' Meirion Thomas saga is complex and I shall not attempt to address all the details in this piece.  For more detail simply use 'google' and read away.  I am no fan of Mr Meirion Thomas, he is in fact no longer entitled to use his honorary 'Professor' title as it lapsed back in 2012.  He has written several rather ill informed and ignorant pieces around the NHS in respected sources such as the Daily Mail and Spectator, there is no hint of sarcasm in my tone I may righteously add.

Whatever one thinks of his words, he is entitled to his opinion, no matter how misogynistic, sexist, ill informed, ignorant, hurtful or just plain wrong that opinion is.  The Spectator ignorantly claims that no one has 'tackled' his comments, a total distortion of the reality which has seen his words totally dismantled by objective evidence based opinion throughout the media.  The Spectator's attempt to obfuscate and hide from the truth are embarrassing, labelling Meirion Thomas a 'whistleblower' is a total abuse of the word.  He is entitled to free speech like all of us, unless he crosses the line of the law.  It is worth bearing in mind that it is not just criminal law that Mr Meirion Thomas has to respect, but he is an employee of the NHS and as an NHS employee he has to respect the law of his contract of employment.  He certainly hasn't been criminal with his ill informed opinion, but has he ignored the terms of his employment contract?  Of critical importance is the clear fact that the Spectator claims that:

"he insisted on being described simply as an NHS surgeon"


This is pivotal.  Firstly it is important to realise that this is what the Spectator claims, I do not know whether the Spectator is being strictly honest with its words.  However it remains abundantly clear that Mr Meirion Thomas effectively used the description of 'NHS surgeon' and it was not made abundantly clear, as any competent Editorial team should well appreciate, that he was speaking as an individual and not for his employer, the NHS.  Therefore Mr Meirion Thomas has ended up on very tricky legal ground with his employer, the Royal Marsden.  This is something the rather biased and stilted Spectator piece completely fails to get across, there is a thing called employment law and although there is a thing called 'free speech', this is not unbounded and it most certainly does not allow one to appear to speak for one's employer in the media, effectively what Mr Meirion Thomas has ended up doing.

So in summary don't believe the nonsense written Mr Meirion Thomas and the dubious justifications of his ignorant scaremongering in the Spectator.  Free speech is not absolute, Mr Meirion Thomas is an NHS employee and as such must respect his contract of employment.  It is is clear that for whatever reason, whether down to Mr Meirion Thomas' ignorance or the Spectator's desire to exploit his NHS reputation for page hits, or perhaps a combination of the two, it was not made clear to readers that Mr Meirion Thomas's writing was merely the opinion of an individual.  I doubt we shall ever hear the truth behind the Spectator's editorial fiddling and what the Royal Marsden has actually stipulated about any future articles by Mr Meirion Thomas.  The Spectator claims that he may not publish again:

"at least not without submitting the text to the hospital’s management for approval"

Without knowing the full details it is impossible to justifiably criticise the Marsden's stance on this.  They are not necessarily seeking to 'shut him up' as the Spectator likely unfairly states, they may very well be simply ensuring that he accurately states he is writing as an individual in the future, so that readers are not knowingly misled into thinking his ignorant ill informed bile may have any kind of NHS seal of approval.

Monday, 15 December 2014

Guest article - the real motives behind Shape?

I’d be careful not to go off on the tangents about the sadistic medical narcissists of the 1970s or to blame “The Trendy Educationalists” for these policies. This is driven, top-down, by the government, hence the involvement of lay folk (plus a few of the usual suspect Quisling collaborators from the ranks of the medical politicians). It is driven by the interests of the corporates, who have the politicians in their back pocket. We all dislike the bullying tyrannical consultants of the 1970s & the patient-averse oafs pushing Mickey Mouse reflective learning, but they are not the enemy here.

The aim is to have a very large pool of medics, who have no job security, self-confidence or professional self-respect. They needed to smash the old UK medical firm model to create US/EU-style 'attendings'. The old UK consultant grade had its faults & was by no means perfect. It was suited to the days of “generalism” in the truest (i.e. non-corrupt) sense of the term. Bottle necks & folk, lying in wait for uber-specialised tertiary centre posts, meant that, perversely, it took longer for folks to become sub-specialists, doing a very narrow range of repetitive tasks, than it took people to become generalist consultants. The consultant model was better suited to medicine than some other specialities, it did a lot for my speciality (anaesthetics), raising standards big time. But, it has had its day. The one size fits all MMC approach confused public health training with clinical training. You can randomly allocate a mixed bunch of medics & non-medics to random sequences of QUANGO placements & give them a chit saying they are trained PH specialists after 3 years, you cannot do the same with surgeons. Yet, this is precisely what is being proposed.

The stuff about generalists is despicable. They have realised that subspeciality–ologists don’t do acute medicine & they reckon that scrapping current training pathways to create a pool of accredited generalists will make it easier to staff the few remaining major acutes that will be left when cold sites are sold to private companies, who’ll run them to rake it in from elective surgery. A few doctors will get work in these privatised units, the rest will just end up doing shifts in the major acutes. This is about herding doctors to do skivvy work instead of the work we aspire to.

Expect the medical establishment to make it as difficult as possible for generalists (and existing consultants) to acquire the essential credentials to do any meaningful elective surgery/-ology. Such restrictive practices will suit the corporate subcontractors, who’ll have no problem recruiting medics, with huge student debts & mortgages, who’ll be desperate for scraps of work. Doctors are like scouts, they love acquiring silly badges, without realising how tacky it looks & how their egos are being massaged. It is a con trick. Credentials are barriers, limiting people in what they can do. Fools think they'll do an online course & get the leadership credential, enabling them to lord it over others. Dream on! Things like Breast Surgery will get credentialed, which means that generalist cst-holders wont be able to do it, so they'll have to continue doing shifts at the local major acute, doing back-to-back abscesses & laparotomies. The "Sell" is that this system will enable you to show what you can do. In reality, it will prohibit what you can do & you'll be at the mercy of employers, who'll offer you jobs doing stuff you'd rather not do (shifts in acute surgery) rather than training for the stuff you want to do (major cancer work, bariatrics etc). Create cst oversupply & then limit what cst-holders can do [most elective work]. Suddenly, acute surgery gets a whole lot cheaper!

Remember, the DH know that doctors will always happily kick the ladder away from their younger colleagues. They figure that we'll figure that a new SR grade would be in our interests. It is classic divide & rule. Naturally, the guys doing their PhD in the cell biology of Ca Oesophagus will figure that they will effortlessly become credentialed in upper GI surgery, whilst their younger colleagues will simply become shift-working acute surgeons, doing the "low level" stuff. Please lets not fall for this. They are not redesigning medical careers to cater for some indescribably brilliant doctors, who are head & shoulders above their peers. They are doing this to undermine the profession & make it easier to errode our pay & T&Cs to suit their corporate cronies, who'll end up hiring (and firing) doctors. Wake up!

We need to oppose this & that means jettisoning the traitors in the medical establishment, who’ll try to stifle proper resistance.

Wednesday, 10 December 2014

The Shape of Training - first do no harm Prof Reid?

 
I have been doing a fair bit of thinking about medical training reform in recent days, things appear to steaming ahead at full speed despite a rather large number of valid concerns that remain totally unaddressed by those pushing Shape through.  I stumbled upon an interview with HEE chief Wendy Reid and it is worth reading.

I think this brief interview succinctly demonstrates a number of crucial problems with those railroading through the review.  Firstly there is a lot of talk of doing good things:

“We’re really pushing forwards saying, ‘Here are explicit ways in which you can become this,’ and that’s where I think we will make a big difference.”

The meaning of such statements is debatable, it is actions that matter as cynics like myself have come to realise after years of seeing harmful reforms being packaged in a sheep's clothing.  Wendy Reid says we need major structural reform to meet the expectation of patients:

Reid says the structure of doctors’ training, which has remained the same for decades, needs to change to keep up with patients’ expectations. “We have doctors training in a system that is pretty much the same as it was in the 1980s when I trained”

This is a total straw man argument.  It is strange to say that our training system has not been changed since the 1980s, perhaps Wendy failed to notice the structural reform of MMC, I do not know.  Anyhow the argument that we need major structural change because we have not had it for a while is incoherent and sloppy.  In medicine 'first do no harm' is a fantastic motto by which to practice which serves one well in many different aspects of life.

As I have summarised in more detail here, the case for major structural reform has not been made by the review and no solid evidence is put forward to justify this needless change for the sake of change.   The reason for 'first do no harm' is that change is expensive, time consuming, disruptive and results in many unintended negative consequences.  The straw man has been exposed, so what are the real motives behind this major structural change Wendy?

Wendy Reid tries to answer "What is the future doctor going to be doing?".  I would suggest that if Shape is implemented then the future will see many quit medicine and emigrate to work in health care systems in which they can be trained to the level of a proper consultant.  Also how does dumbing down the consultant grade meet the expectation of patients?  Do patients want a lower quality service delivered by sub consultants?  Do they want a system of care delivery that traps a large cohort in the sub consultant grade and forces much top talent abroad and out of medicine?  These are the questions I would like Wendy Reid to answer.





Monday, 8 December 2014

Contact your MP to protest against cuts to medical training........


The Shape of Training is a review of doctor training that is currently being forced through despite many significant safety concerns being expressed by the majority of doctors' professional bodies.  If you want your doctors to be trained properly then it is essential that you take the time to write to your MP to express your opinion on this matter, below is a template letter you may use:



"Dear <INSERT HERE>

I am writing regarding the ‘Shape of Training’ review of doctor training that recently reported and is currently in the process of implementation.  I have grave concerns about several specific recommendations made by the review. 

Firstly the recommendation to shorten the training time of hospital consultants and will create a ‘sub consultant’ grade.  Such a move is incompatible with maintaining high quality patient care and would have a significant negative impact on patient safety.  Secondly moving the point at which doctors register fully with the GMC to medical school graduation is unsafe.  

In addition to these significant concerns surrounding both short and long term harms relating to both doctors and patients, the review has ignored the majority of respondents to its consultation who felt the training of hospital consultants should not be shortened:

most individuals and organisations argued that generalists would require a longer training period or reconstruction of training to capture the breadth of experiences needed to provide competent general care”

As things stand the Shape of Training review’s implementation is incompatible with maintaining high standards in medical education and patient care and as a result I would be very grateful if you could look into what can be done to halt this review's dangerous implementation,


Yours sincerely"

Write to your Royal College or Trainee Organisation


Here is a letter template that I have crafted for anyone to send to their Royal College or trainee organisation.  If you agree that Shape takes medical training in the wrong direction, please take the time to do this, we must hold these organisations to account and tell them what we think. Every little really does help.

"Dear <INSERT HERE>

I am writing regarding the ‘Shape of Training’ review that recently reported and is currently in the process of implementation.  I have grave concerns about several specific recommendations made by the review. 

Firstly the recommendation to shorten training time before CST that will inevitably lead to the creation of a ‘sub consultant’ grade.  Such a move is incompatible with maintaining high quality patient care and would have a significant negative impact on patient safety.  Secondly moving full GMC registration to medical school graduation is unsafe.  Thirdly the introduction of post-CST ‘credentialing’ is not based on any evidence base and it would lead to serious problems with workforce planning as local employers would be left in control.  Fourthly the combination of a ‘sub consultant’ grade with post CST credentialing controlled by employers would lead to significant staffing issues as a result of its negative impact on staff retention and recruitment.  It will become harder for medicine to attract high quality applicants when a large number of post CST ‘sub consultants’ become trapped in a non-training service delivering graveyard, while currently the recruitment environment is worryingly brittle in many areas.

In addition to these significant concerns surrounding both short and long term harms relating to both doctors and patients, the review has ignored the majority of respondents to its consultation who felt training should not be shortened:

“most individuals and organisations argued that generalists would require a longer training period or reconstruction of training to capture the breadth of experiences needed to provide competent general care”

As things stand the Shape of Training review’s implementation is incompatible with maintaining high standards in medical education and patient care and as a result I urge you to openly reject the review’s implementation.  The failure to withdraw from such a failed process will only be used to by those forcing it through as a signal of tacit approval and this must be avoided at all costs,


Yours sincerely"

Friday, 5 December 2014

The Shape of Training - the Key Issues


I am sure many of you are aware of the small details of the Shape of Training (ShoT) review, however many of you probably are not, therefore I though I would write a brief piece to summarise the key issues in training, how the review has gone about their business and the many issues that have been left completely unanswered.

Firstly medical training was subject to huge structural reform less than ten years ago in the form of Modernising Medical Careers (MMC).  It is worth remembering that history proved MMC was a covert and dishonest attempt by the Department of Health to force through a 'subconsultant grade' through the back door.  This was never admitted before or during MMC, I obtained documents after the event proving that the DH and senior medical leaders had misled doctors and patients by claiming the review was just about 'better training', it clearly was not, it was dressed up as something that it overtly was not.  This context of government dishonesty is hugely important.

The ShoT review was led by a Professor of economics with no medical experience and supported by the GMC, it gathered 'evidence' over 2013 and released its final report at the end of 2013.  It is currently in the process of being implemented by Health Education England and others including the GMC, of note not a single trainee is involved in implementing this reform, ironic that.  Now let us consider the review from a series of angles that any sensible individual would see as vital, let us examine the need and rationale, the evidence, the consultation, the review's recommendations, the views of professional bodies and the future.

The rationale and evidence for major structural change. Firstly it is necessary to move forward and back, the review claimed from the start that it would not necessarily lead to major structural changes, but this is precisely what the review has recommended, major structural changes, and very soon after MMC.  It is clear from the evidence reviewed by the ShoT team that there was no solid evidence to justify major structural change, it is this simple.  Not only that but the evidence gathering appeared biased and the opinions on the evidence predetermined. The argument from some Shape proponents that 'reform is needed' is bogus, reform is always needed as reform is improvement, but this does not justify the negative destructive proposals of Shape.  The current system needs gentle tweaking in some areas, minimal change in others, what was not needed was blanket reform of a major structural nature.  Verdict: inadequate evidence to justify major structural change.

The consultation.  The consultation was poor in many ways.  Firstly the consultation was extremely biased and leading in the way in which questions were asked to respondents.  Shape deliberately tried to push people into certain boxes with their answers on the future of training, in a way that appeared they wanted a certain answer even before the evidence had been gathered.  The consultation was also extremely poor in terms of the total number of responses, less than 200 responses were gathered from doctors.  Interestingly the majority of opinion was also ignored on key issues such as time to train:

“most individuals and organisations argued that generalists would require a longer training period or reconstruction of training to capture the breadth of experiences needed to provide competent general care”

Verdict: poor quality consultation which ignored majority of opinion on key issues.

The review's key recommendations.  Much like MMC many of the vague comments of the review appear reasonable at first glance, more 'flexibility', 'quality training' and 'generalism'.  Sadly when one takes the time to analyse the practical reality of the review's key recommendations then not only are some of the ideas extremely vague, but some of the clear points are quite frankly unsafe for patients.  The two most reckless and harmful recommendations are to reduce training time for hospital specialities and to move full GMC registration to the end of medical school.  It is of key importance that these two dangerous ideas are not justified or even explained in any sufficient detail within the review's text.  It is also key that there was significant majority of opinion against shortening training time.  The reality of shortening training time is a less skilled consultant grade, this is to all intended purposes a 'sub-consultant grade', and something that many professional bodies have argued strongly against, based on some very strong evidence I should add

The review does not justify why it is necessary to dumb down the consultant grade, and it introduces the highly bureaucratic idea of post CST (completion of specialty training) 'credentialing'.  Hence 'sub-consultants' may well not be able to do some things unsupervised as they are not proper consultants as the public would understand, but they would also not be guaranteed training to the proper consultant level, this would be left up to individual employers via the 'credentialing' pathway.  Another problem with this development of responsibility of training to employers is the lack of ability to plan the shape of the workforce in the long term.  The RCP/RCS/BMA have all spoken eloquently about the dangers of these key recommendations.  The review also fails to address the really key problems in training today such as reducing experience levelsVerdict: valid patient safety concerns from several professional bodies regarding key recommendations remain unaddressed and key recommendations inadequately justified. 

The future.  The one positive thing about the review is that it has not yet been implemented, it is currently in process.  If I briefly try to sum up the review, it would be that it's proposed aims are inconsistent with the practical reality of its recommendations.  Shape claims one thing but does another, much like MMC before it.  The reality of the review is that unless aborted it will reduce training standards, create a far less skilled and demoralised sub-consultant grade.  Not only will it have direct negative impacts on care quality and patient safety but it will likely lead to more recruitment crises, as intelligent able young doctors are not going to be keen at being trapped in a service providing sub-consultant grade for life.  Do not be conned by the eloquent claims Shape makes of 'excellent patient care', the reality is the opposite, and the highly cynical would see the changes as creating a compliant new service grade doctor to be putty in the hands of an ever expanding network of corporate NHS providers.  The Francis report highlighted huge problems with HCA training, something that has been left to employers, and Shape pushed the training of doctors further towards this disastrous position in which quality will be sacrificed for the artificial financial imperatives of employers.  I shall leave you with a quote that sums up the reality of Shape as things stand:

"The Shape of Training does not lead to 'generalism' as the public would understand it, a new 'generalist' will be trained in less time than currently it takes to train a specialist, essentially Shape's brave new 'generalist' will be a service providing middle grade, someone who is simply not trained to provide the high quality care that a properly trained consultant of today can deliver.  It would be more accurate to call Shape's new version of the consultant a 'registrar‘, essentially the review results in an Orwellian abuse of the English language in order to hide the truth from the public."

Friday, 31 October 2014

Health Education England, Fiona Patterson and Situational Judgement Tests (SJTs)

 
I got a bit distracted this morning when I started reading HEE's latest idea on medical selection called 'valued based recruitment' (VBR).  Here is HEE's long and shiny briefing on VBR.  I was particularly interested to read their evidence summary section in which it is stated:
 
"Situational Judgement Tests: high reliability and high validity"
 
(high validity in context of 'only if based on robust psychometric methodology')

This led me to question just where this evidence has come from.  HEE kindly replied to my query very quickly and gave me the source.  It was also interesting to note that all methods of assessing values were pretty highly susceptible to coaching, I would interpret that to mean that these methods aren't really very good at assessing people's real values as these should not be very modifiable by coaching!

Here is the full evidence section, it is important to note that Fiona Patterson and her Work Psychology Partnership Group (infamous due to their role in the disastrous MTAS selection process) are thanked by HEE for their 'support' in reviewing the evidence.  Of note Fiona Patterson and her group are keen on SJTs, have developed them for use in the UK and have made a lot of money out of SJTs.  No conflicts of interest there then.

Anyway I went on to read the evidence to see where this evidence of high reliability and validity of SJTs has come from.  Well, surprise, surprise, the evidence on SJTs is Fiona Patterson's very own review!  I then read this review to see if the evidence would back up HEE's summary, alas it did not.

Patterson's review shows a very different picture on SJTs to that painted by the HEE summary.  The reliability of SJTs is mixed, certainly some studies have shown good reliability, however many have shown extremely poor reliability (Cronbach's alphas of around 0.2).  The same is true for SJT validity, it is mixed at best, and that is arguably too optimistic, numerous studies have shown extremely poor reliability (spearman's r's of around 0.2/0.3). 

Overall the evidence in Patterson's review does not support the use of SJTs in medical selection, in my biased opinion.  It is worth noting that Patterson appears to be promoting a selection tool from which she stands to make significant amounts of money from. There is one aspect of the review I strongly agree with and that is the solid evidence demonstrating that selection centres are good selection tools:

"When designed appropriately, selection centres are valid predictors of job performance and could be used for VBR. Construct validity is enhanced by ensuring the content is directly relevant to the attributes of interest in the target job role. However, selection centres are relatively expensive to design and deliver. "

The bottom line for me is that SJTs should not be used, they are unreliable and not sufficiently valid.  However for various reasons including a distorted and cherry picking view of the evidence, HEE had come out advising that SJTs be used as part of VBR.  HEE have got it wrong, SJTs should not be used, I am sure Fiona Patterson and her group will do very nicely if SJTs are used though, what a coincidence......

Sunday, 12 October 2014

The Shape of Training review: a bizarre analysis of ‘evidence’


I have taken the time to review the ‘evidence’ that the Shape of Training review has relied upon in order to make some rather sweeping recommendations about the future of UK medical training.  One would hope that such major reforms would have been based on solid evidence, analysed in a robust and systematic fashion.  All my comments relate to the Annexes andAppendices document that is linked here

In the introductory section MMC is described “as a programme of radical change to drive up the quality of care for patients through reform and improvement in postgraduate medical education and training”.  This is plain wrong, MMC was an underhand attempt to introduce the sub-consultant grade by avoiding proper contract negotiations with the BMA.  The review’s distortion of the reality of MMC is either down to ignorance, or something rather more sinister.  The cherry picking of themes from the Tooke and Collins reviews is also unfair and not representative of these reviews’ recommendations.  At this stage it is worth repeating the Shape of Training review’s proposed purpose:

“The purpose of the Review is to ensure that doctors receive high-quality education and training that support high-quality patient care and improved outcomes.”

One would therefore imagine that their evidence review should look for evidence which can prove any changes they propose have proven benefits in terms of outcome.  Let us see if they can support their claimed purpose with evidence.  Interestingly a false dichotomy is proposed very early on:

“Theme 2 - Workforce needs: Specialists or generalists”

This is a false dichotomy and a demonstrative of a massively over simplistic attitude, one size certainly does not fit all.  Hence by proposing this false choice from the start, it is obvious the review intends to try to push us down one avenue, but why?  The lack of any decent definition of a ‘generalist’ is strange, surely this needs defining from the start?  There is also a lot of talk of ‘credentialing’ but a hard definition of what this word means is lacking, almost as if the review intends to use it as a flexible tool with which they can use however they wish at a later date.

Now to the evidence section, the start is fine, it is made clear that much evidence is anecdotal and that many previous reviews of training have been non-systematic.  Sadly things then start to go downhill when the discussion of ‘generalism’ and ‘integration’ is largely based on the work of ‘think tanks’ and some other rather potentially biased sources.  There is also a complete failure to define what ‘generalism’ and ‘integration’ mean, laying foundations of sand from the very beginning.
Sadly despite making the reader aware of the lack of decent evidence available, the review then goes about reviewing the literature and non-peer reviewed work of various think tanks using some rather vague search terms a thoroughly non-systematic methodology, one that is so open to bias it is not true.  Amazingly the review of evidence cites the GMC’s Good Medical Practice, hardly a robust and objective source of evidence.  So in conclusion the ‘evidence’ on generalism and integration has been cherry picked from a wide variety of frequently non-peer reviewed sources and sources that are open to significant political bias such as think tanks.

The first major question asked was:

“What evidence is there for the effectiveness of UK postgraduate medical education? What factors impact on the quality of postgraduate medical education in the UK?”

The search strategy was systematic, the problem with this is that this is entirely pointless if the evidence is not analysed in a systematic fashion, which it isn't, it is analysed in a fashion that resembles cherry picking.  Strangely the review then proceeds to describe some cherry picked qualitative studies relating to the impact of EWTD and MMC.  Numerous other questions are posed and not answered, often because the evidence just isn't there.  It is therefore particular strange to see this lack of evidence summed up in such a skewed and positive fashion:

“The research and discursive literature on post-MMC postgraduate medical education presents a mixed, albeit mainly positive, picture. Concerns have been raised in the major reviews, surveys and smaller research studies about the impact of system demands and working regulations on the training opportunities available to trainees. These concerns come primarily from trainers who compare their personal experience of a previous system of training with medical education as it is now. There is little objective evidence to suggest that training opportunities and experiences are diminishing.”

I find this summary astonishing, there is a clear element of propaganda here, the attempt to spin MMC positively is being repeated and there is a hint of denial when it comes to acknowledging the obvious, MMC and EWTD have hit experience levels and have impacted on training and patients.  It is contradictory that it deems on one hand that ‘there is little objective evidence that training opportunities and experiences are diminishing’ but on the other hand it deems the literature on post-MMC medical education is ‘mainly positive’ despite the lack of objective evidence to back this assertion up.

The reliance on GMC documents in the next section is interesting; there are repeated references to the GMC’s work in relation to the needs of patients/training and service needs.  It is interesting that the author of this ‘review’ has worked closely with the GMC on several projects and has not published on medical education at all from what I can see.  She is also rather obviously not a doctor, rather a large problem in terms of understanding the evidence concerning training doctors.  The second section concludes by again saying the evidence is not great:

“Whilst much has been written about the state of medical education in its current iteration, the evidence regarding whether it meets service, workforce, patient and individual doctors’ needs could be stronger. Much of the work that has been done has been opinion-based surveys.  There is a danger though, in using opinion as a proxy for hard evidence, particularly when current views may not have past counterparts against which comparisons can be made.”

The third section of the review aimed to compare the UK’s training systems to those of other countries, unsurprisingly this section concluded:

“An evidence review of this size could not realistically absorb all of the published literature on PGME….This evidence review did not find any themes in the debates or developments that are underway elsewhere that are not yet taken account of in the Review.”

Bizarrely the review’s third section asked “What do key opinion formers and stakeholders consider to be the future of medicine - pressures, opportunities and developments?”  This is a very strange question to ask and a strange group of people from whom to gather ‘evidence’ (opinion really).  If the Shape of Training review was just about improving training for the best quality patient care, then why are the views of stakeholders of any relevance in terms of evidence?  They are not of relevance; this should be about evidence, not about what stakeholders want as a result of their various vested interests.  The opinion in this section was largely gathered from the King’s Fund, the Centre for Workforce ‘Intelligence’ (my commas) and the RCGP.  There is a very brief mention for the BMA, but it is very brief indeed.  Unsurprisingly the conclusions of this section read much like a piece of think tank dogma “Regarding service demand, the shift in locus from hospital to community and the move to integrated care will impact on medical career options” and in this way stilted opinion has been transformed into evidence. 

The final question is “4: Does current UK postgraduate training give doctors the knowledge, skills and experience to meet future need for patient involvement in their care and treatment?”.  Again the search strategy seems open to bias and cherry picking.  The question appears deliberately vague and non-specific, almost as if it has been designed to generate no solid evidence based answer:

“No papers looked specifically at whether current postgraduate medical education prepares doctors for working with patients and the public in such a way as current trends dictate.” 

A strange comment, especially given that the term ‘way as current trends dictate’ has not been defined by the author.  How could the author even hope to define the ‘way current trends dictate’, this is a massively subjective judgement that is open to huge amounts of bias.  There is much discussion of generalism, but very little mention of how vague and subjective ‘generalism’ is as a term.  Again the conclusions are rather wishy washy and the lack of evidence is mentioned “There is a lack of research in this area, in terms of the impact of changes on the medical workforce, medical education and whole person approaches on patient outcomes and service quality.”

The integration section then reads much like a think tank review “calls for increased integration at all levels and in all areas of health and social care have come from influential bodies”, rather than summarising who has called for the rather nebulous concept of ‘integration’, perhaps the author would have been better to actually define ‘integration’ and see if there is any good evidence that integration as defined is of any benefit to anyone?  As the foundations have not been adequately prepared, this integration section reads like a biased piece of political propaganda, not like ‘evidence’: “An acknowledgement of integration as the direction of travel for UK health and social care is vital to the decisions made in the Review.”  Rather than assessing if there is solid evidence of a benefit of integration to medical training or patient care, the review just accepts that this is the way politicians want to go and doesn't question the dogma.

In terms of consultation the review “asked 19 open-ended questions and allowed for free-form responses” and received less than 400 responses, including both organisations and individuals.  Of note less than a hundred consultants and doctors in training responded in total, hardly an impressive return.  The consultation questions were designed and analysed in a qualitative fashion, almost as if any objective quantitative analysis of opinion had to be avoided.  Strangely the consultation summary frequently attempts to summarise opinion but consistently fails to provide any objective quantitative data to firmly back up these assertions.  Some of the questions were rather leaning in how they were asked, almost as if they were looking for a certain response.  The lack of definition of key terms such as a ‘generalist’ was mentioned but not adequately addressed, strange given how much the review goes on to use this and other such poorly defined vague terms. 

Interestingly this element of the consultation response “most individuals and organisations argued that generalists would require a longer training period or reconstruction of training to capture the breadth of experiences needed to provide competent general care” has been totally ignored by the review’s recommendations to shorten the length of training.  The opinion that training could not be shortened was also ignored “however, many respondents warned general specialty training is not necessarily the shorter training option, particularly in craft or small specialties”.  Broadly the consultation was highly flawed, not only was the response tiny, the vague qualitative responses could be interpreted in any way the review saw fit according to its preconceived ideas.  The consultation summary is much like a religious text, it is so vague and non specific that it can be interpreted exactly as one’s preconceived ideas dictate.

Thus overall the evidence review has totally failed to provide any solid evidence base upon which to justify the massive overhaul of the structure of medical training as recommended by the review.  The confused and almost haphazard way in which vague questions were posed and not answered doesn’t lead to any kind of evidence based conclusions of note.  While the review’s keenness to document the opinion of stakeholders as ‘evidence’ and its decision to include the work of potentially biased sources such as think tanks as ‘evidence’ are arguably highly flawed.  This methodologically dubious merging together of objective published evidence and politically motivated opinion is suspicious in itself; it has allowed evidence free political dogma to be accepted as evidence based fact by the review.  In conclusion the review does not give the impression that evidence has been objectively analysed to meet the long term needs of medical training and patients.  Importantly in this context the review’s recommendations appear misguided, politically motivated and evidence free.

Wednesday, 8 October 2014

Open letter to the BOA president

Dear BOA President,

I am writing to express by disappointment at the BOA's recent move to force Orthopaedic surgeons into training to become BOA members by making BOA membership compulsory in order to sit the UKITE exam.

This move is a regressive and short sighted move that will alienate a whole generation of Orthopods from the BOA.  Instead of forcing trainees to join, you should be trying to work out why trainees are not joining and remedy this, rather than trying to aggressively force their hand.

Do not underestimate the harm you will do if you do not reverse this decision, I would strongly urge you to reconsider as the feeling amongst trainees on this issue is very strongly in disagreement with the BOA,

Kind regards

Ben Dean
Orthopaedic trainee

Friday, 13 June 2014

John Cook's responds but reliability issue is clearly a key study flaw

Here is John Cook's response to my questions:

"Re rater reliability and data breakdowns, we’ve released the raw individual ratings as well as the final ratings of all 11,944 papers at http://www.skepticalscience.com/tcp.php

Re how the endorsement levels were created, this was the result of a long, collaborative discussion between the authors - attempting to resolve the issue that different authors expressed endorsement of the scientific consensus in different ways. By allowing for different expressions of endorsement, it allowed us to have our cake and eat it too.

Re the proportion of studies providing primary evidence, we didn’t tag such papers - but it is an interesting idea worth exploring."
 
The reliability issue appears a big thorn in Cook's side.  In my opinion intra-rater and inter-rater reliability should have been published as part of the original study, the peer review should have picked this up and the fact that it didn't is worrying.
 
Cook's suggestion that the data is there for analysis is potentially misleading in my opinion.  Analysis is not possible without with days and days of re-working the data into a manageable format from which the reliability analysis could be performed.  Also I don't think the data is even there for calculating intra-rater reliability as it appears that raters did not re-rate the studies.
 
In my opinion the data should be provided in a format in which rater reliability can be easily calculated, this is clearly not the case as things stand.  Given that this is something fundamental to the original study and that should have been published in the first place, I find Cook's stance on the data rather unhelpful and the cynic could interpret this as Cook trying to hide the problem that is a far from reliable rating system.
 
At least Cook does acknowledge my point that it would be well worth exploring which studies actually provided strong primary evidence to back up their subjective opinion.  A study that reviewed the primary evidence for man-induced global warming would be far far more valuable than a study which simply surveys subjective opinion that may be based upon no meaningful evidence at all.

Thursday, 12 June 2014

Why won't John Cook reply to my simple questions?


Strangely or not so strangely, having been 'redirected' to send my letter's questions directly to the author, John Cook, I have received a response, but no attempt to respond to my queries about his study's methodology.  John Cook simply asked if I was one of many people 'that referred to Skeptical Science as "That Propaganda Site”?'.

I fail to see what my opinion on John Cook's website has to do with him answering some very simple questions about his study's methodology.  Perhaps Dr Cook should just answer the questions or is there a reason why he cannot?  Here are the questions that John Cook chooses to ignore for whatever reason:

"I read the study by Cook et al with great interest (1).  Firstly the study used levels of endorsement of global warming as outlined in their Table 2, however I can see no mention as to how these levels were created and how reliable they were in terms of both inter-rater and intra-rater reliability (Cohen’s kappa); would it be possible for the authors to clarify?  Secondly the authors ‘simplified the analysis’ by breaking down ratings into three groups, however they have not included the data breaking down the results into the original 7 categories: would it be possible to see this data?  Finally the study showed that 62.7% of all papers endorsed the consensus, but it does not mention how what proportion of these studies actually provided primary evidence to support the consensus: did the authors gather this information? "