The long and tortuous road to publication of the NICE guideline in 2021 was one littered with delays, a last minute ‘pause‘ on publication a day before release, news that not all interested parties endorsed changes to GET and CBT recommendations (including 4 Royal Colleges), a roundtable meeting, and finally after many years, publication.
Yet, publication has not been the end of the saga. July 2023 saw open publication in the Journal of Neurology, Neurosurgery & Psychiatry of an article “Anomalies in the review process and interpretation of the evidence in the NICE guideline for chronic fatigue syndrome and myalgic encephalomyelitis” in which 3 PACE trial authors and over 50 international specialists co-led by researchers from the Institute of Psychiatry, Psychology & Neuroscience, King’s College London, considered that the 2021 NICE guideline changes to GET and CBT recommendations were a “controversial U-turn” from the earlier version of the guideline, and postulated that the controversy stemmed from “anomalies in both processing and interpretation of the evidence by the NICE committee”.
Briefly put (a more expansive explanation is available) it was claimed that the NICE committee:
- created a new definition of CFS/ME, which “downgraded” the certainty of trial evidence;
- omitted data from standard trial end points used to assess efficacy;
- discounted trial data when assessing treatment harm in favour of lower quality surveys and qualitative studies;
- minimised the importance of fatigue as an outcome;
- did not use accepted practices to synthesise trial evidence adequately using GRADE (Grading of Recommendations, Assessment, Development and Evaluations trial evidence);
- interpreted GET as mandating fixed increments of change when trials defined it as collaborative, negotiated and symptom dependent;
- deviated from NICE recommendations of rehabilitation for related conditions, such as chronic primary pain; and
- recommended an energy management approach in the absence of supportive research evidence.
Taking all these errors leads to one conclusion: that NICE has not followed international standards for guideline development, which has led to guidance that could harm rather than help patients. The implications of this are complicated. The flaws really do shake the credibility of this 2021 update of the NICE guideline for CFS/ME.
Paul Garner, Professor emeritus of evidence synthesis in Global Health, and co-author of the review
Such claims were not likely to go without comment as NICE states that its processes and methods are based on internationally recognised standards, on internationally accepted criteria of quality, and on the best available evidence. Thus, NICE swiftly undertook to provide “… a detailed response to this analysis”.
This has now been published. Entitled “NICE guideline on ME/CFS: robust advice based on a thorough review of the evidence”, the abstract trails that the reasoned response “outline[s] how these opinions are based on a misreading or misunderstanding of the guideline process or the guideline, which provides a balanced and reasoned approach to the diagnosis and management of this challenging condition”.
The authors rebuffed each of the 8 points raised with the following being the most interesting points:
- There have been over 25 published case definitions of the disease but with no single ‘gold-standard’ diagnostic test ME/CFS is a diagnosis based on exclusion of other illnesses and the presence of a cluster of characteristic symptoms. However, due to lack of a diagnostic test it was not possible for the NICE committee to assess the validity of criteria in terms of accuracy and precision. NICE examined (using AGREE II) current criteria to decide which would be the most appropriate to use when suspecting or diagnosing ME/CFS. “All criteria were found to have serious or very serious limitations” and whilst most included post-exertional malaise (PEM), cognitive difficulties, and unrefreshing sleep, there were differences in the way such symptoms were defined and whether or not compulsory for a diagnosis. The NICE committee balanced under- and over-diagnosis in coming to its conclusions and decided on a modified version of the Institute of Medicine’s 2015 criteria. The modifications were “derived from assessment of the quality, benefits and harms, and usability of the criteria. It does not represent a new set of diagnostic criteria.”
The 2021 criteria are more restrictive than the 2007 NICE CFS/ME guideline as patients require to present with debilitating fatigue, unrefreshing sleep, PEM, and cognitive difficulties. PEM is required in the Revised Canadian Consensus Criteria, the International Consensus Criteria, as well as in the Institute of Medicine’s. Wider definitions – e.g. Oxford or London (both used by the PACE trial) – do not require PEM and so risk including people with fatigue arising from another illness. In fact, the committee noted that a 2017 study of 6,000 cases found 85% of those identified as having CFS, utilising the Oxford criteria, were actually healthy individuals with mild fatigue who had been incorrectly labelled. Papers that did not use a criteria which included PEM were downgraded due to ‘indirectness’ in GRADE assessment. All papers were looked at but some were downgraded where there was uncertainty as to whether trials reflected accurately the ME/CFS population. - In choosing outcomes and endpoints NICE followed its standard ‘Developing NICE guidelines: the Manual‘ and these were identified before evidence was examined. “Any inference that the committee looked at the evidence review and picked the outcomes which best matched their pre-conceptions is simply incorrect.” The committee found that early positive outcomes were not maintained over time.
- NICE looked at 25 qualitative studies into non-phamacological interventions and approached data on treatment harm using the accepted CERQual methodology. For GET and other exercise interventions, some participants did report a worsening of symptoms after each session and some a worsening of symptoms in the longer term. In one study, up to 74% of participants reported harms after GET. Overall, “the committee was presented with contested evidence of only limited and short-term benefits of GET … benefits which were found to be even less outside of the setting of a clinical trial. This was contrasted against considerable, but methodologically low-quality evidence of patient harm, which by some accounts was quite devastating. After reviewing all this evidence, the committee agreed that GET should not be offered to patients.”
- Fatigue was one of the outcomes which mattered most in the review. Lack of blinding in clinical trials runs the risk of performance and detection bias due to participants’ and/or assessors’ knowledge of the intervention. This is exacerbated where measures of outcome are subjective rather than objective. Therefore, as is standard practice, where unblinded trials regarding fatigue were examined they were downgraded in-line with GRADE. Blinded trials which included fatigue were not downgraded. Again, the evidence was reviewed and not ignored.
- NICE follows its Manual regarding the bringing together of information from a range of sources and disciplines to inform debates and decisions on specific issues (evidence synthesis) and used accepted methodologies – such as GRADE. Applying GRADE methodology meant not rejecting evidence but rather concentrated on assessing its quality. All relevant studies were included in the evidence review and the review itself was quality assured externally. Where evidence was deemed ‘low or very low quality’ this was consistent with other reviews – such as Cochrane’s on exercise therapy for ME/CFS.
- Graded Exercise Therapy is a term which had different definitions which the guideline sought to clarify but in point of fact it reflected the definitions used in the GET evidence review e.g. the GET Therapists manual from the PACE trial.
- NICE recommends different therapies for different conditions even where some illnesses share common symptoms – such as pain. The pain experienced by those with ME/CFS (assessed at <1% of those who report chronic pain) does not correlate with that suffered by people with chronic primary pain which is the focus of NICE’s Chronic Pain Guideline. Exercise has been shown to lessen chronic pain in others but in ME/CFS pain can be exacerbated – both fatigue-related and pain-related PEM -making management strategies more person specific for ME/CFS pain more appropriate than the generality of the Chronic Pain Guideline.
- “There is lack of evidence supporting energy management in people with ME/CFS, and no evidence it is a ‘cure’. But there is also no signal of harm or deterioration with energy management programmes, unlike with GET.”
In conclusion, the authors of the paper emphasised that the guideline had been drafted by an independent committee and followed a transparent process which conformed to recognised standards for guideline development.
Criticisms of the guideline discussed in this paper are misplaced.
NICE guideline on ME/CFS: robust advice based on a thorough review of the evidence
