MethodologyMethods of Formal Consensus in Classification/Diagnostic Criteria and Guideline Development
Section snippets
Defining Consensus
Consensus does not require full agreement among participants (7). A prespecified range is decided by the group running or leading the consensus methodology, according to the needs of the task to be performed. There are several methods for defining consensus with examples below (8):
- •
A final vote determining percentage agreement (eg, 80%) among participants
- •
Rating scale—a specified mean rating must be achieved on each topic for inclusion by group
- •
A majority of participants must give a topic a
Participants
The composition of the group is important in determining the decision reached. Most agree that a participant should be an expert with credibility in the appropriate field (8, 9). An expert can be a clinician with vast clinical expertise, a researcher who is well versed with the current literature, or a layperson or patient who has experienced the impact of the disease or intervention or condition in question (4, 10). Ideal characteristics of a group participant would be researchers and
Importance of Evidence from Literature When Using Group Consensus
Although group consensus participants are generally recruited on the basis that they have superior knowledge of the published literature in the field, it is essential to supplement participants with up-to-date, evidence-based literature. When literature reviews are provided, the evidence is used both in the initial discussion and in later decision-making (14). A literature review (preferably a systematic review) provides a common starting point for the group and increases the perception that
Formal Consensus Methods
The focus will be on 4 types of formal consensus methods used in the health field: Delphi method, Nominal Group Technique, RAND/UCLA Appropriateness Method (RAM), and National Institutes of Health (NIH) consensus development conference methodology (Table 1). In practice, a combination of 2 formal consensus methods or their modifications can be used in a 2-step process, where 1 method is used for item generation or some initial consensus and then the other method is used for final consensus.
Grading of Recommendations Assessment, Development, and Evaluation (GRADE)
This review also discusses GRADE as it has been adopted by scientific world for the development of recommendations. GRADE is not a consensus methodology per se but uses consensus methods discussed above (especially Delphi and NGT) to assess quality and strength of a recommendation. GRADE methodology provides a systematic and transparent approach to rate the quality of evidence and grade the strength of recommendations for patient important outcomes. GRADE was developed by experts with a goal to
Limitations of Review
Although we present a comprehensive review of formal consensus methods, we did not conduct a systematic review. In our comprehensive review, the majority of the articles on formal consensus methodologies are implementations in diagnostic/classification criteria, guidelines, and response indices development.
Conclusion
Formal consensus methodologies have various applications in the medical literature from guideline development to development of criteria sets. These methods are being increasingly used in rheumatology. Each methodology has its unique attributes, and utilization of a particular methodology or combination of methodologies depends on the following: (1) clinical question; (2) audience; and (3) available resources.
When closely tied to evidence-based literature, these summarized statements can
Acknowledgments
We would like to acknowledge Drs. Carol Wallace and Gillian Hawker for providing constructive feedback during the writing of this manuscript.
References (67)
- et al.
Clinical and laboratory follow-up for treating and monitoring patients with ankylosing spondylitis: Development of recommendations for clinical practice based on published evidence and expert opinion
Joint Bone Spine
(2007) - et al.
Measuring the clinical consistency of panelists' appropriateness ratings: The case of coronary artery bypass surgery
Health Policy
(1997) - et al.
Sensitivity and specificity of the RAND/UCLA appropriateness method to identify the overuse and underuse of coronary revascularization and hysterectomy
J Clin Epidemiol
(2001) - et al.
Grades of recommendation for antithrombotic agents: American College of Chest Physicians evidence-based clinical practice guidelines (8th edition)
Chest
(2008) - et al.
The primary and secondary prevention of coronary artery disease: American College of Chest Physicians evidence-based clinical practice guidelines (8th edition)
Chest
(2008) - Lugtenberg M, Burgers JS, Westert GP. Effects of evidence-based clinical practice guidelines on quality of care: A...
How do we decide whether an investigation or procedure is appropriateAppropriate investigation and treatment in clinical practice
AHCPR clinical practice guideline programReport to Congress
(1995)- et al.
Consensus development methods: A review of best practice in creating clinical guidelines
J Health Serv Res Policy
(1999) - et al.
EULAR/PReS endorsed consensus criteria for the classification of childhood vasculitides
Ann Rheum Dis
(2006)
The pediatric rheumatology European Society/American College of Rheumatology/European League Against Rheumatism provisional classification criteria for juvenile systemic sclerosis
Arthritis Rheum
The Delphi method: Techniques and applications
Consensus methods: Characteristics and guidelines for use
Am J Public Health
Consensus methods for medical and health services research
BMJ
American College of Rheumatology 2008 recommendations for the use of nonbiologic and biologic disease-modifying antirheumatic drugs in rheumatoid arthritis
Arthritis Rheum
Team composition in organizational settings: Issues in managing an increasingly diverse work force
Group decision making by experts: Field study of panels evaluating medical technologies
J Pers Soc Psychol
Variations by specialty in physician ratings of the appropriateness and necessity of indications for procedures
Med Care
Evidence and consensus
JAMA
Quality and methods of developing practice guidelines
BMC Health Serv Res
Delphi
The use of the Delphi technique in problems of educational innovations
An experimental application of the Delphi method to the use of experts
Group techniques for program planning: A guide to nominal group and delphi processes
Consensus development methods, and their use in clinical guideline development
Health Technol Assess
Development of a provisional core set of response measures for clinical trials of systemic sclerosis
Ann Rheum Dis
Defining appropriate outcome measures in pulmonary arterial hypertension related to systemic sclerosis: A Delphi consensus study with cluster analysis
Arthritis Rheum
Using Delphi to achieve congruent objectives and activities in a pediatrics department
J Med Educ
Use of delphi methodology to generate a survey instrument to identify priorities for state allied health associations
Allied Health Behav Sci
EULAR recommendations for the management of systemic lupus erythematosusReport of a task force of the EULAR standing committee for international clinical studies including therapeutics
Ann Rheum Dis
Outcome measurements in scleroderma: Results from a Delphi exercise
J Rheumatol
Preliminary criteria for clinical remission for select categories of juvenile idiopathic arthritis
J Rheumatol
International consensus on preliminary definitions of improvement in adult and juvenile myositis
Arthritis Rheum
Cited by (284)
Debriefing Trainees After Global Health Experiences: An Expert Consensus Delphi Study
2024, Academic PediatricsConsensus on occupational health competencies for UK first contact physiotherapists
2023, Physiotherapy (United Kingdom)Recommendations for objective cardiovascular assessment to inform clinical exercise prescription: An Exercise Physiologist and Physiotherapist expert consensus
2023, Journal of Science and Medicine in SportA Subjective Assessment for the Preconstruction Project Plans in Egypt
2023, Ain Shams Engineering Journal
Funding Source: No funding was obtained for the study design, collection, analysis, and interpretation of data; in the writing of the manuscript; and in the decision to submit the manuscript for publication. Dr. Khanna was supported by a National Institutes of Health Award (NIAMS K23 AR053858-04).
None of the authors have competing interests related to the content of this manuscript.
- 1
Both authors contributed equally to the manuscript.