Published Comment and Response: Comparing the USPSTF and GRADE Approaches to Recommendations

First published as a Letter to the Editor in Annals of Internal Medicine 151(5):363-64, September 1, 2009.

Table of Contents

  • Comment
  • Response

Comment

TO THE EDITOR:

In their article on the "insufficient evidence" category of the U.S. Preventive Services Task Force (USPSTF) system,1 Petitti and colleagues make 3 potentially misleading statements about the Grading of Recommendations Assessment, Development and Evaluation (GRADE) (www.gradeworkinggroup.org) approach to rating quality of evidence and strength of recommendations.2 We would like to clarify the GRADE characteristics in question.

First, Petitti and colleagues state that GRADE has no equivalent to the USPSTF "I statement." The I statement accompanies a USPSTF decision not to recommend either in favor or against an intervention because "the current evidence is insufficient to assess the balance of benefits and harms of the service."1 With GRADE, guideline panels may choose to rate the quality of evidence as low or very low, and decide not to offer a recommendation and to provide clarifying statements.3 Thus, GRADE accommodates the situation in which the USPSTF uses an I statement.

Second, Petitti and colleagues accurately state that the GRADE approach to recommendations considers the magnitude of the benefits and harms and the values and preferences patients place on those benefits and harms.4 However, they write that GRADE also takes into account "the importance of the outcome that the treatment prevents, [and] the burden of therapy" as if these were different from the magnitude of benefits and harms and the relative values placed on them. They are not.

Finally, in contrast to what Petitti and colleagues imply, GRADE takes into account all the issues relevant to decisions regarding prevention.5 GRADE considers both the seriousness of a condition and its incidence or prevalence (the 2 factors that determine burden of disease) as factors that policymakers must consider beyond the strength of a recommendation when choosing priorities. From the perspective of individual patients, both the severity and the incidence of a condition (and the proportion of burden of disease that is potentially preventable) are incorporated into estimates of the effect of an intervention and into judgments about the balance between desirable and undesirable effects. This is explicitly addressed in GRADE's approach to determining the strength of a recommendation.4 As with burden of disease, how widely a service is used is a factor that policymakers must consider when choosing priorities.

We trust that this clarification will prevent misunderstanding by any groups considering joining the more than 30 organizations (including UpToDate, the World Health Organization, and the American College of Physicians) currently using GRADE.


Gordon H. Guyatt, MD, MSc
McMaster University Health Sciences Center
Hamilton, Ontario L8S 4L8, Canada


Mark Helfand, MD, MS
Portland Veterans Affairs Medical Center
Portland, OR 97239


Regina Kunz, MD, MSc(Epi)
University Hospital Basel
4031 Basel, Switzerland

References

  1.   Petitti DB, Teutsch SM, Barton MB, Sawaya GF, Ockene JK, DeWitt T; U.S. Preventive Services Task Force. Update on the methods of the U.S. Preventive Services Task Force: insufficient evidence. Ann Intern Med 2009;150:199-205. [PMID: 19189910]
  2.   Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al; GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008;336:924-6. [PMID: 18436948]
  3.   Jaeschke R, Guyatt GH, Dellinger P, Schünemann H, Levy MM, Kunz R, et al; GRADE Working Group. Use of GRADE grid to reach decisions on clinical practice guidelines when consensus is elusive. BMJ 2008;337:a744. [PMID: 18669566]
  4.   Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, et al; GRADE Working Group. Going from evidence to recommendations. BMJ 2008;336:1049-51. [PMID: 18467413]
  5.   Guyatt GH, Oxman AD, Kunz R, Jaeschke R, Helfand M, Liberati A, et al; GRADE Working Group. Incorporating considerations of resources use into grading recommendations. BMJ 2008;336:1170-3. [PMID: 18497416]

Response

IN RESPONSE:

The USPSTF welcomes this opportunity to further explicate the similarities and differences between the GRADE and USPSTF approaches to making recommendations.

Guyatt and colleagues explain that guideline panels that use the GRADE approach may decide not to offer a recommendation and to provide clarifying statements. We agree that GRADE thus accommodates the situation in which the USPSTF uses an I statement. However, the USPSTF "rules of evidence" dictate that an I statement must always be issued if a letter grade of A, B, C, or D cannot be assigned.1, 2 This occurs when the USPSTF judges that the evidence about health benefits and harms does not permit at least moderate certainty that the balance of health benefits and harms is favorable, that no net benefit exists, or that the harms of the intervention outweigh the benefits.3 The values and preferences that individual patients place on benefits and harms are not considered by the USPSTF when deciding whether to assign a letter grade to a recommendation. The USPSTF also does not consider the cost or cost-effectiveness of the service. These are important differences between the USPSTF and the GRADE approaches.

Guyatt and colleagues state that the importance of the outcome the intervention prevents and the burden of the intervention therapy are no different from benefits and harms. The USPSTF agrees that importance and burden could be conceptualized as benefits and harms, but not that they must be.

The USPSTF agrees with Guyatt and colleagues' interpretation of the implications of the USPSTF insufficient evidence publication4—that the differences between treatment and diagnosis on the one hand, and prevention on the other, may make GRADE less applicable to prevention decisions. The USPSTF believes that certainty of a net health benefit must be higher when asymptomatic persons are encouraged by their physician to undergo a preventive intervention than when patients seek help for a problem and are advised to undergo a diagnostic intervention or to partake of a treatment. We welcome Guyatt and colleagues' comment on the implications of this belief.

The USPSTF did not intend to imply that GRADE does not consider the 4 issues the USPSTF describes as pertinent to decision making in the face of insufficient evidence. The GRADE Working Group has published widely about its methods, and these publications describe the uses of information on burden of illness, cost, and use in the community.5-12

The difference between the USPSTF and the GRADE approaches lies in the role that burden of illness, cost, and community use play in moving from evidence about net harms and benefits to suggested actions. For the USPSTF, the assignment of a letter grade to a recommendation carries a suggested action: "Do it routinely" (grades A and B), "Don't do it routinely" (grade C), or "Don't do it" (grade D).1 Topics not assigned a letter grade—all topics with an associated I statement—do not have a suggested action.

It is our understanding that, in addition to information on the quality of evidence about net benefits and harms, GRADE could use information on patient preferences, burden of illness, cost, and use in the community to classify a recommendation as weak or strong. It is also our understanding that a GRADE classification as a weak or strong recommendation determines the linkage of the recommendation with suggested actions.5, 6 This is indeed a large difference between the USPSTF and GRADE approaches. We believe that the USPSTF approach to insufficient evidence is well suited to the needs of the primary care community, the public, and policymakers in the United States for topics in prevention. This does not mean that the USPSTF approach is applicable outside the United States or that it is applicable to diagnosis or treatment.

The USPSTF is aware of the important work that GRADE is doing to try to reduce inconsistencies among different approaches to assessing the quality of evidence and moving from evidence to recommended actions. The USPSTF and GRADE use nearly identical criteria to rate the quality of studies that provide information about health benefits and harms.2, 7-11 The USPSTF and GRADE both attempt to make their methods and processes transparent and to remain scrupulously free of financial conflicts of interests. The USPSTF looks forward to further productive dialogue about and convergence of the methods and processes of the USPSTF, GRADE, and other authoritative groups. We are eager to engage in an ongoing inquiry about how best to rate, review, and use evidence and about the kinds of evidence that should figure in suggestions for action.


Diana B. Petitti, MD, MPH
Arizona State University
Phoenix, AZ 85041


Steven M. Teutsch, MD
Los Angeles County Department of Public Health
Los Angeles, CA 90012


Mary B. Barton, MD, MPP
Agency for Healthcare Research and Quality
Rockville, MD 20850


George F. Sawaya, MD
University of California, San Francisco
San Francisco, CA 94143


Judith K. Ockene, PhD, Med
University of Massachusetts Medical School
Worcester, MA 01655


Thomas DeWitt, MD
University of Cincinnati College of Medicine
Cincinnati, OH 45229

References

  1.  Barton MB, Miller T, Wolff T, Petitti D, LeFevre M, Sawaya G, et al; U.S. Preventive Services Task Force. How to read the new recommendation statement: methods update from the U.S. Preventive Services Task Force. Ann Intern Med 2007;147:123-7. [PMID: 17576997]
  2.  U.S. Preventive Services Task Force Procedure Manual. AHRQ Publication No. 08-05118-EF. Rockville, Maryland: U.S. Preventive Services Task Force, July 2008. Accessed at https://www.uspreventiveservicestaskforce.org/uspstf/procedure-manual on 7 July 2009.
  3.  Sawaya GF, Guirguis-Blake J, LeFevre M, Harris R, Petitti D; U.S. Preventive Services Task Force. Update on the methods of the U.S. Preventive Services Task Force: estimating certainty and magnitude of net benefit. Ann Intern Med 2007;147:871-5. [PMID: 18087058]
  4.  Petitti DB, Teutsch SM, Barton MB, Sawaya GF, Ockene JK, DeWitt T; U.S. Preventive Services Task Force. Update on the methods of the U.S. Preventive Services Task Force: insufficient evidence. Ann Intern Med 2009;150:199-205. [PMID: 19189910]
  5.  Guyatt GH, Oxman AD, Kunz R, Jaeschke R, Helfand M, Liberati A, et al; GRADE Working Group. Incorporating considerations of resources use into grading recommendations. BMJ 2008;336:1170-3. [PMID: 18497416]
  6.  Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, et al; GRADE Working Group. Going from evidence to recommendations. BMJ 2008;336:1049-51. [PMID: 18467413]
  7.  Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al; GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008;336:924-6. [PMID: 18436948]
  8.  Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ; GRADE Working Group. What is "quality of evidence" and why is it important to clinicians? BMJ 2008;336:995-8. [PMID: 18456631]
  9.  Atkins D, Briss PA, Eccles M, Flottorp S, Guyatt GH, Harbour RT, et al; GRADE Working Group. Systems for grading the quality of evidence and the strength of recommendations II: pilot study of a new system. BMC Health Serv Res 2005;5:25. [PMID: 15788089]
  10.   Atkins D, Eccles M, Flottorp S, Guyatt GH, Henry D, Hill S, et al; GRADE Working Group. Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group. BMC Health Serv Res 2004;4:38. [PMID: 15615589]
  11.   Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, Flottorp S, et al; GRADE Working Group. Grading quality of evidence and strength of recommendations. BMJ 2004;328:1490. [PMID: 15205295]
  12.   Schünemann HJ, Oxman AD, Brozek J, Glasziou P, Jaeschke R, Vist GE, et al; GRADE Working Group. Grading quality of evidence and strength of recommendations for diagnostic tests and strategies. BMJ 2008;336:1106-10. [PMID: 18483053]

Current as of: September 2009


Internet Citation: Published Comment and Response: Comparing the USPSTF and GRADE Approaches to Recommendations. U.S. Preventive Services Task Force. February 2014.