Home > FT > Is the Court Encouraging Agencies to Rely on Junk Science? Biestek v. Berryhill

Is the Court Encouraging Agencies to Rely on Junk Science? Biestek v. Berryhill

/
23 Apr 2019
, , ,

April 23, 2019

Biestek v. Berryhill, 587 U.S. ___ (2019) (Kagan, J.).
Response by Richard J. Pierce, Jr.
Geo. Wash. L. Rev. On the Docket (Oct. Term 2018)
Slip Opinion | SCOTUSblog

Is the Court Encouraging Agencies to Rely on Junk Science? Biestek v. Berryhill

It would be easy to interpret the six-Justice majority opinion in Biestek v. Berryhill1 as an invitation to agencies to make important decisions based on junk science, i.e., opinions of putative experts that are not supported by reliable data or analysis. I hope that agencies and lower courts resist that interpretation of the case, but it is an entirely plausible interpretation.

The facts of Biestek are simple.2 Biestek applied for Social Security disability benefits. At a hearing before an Administrative Law Judge (“ALJ”) Biestek claimed that he was so disabled that he could not perform the functions required by any job that is available in significant numbers in the U.S. economy.

In such a hearing the applicant has the burden of proving his health state, while the government has the burden of proving the availability of a significant number of jobs that can be performed by someone with the age, education, experience and health state of the applicant. The Social Security Administration (“SSA”) attempted to prove the availability of jobs that Biestek could perform by introducing the testimony of a vocational expert. The witness testified that there were 120,000 jobs available for “sorters” and 40,000 jobs available for “bench assemblers” that Biestek could perform.

Biestek’s lawyer asked the witness to describe the basis for the remarkably specific number of jobs of each type that the witness claimed to be available. The witness referred to two sources—the Bureau of Labor Statistics (“BLS”) and private surveys the witness had conducted for other clients. Since BLS does not report job availability statistics with the specificity required to support the number of jobs of a particular type the witness claimed to be available, the only possible basis for those numbers were the private surveys.

Biestek’s lawyer asked the witness to provide the private surveys. The witness refused on the basis that they were part of her confidential client files. Biestek’s lawyer then asked the witness to provide redacted versions of the surveys that excluded any client-specific information. The witness should have been prepared to comply with that request. SSA’s Vocational Expert Handbook instructs witnesses to “have available, at the hearing, any vocational resource materials that you are likely to rely upon” because “the ALJ may ask you to provide relevant portions of [those] materials.”3

We will never know whether the witness had a redacted version of the surveys with her because the ALJ then interrupted the cross-examination and stated that he would not require the witness to provide a redacted version of the surveys. The ALJ then relied solely on the testimony of the vocational witness as the basis for his decision to deny Biestek’s application for benefits based on his finding that there were a significant number of jobs available that Biestek could perform.

Of course, we will also never know whether the private surveys exist. If they exist, we will never know whether they support the witness’s testimony. If they support the witness’s testimony, we will never know whether they are based on reliable data. If they are supported by reliable data, we will never know whether the methodology the witness used to draw inferences from the data was reliable.

No court would admit evidence of the type the ALJ relied on as the basis for his critical finding that there are a significant number of jobs that Biestek can perform. If a district court admitted evidence of that type and relied on it as the basis for a finding in a bench trial, its decision would be summarily reversed on appeal.

In its famous opinion in Daubert v. Merrell Dow Pharmaceuticals,4 the Supreme Court took a major step toward assuring that our legal system functions on the basis of sound scientific principles. The Court held that judges must apply criteria based on sound scientific principles when they decide whether to admit expert testimony. Daubert was a reaction to the well-documented problem of court decisions that are based on “junk science.” That term refers to opinions offered in evidence by supposed experts that were not supported by reliable data and analysis but that often were the basis for jury verdicts before the Court decided Daubert.

Daubert requires an expert witness to disclose to the court and to opposing counsel the data and methodology the expert relied on as the basis for any opinion the expert proposes to offer in court. The judge is then assigned the task of deciding whether the data and methodology are sufficiently reliable to support a decision to admit the expert’s testimony. If the reliability of the data or methodology is challenged by opposing counsel, the judge conducts a voir dire hearing during which opposing counsel have the opportunity to test the reliability of the data and methodology by cross-examining the witness. No expert witness is permitted to testify unless the witness has disclosed the data and methodology that are the basis for the opinion offered by the witness.

Three Justices expressed the view that the ALJ’s decision to deny Biestek’s application for disability benefits based solely on the totally unsupported testimony of the vocational expert should be reversed for all of the same reasons that any court would reverse the decision of a district judge in similar circumstances.5 Unfortunately, they had to express that view in dissenting opinions.

The majority upheld the ALJ’s decision. The majority held that the ALJ’s finding was conclusive because it was supported by substantial evidence.6 The majority reasoned that the ALJ could rely on the unsupported opinion of the witness and deny Biestek access to all of the means through which he could challenge that testimony for two reasons—the evidence satisfied the substantial evidence test,7 and the Federal Rules of Evidence (“FRE”) do not apply to SSA disability hearings.8 The first reason is based on a misunderstanding of the substantial evidence test, while the second is based on a misunderstanding of the reasons why Congress decided that the FRE do not apply to agency hearings.

The majority argued that it would make no sense to conclude that an expert witness’s opinion qualifies as substantial evidence when no one asks the expert to provide the data and analysis to support the opinion but that it does not qualify as substantial evidence when someone asks for the supporting data and analysis and the witness refuses to comply with the request.9 That reasoning is inconsistent with the Court’s well-reasoned opinion in Universal Camera v. NLRB: “The substantiality of evidence must take into account whatever in the record fairly detracts from its weight.”10 A refusal to provide support for an expert opinion clearly detracts from its weight.

The inapplicability of the FRE to agency hearings is unrelated to the wisdom of the Court’s decision in Daubert to keep juries from deciding cases based on junk science. The substantive reasoning in Daubert is equally applicable to agency hearings. Like juries, agencies should not be allowed to make decisions based on junk science. The difference in context bears only on the mechanism that courts can and should use to implement the Daubert principle. In the context of a jury trial, the Daubert criteria must be applied at the time that an expert opinion is offered in evidence. In the context of an agency hearing, the Daubert criteria can be applied in the process of judicial review of the stated basis for an agency’s decision to rely on expert testimony as the basis for a finding.

The FRE are designed for application in jury trials.11 They are essential in that context because juries are not required to state reasons for their decisions. Thus, if we want to ensure that juries do not make decisions based on unreliable evidence, we must preclude them from having access to the types of evidence that are likely to be unreliable. The FRE are not appropriate for application in agency hearings because agencies must give reasons for their findings of fact. Thus, if an ALJ admits evidence that the ALJ later determines to be unreliable, the ALJ can decline to rely on it as the basis for a finding. If an ALJ erroneously relies on unreliable evidence as the basis for a finding, a reviewing court can detect the error and reject the finding as not supported by substantial evidence.

The Court’s opinion in Biestek has the potential to produce an administrative state in which many important decisions are based solely on junk science. As Jason Johnston has explained, many agencies have relied on junk science as the sole basis for decisions that have billions of dollars of consequences.12 Thus, for instance, the sole basis for decisions in which the Environmental Protection Agency (“EPA”) has required firms to incur billions of dollars in costs are findings contained in studies conducted by researchers who have refused to provide anyone, including EPA, with the data and analysis on which the findings are based. It is impossible for anyone, including EPA personnel, to know whether those findings are supported by reliable data and analysis.

The Biestek majority refers to the credentials and reputation of an expert witness as an adequate independent basis to conclude that the opinions of the expert are sufficiently reliable to support a finding of fact even if the expert refuses to provide anyone access to the data and analysis that she relies on to support her opinion.13 That is a dangerous method of reasoning that is inconsistent with Daubert.

The Daubert Court did not indulge the naïve assumption that a putative expert should be believed because she has impressive credentials and an impeccable reputation. There is a mountain of evidence that contradicts that assumption. Thus, for instance, in just a two month period, Harvard, Cornell, Dartmouth and Sloan Kettering were forced to announce that some of their top researchers had committed academic fraud in conducting some of the most important studies that those prestigious institutions have produced.14

Attempts to replicate the findings of hundreds of important studies have produced powerful circumstantial evidence that academic fraud in conducting studies is a widespread problem. Thus, for instance, four attempts to replicate the findings of hundreds of studies published in prestigious journals found that only 36% to 67% of the findings could be replicated using the data and methodology that the researchers claimed to have used.15 It is fair to infer that the other 33% to 64% of the findings were the product of either academic fraud or incompetence.

Academic fraud can be extremely costly to society. The measles epidemic that now afflicts the nation16 is the result of an academic fraud that took place over twenty years ago. A team of twelve distinguished researchers published a study in 1998 in Lancet, the most prestigious British medical journal, in which they reported the results of a study of the relationship between vaccines and autism.17 They found a correlation between taking a vaccine and being diagnosed with autism so high that the lead author used the finding as the basis for an international speaking tour in which he warned that vaccines cause autism.

The vaccine/autism study was a fraud perpetrated by the lead author, who was well paid by attorneys who wanted to have access to a study they could use in lawsuits against vaccine manufacturers.18 The fraud was not detected and publicly disclosed until 2011, however. Each of scores of studies conducted since 1998 have found no relationship between vaccines and autism. The fraudulent 1998 study caused a dramatic decrease in the proportion of the population of parents who vaccinate their children, and a corresponding increase in the number of children who suffer from a variety of potentially lethal communicable diseases like measles. Some parents still cling stubbornly to the belief that was instilled by the fraudulent study, with adverse effects on vulnerable populations all over the world.

We need to act aggressively to reduce the risk of damage caused by junk science. Daubert was an important step in the right direction. Biestek was an unfortunate step in the wrong direction.

The dissenting Justices expressed the hope that lower courts will not interpret the majority opinion as an invitation to uphold all uses of junk science by agencies. They suggested that the majority opinion might be just the first step of many in which the majority will provide agencies and courts with guidance that will discourage, if not eliminate, the risk that they will base their decisions on junk science.19

The reasoning in the majority opinion can be interpreted narrowly. At the end of its opinion, the majority stated that it was only rejecting a “categorical rule” that the petitioner’s lawyer had urged the Court to adopt.20 It went on to disavow any intent to decide “whether, in the absence of that rule, substantial evidence supported the ALJ in denying [Biestek] benefits.”21 That is a strange statement that suggests that the majority did not do the one thing that every court is required to do—resolve the case before the court based on the facts of the case.

I hope the optimism of the dissenting Justices proves to be accurate, but it would be easy for lower courts to interpret the majority opinion as an invitation to allow agencies to rely on junk science in virtually all cases. The Court needs to follow Biestek with a series of opinions in which it makes it clear that an agency can rely on the unsupported opinion of a putative expert only in rare circumstances in which the agency has no other viable option and the agency has good reasons to excuse the expert from providing support for the opinion.

Professor Richard J. Pierce, Jr. is the author of over twenty books and 130 articles on administrative law, government regulation, and the effects of various forms of government intervention on the performance of markets. His books and articles have been cited in hundreds of judicial opinions, including over a dozen opinions of the U.S. Supreme Court.

    1. No. 17-1184 (U.S. Apr. 1, 2019).
    2. Biestek, slip op. at 1–4.
    3. Soc. Sec. Admin., Vocational Expert Handbook 37 (Aug. 2017).
    4. 509 U.S. 579 (1993).
    5. Biestek, slip op. at 1–3 (Sotomayor, J., dissenting); 1–9 (Gorsuch, J., dissenting).
    6. Id. at 1 (majority opinion).
    7. Id. at 8–11.
    8. Id. at 6–7.
    9. Id. at 8–11.
    10. 340 U.S. 474, 487–88 (1951).
    11. See 2 Kristin E. Hickman & Richard J. Pierce, JR., Administrative Law Treatise ch. 9 (2019); Richard J. Pierce, Jr., Use of Federal Rules of Evidence in Federal Agency Adjudications, 39 Admin. L. Rev. 1 (1987).
    12. See Jason S. Johnston, Restoring Science and Economics to EPA’s Benefit Calculation, The Regulatory Review (Oct. 15, 2018), https://www.theregreview.org/2018/10/15/johnston-restoring-science-economics-epa/.
    13. Biestek, slip op. at 7–9.
    14. See Benedict Carey, Prominent Cancer Researcher Resigns from Dartmouth Amid Plagiarism Charges, N.Y. Times (Sept. 14, 2018), https://www.nytimes.com/2018/09/14/health/welch-plagiarism-cancer-dartmouth.html; Gina Kolata, He Promised to Restore Damaged Hearts. Harvard Says His Lab Fabricated Research, N.Y. Times (Oct. 29, 2018), https://www.nytimes.com/2018/10/29/health/dr-piero-anversa-harvard-retraction.html; Anahad O’Connor, More Evidence that Nutrition Studies Don’t Always Add Up: A Cornell Food Scientist’s Downfall Could Reveal a Bigger Problem in Nutrition Research, N.Y. Times (Sept. 29, 2018), https://www.nytimes.com/2018/09/29/sunday-review/cornell-food-scientist-wansink-misconduct.html; Charles Ornstein & Katie Thomas, Top Cancer Researcher Fails to Disclose Corporate Financial Ties in Major Research Journals, N.Y. Times (Sept. 8, 2018), https://www.nytimes.com/2018/09/08/health/jose-baselga-cancer-memorial-sloan-kettering.html.
    15. Colin F. Camerer et al., Evaluating the Replicability of Social Science Experiments in Nature and Science Between 2010 and 2015, 2 Nature Hum. Behav., 637 (2018).
    16. Measles Cases and Outbreaks: Measles Cases in 2019, Ctrs. For Disease Control and Prevention (Apr. 15, 2019), https://www.cdc.gov/measles/cases-outbreaks.html; Tyler Pager & Jeffery C. Mays, New York Declares Measles Emergency, Requiring Vaccinations in Parts of Brooklyn, New York Times (Apr. 9, 2019).
    17. A.J. Wakefield et al., Ileal-lymphoid-nodular Hyperplasia, Non-specific Colitis, and Pervasive Developmental Disorder in Children, 351 The Lancet 637–41 (1998).
    18. The Coll. of Physicians of Phila., Do Vaccines Cause Autism? The History of Vaccines (Jan. 25, 2019), https://www.historyofvaccines.org/content/articles/do-vaccines-cause-autism.
    19. Biestek, slip op. at 20–22 (Gorsuch, J., dissenting).
    20. Biestek, slip op. at 11 (majority opinion).
    21. Id.

Recommended Citation
Richard J. Pierce, Jr., Response, Is the Court Encouraging Agencies to Rely on Junk Science? Biestek v. Berryhill, Geo. Wash. L. Rev. On the Docket (Apr. 23, 2019), https://www.gwlr.org/is-the-court-encouraging-agencies-to-rely-on-junk-science-biestek-v-berryhill/.

About Post Author