Academic Research Controversies

In early June 2012, investigative reporter Michael Patrick Leahy at Breitbart.com[1] revealed a scathing critique of Warren published by highly regarded Rutgers Law Professor Philip Shuchman which was uncovered in the 1990-1991 edition of the Rutgers Law Review.[2]

The 60-page analysis concerned a book Warren co-authored, As We Forgive Our Debtors: Bankruptcy and Consumer Credit in America.  Shuchman wrote:

Most of their study replicates several earlier research publications. These are hardly mentioned. The writers make extravagant and false claims to originality and priority of research. There appear to be serious errors in their use of statistical bases which result in grossly mistaken functions and comparisons. Some of their conclusions cannot be obtained even from their flawed findings. The authors have made their raw data unavailable so that its accuracy cannot be independently checked. In my opinion, the authors have engaged in repeated instances of scientific misconduct. [emphasis added]

Professor Shuchman argued that Warren and her co-authors jumped to conclusions, proclaimed new findings which were not new, and most importantly, ignored or did not accurately reflect data.

On page 240 of the Rutgers article, Schuchman asserted that some of the finding reached reached conclusions which were  not supported by the data:

Burgeoning consumer credit and increased volatility, they say, are the two primary systemic factors that caused the “startling increases” in consumer bankruptcy in the 1980′s. But these increases are less than in other recent periods and are only about one-sixth more than the increases in business bankruptcies, which suggests that consumer credit is not the cause of the increase in personal bankruptcy filings. [footnotes omitted]

Professor Shuchman went even further (at pp. 243-244), and suggested that the data was presented in such a way as to preclude verification:

This book contains so much exaggeration, so many questionable ploys, and so many incorrect statements, that it would be well to check the accuracy of their raw data, as old as it is. But the authors arranged matters so that they could not provide access to the computer printouts by case, with the corresponding bankruptcy court file numbers, thus preventing any independent check of the raw data in the files from which they took their information. [footnotes omitted]

Leahy followed up with an examination of whether, as some had claimed, Warren and her co-authors were cleared of the charges made by Professor Shuchman:[3]

Twenty-two years later, Professor Shuchman’s charges of “scientific misconduct” against Elizabeth Warren and her co-authors remain publicly unanswered and unresolved. These unresolved charges associated with her first major book raise continue to raise questions that hang over Elizabeth Warren’s entire body of academic work.

The next three articles in this series will address the conduct of three institutions enmeshed in this scandal: The University of Texas, the National Science Foundation, and Harvard University.

Leahy looked into investigations conducted by The University of Texas, and concluded that the investigations were inadequate:[4]

The lack of serious scrutiny of Ms. Warren’s academic research has continued for the subsequent two decades. Questions about Ms. Warren’s empirical studies have not been fully explored, and specific policies she has promoted–in particular those that led in 2010 to the passage of the Dodd-Frank Act and its onerous Consumer Financial Protection Bureau–have been imposed upon the public.

Breitbart News is not alleging that Sullivan, Warren, and Westbrook engaged in scientific misconduct. We are, however, presenting evidence that suggests the 1991 investigation conducted by the University of Texas into the allegations brought by Philip Shuchman in his scathing sixty page review[5] of the book Sullivan, Warren, and Westbrook co-authored in 1989 was neither thorough nor exculpatory.

Warren’s research also has been criticized by law professor Todd Zywicki.[6] In September 2010, before Warren was a declared candidate for political office, Zywicki wrote[7] that Warren’s much publicized findings as to medical bankruptcies was suspect:

Consider Ms. Warren’s much-ballyhooed study on the alleged link among health problems, medical expenses and personal bankruptcy filings. Published in the February 2005 issue of Health Affairs, the report was timed to head off bipartisan bankruptcy legislation that was enacted later that year. Ms. Warren and her co-authors claimed that “at least” 46% of personal bankruptcy filings in 2001 (the year from they collected the data) were the result of “medical causes,” and that this represented a 23-fold increase over 20 years.

Both conclusions are extremely suspect. First, the study provided an implausibly broad definition of “medical bankruptcy”—including any filer who reported uncontrolled gambling, drug or alcohol addiction, or the birth or adoption of a child.

Equally dubious, the authors classified a bankruptcy as having a “major medical cause” if the individual had accumulated more than $1,000 in out-of-pocket medical expenses (uncovered by insurance) over the course of two years prior to filing—regardless of income, and even if the debtor did not cite illness or injury among the reasons for bankruptcy….

In contrast to Ms. Warren’s studies, a battery of analysis, including research done by the Department of Justice’s Executive Office of the United States Trustee (which oversees the administration of bankruptcy cases), and by David Dranove and Michael Millenson of Northwestern University, concluded that fewer than 20% of bankruptcies are caused by health problems or medical expenses.

Last year Ms. Warren and her co-authors were back with an even more dramatic study, in the American Journal of Medicine, timed to promote President Obama’s health-care reform law. Drawing on 2007 filings, the authors concluded that 62% of bankruptcy filings were the result of medical issues and that the odds that a bankruptcy had a medical cause had doubled between just 2001 and 2007. This study was also flawed.

After Congress made it harder for people to skip out on their debts in 2005, the number of bankruptcy filings plummeted. In 2001, the year Ms. Warren used for the first study, there were 1,452,030 personal bankruptcy filings; in 2007 there were 822,590. Even if we are to accept the methodologies of the two studies for the sake of argument, there were 670,838 “medical bankruptcies” in 2001 and 510,828 medical bankruptcies in 2007—a drop of 160,000 per year. Yet Ms. Warren’s article nowhere acknowledges that the absolute number of bankruptcies and purported medical bankruptcies declined.

Smilar criticisms of Warren’s methodologies and conclusions were made by writer Megan McArdle:[8]

Does this persistent tendency to choose odd metrics that inflate the  case for some left wing cause matter?  If Warren worked at a think tank, you’d say, “Ah, well, that’s the genre.”  On the other hand, you’d also tend to regard her stuff with a rather beady eye.  It’s unlikely to  have been splashed across the headline of every newspaper in the United  States.  Her work gets so much attention because it comes from a Harvard professor.  And this isn’t Harvard caliber material–not even Harvard undergraduate.

So I think it matters on two levels. One, it matters how we evaluate the work–and I’ve been disappointed at how uncritically some people I  really respect have been willing to accept the 2001 and 2007 findings.   Not because I don’t think that there are medical bankruptcies in the  United States; I do!  I think medical bills are certainly the primary  cause of some of those filings, though I don’t know how many, and I’ve  also been writing about bankruptcy long enough to know that assigning  any one cause to the ultimate financial meltdown is, in many cases,  impossible.  (If you have nice consumer goods and no health insurance,  does a car accident count as a “medical bankruptcy” or a budgeting  deficiency?  If you lived right up to the edge of your income, was a job loss, or your spending pattern, to blame?  How you answer these  questions depends on a large number of prior value judgments that are  hotly contested in our society.)

It matters that we get this stuff right.  I am among the majority who  would like to see bankruptcies reduced in this country, and we’re not  going to be very effective at that if we run around thinking we can cure 2/3 of them by putting a national health care system in place, when in  reality a third or less have any strong causal relationship with medical bills.  Obviously, this was also held out as an argument for PPACA,  making an implicit promise to the American people which I believe to be  false.

But it also matters because a large part of Warren’s prominence comes  from the fact that she’s an academic.  If she came from . . . well, the  sort of think tank that publishes this sort of advocacy science . . .  she would have considerably less glamor, and power.

And perhaps it mattes most of all because this woman is now under consideration to head a powerful new agency.  If this is how she evaluates data, then isn’t that going to hamper her in making good policy?  If we’re going to have a consumer financial protection agency, I want one that has a keen eye to the empirical evidence on consumer welfare–not one that makes progressives most happy by reinforcing their prior beliefs.

Megan McArdle[9] pointed out the weakness of making the causal connection on such a wide scale of medical bills and bankruptcy, since bankruptcy filers themselves did not identify medical expenses as the cause of the bankruptcies (emphasis mine):

Elizabeth Warren has another study out showing that medical expenses contribute to more than half of all bankruptcies–indeed, this time, it’s 70%, up from the 50% she found in 2001.

Now, it is possible that this is true. The fact that it seems to disagree with every other study I’ve ever read that is not authored by Elizabeth Warren, and also, the self-reports of the people in her study (only about a third of whom attribute their bankruptcy to a health problem) could just be a fluke. It doesn’t necessarily mean that it’s wrong.

Yet upon closer examination, it turns out that it is not just wrong, but actively, aggressively wrong. Warren and her co-authors have obscured important and obvious facts that call the integrity of the work into serious question.

The text itself raises a huge red flags. It’s hard to believe that more than half of people who have been pushed into bankruptcy by a medical issue don’t understand this fact. Perhaps they are not the brightest bulbs on the Christmas tree, but could it really be true that most people catapaulted into a financial crisis by their medical bills don’t even notice that health care expenses are their main problem?

McArdle elaborated on other allegedly misleading aspects of the study, and then followed up with:[10]

She’s a professor at Harvard, and the head of the Congressional TARP oversight panel.  This conveys a certain responsibility to present data in the most illuminating way, not in the way that will induce journalists to say things that aren’t true.

The post 2005 increase in bankruptcies isn’t being driven by medical bankruptcies.  It’s simply rebounding from what every single analyst at the time, including Elizabeth Warren, agreed was an unsustainable drop.

References

  1. ^ Michael Patrick Leahy, Warren Accused of 'Repeated Instances of Scientific Misconduct' Before Harvard HireBreitbart.com, Jun. 4, 2012
  2. ^ Philip Shuchman, Review of As We Forgive Our Debtors: Bankruptcy and Consumer Credit in AmericaRutgers Law Review, 1989
  3. ^ Michael Patrick Leahy, The Academic Scandal Elizabeth Warren and Harvard Don't Want You to Know AboutBreitbart.com, Jun. 11, 2012
  4. ^ Michael Patrick Leahy, University of Texas Whitewash of Elizabeth Warren Scientific Misconduct Charge Entangles UVA, UC PresidentsBreitbart.com, Jun. 25, 2012
  5. ^ DocStoc, rutgers-shuchmanDocStoc, Jun. 4, 2012
  6. ^ George Mason University School of Law, Todd J. Zywicki bio pageGeorge Mason University School of Law website, Jan. 19, 2013
  7. ^ Todd Zywicki, In Elizabeth Warren We Trust?Wall Street Journal, Sep. 30, 2010
  8. ^ Megan McArdle, Considering Elizabeth Warren, the ScholarThe Atlantic, Jul. 22, 2010
  9. ^ Megan McArdle, Elizabeth Warren and the Terrible, Horrible, No Good, Very Bad, Utterly Misleading Bankruptcy StudyThe Atlantic, Jun. 24, 2009
  10. ^ Megan McArdle, Why Elizabeth Warren's New Bankruptcy Study is So BadThe Atlantic, Jun. 5, 2009
Last Updated: January 30th, 2013