Today Vox had an article about a new study of health insurance in America that annoyed me in a number of ways, and highlighted both Vox’s patronizing know-it-all style, and the simplistic economics-worship of some of its writers (in this case Ezra Klein). It’s one of those Economics-101 “Yes, you think A but really this [insert shallow confounder] means not A! Wow!” arguments that are beloved of pop economists, and it’s really frustrating to see it being trotted out now by Vox at a time when the USA is going through a major ideological battle over universal health coverage.

This post may turn out to be a bit long and kind of technical, depending on how frustrated I get reading further on the topic as I write …

The basic argument

Klein has taken a new working paper by Finkelstein et al and used a few of its apparently central findings to build up a story around a question. Finkelstein et al analyzed the Orgeon Health Insurance Experiment to find out how much money medicaid recipients were willing to give up in exchange for medicaid, and how much utility they get from their health insurance. As part of this they found that the uninsured actually don’t pay for much of their treatments: only 20% of their out-of-pocket expenses are paid by them, the rest being shouldered by someone else. This is a central part of Klein’s discussion and, in my opinion, a terribly uninformative finding. Klein has a whole section of the article about how the uninsured are actually “kinda-insured,” which is kinda-true but also kinda completely misses the point, in a very important way that, in my opinion, says a lot about the reasons Americans are having so much difficulty with this whole universal health coverage (UHC) thing. He then moves on to a discussion of the findings of the original Oregon Health Insurance Experiment paper in the New England Journal of Medicine, which found that medicaid wasn’t actually that good for a lot of its recipients; there is a lot wrong with this paper and a lot of reasons its findings need to not be over-stated, but Klein doesn’t really consider them, and gives the study findings more weight than (in my opinion) they deserve. He then goes on to one of those discussions that only economists have, which I guess they expect the rest of us to take seriously, that are deeply poisonous in their basic assumptions, and often wrong: “is health insurance worth it?” This is like the classic economics paper on why voting is a waste of time: superficially appealing but absolutely and completely wrong. He finishes with an important statement, that health insurance should be assessed in terms of the value it offers people, but then juxtaposes value with cost-control as if the two things are mutually contradictory. Pretty much everything in this Vox article is superficially right but deeply wrong, and I want to talk about why it’s wrong and what this means for the health insurance debate.

The Oregon Health Insurance Experiment

The Oregon Health Insurance Experiment (OHIE) is at the heart of the findings in this Vox article, but it’s probably not something we should put too much faith in. Basically, the Oregon state government expanded medicaid places a few years ago, but it could only expand to 30,000 so it ran a lottery for the 70,000 potentially eligible people. The 30,000 potentially eligible people then applied for medicaid, with many getting rejected, and Finkelstein cunningly convinced the government to let her study the results. This is a joyous opportunity for health insurance research because it offers a randomized controlled trial (RCT) of access to health insurance: the gold standard of medical research, enabling us to eliminate a whole bunch of confounders and explore only the effect of health insurance.

Unfortunately there are many problems with the Oregon Health Insurance Experiment and the original paper which launched it to fame. First and foremost, although 30,000 people won the lottery, winning the lottery only increased the probability of accessing medicaid by “25 percentage points” because many didn’t apply or were ineligible, and many non-winners somehow finnagled their way into medicaid. Thus the “Experiment” suffers from massive contamination of the kind that usually renders an RCT ineligible for publication, because most of the intervention group ended up as controls and some of the control group ended up as interventions. While the process of assignment to these two groups was random, the process of transition between groups and final allocation was not, and in fact is decided by a very clear set of factors with a high risk of confounding, such as age, unemployment, etc. The second big problem with the OHIE is that the follow-up period was only 2 years, but lottery winners went on a waiting list, so the actual follow-up time from starting medicaid to study end was less than 2 years, but many of the outcomes they studied (blood pressure awareness, treatment and control, for example) require long follow-up, and key outcomes such as financial catastrophe (see below) are dependent on much longer follow-up times and/or retrospective analysis. Note that the non-medicaiders received a full 2 years follow-up, another minor source of bias. The third problem is that many lifestyle and consumption variables that are crucial to understanding the home-financing impacts of health insurance were obtained from a mailed survey with 15,500 respondents (out of 70,000 in the original study!), one of the most infamous ways of introducing bias into studies (respondents to mail surveys are even less normal than you, dear reader(s)). In contrast, surveys of health financing issues in developing nations (which in my opinion are the gold standard of health financing surveys) routinely get 95-98% follow-up in detailed, complex door-to-door interviews. I have said before on this blog that I think American health finance researchers could learn a lot from what the developing world is doing, and this is another example. The fourth problem is the choice of outcomes: even in systems that are completely free (such as the NHS), health outcomes that can be analyzed over just two years of follow-up are heavily dependent on health-seeking behaviors and non-financial access barriers (e.g. work and time off), and the best measure of health success in a health insurance plan is in serious but often rare outcomes – all-cause mortality, hospitalization, that sort of thing. Also, the OHIE didn’t do much analysis of financial outcomes, which are the main point of health insurance programs. Finally, the study is only ethical if you squint and tilt your head: randomizing people to receive health care is not ethical, and the only reason this study gets grace on that count is that America’s system is insane, but the general ethical view of the medical establishment is that just because the state does something convenient, that doesn’t mean it’s ethical to participate in studies of that thing (see e.g. debate in the British Medical Journal for the Godwin-level examples). Regardless, most people accept the validity of the OHIE, so let’s run with it for now, bearing in mind its flaws: flawed papers often still have a lot to tell us.

The uninsured are “kinda-insured”

In my view the central flaw of the Vox opinion piece lies with its uncritical acceptance of the working paper’s finding that only 20% of expenses were paid for by people without insurance, and the implications of this. The Vox article states:

It’s perhaps easiest to explain this through example. Imagine John breaks his leg. If John is uninsured, his brother, Mike, pays for his medical care. But if John has Medicaid, then the government pays for his care. John got medical care either way. So in this case, Medicaid’s money actually didn’t go to John so much as it went to his brother, because it was his brother who actually would have ended up paying the tab.

This is the kind of superficial gotcha that economists like Ezra Klein love, and it’s annoying and … superficial. There is a large body of research on the health financing aspects of health insurance, and a key concept used in that literature is distress financing. In developing nations, distress financing is defined variously as using any of the following strategies to pay for medical care: selling assets from the home or family business, using savings, calling on family members for financial support, or withdrawing children from school to work [yes, you read that right: this is what lack of health insurance does]. What John did was distress financing, and one of the goals of universal health coverage is to reduce or eliminate the incidence of distress financing. Sure, Mike is better off if John gets medicaid, but in health financing we don’t care about Mike, Tom, Dick or Harry: we are designing a system that protects John from financial catastrophe and distress financing. This is because it is of no interest to us if Mike spends his money on a plasma-screen tv or his brother’s appendix or indeed his own, the purpose of health insurance is to pool risk, that is to ensure that no person – whether directly afflicted or not – has to spend unexpected amounts of money on health care. No doubt there are people out there whose monthly premiums are paid for by friends, sugar daddies or family. We don’t care. The important point is that we have established a universal risk pool into which everyone pays, and everyone draws. It’s no concern to us whether Mike pays for John or John pays for John or John’s sugar daddy pays for John, and typically health insurance research doesn’t ask about how premiums are paid, so why should we care how out-of-pocket expenses are paid? So Klein’s example completely misrepresents the moral purpose of health insurance, by assuming the wrong things about why we have health insurance, and misunderstanding the tools that are available to understand how health insurance works.

I also think Klein has misunderstood the working paper on this issue, because I don’t think the working paper makes as big an issue of this distribution of costs as he does. Finally, if John and Mike are sharing the cost of their health care, then really what’s happening there is that they are establishing a very inefficient, unregulated risk-pooling mechanism – a private version of medicaid. When John gets medicaid we aren’t seeing a situation where suddenly Mike is better off because John can pay for his own care, we’re seeing a situation where Mike is better off because John has been drawn into a larger, better-managed, better-regulated risk pool.

Estimating the utility of health insurance

The working paper is largely aimed at estimating the utility of health insurance, and it uses techniques from economics that I’m definitely not qualified to critique. I know nothing about utility functions or their optimization, so a lot of the language and techniques are a mystery to me. However, there seem to be a couple of aspects of their analysis that insert strong biases. For starters, their assumption 3 on page 8:

Individuals choose m and c optimally, subject to their budget constraint

which is explained as:

The assumption that the choices [of some functions] are individually optimal is a nontrivial assumption in the context of health care where decisions are often taken jointly with other agents (e.g., doctors) who may have ddifferent objectives and where the complex nature of the decision problem may generate individually sub optimal decisions
This assumption ignores the possibility that individuals choose not to consume health care, a common problem amongst the uninsured. It’s also a particularly dubious assumption about the poor, who are often not able (through resource constraints and e.g. work situations) to make optimal decisions. A good example of this is abortion: the welfare-maximizing decision might be to have a legal abortion, but there are many states in the USA where this is becoming increasingly difficult for poor people to do for non-financial reasons (travel requiring time off work, the risk of humiliation in small towns, etc.), while for the wealthy it remains a simple utility-optimization decision. A poor person might make the extremely risky decision to have an illegal abortion, which has stochastically-varying risks (mostly none, occasionally many). It’s not enough, in my opinion, to talk about this as a limitation: it needs to be carefully modeled.
The working paper also uses an unorthodox method for assessing income, basically dividing household income by family members[1], and doesn’t consider the issue of disposable income. Typically studies of this kind use the family’s disposable income (or some similar measure of available consumption) during analysis, because people have other fixed expenses (most especially, a house) that they can’t fiddle with.
As a result of these assumptions and estimation processes the working paper comes up with a finding that individuals would be indifferent to giving up medicaid or consumption of about $1000 – $1500. This seems to be actually an astounding finding, given that average income in the people receiving medicaid is $3800. Would you give up just under half of your income for health insurance? Is this an indication that the health insurance is of low utility, as Ezra Klein concludes? Note also that there is no assessment here of financial catastrophe, which is important because these people only need to spend about $700 a year on health care to be in the catastrophe zone (usually about 25% of disposable income, which seems to be about $2800 in the assumptions of this study, though I may have misunderstood it). In order to understand the benefits of health insurance properly in this community we need to understand what their risk of financial catastrophe and distress financing is and what proportion of that risk they are protected against by medicaid; but we are instead treated to a completely irrelevant estimate of what amount of money they are “indifferent to”, based on income and expenditure information from a very small sub-sample of the people originally eligible for the trial.
I’m not convinced that the OHIE is capable of answering the questions we need to know about health insurance coverage, or that this study adds anything except to tell us that poor people will use up to nearly half of their income to purchase health insurance.
Vox’s conclusions
Klein concludes that maybe health insurance isn’t that great, and we need to make it more appealing, or something:
That isn’t to say health insurance is useless, or that medical care doesn’t help. But we’re probably paying too much and getting too little, and now that we’re a lot closer to a world where every American who wants health insurance can afford it, we should be focusing on making sure that all that health insurance we’re buying is actually delivering the health we’re expecting.
Much of this paragraph is based on the published NEJM paper from the OHIE, which as I showed above is not very informative about the health benefits of health insurance. The subsequent working paper hasn’t told us much about its health insurance benefits either, because it was misdirected. So how can Klein conclude from this study that we’re probably paying too much and getting too little, and why would anyone conclude that from medicaid, which is a specific system for the very poor in America? The reasons why medicaid is ineffective are probably closely related to social determinants of health; the reason why standard health insurance plans (or the Obamacare bronze plans) are ineffective probably has a lot more to do with access issues, arbitrary payment systems, and high overheads. This seems like really weak sauce to conclude with, and as a remarkable economic finding it also seems kind of empty. If you went out to buy a plasma-screen TV, I’d tell you to find the best one you can for the lowest price you can, and definitely make sure it works. Klein’s conclusion in this article is that … the community should buy the best health insurance it can for the lowest price, and make sure it works.
So somehow Klein has gone from a gotcha based on a flawed study (oooh look! you thought health insurance works but this study showed it doesn’t!) to saying … we need to make sure we spend money on health insurance wisely. While the rest of the world continues with its process of spending less money than the USA on health insurance, and getting better results.
It’s not really a very helpful conclusion is it?
What is the relevance of this to America’s health debate?
Assuming Vox has any relevance America’s health debate, or to anyone anywhere, that is. This whole article seems to me to be a representation of the strange atmosphere of debate about health insurance and health care in America. First of all it is a discussion of a set of studies trying to find out whether health insurance works, something that the rest of the world takes for granted. Secondly, it buys into a strange economists’ logic of who benefits from health insurance that is almost completely ignorant of a large body of research on health insurance outcomes, and also seems to see health insurance as a consumer good rather than a risk-pooling strategy – i.e. it frames the entire health insurance debate in terms of something that people want to buy, rather than something society thinks everyone should have. It’s another example of how America’s intellectual elite seem to be really clueless when it comes to health care, and it’s a real worry that a site that is supposedly informative is publishing articles by a major economic pundit about one of America’s central social reform issues that are largely clueless about the central debates in that issue. How is the general American public meant to understand a fractious, long drawn-out healthcare debate if public intellectuals like Klein are missing the key issues and presenting the framework of that debate in a completely erroneous and misleading way? Healthcare policy is far from simple and there’s no reason to think ordinary people should understand it without help, but here we have a major public intellectual and economist completely misrepresenting the core elements of the debate, running his readers off down the wrong track into a loopy set of conditions on health care (and a really weird definition of who benefits and who we want to benefit) before ending with a completely uninformative and vapid conclusion (we need to buy more for less!) Is this really the standard of public debate on healthcare in the USA?
Let’s hope the Supreme Court don’t read Vox, and be glad it wasn’t around when Obamacare was first developed!
—
Fn1: The correct method is to scale the household size to consumption equivalents using a power law, the value of the power being estimated from a regression model: for the USA the consumption-equivalents scale as approximately the square root of household size. This is perhaps not a very important flaw in the paper but it points to a bigger flaw: none of the standard experts on health financing from the broader health financing field are referenced and Ke Xu, the world-recognized expert on this, is not included in the reference list. Once again: researchers in the USA could learn a lot about the best methods to study health financing from those who are doing very serious work on UHC in developing nations.