Last week, a new study by Janet Currie, Michael Greenstone, and Katherine Meckel, published in the journal Science Advances, examined the effects of shale gas development on the birthweight and health of newborns in Pennsylvania. The study has received substantial attention, including widespread media coverage and critiques from industry advocates.
Currie et al. make a valuable contribution to the evolving literature on the potential health effects of oil and gas development, presenting a high-quality study with a large sample size and some compelling, statistically significant findings. However, there are a number of reasons why these results should not be seen as a “red flag” providing definitive links between health impacts and shale development. In general, pinning down any health effect from industrial activity such as shale development with fracking has been difficult. Plus, the types of data and analysis used in this study (and many others) do not identify the mechanism—such as air pollution from truck traffic or sleep disturbances from noise and light pollution—through which shale gas development impacts newborn health outcomes. Thus, even if the link between health impacts and fracking were definitive, the policy implications for mitigating these impacts would not be. Without knowing what causes the health impact, policymakers and companies cannot take action to reduce the risks. In sum, we believe this study and others like it should be interpreted as a “yellow flag,” indicating real concerns, but also highlighting the need for additional research.
Understanding the Study
The study uses confidential Pennsylvania data to link each mother’s location, personal characteristics, and birth outcomes—birth weight being the most prominent—to data on the date and location a gas well was spudded (the time at which drilling began). One of Currie et al.’s most significant contributions to the literature—its attempt to control for the characteristics of mothers—surprisingly sets the stage for substantial confusion over its interpretation.
The key issue for a study like this is controlling for factors such as a mother’s health or her choice of housing location. If these elements are not properly accounted for, the study may otherwise falsely link newborn health with natural gas development. We would not know if the differences in health outcomes for newborns are actually driven by the health effects of shale development or the differences between mothers who live or move closer to wells and those who do not. Lower housing prices near wells have been observed, and could attract mothers with lower socioeconomic status.
The authors address this challenge in two ways:
- They compare birth weight and other characteristics of babies conceived near shale gas wells to those born farther away both before and after drilling began at each site. They control for the observable characteristics of mothers by including variables for the mother’s educational attainment, age, marital status, and other socioeconomic characteristics (notably, smoking status, alcohol consumption, and income are not available).
- They compare siblings with the same mother who were conceived before and after the spud date, including a constant term for every mother, which should control for any remaining unobservable characteristics of the mother.
The first approach yields significant results, finding negative health impacts for in utero babies exposed to shale development living within one kilometer of a well. Such babies tend to be about 39 grams lighter than those who are not exposed to shale development, a difference of about one percent. This tiny effect may be masking a large effect on lighter babies, as the average baby within one kilometer is 25 percent more likely to be born with low birth weight (given the standard definition) if conceived after the spud date.
However, the second approach—which is arguably the better of the two because comparing siblings allows the mother to serve as her own control—finds insignificant effects (to see the difference, compare column 2 with column 3, which reports this second approach, in Table 2 of the paper). Yet the authors confuse the reader by highlighting the second approach in the paper’s abstract, then focusing their analysis on the results from the first approach.
When discussing the results of the siblings approach, the authors point to the small size of the treatment group—just 594 sibling pairs—as the reason for the insignificant results. The unexamined implication is that the statistically significant effects found by the first approach are in fact driven by the unobserved differences between mothers, but when the second approach controls for those differences, the results are statistically insignificant. These estimates are likewise often smaller than the statistically significant estimates in the other columns.
Another challenge for this study is its choice of proxy for measuring exposure to shale development. Because gathering data for every household’s air quality, water quality, and other potential exposure pathways is very difficult, all studies to date use proxies. Currie et al. define exposure to shale development as whether a shale well was spudded before the newborn’s conception. That definition, however, includes mothers living near wells that may have been drilled up to 10 years before.
This proxy, at a minimum, confounds the well development stage (from spudding to well completion) with the production stage. The drilling, fracturing, and completion aspects of well development are likely far more intrusive to mothers, and involve several thousand truck trips and dozens of diesel engines bringing noise, light, congestion, and air pollution to the well pads for 24-hour operations lasting weeks to a month at a time per well. The well development phase also includes flowback, a period when fracking fluids and potentially harmful gases return to the surface from deep underground.
As a result, Currie et al.’s proxy for exposure may in fact understate the impacts of shale development on birth outcomes, since the treatment includes the production phase, which is potentially less harmful than well development. Including a cutoff date for exposure, such as the completion date, could mitigate these issues, but may also reduce the treatment group to a small size unsuitable for statistical analysis. The exposure proxy may likewise understate the impacts of shale development because the control group includes babies that were conceived before the spud date but were still exposed to shale development in utero. Using mutually exclusive categories (with the unexposed group defined as born before spud date and the exposed group defined as conceived after the spud date) may provide better estimates for these impacts, though the sample size would become smaller as well.
Finally, Currie et al. include little discussion of the surprising result that smaller and less significant health effects occur between one and two kilometers of a well site compared with those between two and three kilometers (these results are apparent in Figures 3 and 4, as well as in Table 2 columns 4, 5, and 6). While the number of observations in the treatment is small and the distance categories are not mutually exclusive (in that a mother can be one kilometer from one well and three kilometers from another), the authors do not discuss the implications of this curious finding.
A View from Industry Advocates
In a widely circulated response to Currie et al., the industry advocacy group Energy in Depth (EID) published an article entitled “Why You Shouldn’t Believe a New Study Linking Fracking to Infant Health Problems.” EID regularly critiques studies that raise questions over the environmental and health risks of shale development, though—perhaps unsurprisingly—they rarely apply the same level of rigorous analysis to studies trumpeting the benefits of the shale revolution.
While EID makes one valid point regarding the authors’ findings for one and two kilometers (which we note above), other critiques are far less compelling. In one example, EID argues that the results should not be trusted because the baseline characteristics of mothers differ in ways that could bias the results. However, in the study, Currie et al. clearly and uniquely account for many of these differences in their regression analysis (presented in Table 2 of the study) by including controls that should negate any baseline differences in the educational attainment, marital status, and other socioeconomic differences between mothers.
Other arguments from EID are even more questionable, including a critique of the authors for, in other work, describing the air quality benefits from natural gas’s displacement of coal in the electric power sector. This criticism embodies the black-and-white nature of the debate over fracking (a theme explored in depth in the forthcoming book, The Fracking Debate), where EID fails to understand that shale development includes both benefits (e.g., health benefits from displacement of coal) and risks (health risks from living near well sites).
Concluding Thoughts
Despite our concerns over this paper, we disagree with the categorical dismissal of the study presented by EID. Currie et al. provide another valuable, if imperfect, data point in the evolving literature over the potential effects of shale development on the environment and human health. Because the research gold standard of randomized controlled trials is impractical (and would likely be unethical) in this context, it is unrealistic to expect studies to provide bulletproof causal linkages between shale development and human health.
Keeping up with the evolving literature on energy and the environment is a challenging task, made even more difficult when the results of a study are not presented as clearly as possible, either by the authors or by industry advocates. We hope this post helps to clarify what this study tells us, and what it doesn’t.
