Over at the Woodrow Wilson Center’s New Security Beat, I have a new post about a recent Science article by several economists on the connections between climate change and conflict. That article, which featured a reanalysis of sixty studies, sought to unambiguously demonstrate the climate effects contribute to conflict across a range of time periods, levels of conflict, and countries (see my previous Duck post with links to other critiques). However, as I suggested, the lack of serious attention to causal processes and the lumping together of quite distinct phenomena under a broad umbrella missed the point that climate effects are likely to have heterogeneous links to conflict. Rainfall abundance, rainfall scarcity, extreme weather events, and temperature may operate through different processes and in some cases may help ameliorate conflicts rather than foment them. Now, there is a section of that blog post that ended up on the cutting room floor, and I wanted to try to get some of it out here, mostly to elicit feedback from more quantitatively inclined social scientists who might be able to weigh in and explain whether what they did was kosher. Hsiang and his co-authors make an interesting choice with respect to statistical controls. They suggest that if we include variables like income per capita as controls that might also be correlated with conflict, we run in to the problem that climate indicators themselves affect income and thus inclusion of income would underestimate the climate effect. Their choice, if I read their article correctly, is not to include those standard controls but to try to account for them through fixed effects, bothlocation-specific fixed effects and temporal ones. Here they write about fixed effects:
If different locations in a sample exhibit different average levels of conflict – perhaps because of cultural, historical, political, economic, geographic or institutional differences between the locations – this will be accounted for by the location-specific constants μ (commonly known as “fixed effects”). Time-specific constants θ (a dummy for each time period) flexibly account for other time-trending variables such as economic growth or gradual demographic changes that could be correlated with both climate and conflict.
The on-going floods in Colorado
By dumping in all the things we as political scientists care about into dummy variables, what can we learn about causal processes and the conditions under which climate effects lead to violence? They elaborate on their choice:
Nonetheless, some studies use cross-sectional analyses and attempt to control for confounding variables in regression analyses, typically using a handful of covariates such as average income or political indices. However, because the full suite of determinants of conflict are unknown and unmeasured, it is likely impossible that any cross-sectional study can explicitly account for all important differences between populations. Rather than presuming that all confounders are accounted for, the studies we evaluate only compare Norway or Nigeria to themselves at different moments in time, thereby ensuring that the structure, history and geography of comparison populations are nearly identical.
So, I get that they don’t want purely cross-sectional studies but they are not including just studies of single countries. I guess the fixed effects are doing the work to render a clean shot at isolating the climate effects in different time periods. But, by not including traditional statistical controls and solely relying on fixed effects, it occurs to me that we cannot know much about the relative role played by critical indicators associated with adaptive capacity that are likely to mediate between climate indicators and conflict. In another paper, Burke, Miguel, and co-authors defend the choice of fixed effects:
One potential downside of the fixed effects approach is that it is difficult to estimate the direct impact of other potential explanatory variables of interest (e.g. perhaps consistently democratic countries have less conflict). The advantage – overwhelming, in our opinion – is that our coefficients of interest are unlikely to be biased by either omitted variables or endogeneity.
I see a bigger downside. Understanding whether and how differential economic growth rates, infant mortality, and governance affect the likelihood of conflict are absolutely critical because these are the areas, outside of a dramatic climate change mitigation effort, where human agency can make a difference. Given that some of these indicators are also influenced by climate, I see that they want to isolate the magnitude of climate effects on conflict, but aren’t there other ways to deal with endogeneity other than fixed effects models like two-stage models or instrumental variables approaches? Maybe I just have a poor grasp of what fixed effects can do for you, but to me, it’s what bundled under the umbrella of fixed effects that are interesting and possibly the source of interaction effects and other substantively meaningful causal connections.
You make a very good point about losing the ability to get at some of the variables we care about. On the other hand, it doesn’t seem like we’re quite yet at a consensus that climate change is associated with violence in the first place. Personally, I think there is space in the literature for both approaches, but one advantage I see in the near term for the fixed-effect approach is that it allows us to simplify. Countries are obviously super-complex beasts and unless we know with some level of specificity (and can operationalize) the interactions we’re interested in, it may be best to set aside these concerns for now. In my dissertation I’m planning on using a dynamic panel approach which produces only one set of coefficients and essentially subtracts out any ineffable uniqueness of a location (using temporal first differences, or fractional differencing). The downside is that suitable data is hard to come by, but on the upside it still allows an analyst to take into account some cross-sectional comparisons.
It’s possible to standardize the rainfall or temperature data in a way that compares apples to apples (i.e., dry conditions in Nigeria now with comparatively wetter conditions last year) while still testing for other covariates of conflict: simply transform the rainfall and temperature levels for each country into deviations from their country-level means. This is exactly what the fixed effects estimator does, except it does it for all RHS variables. I do this in my work as a matter of course, though I often include fixed effects (Idean and I are cited in the Science paper) both for its statistical properties and, frankly, because peer reviewers want it.
I am with Hsiang et al. on this one. It seems to me that trying to isolate the interactive effect of governance and climate on conflict is pretty much hopeless in a time-series cross-section setting. There is just no way to cleanly identify the interactive effect, given how complex comparisons across countries are. Even if you include country fixed effects, governance is surely endogenous to both conflict and expectations thereof. Moreover, governance itself depends on other plausible modifying variables, such as income and culture. Traditional hypothesis testing using partial correlations from a regression can be insightful, but then the word “effect” should not be used. Correlation does not equal causation, and causal analysis requires identification.
For studying the modifying effects of governance on the climate-conflict relationship, I would have more faith in a micro study that looks at exactly how countries with different institutions respond to sudden climate shocks. Such a study could investigate specific causal mechanisms and test precise hypotheses.
It’s a mistake to assume this is an either/or decision. Using location
fixed effects appropriately controls for geography, but this does not
preclude additionally controlling for country income, inequality, etc.
The researchers would just need to be clear that these variables have to
be interpreted as values relative to the country means. In the authors’
defense, omitting these variables is unlikely to confound the results,
since climate variation (from the mean) is presumably random.
Nevertheless, it would be interesting to see how climate’s impact stacks
up against these variables, and whether there’s an interactive (or
mediation) relationship.
There is nothing to say you cannot include control variables in a fixed effect model, as long as they are not static. Static variables, though, can only get at cross-sectional differences and don’t tell us very much about how climatic shocks affect conflict behavior. Now, it could be the case that climate shocks have different effects in some countries versus others, in which case interactions are appropriate. I think the bigger conceptual issue is what Hsiang et al term “bad controls” (for example, not including GDP growth because climate may work through GDP). Some of the work being done on causal mediation tests can help account for this, although it is hard for me to think about all potential causal pathways from climate through to conflict.
There is nothing to say you cannot include control variables in a fixed
effect model, as long as they are not static. Static variables, though,
can only get at cross-sectional differences and don’t tell us very much
about how climatic shocks affect conflict behavior. Now, it could be
the case that climate shocks have different effects in some countries
versus others, in which case interactions are appropriate. I think the
bigger conceptual issue is what Hsiang et al term “bad controls” (for
example, not including GDP growth because climate may work through GDP).
Some of the work being done on causal mediation tests can help account
for this, although it is hard for me to think about all potential
causal pathways from climate through to conflict.