One of the few items recently that has caused me to emerge from my nothing-but-Friday-nerd-blogging temporary hiatus was this article on civilian war deaths by Michael Spagat and his collaborators. I wrote a post with some praise and some questions, and recently received a thoughtful response by email from Michael and his crew in which they further detail the coding methods used in the project. Since the original thread generated some interest, I’ve decided to post their response here.

Civilian Targeting Index Clarification
by Madelyn Hicks, Uih Ran Lee, Ralph Sundberg and Michael Spagat

Since publishing our paper in PLoS ONE on the Civilian Targeting Index we have received some interesting feedback both in emails and on the “Duck of Minerva” blog concerning the nature of the one-sided violence data we use from the Uppsala Conflict Data Program (UCDP). In particular, some readers wonder how solid the underlying evidence of intentionality can really be for incidents coded as one-sided violence (i.e. ’civilian targeting’). We would like to take this opportunity to clarify this important coding issue in some detail.

First, here is a short general discussion. UCDP coding of ‘deliberate’ or ‘intentional’ civilian targeting is not a judicial assessment (e.g. of manslaughter or murder), nor is it an attempt to ‘know’ a perpetrator’s motivations. Instead, UCDP coding methodology assesses whether particular conflict-related deaths were likely to be one-sided or battle-related based on a combined review of: the plausible target, the method by which a killing was carried out, presented evidence, and credible statements or attributions of guilt. In each situation of violence the human coders, using what evidence is at hand, first attempt to identify a likely target. Coders also consider what method of attack was used (bombing, shooting, IED, etc.).

It is from this analysis of available information that a coder draws a conclusion regarding whether this was a likely ‘intentional’ one-sided death.
Often Uppsala infers the intention to kill civilians – and only civilians – from the absence of a possible military or conflict-related target. Conversely, the presence of such a target will be sufficient to force a coding of battle death (rather than one-sided death). This means that the bar is set fairly high for classifying deaths as one-sided since for many incidents there will be plausible military or conflict-related targets. Thus, many killings that could in reality have been largely intentional will, nevertheless, be classified as battle deaths because the available evidence will not be considered strong enough to code these death as one-sided.

Note, however, that there are some coding outlets for expressing uncertainty over intentionality. For example, when there is weak evidence of intentionality, deaths can be placed into the “high” category of one-sided deaths rather than into the “best” category. Such coding might be reconsidered later as further evidence comes to light. For example, in Burundi some such deaths were later transferred from “high” to “best” as new evidence became available that these events were really massacres.

We now turn to specific examples of coding in practice, focusing on the borderline between one-sided (i.e., intentional civilian targeting) fatalities and battle deaths. First, in many cases, the correct coding is fairly obvious due to the method used in killing, as reported by credible sources. For example, many one-sided deaths were attributed to the Lord’s Resistance Army (LRA) based on a Human Rights Watch Report (“The Christmas Massacres”, February 16, 2009) that contained graphic descriptions such as the following: “LRA combatants hacked their victims to death with machetes or axes or crushed their skulls with clubs and heavy sticks.” In Colombia there were many incidents of armed groups entering villages and then massacring people by shooting them in their heads at close range. In Iraq there were numerous incidents of bodies of executed individuals found, often with their hands tied behind their backs, shot in the head and bearing marks of torture. Such killings are unambiguously one-sided deaths.

The Iraq example does, however, raise an important point about UCDP coding which everyone should bear in mind when interpreting the data; the coding scheme only admits deaths that can be attributed to particular armed groups. Thus, many execution deaths in Iraq (and elsewhere) are not included because the perpetrating groups are not known. (These deaths can be added later if the identities of perpetrators are discovered.) Of course, the requirement of identifying perpetrators limits the coverage of the UCDP data. More importantly, it affects the tallies for some groups differently than it affects others. This biases the data and we need to think through the implications of these biases when we make interpretations. It is, for example, worth knowing that the Taliban often claim credit for executions of teachers and unmarried couples while groups in Iraq typically execute their victims anonymously.

Of course, many events are much more ambiguously situated on the battle death/one-sided death borderline than the ones described so far. For example, IED-caused deaths of civilians are coded as battle deaths because one cannot rule out the possibility that the intention of the perpetrators was to attack NATO or Afghan forces. Mortar fire is treated similarly; since mortars are hard to control it is difficult to conclude that a mortar was intended to hit civilians only even if it does only hit civilians. Fatalities from rockets fired from the Gaza Strip and southern Lebanon are coded as battle deaths for the same reason. Civilian deaths in aerial bombings, which usually have plausible military objectives, are also normally coded as battle deaths unless there is clear evidence of intentional targeting of civilians. Checkpoint killings in which soldiers fire on an approaching vehicle after going through a sequence of warnings will usually be classified as battle deaths based on statements of soldiers that they believed they were firing on hostile individuals inside the attacked vehicle, and in the absence of evidence to the contrary.

It is clear from the examples of the previous paragraph that the battle death/one-sided death distinction, and hence the Civilian Targeting Index (CTI), is of no use for analyzing the reckless endangerment or indiscriminate killing of civilians, as we point out in our paper. Other tools, such as the Dirty War Index [Hicks and Spagat (2008)] are required for this purpose, with a recent example of measuring indiscriminate effects on civilians in Iraq [Hicks et al. (2011)].

Another important point is that UCDP systematically reexamines incidents as new evidence appears. For example, there was an incident in Afghanistan in March of 2007 in which US Marines killed 12-19 civilians following ambush. Those civilians were killed as the Marines sped off from the scene of the ambush, peppering passing vehicles with gunfire even though they were already clear of danger. When a US military court ruled that the soldiers had used ‘excessive force’, these deaths were transferred by UCDP from battle deaths to one-sided deaths. New evidence also comes from bodies like truth commissions, forensic investigations, court proceedings (including tribunals for the former Yugoslavia, the ICC, military courts and criminal courts) and NGOs. Note the wide range of sources that UCDP brings into play in compiling its data. The UCDP data are not based only on media surveillance as many people seem to believe.

Soon UCDP will start releasing its data at the incident level. At that stage, anyone will be able to inspect the data incident by incident and develop a much stronger understanding than we are able to provide in this short comment. Moreover, people will be able to question incidents and to argue that codings should be changed. Thus, over time UCDP data should improve while we simultaneously improve our understanding of its strengths and weaknesses.