So Charli — who did, in fact, attend our ISA Battlestar Galactica panel in costume — has posted elsewhere about the panel. Though she calls it the “best event [she] attended at ISA this year,” she does express some disappointment about the content of the panel when compared with the panel’s title:

Unfortunately the panel turned out to be misnamed however, for none of the papers really spoke to the question of whether BSG has an impact on actual world politics. … Admittedly, the papers weren’t really trying to do that kind of explanatory work – the panel really was misnamed – so this isn’t a criticism as much as an observation.

Certainly, as one commentator over at LGM has already pointed out, there’s sufficient ambiguity in the word “explains” that it need not quite mean “influences” or “impacts.” But what’s intriguing to me, methodologically speaking, is that Charli, along with not a few others in our field, tend to go directly from “explain” to “impacts,” and in particular, to “exerts an independently-measurable causal impact on.” That says something revealing about the field, and it might just help to explain why the study of popular culture continues not to make a lot of headway in the disciplinary mainstream.

Some definitions first. In Charli’s formulation, for BSG to “explain” world politics, the show or the viewing of the show has to function as an independent variable, which means that in order for it to matter it has to be shown to be correlated in some relatively robust way with a measurable outcome. In that way, BSG could “explain” observed variance, perhaps best visible if we compared a BSG-watching-and-discussing national or international security bureaucracy with a non-BSG-watching-and-discussing one, or a BSG-watching public with a non-BSG-watching one. This does set up some tricky measurement problems — how do we count watching and discussing, how do we specify the intervening steps between the show and the outcome — and raises the specter of spurious correlation all over the place, but in principle there’s nothing altogether impossible about conducting a study like this. The challenge would be to isolate the impact of BSG, which would probably require a more elaborate theoretical account of human perception than we have at present; it’s one thing to see people in their office cubicles discussing BSG and then writing counter-terrorism policies, and another to demonstrate that their discussing BSG exercised an independent effect on the subsequent policies.

[And parenthetically, even if BSG were subsequently cited — either in the final policy statement, or in interviews after the fact — that would tell us precisely nothing except that some people who participated in the process thought and think that BSG was important. That doesn’t make them right for thinking so. If, hypothetically, BSG was referenced when the policy was concluded and presented, we’d have a heck of a time determining whether this was a strategic rhetorical move designed to appeal to the audience (think of Reagan’s use of the phrase “star wars” here) or some kind of an expression of a sincere belief, Similarly, if people interviewed later on cited BSG, or 24, or some other TV show as influential in their thinking, we’d have a heck of time distinguishing between a) strategic re-presentation of events in the light of contemporary concerns; b) mis-remembering what happened, perhaps benignly and perhaps more craftily; and c) actual influence. So that kind of primary source evidence isn’t enough, in any case, to establish that BSG had an effect; instead, we need some hypothesis about what viewing and discussing BSG would imply behaviorally, so that we could look for the appropriate kinds of indicators not in what people said about why they did what they did, but in what they actually did.]

But fortunately for the rest of the panel, “explain” doesn’t just mean this kind of neopositivist methodological strategy. Indeed, it’s even something of a misnomer to say that an independent variable “explains” an outcome; strictly speaking, the whole statistical model explains the outcome, and the various independent variables participate in or contribute to the explanation — or, perhaps, they explain a certain portion of the observed variance. So if we wanted to be precise here, even if we were following Charli’s neopositivist implicit definition of explanation, we would have called the panel “How Battlestar Galactica Contributes to the Explanation of World Politics.” This inelegant reformulation does, however, permit a number of things to fit more easily underneath its umbrella, including the paper that I presented on that panel on “Battlestar Galactica as Methodology” in which I suggested that social scientists could learn something from BSG about the way we construct our explanations of world politics. That would not be BSG as causal factor — truth to tell, I think that the measurement and conceptualization problems with the BSG-as-IV thing that Charli proposes are actually insoluble, and pretty much everyone doing audience-reception studies agrees that drawing an independent causal connection between something that you see on TV and something that you subsequently do is pretty much impossible — but BSG as inspiration, BSG as an exploration of a certain ideal-typical formulation of values as they confront a variety of fantastic empirical situations, and BSG as a model of how social scientists ought to analyze concrete cases in terms of how the logical implications of a given set of values are or are not realized in practice. (My remarks on the panel were recorded; podcast here, if anyone wants to check them out.)

But the bigger issue here is not whether Charli or I are correct about what to do with BSG, especially since I don’t think that there’s anything illegitimate about either of these methodological strategies. The issue is that we don’t have a very good lexicon in IR for discussing what to do outside of the hypothesis-testing, IV-DV kind of explanation that Charli is proposing. I worry about that for a lot of reasons, not in the least because I think that a claim about the independent causal impact of popular culture is kind of foredoomed to failure and I like studying popular culture (among other things) because I think it does matter — but it “matters” in a different way. I have written a book trying to address this situation, and I want to do something else about it too: a new department here at the Duck that I’m calling “Methodology411.” I will post something launching that department sometime in the next couple of days, when I will also explain where the name comes from.

But for the moment, let me just post an excerpt from an e-mail exchange that Charli and I had about a slightly different point concerning methodology and BSG, by way of showing that the number of methodological issues involving the study of things like popular culture is actually quite a large one, and gets larger once you plunge into actual empirics. Food for thought, and perhaps, for subsequent discussion.

Charli: I have been thinking more about your argument re. humanity using the example of the moment with Helo. The more I think about it the more I don’t buy it. I remember the problem with that scene was that Helo is in a love relationship with a Cylon and his child is Cylon. I think this really dilutes the moral strength of his argument as he appears in the episode to simply be promoting his own self-interest – not arguing in favor of sacrificing self-interest on moral ground to include “the other.” For him, Cylons are part of the “in-group” already, so it’s not exactly a hard case…

PTJ: Your analysis here makes two unfortunate conflations — conflations that put your objections on a page quite different from my original claim [from the panel]. Substantively, you conflate an analysis of Helo’s motives with an analysis of the social meaning of Helo’s actions, and methodologically, you conflate an analysis of the characters in BSG with an analysis of BSG as a whole narrative product.

Substance first. In your objection you have subtly shifted the question from the terms of Helo’s argument to the reasons that he might have for making it. This is ‘reductionism’ in the precise terms that Ken Waltz meant it: you take the social and reduce it to the individual, as though social action were merely the aggregate of individual behavior. But social action is meaningful, which means — by definition — that it includes a component that is not reducible in this way, since it involves shared sensibilities. ‘The social’ is not just a bunch of individuals, in the formulation I’m using; logically speaking, the social *precedes* the individual, and in fact ‘the individual’ is a site or a node in a structured social network. Why one individual does one particular thing, and how that thing was possible in the first place, are different kinds of questions, and they don’t preclude one another; Helo might indeed have been motivated by the kinds of concerns that you suggest motivated him, but that is strictly speaking irrelevant to my argument. ‘Sincerity’ is not an empirical phenomenon, but a normative judgment, and it doesn’t matter what Helo was really thinking; what matters is what he said (the socially sustainable vocabulary that he used), how people made sense of what he said (which might involve *their* judgments on his sincerity, but those would tell us precisely nothing at all about whether or not Helo really was or really was not sincere), and what resulted from his intervention. And in those terms, what’s fascinating about Helo’s action is that it might have involved a claim that the non-human Cylons had, in some sense, humanity — that they were what Orson Scott Card (in his brilliant novel Speaker for the Dead) would call “ramen,” or humans of another species. Motive doesn’t matter; socially meaningful action does.

Now, methodology. You seem to be focusing on this scene in isolation from the rest of the text, as though BSG as a whole could and should be treated as a novel set of empirics to be analyzed in the same way that we might analyze the historical events of our ‘regular’ world politics. But BSG is, of course, *fictional*, which to me means that it has a necessarily world-constituting quality that actual historical events don’t necessarily have. When we analyze world politics, when we are engaged in producing social-scientific accounts of world politics, we are of necessity treating world politics as the raw material from which we produce our accounts. There’s no ‘authorial intent’ to worry about, and no pre-existing plotline save the one that our methodology and our analysis helps to disclose. BSG is different: not because what Ron Moore and company think about BSG should necessarily be controlling for our interpretations, but because the story is a *story* and as such has a plot of its own. Ignoring that — which is what modern-day realists do when they mis-read Thucydides as advocating the Athenian philosophy “the strong do what they can and the weak suffer what they must,” since Thucydides then goes on to depict the tragic consequences of this bit of hubris as the book unfolds — is problematic. When I say that BSG’s value-system is critical humanist, I do not mean that any particular character is a critical humanist (although some are or become so, including Helo, and this scene is a pivotal moment when he consolidates that position), but that the series as a whole expresses critical humanism. That expression is sometimes found in dialogue, but it is more often found in plot and outcome, especially the show’s relentless demolition of every particular definition of ‘the human’ and its replacement by something even more encompassing — culminating, of course, in the revelation of ‘mitochondrial Eve’ as a Cylon/Colonial hybrid.