It looks like the Obama administration has secured 42 votes for the Iran deal in the Senate, enough to filibuster even a vote, and despite today’s machinations in the House, the Iran Deal will likely go through. Indeed, when Republicans agreed back in May to a review process that would require a super-majority in both chambers to overturn the deal, the die was already somewhat cast.  Still, I’m thrilled that supporters have been able to hold the line in the face of a multi-million dollar campaign against the agreement. In the final days, we’re seeing a surge in efforts to get views on the table from supporters and opponents.

In a stock-taking exercise, Foreign Affairs released the results ($) of a survey of a “broad pool of experts” about whether Congress should approve the Joint Comprehensive Plan of Action (JCPOA) or the Iran Deal for short. 32 of the 52 experts — 62% – answered  “strongly agree” in support of Iran Deal. Adding in the “agree,” support rises to 72%.

The virtue of this survey is that individual respondents are on the record about where they stand. For many of them, there is a bit of explanatory text about their reasons. As the graphic above shows, they are asked to rate their level of support on a 5-point Likert scale ranging from “strongly agree” to “strong disagree.” They were asked to provide their confidence in their assessment on a 10 point scale, which, as my collaborator Craig Kafura noted, showed an extraordinarily high level of confidence in their judgments.

The public stance of individual respondents is important. Other magazine elite surveys will reveal who was surveyed but not their answers to particular questions so you can’t create a dataset to examine crosstabs between salient demographic characteristics and survey answers. Even anonymized versions of answers are almost never provided so you can read the write-up but not much more; the recent release of the revised Chicago Council elite surveys in which I participated is an exception.

Still, some of the attributes of this particular survey raise important questions about representativeness and who is actually being surveyed. I took the liberty of coding all the respondents’ responses, their gender, citizenship (to the extent this was easy to find), and I made some preliminary efforts to code partisanship (a Google doc is here, and if useful, I could crowd-source the partisanship field).

Congrats – You Have an (Almost) All Male Panel

Only 3 of the 52 — roughly 6% — respondents are women (including former head of the Carnegie Endowment Jessica Mathews, European-based Iran analyst Ellie Geranmayeh, and CNAS’ Elizabeth Rosenberg). Now, I know that international security and nuclear policy are male-dominated areas, but there were some obvious omissions of women expert in this arena, Cheryl Rofer and Kori Schake for starters, who could have been surveyed. To be fair, Foreign Affairs might have asked them or other women to participate and just had to go to press with whoever responded. That said, some of my concerns about the pool go beyond gender and raise other questions about how to draw inferences from surveys of elites in general and samples of convenience in particular. [Addendum: Foreign Affairs reached out to comment that the low response rates among women accounts for the final tally. They wrote: “In fact, we asked nearly a dozen women to participate in that survey–mostly actual Iran experts, fwiw–but only three of them responded.” The wider survey included people with deep expertise in nonproliferation or Iran, with a few prominent general figures of authority.]

I don’t mean to just beat up on Foreign Affairs on this sampling strategy, as the same series has published three pieces on elite surveys I carried out with collaborators (here, here, here, $). Still, 3 women out of 52 people is really low.

Who Are The Elites?
In thinking about representativeness, we have to ask what is the pool of people this survey is meant to represent? Since we don’t have much more to go on than “broad pool,” we have to look at the pool itself to imagine the intent.

Non-Americans
One thing that is immediately obvious is that it isn’t all Americans. There are a number of Israelis and Iranians in the mix. For example, on the Israeli side,  Amos Yadlin, former chief of Israeli intelligence is surveyed as are several scholars at Tel Aviv University. On the Iranian side, Seyed Hossein Mousavian is surveyed, who served as Iran’s Ambassador to Germany from 1990 to 1997, as is the Iranian dissident Akbar Ganji. By my count, there are at least 12 non-American respondents, of which 5 appear to be Israelis. People may have dual citizenship, and I tried not to just profile people based on names of non-American dissent. If someone was the former Iranian ambassador to Germany though, chances are…

So, if this rough and ready portrait of citizenship is accurate, American views are either firmly in support (27 or nearly 68% said strongly agree) with a minority  of 5 strongly disagreeing. By contrast, only 5 of the 12 strongly agree among those I counted as non-Americans (42%) with none “strongly disagreeing.” That includes all of the Israelis who are pretty split, 1 of them agrees with the Iran deal (Bermant), two are neutral (Yadlin, Bergman), and two disagree (Golov, Guzansky). What a sample of 5 Israelis tells us about Israeli expert opinion on the nuclear deal I don’t know.

The sample doesn’t include views of European allies (save for Josef Joffe whose views are likely idiosyncratic within Europe), nor really anyone other than Americans, Israelis, and a few Iranians.

U.S. Elites in the Foreign Affairs Poll

I think the more important question is what this survey tells us if anything about the spread of elite U.S. opinion. Here, I’m also not sure. From the survey, it is pretty easy to tell that the Council brought in views of some of its own, Jim Lindsay, Elliot Abrams, Phil Gordon, but it doesn’t include everybody in that universe, no Max Boot, Micah Zenko, nor Stewart Patrick. The poll also includes some of the leading security practitioners and those engaged in nuclear diplomacy such as Robert Einhorn, James Dobbins, John Deutch, among others. There are some folks such as Jeffrey Lewis missing from this sample so I wonder how folks were invited to participate. At the same time, sitting policymakers, such as Colin Kahl, whose views on the topic are well-known are missing from the sample.

Clearly, Foreign Affairs also wanted to make sure that some contrarian perspectives were included so among those with the strongest views opposed to the deal are former members of the George W. Bush administration and neoconservatives such as Daniel Pipes, Michael Doran, Mark Dubowitz, Eliot Cohen, and Michael Rubin. It’s unclear given the sample frame whether Foreign Affairs has the right number of those folks included in the mix. Perhaps it is more important the reasons on both sides for support or opposition, though these are pretty familiar talking points.

There are some of the grand figures in international relations and foreign policy – Mearsheimer, Walt, Posen, Betts, Desch, Nye, many of whom are realists and unsurprisingly supportive of the deal. Many of them (and others) recently signed on to a letter that appeared earlier this week in the New York Times.  There are also a few younger  people such as Daniel Byman, Matthew Kroenig, and Brent Sasley whose views and expertise on the topic are well-known.

The political scientists sampled here likely reflect the views of the wider academy, though their rationale within the academy may have some variation between realists and more liberal internationalists.

A March 2015 snap poll from the TRIP Survey captured likely US academic sentiment before the final deal was announced, with nearly 80% of some 1,000 U.S. IR scholars seeing the deal as helpful or more helpful than harmful in theory. That’s fairly consistent with a July 2015 snap poll from trip that found 69% of IR academics thought the deal would increase regional stability and 85% thought unilateral withdrawal would make Iran more likely to restart its nuclear program.

The TRIP snap polls have the virtue of being comprehensive with a sampling frame that encompasses all US-based academics with expertise in IR as well as other countries. However, if we wanted to know just what people with expertise in the nuclear arena thought, then a smaller subset of views might be necessary. Of course, academic attitudes may be unrepresentative of wider thoughts of “elites” whose views actually influence policy.

Elites Attitudes More Broadly

To get a more comprehensive view of a wider set of elites, we can also look to other sources. A 2013 poll of members of the Council on Foreign Relation was completed just prior to the completion of the interim nuclear agreement but did ask member whether they approved of Obama’s Iran policy, of which 72% did (though nearly half didn’t trust Iran’s intentions on nuclear weapons). Still, I wonder about the representativeness of CFR members (I am a Life Member). CFR Members may tend to represent financial and government elites on the East Coast, so I’m not sure if they are as representative of wider elite sentiment.

Another approach comes from the 2014 elite survey from the Chicago Council on Global Affairs. In that survey, we asked ten groups of elites, including folks from the executive branch, Congressional staff, think tanks, NGOs, the media, the business community, academics, labor, churches, and retired military.

We asked those folks the following question:

As you may know, the U.S. and other countries have reached an interim deal with Iran that eases some of the international economic sanctions against Iran. In exchange, the deal requires that Iran accept some restrictions on its nuclear program – but not end it completely – and submit to greater international inspection of its nuclear facilities.  Do you favor or oppose this interim agreement?

When you weight all groups equally or weight the groups based on past responses to the survey, the deal received overwhelming support by more than 80% of the respondents.

If you start to prize apart different groups, then you see more differences. Our former military sample contained more than 500 people, mostly guys in the 70s who attended defense universities. They trend much more conservative than the rest of the sample. Even among those, support for the interim deal exceeded 60%.

Our Congressional sub-group is the other group which was had more Republicans than other groups. Here, 64% supported the deal, but among the 22 Republicans in that sample, the results were flipped with 68% opposing the interim deal. Among the 200 plus self-identified Republicans in the military group, 57% opposed the interim deal.

Whose Views Matter?

All of this raises the question of whose views matter. What value-added do elites provide? Are we interested in their attitudes because of high information? If so, are they any different than high knowledge voters? If we are interested in the specific knowledge of those with particular expertise over nuclear matters, what is the universe of people whose views matter?

Emily Hafner-Burton and co-authors in a 2013  Perspectives on Politics piece explored some of these issues, noting that it isn’t simply knowledge that elites possess:

Following the literature, we adopt the working definition that elites are individuals with considerable influence within their domains.We argue that experience is the driving force in explaining why elites have different decision-making skills and so throughout this essay we refer to “experienced elites.”

Elites possess greater experience which gives them such influence, but this has some pathologies which we see in the CFR sample, namely overconfidence:

Highly experienced elites are more likely to exhibit the attributes of rational decisionmaking than samples with less germane experience. However, elites are also more likely to suffer overconfidence, which degrades decisionmaking skills.

They write that elites include:

Elites are broadly defined as the small number of decisionmakers who occupy the top positions in social and political structures; those who “have the highest indices in their branch of authority” and “exercise significant influence over social and political change.”

Now, this might be a narrow definition of elites and not fully include all the influencers. On this issue, if you are trying to see how members of Congress might vote before they announce their support for a policy, you might look at the networks of potential influencers out there. However, it begs the question what is the relevant universe of people you should include, but for those designing surveys where aggregate results are presented, clearly one needs to think carefully about who to include

These are the kinds of questions I am exploring with Jon Monten, Jordan Tama, and Craig Kafura in follow on work on elite attitudes and the use of force. So stay tuned.