This is a guest post from William G. Nomikos, Assistant Professor of Political Science at Washington University in St. Louis. Follow him on Twitter @wnomikos.

Recent relations between North Korea and the United States suggest a puzzle for International Relations. The Trump administration has relied on what it has called a “maximum pressure” campaign—a set of sections and threats of military escalation—to prevent North Korean nuclear missile tests and to roll back North Korean nuclear proliferation.

According to the prominent theory of International Relations known as “audience costs,” North Korean leader Kim Jong-un should take maximum pressure threats seriously. President Donald Trump is facing a challenging re-election campaign and voters should punish him if he makes a threat but subsequently backs down. Yet North Korea has shown very little signs of taking these threats seriously. If anything, North Korea has advanced its nuclear weapons program.

What explains this divergence between a well-established theory, a robust set of empirical evidence and North Korea-U.S. relations? Do audiences care about reputation in foreign policy? Probably not. In my recently published research with Nicholas Sambanis, we find that domestic audiences care more about a leader’s perceived competence than their ability to manage the nation’s reputation. In this study, we identify a new mechanism by which audiences evaluate leaders in foreign policy crises and find that existing research overestimates “audience costs.”

Audience costs are generated when a leader backs down in a crisis after making a threat. A leader’s approval suffers because of the inconsistency between what they said they would do (military action in the threat) and what they actually do (no military action after backing down). The mechanism is domestic perception of the leader after they back down. In the words of James Fearon, the leader “engages the national honor” when they make the original threat. When they back down, the audience perceives the leader to have compromised the nation’s reputation and supports the leader less than they had before, or so the argument goes.

We contend that approval losses for the leader depend on evaluations of the leader’s competence, which we refer to as “incompetence costs.” We define competence as a leader’s ability to achieve the outcome the audience prefers or the outcome the audience expects, given prior beliefs. We make the case that audiences judge leaders’ competence by assessing the outcome of the crisis. The conventional wisdom misses this component and, thus, a large part of the puzzle.

This logic is consistent with previous research which shows that leaders back down during electoral cycles and that leaders can justify backing down from crises. In the case of North Korea, backing down from military escalation is unlikely to affect Americans’ opinion of President Trump’s competence relative to not making a threat in the first place. If anything, large majorities of Americans would prefer no conflict with a nuclear-armed North Korea. As a result, given that backing down will not generate any incompetence costs, North Korean leadership knows that the President’s threats will not be credible.

The question of audience costs due to inconsistency remains, however. Might it not be the case that backing down from threatening North Korea could generate disapproval from large swaths of Americans and, consequently, make the Trump administration’s threats credible? Existing research has offered robust evidence from the real world for this claim. Yet such evidence is ultimately indirect at best.

The problem is that the higher the expected audience costs, the less likely it is that rational leaders would make empty threats. And so, observational studies will likely observe a biased selection of cases in the real world. Survey experiments offer a solution by randomizing whether a leader backs down in a crisis or stays out altogether. That difference in mean approval is audience costs. Indeed, several studies have used this experimental approach to argue that audience costs exist. Yet, there are at least three reasons to be skeptical of these findings and the claim that they offers evidence in favor of reputational costs due to inconsistency in foreign policy crises.

First, survey respondents demonstrate a general aversion to inconsistent politician behavior. As a consequence, if presented with a leader in a survey—like the leader in the standard audience cost experiment—that behaves inconsistently, they are primed to disapprove of that leader’s behavior regardless of the domain. To test this claim empirically, we conducted a placebo test in which we randomly assigned American respondents to one of two types of vignette: (1) standard audience cost vignette in which the leader is the U.S. President and (2) audience cost vignette in which the leader is from a foreign state, which we name.

If the mechanism underlying audience cost theory concerns reputation, we should only find audience costs when respondents are told the leader is from the United States, not from a foreign state. However, we find evidence of audience costs in both types of vignette. Moreover, contrary to the theory’s predictions, the decline in audience costs is driven almost entirely by a drop in approval for foreign leaders that stay out of a crisis. In other words, American respondents disapprove of leaders that back down from threats regardless of nationality of the leader, a finding that is inconsistent with the logic of reputational audience costs.

Second, the existing benchmark for experimental studies, a variation of which almost all experimental studies of audience costs use, biases in favor of finding audience costs. According to the conventional approach, a vignette outlines a foreign policy crisis. The researcher then randomizes whether the leader in the vignette escalates the crisis and then backs down or stays out entirely. The problem is that this assumes that the treatments (known as “backing down” or “staying out”) don’t have a separate impact on approval through another path. When we incorporate incompetence costs, we see such a back-door path emerge. This introduces unknown levels of statistical bias.

When a survey treatment tells a respondent “the leader will stay out/back down,” it interacts with the respondents’ prior beliefs in different ways. Did the leader stay out for a reason? Did the leader back down because they know something about the foreign adversary? The respondent does not have enough information and, as a result, forms new beliefs about the leader’s competence.

We diagnose this issue empirically in a survey experiment in our study. We find that incompetence costs account for a 28% drop in approval, dwarfing drops in approval due to inconsistent behavior. This suggests that when researchers have found approval losses due to inconsistent leader behavior, they are likely finding approval losses due to incompetence costs instead.

We offer a solution for future work—researchers should provide respondents with information pertinent to their priors and the crisis outcome about the crisis in the vignette. When we do this, depending on what we tell respondents, audience costs change, suggesting yet again the role of bias.

We experimentally manipulate respondents’ beliefs about the likely outcome of the crisis by telling them whether “the President’s advisers in the Pentagon” believe the intervention will succeed or not. In this way, we ensure that incompetence costs are the same in the backdown and stay out condition. And so, for example, when respondents believe an intervention is likely to succeed, approval ratings for staying out are lower than we would see in a standard audience cost experiment, resulting in lower approval losses from backing down.

Third, it is not at all clear that all domestic audiences would necessarily punish inconsistent behavior. Josh Kertzer and Ryan Brutger have shown that not all groups of subgroups of respondents feel the same way about reputation. For example, liberals are less likely to care about inconsistency than conservatives. Joslyn Barnhart and her co-authors show that women systematically prefer more peaceful options in international crises than men. In ongoing work with Claudia Alegre and Elana Resnick, I find that Latino respondents care substantially less than Caucasian respondents about the reputation of the United States. Combined, this research suggests that the reputational mechanism likely does not apply to the majority of individuals in a domestic audience.

Ultimately, audience costs are about creating sufficient costs as to make a threat not to back down seem credible. Since the reputational component of these costs is likely not as high as we thought, leaders may not be good at making credible threats via public threats.

An important caveat is that leaders can change the framing of a conflict and generate support for conflict. Consider President Trump “maximum pressure” strategy to try to contain Iranian nuclear proliferation. Should Iran find his threats credible? The conventional wisdom would suggest that he is unlikely to back down given that he would face substantial losses at the ballot box should he back down. Knowing this, Iranian leadership should take President Trump’s threats seriously.

However, what we find in our study is that domestic audiences punish leaders because of their perceived incompetence, not the inconsistency of their word and deed. As long as Americans don’t want the United States to attack Iran—according to an IPSOS/Reuters poll, only 12% of all Americans would support a “preemptive” attack on Iran—voters will not punish any inconsistency or belligerent rhetoric that does not result in war. In short, recent research casts serious doubt upon the idea that audiences care about reputation and, in turn, that leaders can use potential audience costs to make their threats credible.