Policy diffusion and survey experiments

When studying policy diffusion, establishing causality is even trickier than usual because interdependence violates an important assumption of the Rubin causal model, namely the so-called “stable unit treatment value assumption” (SUTVA), which requires that the outcomes in one unit do not depend on the treatment status of other units.

With observational data, going around this problem is very hard. In an forthcoming article just published in the American Journal of Political Science, Katerina Linos uses a survey experiment to address this issue. Survey experiments have become increasingly popular in political science but, as far as I am aware, Linos is the first to apply this method to policy diffusion.

The study asked respondents (a representative sample of Americans) whether they agreed or disagreed that “the United States should increase taxes in order to provide mothers of newborn children with paid leave from work.” Respondents were assigned randomly to either this baseline question or to one of four treatments, which consisted of statements that (1) “Canada” or (2) “most Western countries” already have similar policies, or that the policy is recommended by (3) the United Nations or (4) American family policy experts. The main results are shown in this graph (own elaboration of Table 1 in the article):

The effect of all four treatments is large and also remarkably similar. In additional analyses, Linos shows that these effects decrease with the level of information of respondents, which suggests that the experience of other countries and the advice of authoritative people or organizations influence opinions by supplying new information and not, for instance, by setting certain normative standards to which it feels appropriate to conform.

Interesting work, which should encourage other diffusion scholars to use these tools both to replicate these findings and to explore other questions.

The Electoral Effects of Arguments and Partisan Cues: A First Peek at Experimental Evidence from the Swiss Deportation Initiative

The Swiss accepted a controversial anti-foreign crime initiative in a direct democratic vote last month. The so-called deportation initiative will soon keep a group of experts busy who have to prepare a detailed legislative proposal to be submitted to parliament. However, already today the deportation initiative keeps at least four political scientists busy. Jens Hainmueller, Dominik Hangartner, Marc Helbling and I have started to analyze experimental data generated in the context of the deportation initiative.

We conducted a large-scale survey experiment in the week prior to the referendum to explore the effects of arguments and party cues on citizens’ preferences over the deportation initiative.[1] We randomly exposed respondents to one of the two key arguments that dominated the highly polarized pre-referendum campaign. The Swiss People’s Party (SVP), which instigated the deportation initiative, claimed that immigrants to Switzerland are disproportionately responsible for crime. The SVP argued that the deportation initiative would significantly reduce crime rates and thereby add to public security (pro argument). The Social Democratic Party (SP) said the deportation initiative would violate basic rights guaranteed in the Swiss constitution and even international law (con argument).

The preliminary results look as follows: We find that these arguments had no overall effect on citizens’ vote intentions. Those receiving the pro-argument were no more likely to approve the deportation initiative than citizens who received the con-argument. We also randomly provided respondents with information about which party supported (SVP) or opposed (SP) the deportation initiative. Again, we do not find any difference between these two groups.

However, the preliminary results suggest that may party cues may act as a negative heuristic: Voters who disliked the SP were significantly more likely to vote in favor of the deportation initiative if they received information about the SP opposing the initiative. To track this effect over time, we conducted a second survey in the week after the referendum, in which we called the same individuals. We find that the effect persists. Even two weeks after SP-nonidentifiers had received the SP cue, these citizens opposed the deportation initiative more strongly and had voted significantly more often in favor of it than voters who had received no such information.

The group of experts will have to submit their legislative proposal to the Swiss parliament by July 2011. We have decided to beat the experts at least once by delivering a working paper until June 2011.

[1] We thank the Wissenschaftszentrum Berlin and DemoSCOPE for financial support.

The Effectiveness of Arguments in Direct-democratic Decisionmaking: Evidence from the Swiss Deportation Initiative?

On Sunday the Swiss decided to automatically expel foreign nationals guilty of serious crimes including murder and drug dealing. The vote on the deportation initiative has triggered strong reactions. Unsatisfied with the outcome of the referendum and allegedly ever more xenophobic Swiss immigration policies, opponents of the deportation initiative even engaged in partly violent demonstrations in several towns in Switzerland, a rarity in a country known for its peaceableness.

Two arguments dominated the public debate prior to the referendum, each attached to one of the two political camps. The Swiss People’s Party (SVP), which instigated the deportation initiative, ran a costly and highly professionalized campaign, claiming that immigrants to Switzerland are disproportionately responsible for crime. The SVP argued that expelling foreign criminals would significantly reduce crime rates and thereby add to public security. It also played to stereotypes. The “Ivan S., rapist, soon a Swiss citizen?”-flyer invoked the well-known Swiss antipathy against immigrants from former Yugoslavia and presumably also alluded to Ivan IV of Russia known as Ivan the Terrible.

SVP Deportation Initiative Flyer. Source: www.kriminelle-nein.ch/downloads/index.html

SVP Deportation Initiative Flyer. Source: http://www.kriminelle-nein.ch/downloads/index.html


Opponents like the Social Democratic Party (SP) said the deportation initiative was too harsh. They argued it would violate basic rights guaranteed in the Swiss constitution and even international law, since in some cases individuals would have to be deported to countries that practice torture or the death penalty.

Which of these arguments was more effective in changing people’s minds about the deportation initiative? This question is not only interesting for practitioners and scholarship on electoral behavior and immigration. Immigration constitutes a key component of globalization, a process in which countries remove legal restrictions on the flow of capital, goods, services, and, last but not least, labor. Immigration has therefore also become one of the rising themes in international political economy. The Swiss vote on the deportation initiative presents a valuable case to learn about the politics of immigration and the effectiveness of arguments in changing people’s minds about immigration policy.

From a political science perspective, examining the effectiveness of these two arguments poses a considerable challenge. Clearly, asking people about their preferences over the deportation initiative and having them indicate how convincing they find one argument or the other fails to provide any credible evidence. Individuals could hold an opinion because they find an argument convincing, but they could as well find an argument convincing because it supports their own opinion. We also have far too limited knowledge about the large number of other factors that potentially affect both citizens’ preferences over the deportation initiative and whether they find an argument convincing or not.

Jens Hainmueller, Dominik Hangartner, Marc Helbling and I have devised an empirical test that allows us to get an impression of each argument’s causal effects on citizens’ vote intentions. In the week prior to the referendum we conducted a large, representative telephone survey in which we embedded a randomized experiment.[1] We randomly allocated respondents into three groups. We exposed one group to the security argument and the second group to the non-discrimination argument. A third set of respondents received no argument and therefore served as a control group. To evaluate the effects of partisan cues we designed additional statements that individuals were exposed to randomly.

After providing respondents with an argument, we asked them to indicate their preferences over the deportation initiative and to answer more general questions about their attitudes toward immigrants. Any difference in respondents’ preferences between these different groups would be entirely attributable to the exposure to different arguments. Also, we are currently conducting a post-referendum survey in which we call the same individuals and ask them about their voting behavior and attitudes toward immigration to see whether the exposure to one of these arguments has any effects that last beyond the vote on the referendum.

We are currently conducting the last 300 interviews of the post-referendum survey and will post first results in this blog soon, so stay tuned.

[1] We thank the Wissenschaftszentrum Berlin and DemoSCOPE for financial support.

We matter, but maybe not as much as we think

What difference does the quality of education make for student outcomes? Establishing a causal relationship is difficult especially for PhD students, because it is likely that the best students are drawn to the best departments, so that a positive correlation between departmental strength and students’ success is at least in part due to a selection effect. An article in the latest issue of the Journal of Political Economy uses a smart identification strategy, namely the expulsion of mathematics professors in Nazi Germany as an exogenous shock to faculty quality. The abstract sums up the main results:

I investigate the effect of faculty quality on PhD student outcomes. To address the endogeneity of faculty quality I use exogenous variation provided by the expulsion of mathematics professors in Nazi Germany. Faculty quality is a very important determinant of short- and long-run PhD student outcomes. A one-standard-deviation increase in faculty quality increases the probability of publishing the dissertation in a top journal by 13 percentage points, the probability of becoming a full professor by 10 percentage points, the probability of having positive lifetime citations by 16 percentage points, and the number of lifetime citations by 6.3.

These effects look sizeable but maybe not as large as we, as professors, like to think. Self-selection is probably as strong a determinant of success as departmental quality, possibly even more.

As an aside, this research is a nice illustration of the causal inference trend that is well established in economics and is catching up quickly also in political science. In this regard, see also this previous post.

Floods and Elections: How Lasting is Voter Gratitude for Disaster Relief Spending?

The recent flood in Poland has taken a heavy toll. More than 14 people have been killed and thousands evacuated. The damage from the Oder flood is estimated to exceed $2.5 billion. The Polish government has already promised to disburse $46 million to people in the most heavily affected regions, about $1,800 per household. Such disaster relief provided by the government is clearly vital to help those affected by the flood. But do affected citizens electorally reward such rapid policy responses to natural disasters and how long will their gratitude last?

Jens Hainmueller (MIT) and I address this question in a study that exploits the 2002 Elbe flooding in Germany as a natural experiment to estimate the short- and long-term electoral returns to disaster relief. Given the well known “peak-and-end” rule for forming retrospective judgments, the 2002 Elbe flooding provides an ideal case to establish an upper bound for the longevity of voter gratitude, as the incumbent’s policy response to the flood was both “peak” and “end” — it was massive and occurred immediately prior to a federal election.

Affected versus Unaffected Electoral Districts in the 2002 Election

Continue reading