Semantic polling: the next foreign policy tool

26 August 2011, 1233 EDT

George Gallup –
what have you started?

The traditional methods for a state to know what overseas publics are thinking are changing. Instead of relying on your embassy staff’s alertness, your spies’ intelligence and the word of dissidents, we’re reaching the point where foreign policymakers can constantly monitor public opinion in countries in real-time. The digitization of social life around the world  – uneven yes, but spreading – leaves ever-more traces of communications to be mined, analysed and acted upon.  In a paper that Nick Anstead and I presented in Iceland this week, we called this ‘semantic polling’, and we considered the ethical, political and practical questions it raises.

Semantic polling refers to the use of algorithms and natural language processing to “read” vast datasets of public commentary harvested from the Internet, which can be disaggregated, analysed in close-to-real-time, and which can then inform policy. It can give a general representation of public opinion, or very granular representations of the opinion and behaviour of specific groups and networks. Multi-lingual processing across different media platforms is now possible.  Companies already provide this service to pollsters and parties in domestic campaigns, and NGOs make use of it for disaster response monitoring. Given how public diplomacy has adopted many techniques of the permanent campaign, it will be no surprise to see semantic polling become part of the foreign policy toolkit.


The semantic web is the standardization of protocols so that everything on the web becomes machine-readable. This means semantic polling is about more than reading social media data. In principle, our shopping, driving, social media, geolocation and other data are all searchable and analyzable. It is only a matter of computing power and integration of data streams for this method to profile to the individual behavioural level. This also enables predictive engagement: if Amazon thinks it knows what you want, then a state, with access to more data streams, might be use semantic polling and think it knows who will support an uprising and who will not.
Ethically, do people around the world know their tweets, public facebook data and comments on news blogs are being used to build a picture of their opinion? How should journalists report on this when it happens? Politically, how will states and NGOs use semantic polling before, during and after crises and interventions? Is it predictive, valid and reliable? Will semantic polling’s real-time nature further intensify the pressures on policymakers, since the performance, credibility and legitimacy of their policies can be visualized as they are enacted? Will publics resist and find ways to circumvent it? And given that it is barely regulated at the domestic level, how could or should it be policed in international affairs?
When we thought of this paper it seemed a little bit like an exercise in science fiction, but interviews with the companies, pollsters and social scientists driving this has convinced us this is developing quickly. Our political science audience in Iceland seemed positive about this – semantic polling offers relatively cheap, unobstrusive and ‘natural’ data that might provide insights into human behaviour existing methods cannot give. Perhaps a good first step would be for people around the world to understand how semantic polling works, so they can decide what they think about it, since it is their lives that are being monitored.