Network of Excellence in Internet Science

From Mantra to Metrics: Measuring Multi-Stakeholderism?

The multi-stakeholder (MSH) approach to Internet Governance is hailed by many as the innovative policy process to manage the global Internet ecosystem. Especially keynote speakers at the Internet Governance Forum (IGF) were keen to praise the open discussion forum between all stakeholders enabled at the IGF. Others are more critical (pdf) of the approach, considering MSH to be ineffective and highlighting its possible misuse to legitimise Internet policies that are developed behind closed doors and only benefit special interests. In their view, MSH is a veneer of fake consultation used by powerful governments to obscure the real corporate-government discussion. The first morning of the IGF pre-event was dominated by Professor Ron Diebert’s exposure of the fake MSH discussion at the High Level Leaders Meeting in which he used the E-word (Edward Snowden), setting the scene for the afternoon workshops.

The Joint Research Activity on Internet Governance, Regulation and Standards of the European 7th Framework Programme academic Network of Excellence on Internet Science (EINS) organised a workshop on Day 0 of the IGF to explore how to measure the impact on this multistakeholder approach on both technical standard setting and law making processes. The workshop was led by Dr Meryem Marzouki (CNRS & UPMC Sorbonne Universités) and Professor Chris Marsden (Sussex University), JRA leader. Two presentations by further EINS members explored which metrics and methods could be used to study the extent to which the multistakeholder process is effective on the workings of the Internet.

An Ecology of Values
Initiating the workshop, Dr Alison Powell (LSE) suggested mapping the “ecology of values” among the various stakeholders in order to investigate the normative aspects of the multi-stakeholder approach in Internet governance. This method takes the normative positions of the various stakeholders as the organising principles for governance debates, and as such provides a way of mapping controversies within governance and policy-making processes.

The mapping exercise allows for an analysis of how various values play a role in the multi-stakeholder approach of the Internet governance discourse. Through comparison the researcher can test the hypothesis that different types of stakeholders and ecologies of values appear and evolve in the various thematic discussions.

Insights from practitioners
The two multi-stakeholder panels – with representatives from government, civil society, business and academia – reflected on their experiences with the impact of the multi-stakeholder approach, and discussed how and what could be measured.

Michael Niebel (European Commission) explained that several totally different policy processes are labelled as a “multi-stakeholder process”, which threatens to render the term meaningless. Michael agrees with the need to move from “mantra to method”, to find out which multi-stakeholder processes are effective, and which are not.

Jan Malinowski (Council of Europe) pointed out that the multi-stakeholder approach presents a useful dynamic dialogue to discuss the tensions of competing interests in the Internet policy space. The Council of Europe has adopted principles and recommendations with regard to Internet Governance and on the openness of the Internet, which are the result of multi-stakeholder processes. There are some efforts underway to monitor how legislative bodies slowly but surely adopt these outputs.

MEP Amelia Andersdotter (European Parliament) reminded the workshop that not all stakeholders are equal, since business representatives and government officials have more resources than civil society organisations to travel to the various multi-stakeholder forums and take part in the shaping of legislation and standards. Anne Carblanc (OECD) explained that it has been a challenge for the OECD to determine who should be representing civil society at negotiations and how their weight should be measured.

Anriette Esterhuysen (APC) explained that her organisation uses policy-mapping tools for stakeholder analysis, which are useful to learn how an organisation can most efficiently affect policy change. However, the values and interests must be clearly and precisely defined. Every stakeholder will express their commitment to human rights, for example, but tensions will exist in the way these rights are respected in practice.

Rafik Dammak, of the ICANN Non-Commercial Users Group, explained his concern that the much-heralded ICANN strategy panels were arranged with insufficient MSH process, and that they may produce a fait accompli in terms of ICANN’s plans to provide a new governance solution at the Rio summit 2014.

Sebastian Bellagamba of the Internet Society, explained that from the South American perspective, the revelations of surveillance including those released by Edward Snowden had created a tremendous momentum for moving away from the previous United States-controlled legal environment for ICANN. Whether that proved to be a truly MSH reform, or a more multilateralist approach with lip service paid to MSH, was of concern to several panellists.

Tomoake Watanabe, of GLOCOM research at the International University of Japan, explained that MSH processes are developing despite a lack of dynamism in civil society in Japan. His research into the Creative Commons movement, opening of government data and network neutrality showed that, while MSH processes are under-developed, Japanese civil society is slowly developing a policy input into government-corporate policy development. Much remains to be done, in Japan as elsewhere.

Meredith Whittaker (Google Research) highlighted the statistical wisdom that correlation is not causation, which will make a measurement of the impact of a multi-stakeholder process tricky. For example, it may be far fetched to claim that a policy change encouraging specific online behaviour results in more of that behaviour, if it was a growing phenomenon already. And how would one measure the impact of the recent Montevideo statement? The panel agreed that measuring the influence of individual stakeholders would greatly enhance our understanding of MSH processes.

Multi-stakeholder modelling
Tamas David-Barrett (Oxford) of EINS JRA6 Virtual Communities Research rounded off the workshop with a presentation in which he suggested applying the behavioural synchronization theory he and Robin Dunbar (known for their work on socialization processes including the ‘Dunbar Number’) developed for experimental psychology to model and measure group coordination within the multi stakeholder process of Internet governance. The method is based on dyadic synchronization in a non-panmictic, structured network. The necessary input information for the model may be derived from Powell’s “ecology of values” mapping exercise.

The behavioural synchrony model measures how behaviour is coordinated in a network, and to the extent multi-level networks can be optimal structures from the point of view of the collection of individuals that make up the group. For this, David-Barrett assumes that the objective of a group of agents (i.e. stakeholders) is a collective action that is conditional on the agents’ behavioural synchronicity.Tamas compared three coordination protocol scenarios in large-scale network that is structured to model the global economy using the internet to coordinate economic action: (A) each subnetwork determines their own coordination protocol, (B) all subnetworks adopt the protocol of only one subnetwork unit, (C) a collectively agreed protocol adopted by each subnetwork. The model showed that the highest efficiency from a multi-stakeholder process is achieved when the dominant parties do take into account the positions of other, less powerful parties.

The proceedings of the workshop made clear that measuring the impact of the MSH approach to Internet governance is a challenging academic and urgent practical task. Although the exchange of ideas at the IGF, along with the social aspects and networking opportunities between stakeholders are important for mutual understanding in the complex process of Internet Governance, it remains important to find out to what extent the different variations on MSH has an effect on real standard setting and policy making, which influences daily use of the internet.

The discussion on metrics and methods to measure the impact of the MSH approach in Internet governance has only commenced at the IGF2013 and will be continued in more depth in JRA4 working in partnership with JRA6 and other interested parties.

Watch the video of the workshop here.