A Decade of Leveraging Big Data for Sustainable Development

Pulse Lab Kampala is working on a project to gauge via radio the quality of health-service delivery in Uganda, March 2017.           ©UN Global Pulse

Advances in information and communication technologies (ICTs) are driving global changes in our society—from the way we communicate with each other to the forces that shape our economy and behaviour. Insights generated from big data are already transforming many domains. Mobility data from mobile phone networks can reveal the extent of displacement after a disaster and help predict the spread of infectious diseases, while mobile airtime purchases can help track food consumption. Roofing materials visible from space serve as a proxy for poverty, changes in debit card usage indicate the impact of a crisis, and postal records have been used to estimate trade flows. At the same time, the rapidly evolving capabilities of artificial intelligence (AI) offer new opportunities to unlock the value of big data for more evidence-based decision-making that can accelerate progress towards the Sustainable Development Goals (SDGs).

The United Nations has been advocating for a data revolution since before it had a name, and began working on big data applications just as the concept was emerging. By the time the global economic crisis (GEC) peaked in 2009, the international community had already been working to achieve the Millennium Development Goals (MDGs) for almost a decade. The GEC awakened many at the global level to two significant challenges: not only were we unable to anticipate where and when global hazards would originate, but we also could not predict who they would impact or how. Fundamentally, comprehensive information about critical stressors among vulnerable populations was unavailable in a timeline that would allow policymakers to respond with effective interventions.

UN Global Pulse was created in 2009 to assist the United Nations in leveraging big data and data analysis for sustainable development and humanitarian action. We began experimenting with mobile data and social media to understand how to turn them into actionable information. In 2011, we produced the first seminal reportthat conceptualized how big data can be used to inform development practice. To ensure that our research was grounded in the realities of communities we were trying to assist, we developed a network of innovation centres that we call Pulse Labs. Opened in 2009, Pulse Lab New York serves as our headquarters. Pulse Lab Jakarta was established in 2012, followed by Pulse Lab Kampala in 2014.

In May 2013, the High-level Panel on the Post-2015 Development Agenda called for a data revolution to ensure that no one is left behind. In response, then United Nations Secretary-General Ban Ki-moon formed an Independent Expert Advisory Group (IEAG) to provide advice and recommendations on how to accelerate the data revolution for sustainable development. Several of the key recommendations of the IAEG final report2 are already a reality: a multi-stakeholder Global Partnership for Sustainable Development Data has been created, and the second UN World Data Forum3 was organized in 2018.

With the adoption of the SDGs in September 2015, and the shift to their implementation that followed, we have seen efforts around new sources of data explode. Yet, data-driven innovation and institutional adoption of analytics continue to lag, held back by a number of factors. On one hand, the lack of incentives for the private sector to share data and systematic barriers related to privacy have impeded the emergence of an enabling environment for innovation. On the other, we have too few examples of compelling operational projects that demonstrate how big data applications can be used at scale and replicated.


Advancing Corporate Data Philanthropy

Seven years ago, UN Global Pulse began engaging the private sector in data philanthropy, through which we work with companies to put their data, technology and expertise to use for the public good. In 2013, we partnered with mobile operator Orange to launch the Data for Development challenge.4 For the first time, a mobile database was anonymized and opened to researchers to develop applications for sustainable development.

We now have companies actively seeking ways to put the big data they collect to work for the SDGs. Recently, we worked with nine private enterprises—BBVA bank, Crimson Hexagon, Earth Networks, Nielsen, Orange, Planet, Plume Labs, Schneider Electric and Waze—to open their datasets to the international scientific community working towards the achievement of SDG 13, through the Data for Climate Action innovation challenge.We worked with BBVA bank in Spain, using anonymized and aggregated debit card transactions to measure the impacts of a natural disaster and how long it takes different communities to recover.6 We also signed an agreement7 with Twitter to provide the United Nations system with access to its data, and several United Nations agencies have already included this resource in projects.

At an industry-wide level, mobile technology is furthest ahead. In 2017, the association of mobile operators, called GSMA, partnered with the United Nations on an explicit strategy built around putting anonymized mobile big data to work for the SDGs. There are currently 19 mobile operators that are committed to the GSMA programme on big data for social good, which they call BD4SG.8 This is a promising model that we can use for other industries with high-potential data, such as e-commerce, financial service providers, manufacturing, transportation, etc. A significant challenge, however, is that outside of extreme cases, such as responses to humanitarian emergencies and the prevention of acts of terror, there is no regulatory playbook for data sharing for the public good. Undoubtedly, current efforts such as the European Union’s General Data Protection Regulationare paving the way to using new technologies while mitigating the potential risks and harms of data use.


Privacy, Protection and Ethics in Big Data

At the United Nations, we are actively working to accelerate the discovery, development and adoption of privacy-protecting big data applications that can transform how we operate and help communities everywhere achieve the SDGs. UN Global Pulse co-founded and chairs the inter-agency UN Privacy Policy Group (UN PPG). The Group’s primary objectives are to: (i) advance dialogue and information sharing on key issues related to data privacy and protection within the UN system; (ii) unite existing efforts on data privacy and protection; and (iii) develop a practical UN System-wide framework on data privacy and data protection.10 In 2018, UN PPG developed a set of high-level principles on Personal Data Protection and Privacy,11 which were formally adopted by the High Level Committee on Management (HLCM). In 2017, we spearheaded efforts to develop the document Data Privacy, Ethics and Protection: Guidance Note on Big Data for Achievement of the 2030 Agenda12 which was adopted by the United Nations Development Group. Many of our policy instruments benefit from the input of an expert UN Global Pulse Data Privacy Advisory Group,13 which was established in 2015 and which comprises data privacy and ethics experts, regulators and academics.

We incorporate data privacy, protection and ethics into our innovation projects by conducting a Risks, Harms, and Benefits Assessment14 to identify anticipated or actual ethical and human rights issues that may occur at any stage of a data innovation process. The tool also helps develop a risk mitigation strategy and ensure that the risks do not outweigh the benefits of a given project. We recently released a paper15 with the International Association of Privacy Professionals that provides guidance for organizations intending to operationalize data ethics in big data and AI-related projects.

Scaling Big Data Applications for Doing Good

The good news is that innovation labs across academia, governments, the international development community, civil society, and the private sector are using big data and AI to develop a wide range of applications. These individual cases represent a modest but significant innovation in learning about the world around us. Yet the lack of examples of such applications being scaled or becoming sustainable indicates that there are still gaps in the field.

Over the last two years we have been working, through the Pulse Labs, to develop methodologies and tools that can be replicated and scaled for multiple purposes in different locations. In Jakarta, we developed Haze Gazer,16 a crisis analysis and visualization tool designed to inform official policy on issues related to forest and peatland fires. The tool was adopted and installed in the situation room of the President of Indonesia. Based on this concept, we are now developing CycloMon,17 an interactive platform that ingests social media data and combines it with weather data, which can be adapted at the global level to provide information before, during and after cyclones.

One of the areas we are particularly excited about is speech computing, which many of us have encountered through computer dictation software, or in our interactions with technologies such as Alexa. Three years ago, we started an experimental programme in Uganda to capture people’s voices from radio broadcasts.18 Although UN Global Pulse continues to advance its work in social media analytics, the digital divide remains a reality. In rural Uganda, as in much of the developing world, people rarely have access to social media platforms or smartphones. Instead, they call or text into public FM talk radio and news programmes to share and discuss what is happening in their communities. There are shows for new mothers, shows for farmers and shows for people living with HIV/AIDS. These public discussions are just as filled with relevant information as those on Facebook or Twitter. 

Transforming this content into actionable insights, however, presents unique technical challenges, as these are spoken communications, not written, and they are in indigenous languages, of which there are about 40 within Uganda alone. We recruited a team of speech engineers and AI experts who used convolutional neural networking technology to create the world’s first Automated Speech Recognition (ASR) toolkits for local Ugandan languages, starting with Ugandan-accented English, Acholi and Luganda. We now have a robust set of capabilities that can listen to dozens of radio stations simultaneously, flag relevant content when specific keywords are mentioned, generate a transcript for deeper analysis and drop the audio segments into a queue for human review. The exciting aspect of this is that it is happening in isolated communities, where previously we only received sporadic and unreliable data. Suddenly, we can obtain real-time public information about the impacts of floods on crops, the price of food and attitudes towards vaccination. We are looking to expand the project to other countries where information sources are scarce.

Ten years ago, the term “experiment” generally carried negative connotations when applied to activities within the public sector, implying a risky, irresponsible use of public resources. Today, by contrast, it has become clear that the greatest risk of all is that of not taking a learning-by-doing approach to organizational change. Innovation labs around the world are making tremendous progress within governmental and development organizations. Innovation strategies have matured, moving from vagary and hype to delivering real value in the form of practical, scalable tools and a culture of openness to new ideas and new ways of working.

Earlier this year, at the AI for Good Global Summit,19 and more recently at the United Nations World Data Forum, which took place in Dubai, United Arab Emirates, from 22 to 24 October 2018, we saw how these technologies are complementing statistical information for better planning, faster responses, and more effective and efficient programmes and projects. The Secretary-General’s Strategy on New Technologies,20 launched in 2018, outlines our approach to innovation going forward. Part of the Strategy includes creating an innovation lab within the Secretary-General’s own office, and UN Global Pulse is actively engaged in supporting this effort.

We must, however, be mindful of the potential risks and issues that arise from the increased adoption of big data and AI. To date, no international standards exist for the anonymization, sharing and responsible use of insights from big data in priority industries. At the same time, non-use of these new capabilities represents at least as great a risk to the public. Just as misuse of big data and AI may lead to harm, missed use may allow preventable harms to occur, resulting in lost lives, needless suffering and setbacks in development.

The way forward must be inclusive. Achieving the 2030 Agenda for Sustainable Development requires all stakeholders to come together to create an enabling environment for the adoption of responsible frameworks and the development of inclusive tools. The timely appointment of the High-level Panel on Digital Cooperation21 affords new opportunities to advance these goals. In the end, to fully harness the data revolution, we will need to safeguard the responsible use of new technologies, as well as a rights-centric approach to accountability for both misuse and missed uses of data and analytics, ensuring that no one is left behind. 



1UN Global Pulse, “Big data for development: challenges and opportunities” (New York, 2012). Available at http://www.unglobalpulse.org/sites/default/files/BigDataforDevelopment-UNGlobalPulseJune2012.pdf.  

2Independent Expert Advisory Group on a Data Revolution for Sustainable Development, “A world that counts: mobilising the data revolution for sustainable development”, Report (November 2014). Available at https://repositorio.cepal.org/bitstream/handle/11362/40319/AWorldThatCounts.pdf?sequence=1&isAllowed=y.  

3For more information, see https://undataforum.org/.  

4Jennifer Poole, “Data for development (D4D) challenge at Net Mob 2013” (UN Global Pulse Blog, 1 May 2013). Available at https://www.unglobalpulse.org/D4D-NetMob.

5For more information, see http://dataforclimateaction.org/.  

6For more information, see Elena Alfaro Martinez and others, “Measuring economic resilience to natural disasters with big economic transaction data”, Bloomberg Data for Good Exchange Conference, 25 September 2016. Available at https://arxiv.org/ftp/arxiv/papers/1609/1609.09340.pdf.

7United Nations, “Twitter, UN Global Pulse announce data partnership”, Sustainable Development Goals, 26 September 2016. Available at https://www.un.org/sustainabledevelopment/blog/2016/09/twitter-and-un-global-pulse-announce-data-partnership/.

8For more information, see GSMA, “Big data for social good”, #BetterFuture. Available at https://www.gsma.com/betterfuture/bd4sg (accessed on 16 November 2018).

9European Union, The EU General Data Protection Regulation (GDPR) is the most important change in data privacy regulation in 20 years. Available at https://eugdpr.org/(accessed on 16 November 2018).

10For more information about the UN Data Privacy Policy Group (UNPPG) and its objectives, see the website of the United Nations System Chief Executives Board for Coordination at https://www.unsceb.org/privacy-principles.

11For more information about the principles, see https://www.unsceb.org/principles-personal-data-protection-and-privacy.

12United Nations Development Group, Data Privacy, Ethics and Protection: Guidance Note on Big Data for Achievement of the 2030 Agenda (New York, 2017). Available at undg.org/document/data-privacy-ethics-and-protection-guidance-note-on-big-data-for-achievement-of-the-2030-agenda/.  

13For more information about UN Global Pulse Data Privacy Advisory Group, see its website at https://www.unglobalpulse.org/data-privacy-advisory-group.

14UN Global Pulse, “Data innovation risk assessment tool”, 2016. Available at https://www.unglobalpulse.org/privacy/tools.  

15UN Global Pulse and International Association of Privacy Professionals, “Building ethics into privacy frameworks for big data and AI”. Available at https://iapp.org/resources/article/building-ethics-into-privacy-frameworks-for-big-data-and-ai/.

16For more information, see Haze Gazer, a Real Time Analysis and Visualization Tool for Enhanced Crisis Management at http://hazegazer.org/.

17For more information, see CycloMon, an Analytics and Visualisation Platform Developed by Pulse Lab Jakarta to Monitor Action before/during/after Tropical Cyclones across the World at http://cyclomon.org/.  

18UN Global Pulse, “Using machine learning to analyse radio talk in Uganda”, September 2017. Available at http://unglobalpulse.org/sites/default/files/Radio%20Analysis%20Report_Preview%20%283%29.pdf.

19For more information, see International Telecommunication Union, “AI for Good Global Summit 2018” at https://www.itu.int/en/ITU-T/AI/2018/Pages/default.aspx.   

20For more information, see “Secretary-General’s Strategy on New Technologies”, September 2018 at http://www.un.org/en/newtechnologies/images/pdf/SGs-Strategy-on-New-Technologies.pdf.

21For more information, see United Nations, “Secretary-General’s High-level Panel on Digital Cooperation”. Available at http://www.un.org/en/digital-cooperation-panel/ (accessed at 20 November 2018).