Abstract
In 2017, a prolonged drought severely affected several countries in Africa and the Middle East. For less resilient countries, food shortages escalated into catastrophes, and famine-like conditions were created in Nigeria, Somalia, South Sudan, and Yemen, urging a rapid humanitarian response, but also developmental interventions to create resilience in the long term. The Rapid Emergency Response Survey was designed to collect data to inform the developmental response. Results were generated over an eight-week period, using mobile phone interviews that were designed to be applicable to other emergencies and crises. This chapter describes the methodology, sampling, and key results of the Rapid Emergency Response Survey, and draws lessons for the use of an adaptive questionnaire design to find a better trade-off between depth and comprehensiveness of data.
You have full access to this open access chapter, Download chapter PDF
Similar content being viewed by others
1 The Data Demand and Challenge
In 2017, the United Nations (UN) stated that the world was facing the most serious humanitarian crisis since the Second World War, with over 20 million people at risk of starvation and famine.Footnote 1 The crisis was concentrated in four countries: Nigeria, Somalia, South Sudan, and Yemen. Alongside hunger, large portions of the population in these countries were facing deteriorating living conditions and threatened livelihoods.Footnote 2
The crisis was driven by both drought and conflict to differing degrees in the four countries. In Nigeria, the Boko Haram conflict contributed to poor market access, severe food shortages, and disruption of livelihoods in the North-Eastern States.Footnote 3 For the Somali population, the dry agricultural season contributed to high food prices, livestock losses, and displacement.Footnote 4 In South Sudan, below-average crop production and inter-communal violence contributed to famine in the former Unity State; in addition, 70% of the population of South Sudan was in serious need of humanitarian assistance.Footnote 5 In Yemen, airstrikes and violent clashes on the ground kept food prices high and resulted in high dependency on food imports and emergency aid.Footnote 6
The crisis required a response along the humanitarian-development nexus, to address urgent humanitarian needs while working toward short- to medium-term socio-economic development goals. The UN and the World Bank have worked to synchronize their responses to crises to the greatest extent possible.Footnote 7 Greater development can improve resilience and reduce fragility , so that future shocks do not automatically lead to humanitarian catastrophes.Footnote 8
During the crisis, rapid data collection was required to assess the population at risk of famine. Traditional survey methods were unsuitable for a variety of reasons. First, results were needed urgently, so lengthy household questionnaires were inappropriate. Second, funding constraints meant that costly traditional surveys were also unfeasible. Third, a significant portion of the affected populations was believed to be located in conflict-affected areas, where face-to-face data collection is very risky. Given the context, there was a need for a survey that was low-cost, fast, and technically feasible. Data collection needed to be launched and completed in a matter of days, while also ensuring the safety of the implementing teams. Convincing sampling frames had to be obtained in environments where existing data was scarce. Finally, the crisis was unfolding in four different contexts, and country-specific approaches were required that were both standardized yet adaptive.
2 The Innovation
The Rapid Emergency Response Survey (RERS) was designed with standardized survey protocols that can be implemented quickly in times of crises. It was designed as a phone survey to allow rapid access to populations at risk of famine, and can be carried out by local call-centers at low cost.Footnote 9 During the crisis, enumerators recorded data digitally and uploaded it every day to a cloud-based server, in order to map and update data trends on a daily basis.
The questionnaire was quick to administer, yet still included a broad range of development topics that might need to be better understood during a crisis. A maximum administering time of about 20 minutes was necessary for many reasons: Phone networks often have weak connectivity, making long interviews difficult, respondents have shorter attention spans over the phone compared to face-to-face interviews, and minimizing respondent fatigue was crucial to increasing the accuracy of the data and to avoid burdening potentially stressed respondents. However, the questionnaire must also provide a wide snapshot of the population’s conditions, investigating a comprehensive set of topics including education, livelihoods, health, market access, food security, and water, in order to identify which have been most affected by the crisis and to inform a response.
The survey covered the mobile phone users, with a focus on areas deemed to be in ‘emergency’ or worse by the Integrated Phase Classification (IPC).Footnote 10 In order to participate in the survey, households had to own a mobile phone, have network coverage, and a means to charge the phone. As such, one key limitation of mobile surveys is that it excludes households too poor to have a mobile phone , or households that are too isolated to live in areas with coverage. Despite this shortcoming, the survey allows for an immediate, ground-level assessment of challenges related to the crisis, and the survey’s results can be considered conservative estimates of how the entire population is affected, leading to insightful policy interventions.
Sampling strategies must be adaptable to local contexts. In Nigeria, an ideal starting point was to call respondents from previous surveys who represented the intended population, since phone numbers had been collected for previous waves of this survey .Footnote 11 This approach allowed for the comparison of RERS estimates to estimates from the existing survey, which included the non-phone-using segment of the population. Household characteristics can thus be compared between the sample from the previous survey and the RERS, so that the representativeness could be assessed. About 80% of the phone numbers called resulted in successful interviews.
In the absence of existing surveys, a comprehensive list of phone numbers disaggregated by region would provide the best sampling frame ; however, such lists are often unavailable, unreliable, or outdated. A bulk SMS to mobile phone users asking for consent to participate in a survey can provide an alternative sampling frame, from which respondents can be randomly selected. To ensure that the crisis-affected population is represented, it is crucial that any bulk SMS can geographically target crisis-affected regions: This approach was followed in Somalia. While this methodology is effective, it further compromises the representativeness of the survey by requiring respondents to reply to a text message before being interviewed. More than 65% of the numbers called resulted in successful interviews, allowing for fast execution of the survey. However, the actual number of recipients of the bulk SMS is unknown, making it difficult to calculate the percentage of SMS recipients who were interested in participating.
Random digit dialing (RDD) is a tool that randomly generates phone numbers, and can be a practical solution when phone number lists from surveys or bulk SMS dashboards are not available. This approach was followed in South Sudan and Yemen. Machine-learning algorithms can generate random-digit sequences based on a small set of verified existing numbers to create new numbers that are likely to exist. This reduces the loss of time that results from calling non-existent numbers. However, response rates are still unpredictable, especially if the survey targets specific geographic areas. On average, 10% of the numbers called resulted in successful interviews, prolonging the survey’s duration.
3 Key Results
This section describes the collected data and highlights selected trends, starting with similarities between the four countries, and followed by selective country-specific deep-dives. In Nigeria, the survey involved households located in the North-East, North-Central, and South-South zones; of these, the North-East zone includes states classified to be under the emergency phase as per the IPC. For Somalia and South Sudan, only areas declared to be in a state of emergency or worse were surveyed. In Yemen, the survey covered all regions, stratified into emergency and non-emergency areas: Non-emergency regions are sampled because they had pockets of highly food-insecure households.
The proportion of highly food-insecure households was found to be large, but varied widely between the four countries, ranging from 30% in Somalia, to around 50% in Nigeria and Yemen, to over 70% in South Sudan (Fig. 1).Footnote 12 Higher food insecurity was recorded in those countries that faced conflict during the crisis. A high incidence of conflict was reported in Nigeria, South Sudan, and Yemen, while in Somalia, the crisis was primarily due to dry agricultural seasons and a lack of resilience.
In addition to food insecurity , the populations surveyed also faced a range of developmental challenges. Livelihoods were affected in all four countries, with large portions of the populations (ranging from 31% in Nigeria to 84% in Yemen) facing a decrease in income and a change in their main source of livelihood (ranging from 13% in Nigeria to 31% in Somalia). Poor health, insufficient access to water, and low preparedness for drought were also common in all four countries. Other issues such as school attendance and livestock loss were more context-specific (Table 1).
In Nigeria, one in five households lost income over the previous 12 months. Highly food-insecure households were more likely to experience a decrease in income than food-secure households (39 and 21% respectively; Fig. 2). Over one in three households did not store food for future use. Highly food-insecure households were most likely to not store food (39%) compared to households with low or medium food insecurity (28 and 29% respectively; Fig. 2). Early warning systems for drought preparedness and food-storage capabilities might allow for higher resilience and reduce the need for desperate coping strategies.
In Somalia, surveyed household had lost livestock and changed their employment activities. Over 30% of the Somali population owned livestock in the previous 12 months. However, among households that owned livestock, four in five faced a decrease in livestock holdings, with the primary reason being death or disease (66%). Livestock had also been depleted from being sold off (9% of livestock-owning households; Fig. 3). Economic assistance to compensate for livestock losses was clearly necessary to increase household income. The survey found that about 15% of households changed their main employment activity, with a shift away from farm-based employment (15–10% in farm labor; 20–13% in own-account farming), while the number of respondents involved in non-farm businesses increased sharply (17–28%; Fig. 3). Households responded to the drought by compensating for losses in agricultural income through shifts in the labor market.
In South Sudan, most households rely on markets to buy food. However, while food is generally available in markets, households often cannot afford to buy food because of high prices (Fig. 4). These high food prices are not surprising, given South Sudan’s recent period of high inflation. To improve access to food, the survey findings emphasize the importance of vouchers as opposed to food imports.
Almost 40% of the South Sudanese population owned livestock in the previous 12 months. Of these, almost two in three households lost livestock due to death or disease, and theft (25 and 21% respectively; Fig. 4). Livestock restoration could help livestock-based agricultural livelihoods to be regained faster.
In Yemen, food prices and in particular, and school attendance were found to be affected by the crisis. Most households used markets to access food; as such, an increase in food prices was the key challenge in accessing the market for both food-secure and food-insecure households (Fig. 5). Again, this suggests that future interventions should be based on food vouchers rather than food imports. About one in four households had not sent all their children to school regularly in the previous year, largely due to unsafe routes to school and school closures. Safety issues and school closures resulted in low school attendance for both boys and girls (Fig. 5). This underlines the detrimental effects of insecurity on future generations, and the need to restore educational infrastructure.
4 Implementation Challenges, Lessons Learned, and Next Steps
The results of the Emergency Response Survey showcased above emphasize the importance of understanding the local situation, which is context- and country-specific. A quick turnaround from survey inception to results and analysis is a key factor in the usefulness of the data to inform disaster responses. The survey was completed the quickest in Somalia (2600 interviews in ten days), and slowest in Nigeria (around 600 interviews in 25 days; Fig. 6), with the response rates and the size of the enumeration team being key factors in the speed of data collection. In Somalia, the enumeration team consisted of 25 enumerators, which was five times the size of the team in Nigeria. In South Sudan and Yemen, response rates were less than 15%, which increased the survey’s duration. However, despite these various constraints, data collection was quick enough to generate results for each country in eight weeks. The case of Somalia further demonstrates that very rapid data collection can be done with a reasonably sized team of enumerators, even in a constrained environment.
This survey methodology can be deployed rapidly while keeping costs low. Operating through country-based call centers cost roughly $50,000 per country, and the cost per interview was less than $35 in all countries except for Nigeria. The interviews were most cost-effective in Somalia ($23 per interview) and most expensive in Nigeria ($86 per interview; Fig. 6). The bulk of the costs were fixed, and thus a larger sample size drove down the cost per interview. This fixed-cost structure allowed for increasingly cheap future rounds of the survey once the call center infrastructure has been established.
Statistical infrastructure, such as a list of respondents with phone numbers, can accelerate data collection and improve the representativeness of the survey . In Nigeria, phone numbers collected as part of a nationally representative household survey were used to select respondents. Such approaches can save time in preparing the sampling frame for the survey, compared to negotiating with mobile phone providers to provide randomized lists of phone numbers or to send a bulk SMS. It can also minimize the legal implications of mobile phone-based surveys, as some countries do not allow large numbers of unsolicited phone calls. Using data from a nationally representative survey, as was the case in Nigeria, the representativeness of the collected data can be assessed by comparing respondents who participated in the phone survey with respondents who could not be reached, as well as to respondents who either did not provide or did not have a phone number.
Questionnaire design is a crucial step in preparing a survey . In the absence of quantitative data about the impact of the drought on a variety of areas, the questionnaire was designed to explore diverse developmental topics including education, livelihoods, health, remittances, prices, and market access. Survey data clearly indicated that certain topics were more seriously affected than others, warranting more detailed exploration that was impossible in the context of the initial survey. For example, in South Sudan, more than 90% of households had suffered an illness in the previous three months (Table 1); however, while the questionnaire was able to collect details about the most recent illness in the household, the module was insufficiently deep to allow specific conclusions regarding health interventions to be drawn. In hindsight, additional information on household member-specific and less recent illnesses would have been valuable. However, the design of the questionnaire traded in-depth exploration against comprehensive thematic coverage. Another finding from South Sudan was that remittances were not severely affected by the crisis; again, in hindsight, the questionnaire could have been optimized by adding questions on remittances. However, it is impossible to make these choices a priori, especially during an unfolding crisis situation.
The use of an adaptive questionnaire is a promising approach to escape this limitation. The premise is to adapt the questionnaire while collecting data. The first round of the questionnaire should cover a broad range of developmental topics, with an emphasis on preliminary questions assessing the extent to which each topic is affected. After around 500 interviews, trends from the data collection will indicate which topics warrant more exploration.Footnote 13 A survey conducted through a call center allows for the rapid adaptation of the questionnaire, as well as splitting the sample into individually representative parts at no extra cost. Thus, the questionnaire can be adapted after every 500 interviews, with increasing levels of detail on relevant topics and subtopics (Fig. 7, colored green). Less relevant topics can be dropped to keep the duration of the interview manageable (Fig. 7, colored gray). Even saving five minutes by skipping preliminary questions on irrelevant themes can save crucial time in a 20-minute interview for more in-depth exploration of relevant topics.
Adaptive questionnaire design fits well within the fast, low-cost survey methodology of the RERS. Enumerators can be trained on the full, detailed version of the questionnaire before data collection to allow quick adaptation of relevant and irrelevant topics (Fig. 8). The design will create systematically missing values for detailed questions in interviews conducted at the beginning of data collection, and for explorative questions later in the implementation. The random sequence of interviews, however, ensures that the missing data is not biased and, thus, can be analyzed by ignoring missing values. While this will affect standard errors, a large sample, such as the 2500 interviews in Somalia, can ensure sufficiently narrow confidence intervals, even after several adaptations of the questionnaire.
By presenting the example of a pilot survey , this chapter provides proof-of-concept that quantitative data collection via phone surveys is feasible, cost-effective, and informative in the context of shock responses. The pilot highlighted the importance of using an effective questionnaire design, like adaptive questionnaires, to balance the need to comprehensively cover a wide range of topics with the need to collect detailed information on specific sub-topics identified as part of the survey. Although smart and innovative designs can optimize trade-offs, emergency response surveys are no substitute for face-to-face household surveys based on representative sampling frames , ensuring that everybody is included, even the poorest and most vulnerable, who might not own mobile phones . However, compromises need to be made to provide a timely response following a shock, creating a niche for emergency response surveys as presented in this chapter.
Such emergency phone surveys can be prepared and implemented at global and national levels. At the global level, an adaptive questionnaire template can be prepared before emergency situations occur. This will reduce the preparation time needed to adapt the questionnaire template to a country and a specific crisis. At the country level, the groundwork for a survey can be prepared by collecting phone numbers of potential respondents. Questions about phone numbers and the willingness to participate in future survey interviews should be included by default in nationally representative surveys at both the household and firm levels. Lists of phone numbers of respondents who are knowledgeable about specific topics can further add value and allow for more in-depth interviews. These can be obtained by reaching out to sector ministries to collect phone numbers for their staff, which may already be integrated into the HR system. The ability to call, for example, health workers or police officers across an entire country allows for many more monitoring options that would not only be relevant in emergency situations. National statistics offices often maintain such (sufficiently anonymized) phone number databases, and provide them in emergency situations. In crisis-prone countries, the establishment of a call center and, potentially, the use of continuous phone surveys can also further accelerate implementation and provide baseline data.
Emergency phone surveys can play a critical role in crisis analytics, especially if integrated with other data sources that are either recurrently collected in a country, available upon demand, or typically collected during a crisis. For example, market price data is collected in most countries, whether by national statistics offices, UN agencies, or both, and can be triangulated geographically with household interviews. Satellite images can also provide additional context specifically to household interviews, for example, by gathering information about agricultural activity or damage to physical infrastructure. Furthermore, social network data or mobile phone usage data can provide invaluable insight. However, access to such datasets often takes time, making pre-emergency agreements necessary. UN agencies have developed standard data collection methodologies for crisis situations, including the WFP’s Vulnerabilities Assessment and Mapping (VAM), and IOM’s Displacement Tracking Matrix (DTM). For a meaningful integration, the different underlying data sources should be readily available and anonymized at the micro level, including sufficiently disaggregated geographical indicators. This requires pre-emergency agreements between involved agencies, ideally at both global and national levels. This is a valuable effort for the avoidance of redundancies and the creation of new synergies. Thus, technological advancements make crisis analytics an extremely powerful tool to inform crisis responses. To harvest their full potential, more pilots must be carried out, and specific statistical infrastructure as well as collaborations and agreements between different stakeholders will be needed.
Notes
- 1.
- 2.
Food Security Outlook Update Nigeria, Famine Early Warning Systems Network (2017); Post-Gu Technical Release Somalia, Food Security and Nutrition Analysis Unit and Famine Early Warning Systems Network (2017); Food Security Outlook Update South Sudan, Famine Early Warning Systems Network (2017); Food Security Outlook Update Yemen, Famine Early Warning Systems Network (2017).
- 3.
Food Security Outlook Update Nigeria, Famine Early Warning Systems Network (2017).
- 4.
Post-Gu Technical Release Somalia, Food Security and Nutrition Analysis Unit and Famine Early Warning Systems Network (2017).
- 5.
The UN officially declared famine in parts of Unity State, South Sudan: https://unmiss.unmissions.org/famine-declared-parts-south-sudan; Key IPC Findings: January–July 2017, Integrated Food Security Phase Classification (2017).
- 6.
Food Security Outlook Update Yemen, Famine Early Warning Systems Network (2017).
- 7.
Making the Links Work: How the humanitarian and development community can help ensure no one is left behind, Inter-Agency Standing Committee (2014).
- 8.
New Way of Working, United Nations Office for the Co-ordination of Humanitarian Affairs (2017).
- 9.
The call centers were located in-country for Nigeria, South Sudan, and Somalia, and in Egypt for Yemen.
- 10.
Guidelines on Key Parameters for IPC Famine Classification, Integrated Food Security Phase Classification (2016).
- 11.
General Household Survey 2016, conducted by National Bureau of Statistics of the Federal Government of Nigeria under Poverty and Conflict Monitoring Systems.
- 12.
Food security scores are based on the Reduced Coping Strategies Index Score and adapted to define lower scores for less food-secure households. Reduced Coping Strategies Index Score is calculated using the CSI Field Methods Manual, Cooperative for Assistance and Relief Everywhere (2008).
- 13.
Around 350 observations is the minimum sample size that provides a 95% confidence interval for estimates. Thus 500 is a sufficient sample size to map early data trends.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
The opinions expressed in this chapter are those of the author(s) and do not necessarily reflect the views of the International Bank for Reconstruction and Development/The World Bank, its Board of Directors, or the countries they represent
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 3.0 IGO license (https://creativecommons.org/licenses/by/3.0/igo/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the International Bank for Reconstruction and Development/The World Bank, provide a link to the Creative Commons license and indicate if changes were made.
Any dispute related to the use of the works of the International Bank for Reconstruction and Development/The World Bank that cannot be settled amicably shall be submitted to arbitration pursuant to the UNCITRAL rules. The use of the International Bank for Reconstruction and Development/The World Bank's name for any purpose other than for attribution, and the use of the International Bank for Reconstruction and Development/The World Bank's logo, shall be subject to a separate written license agreement between the International Bank for Reconstruction and Development/The World Bank and the user and is not authorized as part of this CC-IGO license. Note that the link provided above includes additional terms and conditions of the license.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2020 International Bank for Reconstruction and Development/The World Bank
About this chapter
Cite this chapter
Pape, U. (2020). Rapid Emergency Response Survey. In: Hoogeveen, J., Pape, U. (eds) Data Collection in Fragile States. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-030-25120-8_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-25120-8_3
Published:
Publisher Name: Palgrave Macmillan, Cham
Print ISBN: 978-3-030-25119-2
Online ISBN: 978-3-030-25120-8
eBook Packages: Economics and FinanceEconomics and Finance (R0)