This year UNHCR, World Bank, and FCDO commissioned an impressive twenty-six papers on social cohesion.
But how did researchers measure it?
The paradox of social cohesion is that its increasing ubiquity in current policy discussions has not resulted in any consensus on what exactly social cohesion is. Definitions of social cohesion vary across academic disciplines and between defining agencies, and the measurement of the concept is equally as wide-ranging. It is within this challenging research context that the World Bank, together with UNHCR and supported by the UK Foreign, Commonwealth and Development Office (FCDO), commissioned a set of papers exploring the relationship between forced displacement and social cohesion. In this blog I explore their approach to measuring social cohesion and highlight some key takeaways.
You can find my consolidated database of measurement approaches (and notes on coding) on GitHub. I include 21 of the 26 total papers in my analysis, as the remaining five papers explored mediators of social cohesion rather than the outcome itself. In addition, as 2 studies used games or survey experiments, indicators were not explicitly assigned to a single dimension. These studies are excluded in the tables below but highlighted in finding 3 for their innovative measurement approaches.
Findings
What were the most common dimensions and indicators used?
From the 19 papers, freedom from violence or crime was most common overall, particularly with studies using secondary data. Attitudes accepting diversity and trust in individuals were commonly seen as well, also relying heavily on secondary data. Of studies collecting primary data, surveys most frequently included measures of social participation.
Table 1. Number of studies by dimensions measured
Zooming in on the dimensions, I then identified the top four most frequently used indicators:
Table 2. Top four most frequently used indicators with examples
Is social cohesion multi-dimensional?
The vast majority of the 19 studies included indicators from more than one dimension, indeed recognizing the multi-dimensional nature of social cohesion.
However, a lack of available data (particularly in fragile or conflict-affected settings where displacement is most often found) may prevent researchers from being able to measure social cohesion across several dimensions, choosing instead to focus their analysis on one aspect. Graph 1 below describes how frequently these studies include multiple dimensions. Six of the studies looked at only one dimension. Of these, five of them used secondary data, which accounts for why most covered the dimension of ‘freedom from violence or crime’ – this type of data is easily found globally from sources such as ACLED.
Graph 1. Frequency: number of dimensions included in social cohesion measurement
What are some interesting innovations in measuring social cohesion?
Four papers caught my attention for their use of creative methods to capture a difficult concept:
Lab-in-Field Games
While the use of games is not technically new in social cohesion research (this interesting paper from Nepal comes to mind!), I appreciate the way Ferguson, et al. use lab-in-the-field games in their study. Specifically, they use a dictator game and the stag hunt game to measure social cohesion for their study of vocational training involving both hosts and refugees in Jordan and Lebanon. By layering an experiment with the games, they test whether varying the type of game partner (host or refugee) would change respondents’ resulting behavior. The analysis gives interesting insights into the intersection of altruism, acceptance of diversity, and trust.
Creative Survey Tools
Using a conjoint experiment, COMPAS researchers Will Allen and Carlos Vargas-Silva, with colleague Isabel Ruiz tease out which aspects of Venezuelan immigration policy would be more favored by Colombian hosts, and find that those who have less contact with Venezuelans tend to support more restrictive policies. The paper suggests that increasing quality of contact between diverse groups may be one way to foster more positive attitudes toward that diversity.
Social Network Data
Walk, Garimella, and Christia use social network data from Twitter, Telegram, and Facebook to identify how online conversations mediate decisions of Syrian refugees to return to Syria, when low levels of violence are still present. While this research is less explicitly related to social cohesion, it provides a good example of how social network platforms could provide useful data in fragile- or conflict-affected settings.
Sequential mixed methods
Pham et al. challenge the traditional (often Western) definitions of social cohesion by eliciting community members’ own understandings of the concept. The authors conducted focus groups before the quantitative data collection to define their measurement approach.
Conclusion
The papers suggest that scholars agree on certain key dimensions of social cohesion: trust, freedom from violence, and acceptance of diversity are all frequently measured. However, this obscures an underlying need to enhance our measurement for impact evaluations in conflict- and crisis-affected contexts. Choices to include freedom from violence and acceptance of diversity are likely influenced by the fact that these are the types of secondary data most commonly available in difficult contexts (from sources such as ACLED or other general surveys). Measurements of trust, while frequently used in primary collection, are also known to be particularly vulnerable to change, such that an over-reliance on it as a key indicator may not accurately reflect true changes in social cohesion (see COMPAS scholar Neli Demireva’s discussion of this.) A challenge for researchers moving forward, then, is to develop new measurement tools adapted to contexts of displacement, which can provide a fuller understanding of social cohesion.
Kristen McCollum is a DPhil student in Migration Studies and a Grand Union DTP Scholar with an Advanced Quantitative Methods award. She is currently working with the World Food Programme to coordinate a set of experimental impact evaluations to improve programming in conflict- and crisis-affected settings.
