Online Interactive Experiments on Networks
Description
Conducting human experiments using crowdsourcing platforms, such as Amazon Mechanical Turk, has made it possible to collect a much larger amount of experimental data in a much shorter period of time relative to what was possible in traditional physical lab settings. This has provided a new suite of methods for conducting randomized experiments in socio-technical systems, allowing for straightforward causal inference. However, using crowdsourcing platforms to experimentally study real-time interactions between individuals presents numerous practical challenges. These studies need fairly large groups of subjects to be present simultaneously in each session, and outcomes typically occur at the level of the group (i.e., session) rather than the individual. Yet most crowdsourcing platforms are not designed to facilitate simultaneous structured interactions between subjects. Thus, it can difficult (and expensive) to recruit enough participants to achieve a sufficient degree of statistical power (especially for session-level outcomes). In this tutorial, we will discuss best practices for designing and conducting online social network experiments where human subjects (and programmed bots) interact simultaneously within a specified network structure. We will show how the experimental design can be informed by computational models in an iterative process (i.e., using experimental data to calibrate the computational model and use the computational model to optimize the design and find the right parameters for the experiments). We will also introduce additional tools/platforms that facilitate conducting such studies and walk the audience through the implementation steps of a typical experiment on networks using customized and publicly available software.
Schedule and Activities
- Introduction We will talk about using crowdsourcing platforms to run experiment with human subjects, single-player vs multi-player experiments, and setups which require concurrent interactions between human subjects.
- Background We will cover type of studies that can benefit from experimental social network approach, e.g., collective intelligence, collective problem solving, and formation of social norms.
- Design techniques We will talk about implementation techniques such as synching interactions between subjects, designing asynchronous (where subject can decide at any moment of time during the experiment) vs synchronous (round-based games, where all subjects have to make decisions in a given period of time). We then talk about practical aspects, such as over-recruitment, screening, waiting room, structured communication, and drop-outs.
- Implementation Demo We will walk the audience through implementing a typical social network experiment on a software platform (e.g., Breadboard) and illustrate how different stages of the experiment can be implemented within the platform.
Organizer
Mohsen Mosleh, Postdoctoral Scholar, MIT Sloan School of Management, [email protected]
Software
References
[1] Becker, Joshua, Devon Brackbill, and Damon Centola. “Network dynamics of social influence in the wisdom of crowds.” Proceedings of the national academy of sciences (2017): 201615978.
[2] Shirado, Hirokazu, and Nicholas A. Christakis. “Locally noisy autonomous agents improve global human coordination in network experiments.” Nature 545.7654 (2017): 370.
[3] Nishi, Akihiro, et al. “Inequality and visibility of wealth in experimental social networks.” Nature 526.7573 (2015): 426.
[4] Rand, David G., Samuel Arbesman, and Nicholas A. Christakis. “Dynamic social networks promote cooperation in experiments with humans.” Proceedings of the National Academy of sciences 108.48 (2011): 19193-19198.