A Method for Leveraging Public Information Sources for Social Science Research

In recent years, scholars have made great strides using computer-assisted methods to collect and analyze large volumes of machine-readable information from open sources such as online news feeds and the new social media to aid in these efforts. The result has been the production and analysis of massive amounts of data at unprecedented scales and speeds. Individual scholars and research teams have experimented with multiple approaches to collecting and analyzing such data. This experimentation has promoted innovation and exploration, but it has also produced an array of data sets and methodologies that are not well integrated or coordinated. The result is that scholars and policy makers alike are learning less from these data than they could. This project seeks to overcome that obstacle by developing the community and capacity needed to establish common best practices and foster greater integration and coordination regarding the collection, dissemination, and analysis of data. As part of this development process researchers will focus on extracting information from public documents such as news reports. The scope includes events which involve disagreement ranging from nonviolent political protest to terrorism and interstate war - issues which have long been a subject of inquiry in the social sciences.

The project will accomplish this through a series of workshops that will bring together national leaders in the areas of inter-state and intra-state conflict, data management, quantitative methods, and computer science to develop a coordinated strategy for moving forward. In conjunction with these workshops, the project will develop a prototype system of software tools designed to collect and archive such data, as well as conducting extensive educational outreach and training programs in the use of the data and tool sets. The entire project will be supervised an advisory board of specialists.

The project will advance the capacity for learning about the early warning signs of contentious political events, what leads to their escalation or de-escalation, and the resulting consequences of such events. The project will also advance understanding of massive near-real-time data collection for the analysis of conflicts as well as the tools needed to manage, share, and analyze such data sets more generally. More broadly, this project will provide access to a vast new open data resource for researchers from academia, government, industry, and the non-profit sector. It will provide training opportunities for using the data, and also provide the foundation for data-oriented education programs designed for students ranging from middle school through graduate school that can help students learn about the causes and consequences of inter-state and intra-state conflict. Finally, the project will help policy makers make better decisions, which could potentially reduce the frequency and/or scale of political conflicts, and ultimately help to save lives.