Measuring Collective Intelligence

UROP Candidate: Eric Emer February 12, 2013

UROP Faculty Supervisor: Prof. Thomas Malone UROP Direct Supervisor: Lisa Jing, David Engel UROP Location: MIT Center for Collective Intelligence, NE25, 7th Floor

Project Overview

The research at the MIT Center for Collective Intelligence (CCI) intends to answer the question, How can people and computers be connected so thatcollectivelythey act more intelligently than any individuals, groups, or computers have ever done before? By considering the widespread accessibility of the internet and digital tools, as well as the depth of knowledge resources on the internet, the way people interact and communicate is becoming rapidly more connected. This leaves many research questions regarding "collective intelligence" unanswered. One such question that I will help address in this UROP is how to measure collective intelligence. MCI (Measuring Collective Intelligence) builds o a foundational study done at CCI called Evidence for a Collective Intelligence Factor in the Performance of Human Groups" (Woolley et al, 2010). This study established that collective intelligence (CI) is stable factor that explains group performance on a wide variety of tasks much like a group IQ would. Interestingly, it is not correlated with the individual intelligence of members, but with group member qualities such as social sensitivity, conversational turn-taking, and the proportion of women. We hope to examine whether building an ecient online instrument to measure CI is feasible. A laboratory battery has already been established at the CCI which assesses CI. Recently, this oine battery has been adapted into an online battery. We hope that by cross-validating the measurements of the oine CI battery and the online CI battery we can determine the accuracy of the online instrument. Ultimately, we hope to determine whether or not the online battery can be a reliable indicator of CI. There are many considerations we must take into account when looking into measuring collective intelligence. We are also interested in how the group member factors that aect CI online and oine compare. Certainly, there are dierent environmental factors in the oine 1

and online scenario, and we are interested in examining what factors might account for the variability between the two measurements. Within this project, we also hope to investigate whether or not a measured collective intelligence statistic is predictive of group performance in eld contexts, such as corporate teams and university project teams.

Personal Role and Responsibilities

I will be involved in several facets of this research. Firstly, I may be running human subjects online with the MCI battery. To do this, I will rst need to become certied for human subject testing. This will allow the MCI group to expand their current datasets. The current infrastructure for retrieving data from the central server involves a somewhat onerous series of operations. I would aim to streamline this task by creating an automated downloading and archiving tool that retrieves data from the server to a local folder. We expect that by expediting the data retrieval process we remove some of the overhead involved in conducting thorough data analysis. In addition, I will be involved in the task of performing data analysis on the MCI battery results. There are currently several dierent formats of data from the study: statistics, audio les, and chat logs. I will investigate creative methods for using the non-standard data formats for measuring CI. I will also analyze how verbal and non-verbal tasks in the battery compare, and I will examine how predictive they are of CI. Ideally, by extensively analyzing the data recorded from the online battery tests, I will be able to determine which of the tasks in the online battery are most predictive of CI. This may allow us to reduce the duration of the online battery from 60 minutes to a target of 15 minutes.

Goals

I hope to be able to provide a needed service to the MCI group of the MIT CCI. One goal that I have for this UROP is to explore creative means and ultimately to nd a useful method for analyzing and compiling the vastly dierent forms of data available into one comprehensive predictor of CI. I think this would be a good learning opportunity for me. It would allow me to expand my arsenal of data analysis skills. I aim to provide lasting contributions to the MCI groups psychological battery. Additionally, I expect to provide the MCI group with automation tools that will assist with data evaluation. Through these means, I seek to simultaneously improve my quantitative skills and my programming ability.

Personal Statement

This project interests me because it seeks to solve an interesting problem. It is easy to see the benets of being able to accurately measure collective intelligence. This would allow us to optimize the speed and quality of group problem solving, as well as group decisionmaking. This project bridges the elds of applied math, computer science, and psychology. As an individual with a background in mathematical modeling and computer science, as well as a budding interest in psychology, I nd this UROP position to be a good t. Through this UROP, I expect to learn a great deal from my UROP supervisors regarding eective experimentation, professional data analysis, database management, and the many concerns and applications of accurately measuring collective intelligence.