Data Coding Project Overview

Qualitative data are typically very “rich” but also “messy.” These data are often very detailed and diverse,
which allows researchers to deeply understand the phenomena being studied. However, there are often
no established metrics or measures—structure has to be deduced and imposed. This coding and analysis
process is highly iterative. Analysts must review the data carefully to identify important categories, define
and operationalize those categories, and then test how well the categories describe the data (i.e., with
reliability and validity). To reduce the amount of “trial and error,” good analysts are mindful, clear, and
detailed at every stage. Whenever possible, it is ideal to build upon existing theories and methods.

Throughout this course, students will conduct a content analysis of authentic data. These analyses will be
divided into several components to permit frequent discussion and questioning at each stage. This work
will also be collaborative but every student will be responsible for individual work. Students will also be
provided with an initial conceptual framework to guide this process.

Project Stages

(1) Data Review: in this portion of the project, students will iteratively read and review collected data
to make detailed observations related to the conceptual framework. Specifically, students will
examine the data to extract quotes and excerpts that seem to fit theoretical categories. Students
will also begin to develop operational definitions of the categories along with potential coding
rules that allow an analysts to make coding decisions.

(2) Coding Guide: in this part of the project, students will formalize their observations, operational
definitions, and coding rules. The coding guide will serve as an “instructional manual” that
explains to analysts how to categorize the data. The coding guide will build directly upon the data
review, but will provide more formal and comprehensive structure.

(3) Data Coding: in the final part of the project, students will implement a coding guide to formally
code subsets of the data. In pairs, students will independently code a fraction of the data and then
test for inter-rater agreement (i.e., kappa). After an initial round, partners will meet to discuss
and ideally resolve their disagreements, discrepancies, and errors. For the final deliverable,
students will code the full set of data independently.