Exploring Connectivist Massively Open Online Course (cMOOC) Microblogging

Data through a Student Assessment Lens

ELI Annual Meeting, 2015
Laura Gogia, MD
Graduate Fellow
Virginia Commonwealth University ALT Lab
gogialp@vcu.edu
Abstract
In 2014 six Virginia Commonwealth University (VCU) faculty designed and implemented an eight weeklong summer pilot course called Digital Engagement: Living the Dreams, Digital Investigations and
Unfettered Minds, or UNIV 200: Inquiry and the Craft of Argument. While most of the course took place
on open blogging platforms, an associated Twitter community emerged around the hashtag
#Thoughtvectors. Intentionally initiated by course instructors, #Thoughtvectors Twitter activity was
captured in a Twitter Archiving Google Spreadsheet (TAGS) and informally shared with all participants
through the course website. This study explores the TAGS data through the lens of student assessment. It
finds that while it is possible to create dashboards for student assessment from TAGS data, optimal use
might require adapting archiving spreadsheets to capture data specific to formalized connected learning
objectives and activities.
Study Purpose
In todays educational climate, colleges and universities are expected to provide evidence that curriculum
and instructional practices significantly enhance students ability to meet learning objectives and goals.
As the use of connected learning techniques such as blogging and microblogging become more common,
questions related to their evaluation will inevitably follow.
This study was an initial step in a research agenda that aims to advance understanding of how to best
document or assess connected learning in formal higher education settings. Research questions included:

What data are captured by Twitter Archiving Google Spreadsheets (TAGS)?

How might data collected in this format be used to describe the microblogging behaviors of
learning communities, their subgroups, and individual participants?
Is it possible to develop a dashboard for student assessment from data captured in Twitter
Archiving Google Spreadsheets? If so, what are its limitations?

Methodology
A Twitter Archiving Google Spreadsheet (TAGS; https://tags.hawksey.info/) was used to capture
#Thoughtvectors activity from the Twitter Application Programming Interface (API). As shown in Figure
1, the spreadsheet was monitored and maintained from the first use of the #Thoughtvectors hashtag
until one month after the formal course (UNIV 200) had ended.

Figure 1. Thoughtvectors Timeline.

Using public data available on Twitter and the course site for UNIV 200: Inquiry and the Craft of
Argument (Thoughtvectors.net), the Thoughtvectors community participants were placed into the four
subgroups defined in Figure 2. A comprehensive quantitative content analysis of #Thoughtvectors tweets
was performed to extract information related to mentions, retweets, and links. This additional
information was added to the spreadsheet to enhance the analysis. Descriptive statistics were generated
through Excel (Microsoft, 2013). Social network analysis was performed through the open source Excel
template, NodeXL, available through the Social Media Research Foundation and CodePlex
(http://nodexl.codeplex.com).

Students

Instructors

95 VCU students enrolled in UNIV 200; of those, 68

were identified as participating in course-related
Twitter activity.

Thoughtvectors was facilitated by six faculty who

possessed a total of eight Twitter accounts.

Thoughtvectors
Community
Open Participants

Others

35 non-students enrolled in Thoughtvectors and

over a dozen VCU faculty and staff were recruited
to support Thoughtvectors. Of these, 28 engaged
in #Thoughtvectors-related Twitter activity.

480 individuals not enrolled in Thoughtvectors and

not afiliated with VCU participated in
#Thoughtvectors-related Twitter activity.

,
Figure 2. The Thoughtvectors Community

Results
Research Question 1: What types of microblogging patterns can be studied through publicly available
Twitter data?
Table 1 outlines the descriptive statistics and social network analysis metrics made available by the
#Thoughtvectors TAGS data. While these numbers represent the entire time period collected (February
to August, 2014), the tweet timestamps automatically collected by TAGS allows for the application of

dynamic filters and isolation of specific time periods. Figure 3 provides sociograms of the whole
community and student components of the #Thoughtvectors network. Node colors represent subgroups,
while node size correlates to out-degree centrality..
Table 1. Descriptive Statistics and Social Network Analysis Metrics.
STUDENTS

INSTRUCTORS

OPEN
PARTICIPANTS

OTHERS

WHOLE
COMMUNITY

TOTAL TWEETERS

64

28

480

580

TOTAL PARTICIPANTS

68

28

671

775

UNIQUE TWEETS

453

1212

1255

997

3917

MENTIONS*

288

>1090

>868

>957

>3203

LINKS

135

691

762

619

2207

RETWEETS
RANGE OF ACTIVITY (# OF
TWEETS)
MEAN TWEETS/TWEETER

107

534

180

731

1552

1-93

28-775

1-344

1-69

0-775

203

46

MEDIAN TWEETS/TWEETER

94

18

IN-DEGREE CENTRALITY RANGE

0-31

0-198

1-75

0-32

0-198

MEAN IN-DEGREE CENTRALITY

53

13

0-17

0-176

1-83

0-38

0176

44

14

2.16

0-12123

2983-296355 0-88356

0-31248

0-296355

804

56680

263

1286

DESCRIPTIVE STATISTICS

SOCIAL NETWORK ANALYSIS

OUT-DEGREE CENTRALITY
RANGE
MEAN OUT-DEGREE
CENTRALITY
BETWEENESS CENTRALITY
RANGE
MEAN BETWEENNESS
CENTRALITY

11239

*Mention data are estimated because Tweets mentioning more than four individuals were clustered in the analysis. In
general, few Tweets mentioned more than four people.

A. Whole Community Twitter Activity

B. Student Twitter Activity

Figure 3. Sociograms of whole community and student #Thoughtvectors Twitter activity from February to August 2014.

Research Question 2: What aspects of

these data might be useful in
characterizing the behavior of groups
and subgroups within Twitter-based
learning communities?

Who's Tweeting?
12%

25%

Using the descriptive data available, I

was able to answer the following
questions:
In the designated time period*, who
tweeted with the #Thoughtvectors
hashtag? This is important for describing
who is functioning in the community.
(*Note that presented data takes into
account the data collected from February
to August. The capability to narrow the
time frame was present).

31%
32%

Students

70%
60%
Students

% Tweets

40%

Instructors

30%
20%

Open
Participants

10%

Others

0%
0

Open Participants

Others

Figure 4. Subgroup contribution to #Thoughtvectors activity.

Are Tweets Directed Towards People?

50%

Instructors

4+

# Mentions
Figure 5. Mentioning behaviors of #Thoughtvectors subgroups.

Are tweets directed towards individuals? In

other words, do Thoughtvectors participants
mention specific individuals in their tweets? If
so, how many mentions in each tweet, and do
the different subgroups choose to mention in
unique patterns? This may be useful in
identifying the formation of groups, presence
of dialogue, or strategic navigation through the
Twitter landscape. It may also speak to
different levels of digital literacy.
Are Thoughtvectors participants including
links in their tweets? Do the subgroups have
different frequencies of link use in their
tweets? This may speak to the connection of
ideas, examples, or resources outside those
provided. It may also speak to varying levels of
digital literacy.

Who are Thoughtvectors participants mentioning? Do the subgroups have unique patterns in who they
choose to mention? Are Thoughtvectors participants retweeting, and if so, who are they retweeting?

Who is Being Retweeted?

5%
30%
32%

33%

Students

Instructors

Open Participants

Do Tweets Include Links?

Yes

43%

Others

A re Tweet s Ret weet s ?

No

Yes
38%

39%

No
27%

44%
56%

70%

76%

57%

62%

61%

60%
86%
73%

56%

44%

30%

24%

40%
14%

Who Is Being Mentioned?

Students

Instructors

Open Participants

Others

12%
37%

26%

35%

30%

34%

22%

31%

27%

39%

26%

28%
18%

42%

STUDENTS

33%

17%
12%

13%

INSTRUCTORS

OPEN PARTICIPANTS

Figure 6. Additional Characterizations of #Thoughtvectors Tweets

4%
OTHERS

12%
WHOLE COMMUNITY

Research Question 3: What aspects of these data might be useful in characterizing individual
performance within Twitter-based learning communities?
Individuals in the #Thoughtvectors twitter community engaged at different levels. Since each subgroup
and individuals within each subgroup had different motivations for participation, this finding was
expected. Moreover, some instructors required students to engage in Twitter-related activity while others
did not. Nevertheless, the available data and data visualizations allowed for differentiation between
students and their activity in ways that may be meaningful for assessments in more formalized
environments. Figure 7 demonstrates how descriptive statistics and social network metrics might be
compiled into a dashboard to document varying levels of student participation in a connected learning
environment.

Student A
#Tweets: 85
#Mentions: 66
#Links: 11
#Retweets: 64
In-Degree Centrality: 5
Out-Degree Centrality: 17
Betweeness Centrality: 3110

Student B
#Tweets: 24
#Mentions: 5
#Links: 12
#Retweets: 6
In-Degree Centrality: 5
Out-Degree Centrality: 9
Betweeness Centrality: 1108

Student C
#Tweets: 0
#Mentions: 0
#Links:0
#Retweets: 0
In-Degree Centrality: 1
Out-Degree Centrality: 0
Betweeness Centrality: 0

Documenting Student Participation through TAGS Data

Conclusions
This study was an initial step towards understanding how we might document connected learning
in formal higher education settings. It is not our intention to suggest that learning can be documented
wholly through basic descriptive statistics, but rather that these data offer a potentially interesting and
scalable avenue for further research.

Conclusion 1. It is possible to use TAGS data to develop dashboards for the assessment of student
microblogging behaviors.
The data collected from TAGS are appropriate for descriptive statistics and data visualizations consistent
with formative and summative assessments of group and individual student performance.
Conclusion 2. It is better to tailor data archiving spreadsheets to the pedagogical objectives of the
microblogging activity.
In this case researchers hoped to capture the incidence of student connection-making because of the
pedagogical importance placed on making connections across space, time, and disciplines in connected
learning environments. Information regarding mentions, links, and retweets was available through
TAGS, but it had to be mined from text cells in a time-consuming content analysis. Such an analysis
would not be feasible in an authentic, real-time educational context of similar size. Ideally archiving
spreadsheets could be designed to mine much of this information automatically.
Conclusion 3. If assessment strategies around connection-making are to be implemented, instructors
must send clear messages to their students regarding the usefulness of microblogging as a connectionmaking tool.
Individuals in the #Thoughtvectors community tweeted in very different ways. Heterogeneity in student
use might have been related to the lack of consistent and formalized pedagogical messaging around the
use of Twitter for class-related activity. If microblogging is to be seen as a powerful learning tool that
affords (1) connection of ideas across space, time, and disciplines; (2) strategic navigation through
distributed discourse; and (3) signal amplification; and if students are to practice using it as such, explicit
pedagogical messaging around these goals should be in place before the students ability to perform
such tasks are assessed.
Limitations. The current study is limited by the fact that the research was performed only with publicly
available data; identifications of individuals within each subgroup could not be confirmed with official
registration lists, nor could any of the metrics be compared with more traditional assessments such as
final grades. However, given that the primary purpose of the study was exploratory and focused on how
best to design data collection spreadsheets for future research, these limitations were considered
acceptable.
Future Research. Next steps in the research agenda include

Gaining a better understanding of how students and faculty use social media tools in connected
learning environments.
Building evidence to suggest certain digital practices or patterns of practice (e.g. mentioning,
linking) promote the sort of digital engagement, digital literacies, and connection-making valued
in connected learning environments.
Developing archiving spreadsheets specific to the purpose of documenting student learning in
connected learning environments.
Developing meaningful, scalable, and flexible assessment systems for connected learning that
may incorporate the data from digital artifact archiving spreadsheets similar to or based on
TAGS.