Technology and Student Learning

The Impact of Enhanced Podcasts on Student Learning Outcomes

Yianna Vovides (yvovides@gwu.edu), P.B. Garrett
The George Washington University

Research Question: Are there differences in students’ performance on targeted as-

sessment instruments when supplemental course material is presented via traditional
text-based materials versus enhanced podcasts (graphics plus audio)?
Research Design: The study used a quasi-experimental design, specifically, a pre-
post test nonequivalent groups switching replications design. The table below, shows
the study design that was followed for topic 1. For topic 2, the sections that were in
the podcast treatment were assigned to the text-version (control) and vice-versa.
Topic 2 included the same number and type of dependent variables as topic 1.
T opic  1   Topic 1  
T opic  1   Topic 1   Topic 1  
T A  Se ction  T ext‐v ersion  Critic al 
Pre‐T es t  Podc ast  Pos t‐Te st 
(c on tro l)  S um m a ry 
A  O  X    O  O 
1 
B  O  X O  O
C  O  X    O  O 
2 
D  O  X O  O
E  O  X O  O
3 
F  O    X  O  O 
G  O  X O  O
4 
H  O    X  O  O 
I  O  X O  O
5 
J  O  X O  O
6  K  O  X    O  O 
 
About the course/participants: The course had 262 students registered. It con-
sisted of two weekly 75-minute lectures and 50-minute discussion sections between
the lectures. There were a total of 11 discussion sections led by six graduate teach-
ing assistants. The 11 discussion sections met throughout the day. Approximately
80% of the students were under 20 years of age. In terms of gender, males com-
prised 43% of the class with females at 57%.
Procedure & Instruments: The figure below shows the procedure for topic 1. We
collected self-reported surveys (Background, Final, LSI, MSLQ). In addition, a pre-
test and a post-test were administered for each topic. The questions on the test as-
sessed Remembering, Understanding, and Evaluating levels of learning. Final course
grades and
student
level were
also used.
Technology and Student Learning: The Impact of Enhanced Podcasts on Student Learning Outcomes
Yianna Vovides, P.B. Garrett, The George Washington University page 2

Analysis: The factors that were analyzed to examine student variance included age,
gender, student level, learning styles, and motivation and learning strategies. The
statistical procedures used to contact the analyses included Analysis of Covariance
(ANCOVA) using the pre-post difference scores with the pre-test scores as a covari-
ate, and Linear Regression (where appropriate).

Delivery Mode Preference, Engagement, and System Usage: A delivery mode

preference question included in the Background Survey was also included in the Final
Survey. The following table shows the comparison on delivery mode preference from
the Background Survey (beginning of the semester) to the Final Survey (end of the

D e li v e r y  M o d e  B a c k g r o u n d  S u r v e y   F i na l  S u r v e y  
P r ef er e nc e   (B e g i n ni n g   o f  S em e st e r )  (E nd   o f  S e m e st e r ) 
Po d c a st  ( A u d io /V i su a l )  21 62
T ex t ‐v e r sio n  ( P DF )   1 33   1 30  
N o  P r efe r en c e  62   24  
N =  2 16 2 16
 
Learning Outcomes: We found several factors impacting learning outcomes in rela-
tion to the stimulus and control. We highlight key results. The study also included
self-reports on motivation and learning strategies using the MSLQ; the MSLQ scales
are correlated with academic performance. We analyzed these self-reports in relation
to gender. Using Pearson chi-square tests, we found several moderate to strong as-
sociations in both the motivation and learning strategies sub-scales: Extrinsic Goal
Orientation (Motivation sub-scale), Self Efficacy for Learning and Performance
(Motivation sub-scale), Rehearsal (Learning Strategy sub-scale), Help Seeking
(Learning Strategy sub-scale).

We examined gender by modal-

ity for students under 20
(N=115). The interaction be-
tween gender and treatment is
statistically significant with a p-
value of .040, F(1,102)= 4.456
at the Remembering level of
learning. The graph on the left
shows this interaction.
 Text-version (control):
Males (N=22), Females
(N=32)
 Podcast: Males (N=25), Fe-
males (N=23)
Technology and Student Learning: The Impact of Enhanced Podcasts on Student Learning Outcomes
Yianna Vovides, P.B. Garrett, The George Washington University page 3

The graph (left) shows the

estimated marginal means
by treatment in relation to
the MSLQ Motivation Extrin-
sic Goal Orientation sub-
scale for students under the
age of 20. The interaction
between treatment and ex-
trinsic goal orientation levels
has a p-value of .004, F
(2,100)=5.773.
 Text-version (control):
Low (N=15), Average
(N=23), High (N=14)
 Podcast: Low (N=13),
Average (N=24), High
(N=11)
Regression analyses came
back with the Low (p=.005)
and Average (p=.002) levels
showing statistically signifi-
cant results.

The graph (left) shows that

at the Understanding level of
learning there is a statisti-
cally significant main effect
as well as interaction be-
tween treatment and stu-
dent level. The student level
main effect has a p-value
of .048, F(2,98)=3.141. The
interaction between treat-
ment and student level has
a p-value of .031, F(2,98)
=3.621.
 Text-version (control):
Weak (N=19), Average
(N=23), Strong (N=12)
 Podcast: Weak (N=13),
Average (N=22), Strong
(N=9)
Regression analysis came
back with the strong student
rating category as statisti-
cally significant with a p-
value of .028.
Technology and Student Learning: The Impact of Enhanced Podcasts on Student Learning Outcomes
Yianna Vovides, P.B. Garrett, The George Washington University page 4

Conclusion:

We found that there is no statistically significant difference between the two modali-
ties (podcast vs. text) overall. However, when we explored student characteristics in
terms of age, gender, motivation, and learning strategies we discovered that
there are differences in achievement by treatment. Such differences stem from
mostly the Understanding level of learning.

It seems that podcasting, instead of text, can be more beneficial for certain types of
students and that learner differences as well as the levels of learning should be care-
fully considered when designing instructional materials. In terms of motivation, stu-
dents who reported low levels of perceived interest, importance, and utility of the
course subject matter (motivation task value sub-scale) scored higher in the podcast
treatment than the text version. This is consistent with past research that reported
increased levels of engagement with podcasts. In fact, in this research study, the
number of students who indicated that they prefer podcasts over the text-version
jumped from 21 at the beginning of the semester (prior to experiencing it) to 62 at
the end of the semester.

Poster #13
EDUCAUSE 2010
Contact: yvovides@gwu.edu

Technology and Student Learning

The Impact of Enhanced Podcasts on Student Learning Outcomes
Yianna Vovides, P.B. Garrett
The George Washington University