Twitter Basic Training

Implementation and Evaluation Report

for

ACME Chem-Solutions, Inc.

by

Designed for Learning

August 2013

Twitter Basic Training, Evaluation Plan

Contents
Contents .....................................................................................................................................2
Introduction: ...........................................................................................................................3
Module Overview and Description:..........................................................................................3
Description of Implementation: ...............................................................................................3
Analysis of Evaluation Data: .....................................................................................................4
Proposed Revisions and Key Points: (Embedded Audio File) ....................................................6
Reference List: .........................................................................................................................6
Appendices: .............................................................................................................................6
Change Record ......................................................................................................................12

2|Page

Twitter Basic Training, Evaluation Plan

Introduction:
ACME Chem-Solutions, Inc. experienced a decline in product sales over the past few years. The
results of a business analysis pointed to a lack of marketing information reaching potential
customers. The idea of using Twitter was introduced and the company whole-heartedly embraced
the idea as a means of broadcasting information about products to potential customers and also
communicating within the company. In order for Twitter to become an effective marketing and
communication tool, all company users were instructed and evaluated on the use of Twitter (D4L,
2013).

This implementation report contains the processes and results for implementation of the
Twitter Basic Training. It highlights the evaluation used before, during, and after the Twitter
instruction. Kirkpatrick Level 1 and 2 evaluations were utilized to capture learner attitudes and
achievement of the learning objectives. Levels 3 through 5 evaluation instruments should be
utilized at a future date to capture whether the skills have been transferred to the job, whether
there has been a positive effect on the organization, and whether there has been a return on
investment (Simonson, 2007). Only the pilot-course implementation evaluation is included in
this report. Evaluation instruments Level 1 through 3 are included as Appendices. Evaluations
were implemented online through COURSEsites by Blackboard using the Evaluation
Instrument Appendices (D4L, 2013).

Module Overview and Description:

The instruction was designed to educate learners about account sign-up which is required before
any other topics can be covered. How to use their account to send Tweets and follow other Twitter
users was the primary goal of this instruction. Common Twitter terminology was thoroughly
explained (D4L, 2013). The instruction consisted of three modules with several straight-forward
multi-media presentations and screencasts with sound to explain the basics and some advanced
Twitter functions. The instruction also included simple-to-follow job sheets or steps providing
opportunities for the learner to perform them in the actual Twitter environment. This
asynchronous, self-paced instruction was hosted on COURSEsites by Blackboard. Additional Twitter
learning opportunities via links to external internet resources such as YouTube were provided (D4L,
2013).

Description of Implementation:
The instruction was implemented on COURSEsites by Blackboard as an asynchronous online
training course. The course was menu driven with a welcome module, a pre-test, three
modules, a post-test, and a survey. Pilot-course learners were selected and informed about the
need for the instruction and the purpose of the pilot-course in evaluation. They all stated that
they grasped the importance of taking the instruction seriously and completing all modules.

3|Page

Twitter Basic Training, Evaluation Plan

Usernames were setup in COURSEsites for each learner and they were enrolled in the course.
The learners were informed about their usernames and passwords and given one-week to
complete the three modules. Each learner completed the instruction within their own time
schedule. Course pre-test, post-test, and survey data were captured directly on COURSEsites.

Analysis of Evaluation Data:

Before implementation, a Level 2 pre-test was administered to each learner. This test mirrors
the questions asked during the Level 2 post-test. The purpose of these tests and analysis is to
determine if the learning objectives have been attained. The graphs below represent the data
that was obtained from the pre- and post-test.

4|Page

Twitter Basic Training, Evaluation Plan

5|Page

Twitter Basic Training, Evaluation Plan

The overall percentage of learning objectives attained is 70%. This is just under the 71% we
specified as successful in the implementation plan. Analysis reveals that some test questions or
the course content must be revised. These are questions that were missed 50% or more during
the post-test. Of the 25 questions administered, five should be checked for re-wording or to
ensure that adequate content was included to support the correct answers. Also, the data
points to three ELOs (B, J, and K) that fared poorly in the test. We also recommend that
additional questions be added for ELOs with only one question supporting them. However,
overall there was a definite improvement from the pre-test to the post-test with only two
exception (see graphELO F and J).

Proposed Revisions and Key Points: (Embedded Audio File)

The following revisions are strongly advised before another implementation of Twitter Basic
Training to achieve the 71% average of learning objectives attained which was specified in the
Implementation Plan:
1. Add questions to all ELOs that have only one question.
2. Revise questions 5, 23, and 25 or ensure that the content supporting them is correct or
adequately explained.
3. Recheck questions 3, 12, and 22 since they were only 50% attained to ensure that the
question is written correctly or that there is adequate content to support them.

Reference List:
D4L. (2013). Twitter Basic Training, Evaluation Plan. Retrieved from
http://d4l.wikispaces.com/Implementation+Document+%28Final%29
D4L. (2013). Twitter Basic Training, Implementation Plan. Retrieved from
https://class.waldenu.edu/@@/3209B6D6B60DA305BA7128E685B9864F/courses/1/US
W1.41500.201360/attempt/_11243219_1/s/Wk3ProjWrightC.pdf
Morrison, G. R., Ross, S. M., Kalman, H. K., & Kemp, J. E. (2011). Designing effective instruction
(6th ed.). Hoboken, NJ: John Wiley & Sons, Inc.
Simonson, M. (2007). Evaluation and distance education: Five steps. Quarterly Review of
Distance Education. 8(3), 191194.

Appendices:
Appendix A -- Pre- and Post-test Analysis of LOs Coverage (Master)
Twitter Basic Training
Test Item to Learning Objective Analysis

6|Page

Appendix A -- Pre- and Post-test Analysis of LOs Coverage (Master)

August 2013

Lesson

Enabling Learning Objectives Covered

TLO 1: After completing this lesson of instruction, the learner will create Twitter account.
Lesson 1: Create a Twitter
Recall terms associated with a personal Twitter account
Account
Create a personal Twitter account

Number of
Questions

10
2

TLO 2: After completing this lesson of instruction, the learner will demonstrate various functions involved with
composing and sending tweets.
Lesson 2: Compose and
Compose a tweet using Twitter
1
Send a Tweet
Send a tweet using Twitter
2
Demonstrate the function of retweeting with Twitter
1
Demonstrate the function of mentioning with Twitter
1
Demonstrate the function of direct messaging a follower with Twitter
1
Use an established hashtag with Twitter
1
TLO 3: After completing this lesson of instruction, the learner will demonstrate how to follow and unfollow other
Twitter users.
Lesson 3: Follow and
Demonstrate how to search for other users with Twitter
2
Unfollow Twitter Users
Demonstrate how to follow other users with Twitter
2
Demonstrate how to unfollow other users with Twitter
2
Total Number Questions
25

100% of the ELOs are supported by

questions.
Types of Questions used:
Multiple Choice - 25
Fill in the blank - 0
Picture - 0
Answer Frequency:
A = 6 (24%)
B = 7 (28%)
C = 6 (24%)
D = 6 (24%)

7|Page

Appendix A -- Pre- and Post-test Analysis of LOs Coverage (Master)

a. 8
Twitter Basic Training
Question Master
(Pre- and Post-test)
August 2013

b. 6
c. 4
d. 2
6. (TLO 1, ELO 1) What does the term direct
message or DM refer to?

Questions:
1. (TLO 1, ELO 1) The @ sign in Twitter is used
to.
a. identify a special friend.
b. signal the end of a Tweet.
c. call out usernames in a Tweet.
d. block a user who sends offensive
messages.
2. (TLO 1, ELO 1) The # sign in Twitter is used
to.

a. Sending a public message to a user

on Twitter
b. Sending a private message to a user
on Twitter
c. Sending a public message to a user
group on Twitter
d. Sending a private message to a user
group on Twitter
7. (TLO 1, ELO 1) A mention is a.

a. call out usernames in a Tweet.

a. direct messaging technique.

b. find related topics within Twitter.

b. direct marketing technique.

c. search the internet for a related

term.

c. means of crediting an author of an

article.

d. block a user who sends offensive

messages.

d. means of including someone in your

message.

3. (TLO 1, ELO 1) Hashtags can be used to.

a. find trending topics.
b. identify yourself in a Tweet.
c. relate your Tweet to a friends
Tweet.
d. search the internet for a related
term.
4. (TLO 1, ELO 1) Usernames in a Twitter
message are identified by a. symbol.
a. ?
b. @

8. (TLO 1, ELO 1) What does the term follow

mean?
a. You will receive all of their
messages.
b. You will receive all of their direct
messages.
c. You will receive only of their
retweeted messages.
d. You will receive only of their
hashtagged messages.
9. (TLO 1, ELO 1) Usernames are also referred
to as a.

c. #

a. name.

d. %

b. handle.

5. (TLO 1, ELO 1) Twitter recommends no

more than . hashtags per Tweet.
8|Page

c. user account.

Appendix A -- Pre- and Post-test Analysis of LOs Coverage (Master)

d. direct connection.
10. (TLO 1, ELO 1) What does the term
retweet mean?
a. Tweeting a message tweeted by
another user
b. Tweeting a message to a select
group of users
c. Tweeting a private message to
another user
d. Tweeting a private message to a
group of users
11. (TLO 1, ELO 2) What personal information is
required to sign up for Twitter?
a. Full name and Facebook account
b. Full name and email address
c. Email address and birthday
d. Email address and job title
12. (TLO 1, ELO 2) What must be avoided when
signing up for a Twitter account?
a. Using your real name
b. Using more than 15 characters
c. Using any offensive terms in the
username

a. 120-character message sent on

Twitter
b. 130-character message sent on
Twitter
c. 140-character message sent on
Twitter
d. 150-character message sent on
Twitter
15. (TLO 2, ELO 2) Which of the following is
NOT a valid means of sending a tweet?
a. internet
b. text message
c.

voice message

d. mobile device app

16. (TLO 2, ELO 3) Sending a message that
someone already sent is called.
a. retweeting.
b. mentioning.
c. hashtagging.
d. direct messaging.
17. (TLO 2, ELO 4) Including another users
handle in your tweet so that it can be
clicked is called.

d. Using words that do not contain the

word twitter

a. mentioning.

13. (TLO 2, ELO 1) When composing a tweet, it

is important to remember.

c. hashtagging.

a. to always hashtag any person in

your message.
b. to always mention any person in
your message.

b. retweeting.
d. direct messaging.
18. (TLO 2, ELO 5) Sending a message to only
one Twitter handle is called.
a. retweeting.

c. to use a maximum of 160

characters.

b. mentioning.

d. to use a maximum of 140

characters.

d. direct messaging.

14. (TLO 2, ELO 2) What is a tweet?

c. hashtagging.
19. (TLO 2, ELO 6) Including a subject in your
tweet that can be accessed by your
9|Page

Appendix A -- Pre- and Post-test Analysis of LOs Coverage (Master)

followers is called.
a. retweeting.

24. (TLO 3, ELO 3) What must you do to

unfollow a Twitter handle?

b. mentioning.

a. Email the user to stop

c. hashtagging.

b. Select the @ connect tab

d. direct messaging.

c. Go to the users account page

20. (TLO 3, ELO 1) What can be searched for in

Twitter?
a. Topics, hashtags, tweets, or people

d. Click the follow button again

25. (TLO 3, ELO 3) Which ways of unfollowing
below are NOT valid?

b. People, mentions, hashtags, or

topics

a. Clicking on an persons account and

clicking unfollow

c. Mentions, hashtags, topics, or direct

messages

b. Clicking on the # Discover tab and

clicking unfollow

d. Hashtags, topics, tweets, or direct

messages

c. Clicking on Following and clicking on

unfollow

21. (TLO 3, ELO 1) Twitter searches are

primarily conducted on the . tab.
a. Home
b. @ Connect
c. # Discover
d. Me
22. (TLO 3, ELO 2) What must you do to follow a
Twitter handle?
a. Click the users handle
b. Click the # Discover tab
c. Click the @ Connect tab
d. Click on the follow button
23. (TLO 3, ELO 2) When following a Twitter
handle.
a. you will receive all of their followers
messages.
b. you will receive only their
hashtagged messages.
c. you will receive all of their
retweeted messages.
d. you will receive only their direct
mention messages.
10 | P a g e

d. Sending a text message to Twitter

Appendix A -- Pre- and Post-test Analysis of LOs Coverage (Master)

Answer Key
1. C
2. B
3. A
4. B
5. D
6. B
7. D
8. A
9. B
10. A
11. B
12. B
13. D
14. C
15. C
16. A
17. A
18. D
19. D
20. A
21. C
22. D
23. C
24. D
25. B

11 | P a g e

Appendix A -- Pre- and Post-test Analysis of LOs Coverage (Master)

Change Record
Question

12 | P a g e

Change

Comment