Vous êtes sur la page 1sur 18

PEOPLE: International Journal of Social Sciences ISSN 2454-5899

International Journal of Social Sciences ISSN 2454-5899 Mokter Hossain et al. Special Issue Volume 3 Issue

Mokter Hossain et al. Special Issue Volume 3 Issue 1, pp. 489 - 506 Date of Publication: 21st March, 2017

DOI-https://dx.doi.org/10.20319/pijss.2017.s31.489506

2017 DOI-https://dx.doi.org/10.20319/pijss.2017.s31.489506 A STUDY TO EVALUATE USERS’ SATISFACTION OF BLACKBOARD

A STUDY TO EVALUATE USERS’ SATISFACTION OF BLACKBOARD LEARN

Md Mokter Hossain Department of Computer Science University of Alabama, Tuscaloosa, United States of America mokter@gmail.com

Shakil Akhtar Department of ComputerScience & IT Clayton State University, Morrow, GA, United States of America sakhtar@clayton.edu ShakilAkhtar@clayton.edu

Muhammad Asadur Rahman Department of Computer Science & IT, Clayton State University, Morrow, GA, United States of America mrahman@clayton.edu

Abstract

This study summarizes and reports first-time users’ satisfaction of a Blackboard Learn user

interface that had been adopted as an online/blended teaching-learning management tool. Users’

satisfaction was measured in terms of the overall consistency, ease of use, universability, positive

aspects, and problems/limitations of the Blackboard Learn user interface. Data were collected

through a number of 5-point Likert scale type items using an online survey. The survey also

gathered participants’ gender and major area of study to determine if there exists any significant

difference in their satisfaction, among the possible groups in these measures. Results revealed that

first-time Blackboard Learn users are most likely overall satisfied in using it without any

significant difference among male vs. female; and engineering vs. non-engineering major users.

Blackboard users are satisfied with a number of its advanced features, although they have

reported some noticeable problems, limitations, and recommendations.

© 2017 The author and GRDS Publishing. All rights reserved. Available Online at: http://grdspublishing.org/

489

PEOPLE: International Journal of Social Sciences ISSN 2454-5899

International Journal of Social Sciences ISSN 2454-5899 Keywords Blackboard Learn, Shneiderman’s golden rules
International Journal of Social Sciences ISSN 2454-5899 Keywords Blackboard Learn, Shneiderman’s golden rules

Keywords Blackboard Learn, Shneiderman’s golden rules of user interface design, User interface evaluation

1. Introduction

Blackboard Learn is a powerful teaching-learning management system (Anderson, 2009), provided by the Blackboard Inc., an enterprise software company, and primarily dedicated to developing education software, in particular learning management systems. In 1997, Blackboard Inc. was founded as an enterprise software company with its corporate headquarters in Washington, D.C., is primarily known as a developer of education software, in particular learning management systems. Blackboard Inc. was founded in 1997, as a small education technology company; and in 1998 it launched its first course management software (Business Wire, 1998; Washington Business Journal, 2004). After merging with several companies, it became a public company, in June 2004 (Bradford, Porciello, Balkon, & Backus, 2007). In the recent years Blackboard Inc. has acquired some other similar online learning providers such as WebCT, ANGEL Learning and jointly control about 80 percent of the academic course management system market in North America (Bradford et al., 2007). Blackboard Inc. alone is used by more than 70 percent of the U.S. colleges and universities. As of December 2010, Blackboard software and services are used by over 9,300 institutions in more than 60 countries (Blackboard.com; Bradford et al., 2007). There are several e-learning products of Blackboard Inc. system, such as Blackboard Learn, Blackboard Collaborate, Blackboard Connect, Blackboard Transact, Blackboard Analytics, Blackboard Mobile, and Blackboard Engage. Blackboard Learn provides easy-to-use educational instruction, communication, and assessment in online and blended learning course delivery system. These products are offered in 12 languages to over 2,200 learning institutions and contain more than 2,500 supplements from educational publishers (Bradford et al., 2007). The Blackboard Learn comprises of modules based upon: (1) a learning system that provides online course delivery and management for institutions; (2) a community and portal system for use in creating online campus communities; (3) a content management system for centralized control over course content; and (4) a system to record and analyze student assessment

© 2017 The author and GRDS Publishing. All rights reserved. Available Online at: http://grdspublishing.org/

490

PEOPLE: International Journal of Social Sciences ISSN 2454-5899

International Journal of Social Sciences ISSN 2454-5899 results (Blackboard.com; 2010, updated 2017). Students and
International Journal of Social Sciences ISSN 2454-5899 results (Blackboard.com; 2010, updated 2017). Students and

results (Blackboard.com; 2010, updated 2017). Students and faculty get several benefits from Blackboard Learn course management systems. Some of the potential benefits include: (1) increased availability, (2) quick feedback, (3) improved communication, (4) tracking, and (5) skill building (Bradford et al., 2007). The Blackboard Learn version 9.1 has some features that are similar to ANGEL Learning and WebCT platforms as well as, the Web 2.0 technologies and social learning tools. The Blackboard Learn allows instructional developers and instructors extending the functionality of the system; creating customized course management; and delivery by developing software and applications known as Building Blocks (Etesse, 2004). Building Blocks allow third-party developers to create customizations and extensions for Blackboard Learn through open Application Programming Interface and web services (Batson, 2011). Our study was conducted to evaluate first-time users’: students, graders, course developers, and instructors’ experience on using new interface of the Blackboard Learn at the University of Alabama. The rest of the paper presents significance and theoretical framework of the study that includes the research questions; the discussion of results including survey and data collection procedures, validity and reliability of the study, an overview of the satisfaction scores, results specific to the research questions; findings, discussion, and implication, followed by the limitation of the study; and ends with conclusions.

2. Theoretical Framework

This empirical study is designed based on the Golden Rules of Interface Design, proposed by Shneiderman (1998). Our research methodology consisted of a mixed-methods study (Reilley, 2007) evaluating first-time users’ satisfaction in using a new user interface of Blackboard Learn that is used as an online/blended teaching-learning management tool. Users’ satisfaction was measured in terms of the overall consistency, easiness of use, universability, positive aspects, and problems/limitations of the Blackboard Learn user interface; and was compared by participants’ gender and major area of study to determine if there exit any significant difference in their satisfaction among the possible groups in these measures. The study was conducted to answer two quantitative and one qualitative research questions. Data for the study were collected through an anonymous online survey uploaded on

© 2017 The author and GRDS Publishing. All rights reserved. Available Online at: http://grdspublishing.org/

491

PEOPLE: International Journal of Social Sciences ISSN 2454-5899

International Journal of Social Sciences ISSN 2454-5899 Google Drive developed and conducted by the researcher, with
International Journal of Social Sciences ISSN 2454-5899 Google Drive developed and conducted by the researcher, with

Google Drive developed and conducted by the researcher, with an extensive help of the instructor. The Validity of this instrument was measured through the face validity process; and the reliability (Siegle, 2002; Gliem 2003) of this study was measured using Cronbach's alpha methods using SPSS for Windows (George, 2003), that suggests a poor reliability of study result. This might be either due to a poor internal consistency for the Likert-type scale with the participants’ satisfaction scores obtained in this study or due to the small sample size of the study. Data for the quantitative research questions were collected quantitatively through a number demographic and sixteen 5-point Likert-type scale items; and data for the qualitative research question were collected qualitatively through two separate open ended questions using an online survey, developed and conducted by the researcher. These sixteen 5-point Likert-type scale items were combined to attain a median satisfaction score toward the use of Blackboard Learn. Each item was given a possible range of 1 to 5; where 1 indicated Totally Dissatisfies and 5 indicated Totally Satisfied. Users’ satisfaction was measured in terms of the overall consistency, easiness of use, universability, positive aspects, and problems/limitations of the Blackboard Learn user interface; and was compared by participants’ gender and major area of study to determine if there exit any significant difference in their satisfaction among the possible groups in these measures. Thus, the study was conducted to answer the following three research questions:

(1)

Blackboard Learn user interface?

(2)

satisfaction in using the Blackboard Learn user interface?

(3)

interface?

Is there significant difference between male and female users’ satisfaction in using the

Is there significant difference between engineering and non-engineering major users’

What are the major positive aspects and problems/limitations of the Blackboard Learn user

Data for the first two research questions were collected quantitatively through a number demographic and sixteen 5-point Likert-type scale items; and data for the last research question were collected qualitatively through two separate open ended questions using an online survey. In order to collect data for the above research questions, permission was sought from the University

© 2017 The author and GRDS Publishing. All rights reserved. Available Online at: http://grdspublishing.org/

492

PEOPLE: International Journal of Social Sciences ISSN 2454-5899

International Journal of Social Sciences ISSN 2454-5899 of Alabama Institutional Review Board (IRB) to conduct a
International Journal of Social Sciences ISSN 2454-5899 of Alabama Institutional Review Board (IRB) to conduct a

of Alabama Institutional Review Board (IRB) to conduct a survey in the campus, which delayed the collection of data and publication of results.

The survey included four major sections:

(1)

Consent Statement (the beginning page);

(2)

Participants’ demographic information (item numbers: 1- 6);

(3)

Participants’ satisfaction with the Blackboard Learn user interface (item numbers: 7-22);

and

(4)

Open-ended questions regarding the positive aspects, and problems or limitations of

Blackboard Learn user interface (item numbers: 23-25).

The first section was designed to collect participants’ consent in participating in the survey. This section included the basic information about the study required to be information to the participants before starting the survey. Participants, who did agree to participate in the study, by clicking on the “Yes” button were allowed to proceed to the main body of the survey. The second section was designed to collect participants’ demographic information in terms of their college of major area of study, level of study at the university, level of familiarity in suing Blackboard Learn and its predecessor eLearning, and major role in using Blackboard Learn. The third section was intended to measure participants’ overall satisfaction in using the Blackboard Learn user interface through 16 Likert-type items (item numbers 7-22) with a 5-point scale: Totally Dissatisfied, Dissatisfied, Neutral, Satisfied, and Totally Satisfied. The first three items of this section (item numbers 7-9) were about the simplicity of the Blackboard Learn user interface. The next eight items (item numbers 10 -17) were intended to measure participants’ satisfaction in terms of the golden rules of user interface design. Then the last five Likert-type items (item numbers 18 to 22) were intended to measure participants’ satisfaction in terms of its common and advanced features of the Blackboard Learn user interface. Finally, the fourth section was designed with three open ended questions, was intended to collect participants’ written comments regarding the positive aspects, and problems or limitations of Blackboard Learn user interface.

© 2017 The author and GRDS Publishing. All rights reserved. Available Online at: http://grdspublishing.org/

493

PEOPLE: International Journal of Social Sciences ISSN 2454-5899

International Journal of Social Sciences ISSN 2454-5899 In order to recruit participants, a solicitation email was
International Journal of Social Sciences ISSN 2454-5899 In order to recruit participants, a solicitation email was

In order to recruit participants, a solicitation email was sent to a number of instructors of the various departments of the university, stating the objectives and necessity of the study; and the URL of the survey. In the email, a draft of another email solicitation for the prospective participants was sent to the instructors, as an attachment, and the instructors were requested to send emails to their prospective students inviting to participate in the study. The attached email contained the URL of the online survey and a brief statement of purpose and procedure to participate in the study. A similar email attachment was sent the University Student News Service authority with a request to post a message and link of the survey to the general student population through the UA Student News Service. Normally the web and email based surveys are more popular among STEM students (STEM Recruiting, onlinecolleges.org, 2011).

3. Discussion of Results

The survey consists of 82 participants from fall 2012. Out of the acceptable 82 participants 46 (56.1%) were male and 36 (43.9%) were female. Seven (8.57%) of them were from College of Arts and Sciences; eleven (13.4%) were from College of Commerce and Business Administration; five (6.1%) from College of Communication and Information Sciences; sixe (7.3%) from College of Education; 38 (46.3%) from College of Engineering; eight (9.8%) from College of Human Environmental Sciences; five (6.1%) form College of Nursing; and two (2.4%) from Other colleges (see Table 1). Thus, in order to analyze the second research question, all participants are grouped into two major categories: engineering and non-engineering majors that comprised 38 (46.34%) and 44 (53.66%), respectively.

Table 1: College * Gender Cross Tabulation

Major College of the Participants

Gender of the Participants

Total

Male

Female

Arts and Sciences

2 (2.4%)

5

(6.1%)

7 (8.5%)

Commerce and Business Administration

9 (11.0%)

2

(2.4%)

11 (13.4%)

© 2017 The author and GRDS Publishing. All rights reserved. Available Online at: http://grdspublishing.org/

494

PEOPLE: International Journal of Social Sciences ISSN 2454-5899

International Journal of Social Sciences ISSN 2454-5899 Communication and Information Sciences   1 (1.2%)
International Journal of Social Sciences ISSN 2454-5899 Communication and Information Sciences   1 (1.2%)

Communication and Information Sciences

 

1

(1.2%)

 

4

(4.9%)

 

5

(6.1%)

Education

 

2

(2.4%)

 

4

(4.9%)

 

6

(7.3%)

Engineering

28

(34.1%)

10

(12.2%)

38

(46.3%)

Human Environmental Science

 

0

 

8

(9.8%)

 

8

(9.8%)

Nursing

 

4

(4.9%)

 

1

(1.2%)

 

5

(6.1%)

Social Work

 

0

 

2

(2.4%)

 

2

(2.4%)

Total

46

(56.1%)

36

(43.9%)

82

(100.0%)

Among these participants, only two (2.4%) was faculty members; and five (6.1%) were freshmen; 37 (45.1%) were Graduate; 15 (18.3%) were junior; one (1.2%) was non-degree; 15 (18.3%) were senior; and seven (8.5%) were sophomore students (see Table 2).

Table 2: Level of Study * Gender Cross Tabulation

Level of Study

 

Gender of the Participants

 

Total

 

Male

Female

 

Faculty

 

2

(2.4%)

 

0

 

2

(2.4%)

Freshman

 

4

(4.9%)

 

1

(1.2%)

 

5

(6.1%)

Graduate

24

(29.3%)

13

(15.9%)

37

(45.1%)

Junior

 

7

(8.5%)

 

8

(9.8%)

15

(18.3%)

Non-degree

 

0

 

1

(1.2%)

 

1

(1.2%)

Senior

 

6

(7.3%)

9 (11.0%)

15

(18.3%)

Sophomore

 

3

(3.7%)

 

4

(4.9%)

 

7

(8.5%)

Total

46

(56.1%)

36

(43.9%)

82

(100.0%)

Among these participants, 71 (86.6%) reported that their major role(s) of using Blackboard Lear was as student; only one (1.2%) as Course/Section Developer; two (2.4%) as

© 2017 The author and GRDS Publishing. All rights reserved. Available Online at: http://grdspublishing.org/

495

PEOPLE: International Journal of Social Sciences ISSN 2454-5899

International Journal of Social Sciences ISSN 2454-5899 Grader/Teaching Assistant; and eight (9.8%) has had multiple
International Journal of Social Sciences ISSN 2454-5899 Grader/Teaching Assistant; and eight (9.8%) has had multiple

Grader/Teaching Assistant; and eight (9.8%) has had multiple roles such as student, Grader/Teaching Assistant, and/or Section Developer (see Table 3).

Table 3: Major Role in Using BB Learn * Gender Cross Tabulation

Major Role in Using BB Learn

Gender of the Participants

Total

 

Male

 

Female

Course/Section Developer

 

1

(1.2%)

1 (1.2%)

Grader/Teaching Assistant

 

2

(2.4%)

0 (.0%)

2 (2.4%)

 

39

(47.6%)

 

32

71 (86.6%)

Student

 

(39.0%)

Multiple ( Such as, Student, Grader/Teaching Assistant, and/or Section Developer

 

5

(6.1%)

3

(3.7%)

8

(9.8%)

 

46

(56.1%)

 

36

82 (100.0%)

Total

 

(43.9%)

Participants responded to 16 specific questions designed to measure their satisfaction in interacting with the Blackboard Learn user interface. These questions included a 5-point Likert- type scale allowing participants to indicate whether they totally Dissatisfied, Dissatisfied, Neutral, Satisfied, and Totally Satisfied to each item. These sixteen 5-point Likert-type scale items were combined to attain a median satisfaction score toward the use of Blackboard Learn. Each item was given a possible range of 1 to 5; where 1 indicated Totally Dissatisfies and 5 indicated Totally Satisfied. Descriptive statistics (Sirkin, 1999) of these median satisfaction scores yields a mean, median, and mode of 3.88, 4.0, and 4.0 respectively with a standard deviation of 0.77 (Table 4). This indicates that the participants’ typical response is close to agreement that they are most likely overall satisfied with the interaction of Blackboard Learn user interface. We present here the specific results for research questions. Table 4: Descriptive Statistics of the Median Satisfaction Scores

© 2017 The author and GRDS Publishing. All rights reserved. Available Online at: http://grdspublishing.org/

496

PEOPLE: International Journal of Social Sciences ISSN 2454-5899

International Journal of Social Sciences ISSN 2454-5899 N 82 Mean 3.872 Std. Error of Mean 0.849
International Journal of Social Sciences ISSN 2454-5899 N 82 Mean 3.872 Std. Error of Mean 0.849

N

82

Mean

3.872

Std. Error of Mean

0.849

Median

4.0

Mode

4.0

Std. Deviation

0.7690

Results of Research Question 1 The first research question was: Is there significant difference between male and female users’ satisfaction in using the Blackboard Learn user interface? Results of this research question were carried out through a non-parametric Wilcoxon-Mann-Whitney U test (Mehrens, 1987) on the median satisfaction scores grouped by gender, as shown on the left two columns of the Table- AB in the Appendix-B. The Wilcoxon-Mann-Whitney U test result (Table 5) indicates a non- significant difference (N = 82, U = 655.500, p =0.084>.05) that failed to reject the null hypothesis that the participants’ median satisfaction scores did not differ in terms of their gender.

Table 5: Wilcoxon-Mann-Whitney U Test Statistics for Median Satisfaction Scores on Participants’ Gender

 

Median Satisfaction

Mann-Whitney U

655.500

Wilcoxon W

1736.500

Z

-1.725

Asymp. Sig. (2-tailed)

0.084

a. Grouping Variable: Gender

This meant that the study did not find any significant difference among the male and female participants’ satisfaction in using the Blackboard Learn. Again, based on the ranks of the median satisfaction scores by gender, as shown in Table 6, it also determined that male participants had a mean rank of 37.75 and the female participants had a mean rank of 46.29 for their satisfaction toward using the Blackboard Learn user interface. The difference between the

© 2017 The author and GRDS Publishing. All rights reserved. Available Online at: http://grdspublishing.org/

497

PEOPLE: International Journal of Social Sciences ISSN 2454-5899

International Journal of Social Sciences ISSN 2454-5899 mean ranks was not significant, either. It is interesting
International Journal of Social Sciences ISSN 2454-5899 mean ranks was not significant, either. It is interesting

mean ranks was not significant, either. It is interesting to note pervious research shows that women are slowly catching up on the use of Internet (Fallows, 2005; Grove, 2010). However, our research found no significant difference based upon gender.

Table 6: Ranks of the Median Satisfaction Scores by Gender

Gender

N

Mean Rank

Sum of Ranks

Male

46

37.75

1736.50

Female

36

46.29

1666.50

Total

82

   

Results of Research Question 2 The second research question was: Is there significant difference between engineering and non-engineering major users’ satisfaction in using the Blackboard Learn user interface? Results of this research question were carried out through a non-parametric Wilcoxon-Mann-Whitney U test on the median satisfaction scores grouped by college of major (engineering or non- engineering), as shown on the right two columns of the Table-AB in the Appendix-B. The Wilcoxon-Mann-Whitney U test result (Table 7) indicates a non-significant difference (N = 82, U = 714.500, p=0.226>.05) that failed to reject the null hypothesis that the participants’ median satisfaction scores did not differ in terms of their major college groups.

Table 7: Wilcoxon-Mann-Whitney U Test Statistics for Median Satisfaction Scores on Participants’ Major College Groups

 

Median Satisfaction

Mann-Whitney U

714.500

Wilcoxon W

1455.500

Z

-1.209

Asymp. Sig. (2-tailed)

.226

a. Grouping Variable: Major College Groups

© 2017 The author and GRDS Publishing. All rights reserved. Available Online at: http://grdspublishing.org/

498

PEOPLE: International Journal of Social Sciences ISSN 2454-5899

International Journal of Social Sciences ISSN 2454-5899 This meant that the study did not find any
International Journal of Social Sciences ISSN 2454-5899 This meant that the study did not find any

This meant that the study did not find any significant difference among the engineering and non-engineering college major participants’ satisfaction in using the Blackboard Learn. Again, based on the ranks of the median satisfaction scores by major college groups, as shown in Table 8, it also determined that college of engineer participants had a mean rank of 38.30 and the college of non-engineering participants had a mean rank of 44.26 for their satisfaction toward using the Blackboard Learn user interface. The difference between the mean ranks was not significant, either.

Table 8: Ranks of the Median Satisfaction Scores by Major College Groups

Major College Groups

N

Mean Rank

Sum of Ranks

Engineering

38

38.30

1455.50

Non-Engineering

44

44.26

1947.50

Total

82

   

Results of Research Question 3 The third research question was: What are the major positive aspects and problems/limitations of the Blackboard Learn user interface? Qualitative data obtained for this research question were summarized manually and no statistical analysis was performed. After analyzing the survey results in depth, we conclude that participants found the Blackboard Learn user interface highly customizable, faster, user friendly, universal, well organized, very clean with elegant appearance. Blackboard Learn is easy to use, easy to viewing grades; and easy to type in and formatting text and attach assignment file. According to a participant, “It's smoothness and ease of maneuvering to your desired location. You do not have to go through tons of clicks to get to your destination.” According to another participant, “It's nice having mostly everything for all my classes all in one place. I like that when it logs you out, once you log back in it take you to the same page.” Organization of coursework is better in Blackboard Learn, “I like that everything is in one place for each class.”, as mentioned by one participant. Another participant mentioned, “It has more use for larger sectioned courses more so than smaller section courses.” Infact, participants found its user interface as simple, user-friendly, and easy to navigate that does not require expert knowledge of navigating websites or using computers. Its log in is easy and easier and it does not kick the user log out from the system after a certain time.

© 2017 The author and GRDS Publishing. All rights reserved. Available Online at: http://grdspublishing.org/

499

PEOPLE: International Journal of Social Sciences ISSN 2454-5899

International Journal of Social Sciences ISSN 2454-5899 According to some student participants, grade view in
International Journal of Social Sciences ISSN 2454-5899 According to some student participants, grade view in

According to some student participants, grade view in Blackboard Learn is better than eLearning as it “reveals the average grade of each course. Tracks progress as one works through an assignment.” According to another participant, “the Grade Center is much more capable than eLearning. For instance, the ability to ignore an attempt rather than just clear it. Also, jut the general grade entry and column organization.” According to some participant the Blackboard Learn seems more compatible with multiple modern browsers; and seems much more like a modern web-application than eLearning. They found it as better than eLearning, the previous online teaching-learning system. According to the participants, the grade tracking facilities in the Blackboard Learn is well thought out and implemented; and the course layouts are consistence. One of the participants commented as, “Blackboard Learn is very easy to work with. The menu is very easy to use, to figure out where you need to go. Using the Blackboard Learn is very simple. This was my first online course to take and it was easy to figure out.” Another instructor participant commented as, “[It is] easy to organize materials for classes I teach. On-line grading of exams with feedback really simplifies thing for both the instructor and students. Ability the show or hide entire sections is easier in Blackboard.” A number of student participants mentioned that, compared to eLearning, the Blackboard Learn interface is pretty intuitive, less kludge; better to use and for managing multiple course integration; has a much better grade center; and “the load of it is significantly faster than eLearning.” In Blackboard Learn, making comments on grade is more flexible; and it “allows for easy communication with teachers and other students.” Multiple participants found the integration of blogs, wikis, podcasts, and other web 2.0 applications, social networking technologies, as new and appealing. One participant mentioned, “the iPhone application and access through the Alabama iPhone app are useful too.” According to multiple participants, overall “Blackboard Learn is much better than eLearning.” One participant mentioned, “I wish all professors used it, so it was more universal. Also, I wish the My Grades part was always kept up to date.”

According to one student participant, The new Blackboard Learn environment allows each professor to customize their course pages, which is good in some respects. However, this makes it difficult for users who have no

© 2017 The author and GRDS Publishing. All rights reserved. Available Online at: http://grdspublishing.org/

500

PEOPLE: International Journal of Social Sciences ISSN 2454-5899

International Journal of Social Sciences ISSN 2454-5899 standardization between course pages. Also, it seems
International Journal of Social Sciences ISSN 2454-5899 standardization between course pages. Also, it seems

standardization between course pages. Also, it seems professors are only using the course content page to post material and it gets crowded and unorganized. With the old system each there was pages for division of material, for instance, one page for lectures, one page for homework assignments etc. A number of faculty, course/section developers, graders, teaching, and technical support assistants expressed some positive opinion toward Blackboard Learn user interface. For instances:

The Blackboard Learn interface is highly customizable; Easy to use, has never been dysfunctional when I tried to use it; very clean, elegant appearance; Its grade center is much better than eLearning's; Comments on grades is more flexible; etc. One of them mentioned, “I feel the grade tracking facilities were very well thought out and implemented.” Analyzing the qualitative data, we discovered that participants found the Blackboard Learn user interface highly customizable, faster, user friendly, universal, well organized, very clean with elegant appearance. Such as, is easy to use, easy to viewing grades; and easy to type in and formatting text and attach assignment file; etc. Compared to eLearning, the Blackboard Learn interface is pretty intuitive, less kludge; better to use and for managing multiple course integration; has a much better grade center; he load of it is significantly faster than eLearning; allows for easy communication with teachers and other students; etc. Also integration of blogs, wikis, podcasts, and other web 2.0 applications, social networking technologies, as new and appealing; and also the iPhone application and access through the Alabama iPhone app are useful too. The main limitation of this study report is that it presents data found from a small sample 82 participants who have only a limited time experience of using Blackboard Learn. Thus, the findings of this study cannot be claimed to be generalizable. Moreover, given the quasi-interval type data of the dependent variables that were used to measure participants’ satisfaction toward the Blackboard Learn user interface, it was not possible for us to use more powerful parametric tests for analyzing quantitative data. Thus, the non-parametric Wilcoxon-Mann-Whitney U test, which is considered less powerful than the corresponding parametric t-test, was needed to use to analyze the first two research questions (Gay, 2003). Moreover, due to limited time, we could not use any control group for this study. Therefore, we could not compare the participating (treatment group) Blackboard Learn users’ satisfaction with another group (control group) of participants who do not have experience of using Blackboard Learn. Finally, we feel that it would have been

© 2017 The author and GRDS Publishing. All rights reserved. Available Online at: http://grdspublishing.org/

501

PEOPLE: International Journal of Social Sciences ISSN 2454-5899

International Journal of Social Sciences ISSN 2454-5899 much better to collect data from more faculty,
International Journal of Social Sciences ISSN 2454-5899 much better to collect data from more faculty,

much better to collect data from more faculty, course/section developers, graders, teaching, and technical support assistants’ experience and satisfaction to the use of Blackboard Learn.

4. Lessons Learned

In addition to many positive experience, some student participants reported problems/limitations of Blackboard Learn such as: “you have to reload the page many times because it does not show all content at first” One participant reported Blackboard Learn as too slow. Another participant noted, “sometime it is difficult to find help items.” One participant reported Blackboard Learn times out after a while another participant reported is stays logged in for a while after the current window is closed. Another participant noted, it sometimes hangs up; ask for authentication to download files; and does not save username and password in user’s laptop. Another student mentioned, “I can't seem to find a way to remove classes I took over the summer from my "classes" view.” New users found some sort of difficulties in submitting the assignments as the Blackboard Learn does not highlight which assignments have passed the submission deadline, or due soon, or new. A number of student participants mentioned the lack of discussion forums and non- interactive nature and Blackboard email and message communication features. One of the participants expressed his/her opinion on this as follows: Having a forum with appropriate administrative control (ability to create subforums and moderate all forum activities) for instructors would enable very open discussion between students as providing a facility for answering questions posed by a single student but visible to all students. Email is a good candidate for small discussion between a student and the instructor, but I would prefer to also have an open forum for more free discussion between students especially. Institutional email interoperability was found one of the biggest issues in Blackboard Learn. Many student participants noted that the email and message system in Blackboard Learn are not interactive. One participating student noted that the email feature in Blackboard Learn is not very obvious and it feels odd to implement an email system which does not utilize the university email. According to another participant, “I have no way of knowing that I have a message from my teacher unless I look at it every day. Major hassle.” Another participant noted, “Blackboard is not as organized as e-learning. It is hard to find tasks. Also, I like how e-learning

© 2017 The author and GRDS Publishing. All rights reserved. Available Online at: http://grdspublishing.org/

502

PEOPLE: International Journal of Social Sciences ISSN 2454-5899

International Journal of Social Sciences ISSN 2454-5899 alerts you with a green dot whenever you have
International Journal of Social Sciences ISSN 2454-5899 alerts you with a green dot whenever you have

alerts you with a green dot whenever you have a task due.” Another participant mentioned, “Sometimes, it is not possible to add attachments to certain messages or emails sent within Blackboard Learn.” One student participant mentioned:

The internal email announcements process does not alert the student when a new email or announcement is received. If a student doesn't check each class' email box they'll miss it. The group blogs, messaging/discussion boards, etc. work the same way. Some kind of alert process linked to crimson email or text should be in place to let the student know there's new content for them to view. Another student participant mentioned:

According to the help webinar, the University did not buy the license that allows the home page to have notifications of "New" items to review. In eLearning you had a green asterisk that would appear next to email, grades, etc. You knew you needed to review those pages. In Blackboard, you have NO IDEA when something new appears. I had old emails waiting to be read. New assignments that hadn't made the calendar. It was a disaster. Some of faculty, course/section developers, graders, teaching, and technical support assistants mentioned some problems/limitations of the Blackboard Learn. One of them reported that several features of eLearning are not possible with Blackboard such as: “adding auditors and making assignment submissions visible to all” Particularly, one technical support assistant mentioned, “There is no WYSIWYG editor for the modules like there was in eLearning, though I heard this may be provided in an update for our system. Because of this, any minor updates to HTML pages must be done externally.” One of the instructor participants noted an issue of inability to create gradebook columns that use multiplication or multiply two existing columns. S/he found it working offline with the gradebook is a bit of a pain. A technical support assistant mentioned, There are little bugs that need fixed, like no discussion forums to read, yet you still get the icon on the homepage. I felt the biggest issue was the lack of interoperability between Blackboard Learn and the university email (xxx@crimson.ua.edu). The email feature in Blackboard Learn is not very obvious and it feels odd to implement an email system which does not utilize the university email.

© 2017 The author and GRDS Publishing. All rights reserved. Available Online at: http://grdspublishing.org/

503

PEOPLE: International Journal of Social Sciences ISSN 2454-5899

International Journal of Social Sciences ISSN 2454-5899 S/he recommended: In addition to syncing Blackboard Learn
International Journal of Social Sciences ISSN 2454-5899 S/he recommended: In addition to syncing Blackboard Learn

S/he recommended:

In addition to syncing Blackboard Learn email with the university email (xxx@crimson.ua.edu) I would like a forum available which allows communications between students and instructors outside of the classroom and emails. Having a forum with appropriate administrative control (ability to create sub-forums and moderate all forum activities) for instructors would enable very open discussion between students as providing a facility for answering questions posed by a single student but visible to all students. Email is a good candidate for small discussion between a student and the instructor, but I would prefer to also have an open forum for more free discussion between students especially. Another notable issue of the Blackboard Learn is that it is not very friendly to access via mobile devices. However, Blackboard Learn users expect that they should be able to access on it via mobile device. One of the participants noted this issue as follows: “In a perfect world we would all have access to a computer at all times, but emergencies do happen and preventing access to a class via mobile when all other facets of Blackboard are available is shortsighted.”

5. Conclusions

This mixed-methods empirical study evaluated first-time users’ satisfaction of using a version of Blackboard Learn user interface that had been adopted as an online/blended teaching- learning management tool in fall 2012 at The University of Alabama in the southeastern United States. Users’ satisfaction was measured in terms of the overall consistency, easiness of use, universability, positive aspects, and problems/limitations of the Blackboard Learn user interface. The study found that the first-time Blackboard Learn users are most likely overall satisfied in using it without any significant difference among male vs. female; and engineering vs. non- engineering major users. Blackboard users are satisfied with a number of its advanced features, although they have reported some noticeable problems, limitations, and recommendations. In the perspective of mixed-methods study, finding of quantitative and qualitative results field similar or identical results. Although there are some limitations of this study such as small sample size, poor internal validity of the survey instrument and lack of control groups limiting the generalizability of this study, these scopes of study may be continued in future research work. We believe that findings of

© 2017 The author and GRDS Publishing. All rights reserved. Available Online at: http://grdspublishing.org/

504

PEOPLE: International Journal of Social Sciences ISSN 2454-5899

International Journal of Social Sciences ISSN 2454-5899 this study and any relevant future studies will be
International Journal of Social Sciences ISSN 2454-5899 this study and any relevant future studies will be

this study and any relevant future studies will be helpful to justify effectiveness, usability, and limitations/problems of the Blackboard Learning system.

References

Batson, T. (2011). Not Your Grandfather’s Blackboard? My Recent Chat with Bb Learn President Ray Henderson. Retrieved February 4 th , 2017 from

Blackboard.com. Blackboard Locations. Blackboard Inc. Retrieved February 4 th , 2017. Blackboard Inc. 650 Massachusetts Avenue N.W. 6th Floor Washington, DC 20001-3796

U.S.A. Available at: http://www.blackboard.com/Contact-Us/Locations.aspx Blackboard.com. Modules. Blackboard Inc. Retrieved February 4 th , 2017. Blackboard Inc. 650 Massachusetts Avenue N.W. 6th Floor Washington, DC 20001-3796 U.S.A. Available at:

us.help.blackboard.com/Learn/Instructor/Getting_Started/Navigate_Outside_a_Course/Mod ules Blackboard Knows Issues (2016). Blackboard Knows Issues. Available online at:

https://bbhelp.cit.cornell.edu/blackboard-known-issues/ Bradford, P., Porciello, M., Balkon, N., & Backus, D. (2007). THE BLACKBOARD LEARNING SYSTEM. The Journal of Educational Technology Systems, 35(8), 301-314. Retrieved 4 February, 2017, from http://www.gilfuseducationgroup.com/wp-

Blackboard.pdf https://doi.org/10.2190/X137-X73L-5261-5656 Business Wire. (1998). Blackboard Announces Release Of Blackboard CourseInfo 2.0. Business Wire. November 2, 1998. Etesse, C. (2004). Leading the Way on StandardsBased eLearning. Blackboard Overview White Paper. November 2004. Retrieved February 4 th , 2017

© 2017 The author and GRDS Publishing. All rights reserved. Available Online at: http://grdspublishing.org/

505

PEOPLE: International Journal of Social Sciences ISSN 2454-5899

International Journal of Social Sciences ISSN 2454-5899 Fallows, D. (2005). How Women and Men Use the
International Journal of Social Sciences ISSN 2454-5899 Fallows, D. (2005). How Women and Men Use the

Fallows, D. (2005). How Women and Men Use the Internet. Pew Internet & American Life Project. Retried February 4 th , 2017, from

Gay, L. R., & Airasian, P. (2003). Educational research: Competencies for analysis and applications (7th ed.). Upper Saddle River, NJ: Merrill. George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference. 11.0 Update (4th ed.). Boston, MA: Allyn & Bacon. Gliem, J. A., & Gliem, R. R. (2003). Calculating, Interpreting, and Reporting Cronbach’s Alpha Reliability Coefficient for Likert-Type Scales. Paper presented at the Midwest Research-to- Practice Conference in Adult, Continuing, and Community Education, The Ohio State University, Columbus, OH, October 8-10, 2003. Retrieved February 4 th , 2017, from https://scholarworks.iupui.edu/bitstream/handle/1805/344/Gliem%20&%20Gliem.pdf

Grove, J. V. (2010). How Women Use the Web.

Retrieved February 4 th , 2017, from

Mehrens, W., & Lehman, I. (1987). Using standardized tests in education. New York, NY:

Longman. Onlinecolleges.org. (2011). STEM Recruiting: 10 Tools and Toys Bringing Girls to the Sciences. Retrieved February 4 th , 2017. Available online at: https://www.onlinecolleges.org/stem-

Reilly, E. M. (2007). Writing to Learn Mathematics: A mixed method study. (Doctoral Dissertation). Retrieved February 4 th , 2017 from

Siegle, D. (2002). Reliability Definitions at the University of Connecticut.

2017, from http://researchbasics.education.uconn.edu/instrument_reliability Sirkin, R. M. (1999). Statistics for the Social Sciences (3rd ed.). Thousand Oaks, CA: Sage.

Retrieved February 4 th ,

Washington Business Journal (2004). Fastest Growing Companies 2004: Blackboard. Washington Business Journal, 23 (22): p. 16. October 1, 2004.

© 2017 The author and GRDS Publishing. All rights reserved. Available Online at: http://grdspublishing.org/

506