1

Aalborg University Moodle page Information Architecture and Usability Evaluation for Improvement

Didzis Spruds Aalborg University December !1"

Aalborg University Moodle Page Information Architecture and Usability Evaluation For Improvement

#his study assesses information architecture and usability of Aalborg University $AAU% course management system $Moodle%& AAU Moodle is the main information system that university students are using in order to get information regarding courses and communicate 'ith teachers& #herefore( the information architecture and usability issues that are related to effective( efficient and engaging use of the system has a great impact on the )uality of the studies& #his study tries to provide ans'ers to the follo'ing )uestions* +Do the information architecture and usability of Moodle effectively supports the students in their study process, -o' it can be improved,. #he AAU Moodle has different user groups* students( teachers( study secretaries& #his study focuses on the issues that relates to the main user group students& #he study includes evaluation and suggestions for improvements that relates to the current version of Moodle $v&1&/% as 'ell as the ne' version $v& &0% that recently has been created and isn1t yet deployed to all schools 'ithin the university& #he data of this study has been collected using heuristic evaluation carried out by the author( )uestionnaire for the current Moodle users as 'ell as by intervie'ing persons 'ithin the university 'ho are responsible for Moodle system development and management& Heuristic Evaluation -euristic evaluation involves e2amination of system1s interface and 3udging its compliance 'ith recognized usability principles& $4ielsen( 1//5% -euristic evaluation as a research method 'as chosen for several reasons* firstly( heuristic evaluation can provide feedbac6 in short amount of time and can identify obvious usability problems $7aushi6( !!8%9 secondly( it turned out that there isn1t implemented any valuable site usage analytics tool that could provide useful data that could allo'

" to ans'er the research )uestions& #he sub3ect of evaluation 'as the most important functionality : pages of both current and ne' version of AAU Moodle& An issue of information architecture and usability 'as considered anything that prevents tas6 completion9 ta6es someone +off;course.9 creates some level of confusion9 produces an error9 performs the 'rong action& $#ullis < Albert( !1!%& #hese issues 'ere grouped in = categories $consistency( navigation( necessity and usefulness( labeling( search( design% and assessed the severity of each issue& $see Appendi2 A for a full list of criteria that 'ere used in the evaluation%& >n the one side of severity spectrum 'ere high severity issues that has significant impact on usability and on other side lo' severity cosmetic issues& It 'as possible that one issue belongs to several categories& ?elo' are the summary of evaluation results& #he numbers at the categories indicates ho' many information architecture and usability issues : problems 'ere discovered in each of the Moodle versions& #able 1 Information architecture and usability issues found in the evaluation Category @onsistency 4avigation 4ecessity and Usefulness Aabeling Search Design %everity Ao' $cosmetic issue% Medium $medium priority( Moodle version !" / 10 = 15 1 / Moodle version #!$ 1 5 1 1

1" !

" 8

should be noted% -igh $has significant impact on 8 usability% #otal issues 5!

1"

5 #he heuristic evaluation of both Moodle versions revealed that most of the problems of AAU Moodle pages are related to necessity and usefulness of different 'eb page elements $ = issues in the ne' version and 5 issues in the previous%& Bor instance( is that necessary to sho' +Cead more. button at each ne's item if pressing on this button in the ne2t screen appears the same information,9 is that necessary and useful to sho' students help files that are related only to the teachers version of the site,9 is that necessary to sho' menu that allo's to change the listing of replies if there is no possibility for user to add a reply, $Bigure 1%

Big&1* Usability issue from Moodle version 1&/ in the category +4ecessity and usefulness. ; dysfunctional menu&

4e2t category in terms of fre)uency of issues 'as navigation& $10 and

issues% #he main

problem of the navigation of the ne' version of Moodle lies in the global navigation $Morville < Cosenfeld( !!D% in its labels( layout and design elements& It is very li6ely that the main menu categories +My courses.9 +Erofile. and +-elp. are not the categories that people need the most9 one and the same label is mentioned three times in the same menu9 it includes graphical elements that

0 has no function9 it is not possible to distinguish the active menu category& $Bigure % In both versions of Moodle there are also several consistency and functionality problems in the breadcrumb navigation&

Big& * Usability issues in global navigation $Moodle version &0%

Aabeling issues $15 and 1 issue%& Most of the labeling issues are related to necessity and usefulness( consistency and appropriateness of the labels& @onsistency issues $/ and 1 issue% included internal consistency issues li6e consistency bet'een labels and icons and also e2ternal consistency issues li6e placement of search and functioning of navigation menus& Search& $1 and 1 issue% ?oth Moodle pages has problems 'ith search effectiveness as there is not an effective 'ay ho' to search the information that is related to the main content courses& #he previous version allo's to search forums and includes possibility to add +global search. bloc6 that is not functioning& #he ne' version allo's to search by course title and author but it seems that this functionality is relevant only to study secretaries as students usually don1t have more than 8 courses in the list& Significant part of the evaluation are screen;shoots of both Moodle versions& In each screen; shoot there are mar6ers that indicates an issue and comments : )uestions that e2plains 'hy this is

= considered an issue& Screen;shots 'ere mar6ed( commented and managed using 4otism&io design collaboration soft'are& #he mar6ed screen;shoots are available here& #here are several reasons that e2plains 'hy issues li6e these are present in the sites* shortcomings in the implementation( testing and maintenance process9 lac6 of site usage analytics( limitations of pro3ect resources ; time and budget& &uestionnaire and Findings From Moodle Users #he )uestionnaire included 11 )uestions about the usage of different functions that are available in current Moodle version( information architecture and usability issues as 'ell )uestions that as6ed 'hat are the issues that users particularly li6e : disli6e and 'hat 'ould they li6e to see in the ne' version of Moodle& It included closed;ended )uestions( attitude )uestions $semantic differential scales% as 'ell as open )uestions& $@ollins( !1!% $see Appendi2 @ for full list of )uestions and provided ans'ers%& Fuestionnaire 'as distributed to 1= master students of information architecture and human;computer interaction using Gizzualforms&com on;line survey tool& As most of the schools still are using the previous version of the site( the design of the )uestionnaire focused on the version that most of the students are currently using& #he completion rate 'as D1H $1" out of 1= )uestionnaires%& #he ans'ers to the )uestionnaire sho'ed that most of the students use Moodle since their bachelor studies in AAU and they have a longstanding e2perience 'ith it& Moodle is an important part of students daily routine ; on average students use Moodle page several times a 'ee6 or more fre)uently therefore the simplicity( ease of use( accessibility and engagement of the system has a considerable impact on the study process& #a6ing into consideration the e2perience of the users and fre)uent use of the system the efficiency of the system could be improved by providing better navigational shortcuts to the most fre)uently used sections $Saleh( !!/% li6e @urrent @ourses( @alendar( Borums& Students indicated that such sections as My @ourses( Brom the Study Administration(

8 Irades( Sign;Up for Bree Study Activities and @ourse Cegistration are in a great importance for them& #he information architecture and usability of the site 'ould benefit if the lin6s and sections that really are important for the users 'ould be placed first and be more prominent as lin6s not only connects the pages but also reflects the importance and relevance& $7albach( !!D% #he study sho'ed that such sections as ?logs( -elp( Search( 4et'or6 servers and Internal Messages most of the students do not use at all( therefore these sections could be removed or placed a'ay from the main content areas& -aving said that( it should also be said that additional in)uiries and clarifications regarding this important matter 'ould be necessary as the indicated importance of such sections as Irades( Sing up for free study activities and >ffers for students do not correlate 'ith the actual use of these sections& It could be that these really are important sections for the users but they have been poorly implemented or do not contain appropriate content( therefore hasn1t been used& #he clarity and consistency of navigation labels and titles most of the students rate as good( but the search and navigation features that allo's to find content easily 'ere rated less positive as stated by a Moodle user* +I thin6 it can be very time consuming to navigate& Jou need to point and clic6 many times to get any'here& #here is no shortcut bet'een modules you have to clic6 your 'ay bac6 and forth bet'een the homepage& S'itching bet'een forums is also a hassle&&&. #he ma3ority of ans'ers to the )uestions that relates to usability sho's that Moodle allo's the students to reach their goals )uite effectively and efficiently but the e2perience could be more engaging and user;friendly& As for the content that teachers put into the Moodle( most of the students thin6 that the content is clear and formatted neatly and carefully although it lac6s consistency& Ans'ering the )uestion 'hat additional features or improvements students 'ould li6e to see in the Moodle several students has provided comments that relates to consistency of the content* +Cight no' it seems li6e it1s the teacher that sets it up to his:her li6ing.9 +Apparently there are no clear 'ays in 'hich

D professors should present courses and material& It is very confusing and no consistency at all&. Bull list of the findings from the )uestionnaires are available in Appendi2 ?& %trategy for Improvement #he ne' version of AAU Moodle $v& &0% definitely has a potential to become significantly better than the previous version $v&1&/% as it is based on updated Moodle core and at some e2tent incorporates the ne' design guidelines& #he home page is less cluttered 'ith rarely used features and content( it sho's recent activities as 'ell as courses can be categorized by +semester. and +cross;semester.& #hese all are issues that users mentioned that could be improved in the previous version& 4evertheless the ne' version still includes several fundamental IA and usability challenges as 'ell as many minor issues& Some of the IA and usability issues in categories li6e consistency( necessity and usefulness( labeling and design can be improved )uite easily and the short;term effort should be focused on fi2ing these issues& #hough( these fi2es 'ill not resolve the fundamental issues of usability li6e search effectiveness and cross;media support that at some e2tent are limited by hard;coded Moodle modules& #herefore( the long;term strategy should be focused on customizing the modules and if that1s not feasible on implementing or developing a ne' system& At the same time deeper understanding of user needs( implementation of site usage analytics and iterative design and maintenance should help to streamline the change and improvement process& #hat 'ould also allo' to realize some of the most important and sought;after design challenges li6e* ?etter layout( better overvie' Simplicity and user;friendliness Bocus on core functionality #ailoring for user groups Mobile;friendly version More engaging e2perience

@haracter count* 11D/09 #otal number of characters* 5!0=D

'eferences
@ollins( -& $ !1!%& Creative Research: The Theory and Practice of Research for the Creative Industries& ?loomsbury Academic& Cetrieved from http*::goo&gl:Iz@K#E 7albach( L& $ !!D%& Designing Web Navigation& >MCeilly Media& Cetrieved from http*::goo&gl:mF4LKf 7aushi6( A& $ !!8%& Web Analytics: An Hour A Day W!Cd"& Niley& com& Cetrieved from http*::goo&gl:I/#NUI Morville( E&( < Cosenfeld( A& $ !!D%& Information Architecture for the World Wide Web: Designing #arge$%cale Web %ites& >MCeilly Media& Cetrieved from http*::goo&gl:GKU"f' 4ielsen( L& $1//5%& &sability 'ngineering& Elsevier Science& Cetrieved from http*::goo&gl:06yf3e Saleh( 7& A& $ !!/%& %oft(are 'ngineering& L& Coss Eub& Cetrieved from http*::goo&gl:Kz'E-I #ullis( #&( < Albert( N& $ !1!%& )easuring the &ser '*+erience: Collecting, Analy-ing, and Presenting &sability )etrics& Elsevier Science& Cetrieved from http*::goo&gl:zEAKcA

1!

Appendi( A
@ategories of Information Architecture and Usability Issues of AAU Moodle Eages&

1& @onsistency $Is the element and action consistent 'ith other elements in the page and bet'een pages, Is the element consistent 'ith 'eb standards,% & 4avigation $Is the navigation easy to use, Is it 'ell organized and is the relationships bet'een sections clear,% "& 4ecessity and Usefulness $Is the element of the 'ebsite necessary and useful in order to fulfill the purpose,% 5& Aabeling $Are the labels appropriate and consistent,% 0& Search $Is the search functionality easy to use, Is it effective,% =& Design $Is the design appropriate and functional,%

11

Appendi( )
Summary of Fuestionnaire Cesults&

>n average students loo6 into the Moodle several times a 'ee6 or more fre)uently #he importance of Moodle sections for students $most important listed first%* My @ourses9 Brom the study administration9 Administration Irades9 Sign up for free study activities9 @ourse Cegistration9 Borms9 Semester archive9 Study guide9 >ffers for students9 Message boards9 Administration Erofile9 Upcoming events9 Eeople participants9 4et'or6 server9 Search9 ?logs9 -elp9 Internal Messages9 >n;line users&

Sections that most of the people haven1t used or only used once* ?logs9 -elp9 Search9 4et'or6 servers9 Internal Messages&

Most of the students thin6 that the )uality of Information architecture $Aabels9 4avigation9 >rganization% of current Moodle is average&

Most of the students thin6s that such usability cornerstones as effectiveness and efficiency are )uite good in current Moodle but the engagement dimension is poorly satisfied&

Most of the students thin6s that the clarity and layout and formatting of the content that teachers puts into Moodle is )uite good but it lac6s consistency&

Students use both Moodle calendars but the full screen version is used more& Students thin6 that the access and content of forums are confusing9 there are )uite many functions they do not use9 there are no clear 'ay in 'hich professors present course materials9 it lac6s consistency9 the course list is confusing as it doesn1t have the functionality that 'ould allo' to classify the courses&

Students 'ould li6e to see the follo'ing improvements or additional functionality* consistency bet'een courses9 possibility to ma6e homepage more clear and simple9 easier access to forums9 better overvie'9 course list that is divided in categories9 search function9

1 function that 'ould allo' to see 'hat is changed : added( for instance( if teacher has changed literature list& #he most important functionality : content that people 'ould li6e to see in mobile enabled Moodle site : application is an easy to read schedule( upcoming classes( course material( +home'or6s. and forum posts&

Appendi( C
Survey Fuestions and Ceceived Ans'ers&