Criteria: Constructing High Quality Learning Materials—a Checklist

About this checklist

The standards in this checklist were adapted by Nardine Saad, 2nd year Medical Student, UCSF School of Medicine, in the
Summer of 2004 from criteria for the review of digital learning materials being developed by the AAMC Division of
Medical Education’s Digital Scholarship Peer Review Task Force.

Using this checklist

Read this checklist during your planning stage to familiarize yourself with the essential features of high quality learning
materials. While you develop and create your module, use the checklist as a reminder of the important aspects of your
module to work on. As you complete and edit your module, consider every item on checklist and assess your module for
its strengths and weaknesses, using the criteria to target possible areas of improvement. It is often helpful to ask
colleagues or mentors to review your module for you as well. Finally, if in the future you decide to revise your module or
submit it for publication, you may return to these criteria to reevaluate your module and perfect it.

A. Accuracy: Content is valid and reliable.

The author’s description of the material is accurate, up-to-date and complete

Provide your creation and revision dates. If future revisions may be needed, explain this and give a time frame for
future updates. If the content contains treatment guidelines or other recommendations that may become obsolete over
time, these should be updated as needed or a link should be provided to a resource with the most current guidelines.

The content is complete in scope without missing important information

Limiting yourself to a certain amount of content will be a difficult task. Design your content around your learning
objectives. Remember that you can choose fewer learning objectives to keep the module length manageable.

The citations, references, credits or links are appropriate and relevant

The module should offer access to a wide range of resources supporting course content. These sources should be up-
to-date, accurate and relevant. This should include sources used, references and helpful places to find additional
information if interested: textbooks, journal articles, websites, etc.

Content is consistently accurate and up-to-date throughout the material

B. Significance: Content teaches educationally significant concepts, models or skills in the discipline.

Your experience going through the learning process once and your mentor’s experience in teaching the material
over the years will help you determine this.

The content is unique, rare, contributes to the field or is hard-to-find

In other words, the content should not be elsewhere available using the same approach and format, nor should it be
redundant. The material should still be relevant and significant, but is underrepresented in the curriculum or has not
been presented in the same way.

The material facilitates teaching and learning of complex concepts

What approach is most effective in teaching the particular material you have chosen and in meeting the learning
objectives? How will you simplify the material into basic building blocks and then bring it all back together?

The material offers an innovative learning/teaching method

Spend time thinking about how you will use algorithms, cases, assessments, diagrams, images and other technology in
your teaching method. The digital/online format should be a substantial enhancement over what has been or would be
possible in a printed (paper) format.

© 2004 University of California Regents Last Updated 7/8/2004

C. Methods: The material is effective as a teaching tool.

Targeted learners will be able to achieve the learning objectives effectively with this material

Learning objectives are clear and appropriate for the target learners and learning context
Your set of learning objectives is one of the key components of your module. Think about what you would like your
target learners to come away with and what is appropriate for their education level, and write your learning
objectives accordingly. Take care to be completely clear, since students will use the objectives to gauge their own
understanding of the material.

The material accommodates multiple learning styles

The module should provide multiple visual, textual, kinesthetic and/or auditory activities to enhance student learning.
It should also provide multiple activities to help students develop critical thinking and problem-solving skills. (Key:
interactivity and self-assessments!)

The material engages and motivates the learner

A primary means of engaging students is through self assessments that are interesting and thought-provoking (but not
so challenging that they cause frustration) at every major teaching point. Making modules interactive also engages
the learners: for example, on anatomy, histology and pathology slides, clicking for labels allows for interaction and
self-testing; clicking on diagrams that change under different conditions (physiology, disease processes) engages the
learner and allows comparison of different states.

Assessment methods are appropriate to expected learning outcomes

The interactivity (e.g., feedback) is effective and maintains learner dignity

The questions and feedback should be neither too easy nor too difficult. They may increase in difficulty within the
same section; at every major teaching point a final question or two should allow advanced application of the concept
being taught. One common way this is done is through clinical vignettes or case scenarios with diagnosis and
treatment questions.

Multimedia features enhance the learning experience and facilitate achievement of objectives, do not distract and are
used appropriately;

The material can be tailored for various curricula/courses

Consider the learning context. For each application of the module, the course syllabus, online calendar and
professors should clearly identify the role the online module will play in the course as a whole. (Is this module
required or recommended? When should it be completed by? Which lectures and topics does it correspond to?)

The intended use of the material can be replicated based on the information supplied
It should be clear how the learner should use the material based on your introduction and content. If using your
module requires instruction or you anticipate its utilization by other schools, professors or courses, consider
providing a teaching guide or user’s manual.

The effectiveness is consistent throughout the material

Try not to lose steam when you are working on the end of the module! The middle and end should be as interactive
and high in quality as the beginning.

D. Presentation: The material is presented effectively and user-friendly.

Content (including video, audio, photographs, etc) is presented with clarity, focus, and organization

Oral or written communication is easy to understand and employs appropriate vocabulary

Glossary terms should be linked from the content. The links can either take the user to a glossary page with all the
definitions, or open a pop-up window with one definition.
© 2004 University of California Regents Last Updated 7/8/2004
 The information is presented in ways that are familiar to students
All web pages should be visually and functionally consistent throughout the module. (Use the iROCKET template).

The material is easy to navigate

The module should be well-organized and easy to navigate. Learners should be able to easily understand its structure.
This can be achieved in part by using internal navigation (“next” and “back,” or “on to…”), providing a site index,
and using bookmarks on the left hand column.

The design of the material offers appropriate user control (linearity/non-linearity, exit/revisit options, pacing, etc)
Again, a site index helps here.

The material is self-explanatory and intuitive to use

The material offers effective “help” functions

The layout of the materials is well organized without distracting elements

The aesthetic design of each page should present and communicate information clearly throughout the module. A
busy and exciting design layout may be confusing and distracting to learners. Having high quality content, text,
images and interactions will engage learners, not wild layouts. Also, having too many pop-up windows can distract
the user and are an annoyance to constantly close. Reserve these for main concepts and teaching points.

The material appears to have no bugs

Make sure you test the module over and over and, at different stages, get feedback from your mentor(s) and peers.

The material is easy to install, if installation is necessary

Audiovisual quality of the images, sounds, illustrations, videos, etc is good (e.g., good lighting, good sound quality,
adequate staining, well prepared specimen, adequate image resolution, etc).

Sources
1. Peer Review Special Interest Group (HEAL). Preliminary List of Review Criteria, 1/14/2004, and Draft Peer Review
Criteria for Digital Scholarship, 6/2004.
2. CSU Chico Faculty and Staff. ROI: Rubric for Online Instruction,
http://www.csuchico.edu/celt/roi/index.html
3. Peter, David. Course Module Evaluation Rubric.
http://david-peter.com/papers/rubric/course_module_evaluation_rubric.htm
(His resource: Gary S. Moore, Kathryn Winograd, and Dan Lange. You Can Teach Online: Building a Creative Learning
Environment. McGraw-Hill Higher Education, a division of The McGraw-Hill Companies. New York, NY, 2001.)

© 2004 University of California Regents Last Updated 7/8/2004