Vous êtes sur la page 1sur 113

12/11/2016

Training Workshop
on
STUDENT LEARNING OUTCOMES
& ASSESSMENTS

Dr. Mohammed Fadhali


Deanship of Academic Development
Jazan University

First session Second session Third Session


LOs
Institutional Importance & Benefits of LOs Assessments

Introduction to Learning Where do LOs come from?


Outcomes (LOs)
Characteristics of LOs
The difference between
Learning Outcomes LO Process and Levels
and Learning
Objectives NQF Learning Domains and
LO Verbs

1
12/11/2016

Aims of this training workshop


To provide participants with:
Essential concepts of learning outcomes
Adequate skills of using learning outcomes and their relation to
student learning
Opportunity to develop/review learning outcomes within their own
programs and related courses
Adequate skills in mapping of course LOs with the LOs of the
entire programme
Significant awareness of NCAAA Standard 4 and related
documents when using LOs
Significant abilities to assess students achievement through the
specified LOs

Learning Outcomes of this training workshop


By the end of this workshop participants will be able to:

Discuss concepts, issues and best practices related to developing and


approving objectives and LOs at program and course levels for the
different disciplines.

Utilize subject-benchmarks and professional standards and resources to


develop, implement, and monitor student LOs and curriculum considering
NCAAA and international requirement.

Provide support and guidance for faculty, support groups and related
committees in planning and developing LOs for programs and courses in
specified fields of study, mapping them across levels, skills, and courses.

Develop significant awareness and skills related to assessment methods

2
12/11/2016

Session (1) 9:00 10.30

Objectives and Learning


Outcomes

Educational goals , objectives and outcomes are widely used and required
for the development of professional education

Main Pillars of Institutions Whole Quality concern

Infrastructures
&Premises Regulations
Facilities Rules
Academic
Programs

Faculty
members Outputs
Supporting Outcomes
staff Services
Students

3
12/11/2016

The Big Picture Curriculum



Teaching and learning processes
Guiding and advising
Assessment
QA System
Continuous improvement
Other activities

Students


) (
Academic and Main product (Graduates)
supporting Staff Documentations
Facilities Publications
Regulations Nation and
community service

Accreditation &Reputation

Learning

Curriculum

Pedagogy Assessment

Learning
Environment

4
12/11/2016

Educational loop processes


2. Measurable
1. Goals & Student
Objectives Learning Outcomes

5.Conduct assessment and 3. Determine levels of


evaluate how well student Expertise required to
Learning outcomes Achieve learning outcomes
Were achieved

4. Choose
curriculum
and
methods

A DANGER OF
TOPIC OVERLOAD!

5
12/11/2016

Pivotal questions

Why we should write learning objectives and learning


outcomes
The role of learning outcomes in assessment
Why it is important to assess student learning

Its about levels of specificity and purpose


The greater the clarity in specifying objectives and learning outcomes, the
better able we are to teach and assess what our students have learned.

Educational System
1. What do you want the student to be able to do? (Outcome)

2. What does the student need to know in order to do this


well? (Curriculum, resources and facilities)

3. What activity will facilitate the learning? (Pedagogy)

4. How will the student demonstrate the learning?


(Assessment)

5. How will I know the student has done this well? (Criteria)

6
12/11/2016

www.ncaaa.org.sa 13

) (
Outcomes-based Education
:
Use of sentences that describe what is expected from the learner to
know, understand, and able to do at the end

.
Providing learning activities and resource that support the student to get
acquainted with those outcomes
.
Assessing the acquisition of student to those outcomes through certain
criteria
.

7
12/11/2016

Academic framework

University Vision,
Mission and Goals

This model allows you to develop
assessments that measure the
outcomes, and that then connect Program Mission,
directly to the program learning goals Objectives
and Learning Outcomes



these are determined by the faculty Course objectives and
and specify course-level, observable Learning Outcome
products or demonstrations
15


Things that you need to be
aware of..

16

8
12/11/2016

To start with program specification you need to be aware of



University s mission, vision, Goals, Values

Think about
Vision
JAZAN UNIVERSITY
Mission

Goals

Values

17

Vision: is a picture of the future to


create and described in present tense
:
It shows where we want to go What we

will be like when we get there .

Mission
Mission statement defines what an :
institution is
Why the institution exists and its
-
reason for being -
What we are here to do together ! -

18

9
12/11/2016

JAZAN UNIVERSITY Universitys Mission


Achieve academic excellence and prepare graduates to
Vision become regional
Jazan University will be a gateway to the future for and national leaders in business, industry, health,
Jazan Province and the Kingdom, recognized education, and government.
nationally and internationally for academic Serve the community by addressing its problems and
www.jazanu.edu.sa

excellence, competent graduates, high impact supporting its social and economic development.
research, and service that deliver social and Produce internationally recognized research and new
economic impact to the region and the world knowledge that meet the needs of Jazan Province, the
Kingdom, and the world.

19

For any academic program:


- Goals
- Competencies
- Learning outcomes

For any course


- Objectives
- Learning outcomes

10
12/11/2016

For a specific academic program


Program Specification:
- Vision Think about
- Mission your related
program
- Objectives, competencies and student LOs
-
Think about
your related
Course specification
course
Mapping:
- Program LOs with NCAAA domain
- Curriculum mapping
- Intended learning outcomes of each course to the student
or program Learning outcomes
Assessment and evaluation plan and strategy
21

Vision of the program


It is correlated with vision and mission of the institution
It is more focused on the theme of the program
It may be created based on the SWOT analysis

Mission of Program
Mission statement might include history, philosophy of academic, type of
student served ,environment, faculty , curriculum ,contributions to and
connection with the community etc.
Components of Program mission statement

Primary functions: Functions, operations, outcomes, and /or offerings of the program
Purpose: Why you perform your major activities or operations
Stakeholders: those who participate in the program and/or will benefit from it
22

11
12/11/2016

Examples
The mission of the Computer Science Program is to prepare students for
employment in various computer science related areas by
educating them in the fundamental concepts , knowledge , and laboratory /fields
techniques and skills of computing techniques

-
Goals- Objectives- Learning Outcomes
Learning goals are broad statements of the kinds of learning we hope
students will achieve. (Typically for program not courses but some use it)

Learning objectives are narrow statements of what is provided to the

student. (course/instructor-centered)
Learning outcomes are more narrow statements of what will the student
be able to know, do,
(student-centered )

24

12
12/11/2016


How to develop program objectives and program
outcomes?

Is this a chicken
and egg type of
question?


The program outcomes are derived from the program objectives.
PLOs must foster attainment of POs.
It is a topdown process that begins with the Program Objectives.

25

Goals and Objectives


Examples
To Graduate students with a BSc degree of Physics that are equipped with
the essential physics concepts and problem solving skills

Examples of objectives:

13
12/11/2016

Competency based objectives and LO formulation


You should think
Course Objectives and Learning outcomes about before
always link back to a Competency. setting or during
the review of PO &
PLO

27

Competence=Knowledge + Skills
Possession of a satisfactory level of relevant knowledge and acquisition of
a range of relevant skills that include interpersonal and technical components

Knowledge: Familiarity, awareness, or understanding gained through experience or


study
Skill: Ability, proficiency, facility, or dexterity that is acquired or developed through
training or experience

Competencies are more complex (higher level) than learning objectives


(can be regarded as the LOGICAL BUILDING BLOCKS upon which Learning
outcomes and assessments of professional development are based).

14
12/11/2016

MAJOR COMPETENCIES
1. Written and Oral Communication

COMMUNICATION SKILLS [reading, writing, speaking, listening, surfing]

Organize and articulate ideas for a range of audiences and purposes


using written, spoken, and symbolic forms;
Develop understanding, synthesize, and respond to written, spoken,
and symbolic messages;
Use technology to gather, process, and communicate information;
Plan, monitor, solve problem, and evaluate communication
experiences in a variety of situations.

2. Scientific and Quantitative Reasoning

SCIENTIFIC REASONING SKILLS [nature, practice, communication, and


application science]
Comprehend and describe science as a process of generating knowledge that
relies on testing hypotheses, verifying data, and evolving theories that
explain natural phenomena;
Recognize and Apply fundamental concepts in the physical sciences, allowing
informed decision-making based on science and technology

COMPUTATIONAL SKILLS [understanding and applying mathematical


concepts and reasoning, analyzing and using numerical data]
Perform numerical and algebraic skills necessary for the students field of
study without the use of a calculator;
Analyze numerical data relevant to the students field of study

15
12/11/2016

3. Critical Analysis

CRITICAL THINKING AND PROBLEM SOLVING SKILLS [analysis,


synthesis, evaluation, decision making, creative thinking]

Think creatively, critically, and strategically using problem-solving


strategies.

Demonstrate an ability to make effective decisions, solve real-life


problems relevant to the students field of study, and achieve goals in a
variety of situations.

4. Technological Competency

TECHNOLOGY SKILLS [computer literacy; Internet skills,


retrieving and managing information via technology]

Use available technology effectively and efficiently to locate,


retrieve, and manage information relevant to the students
field of study;

Develop new strategies as technology evolves.

16
12/11/2016

Examples
Program Competencies

A. Generic Skills
1. Investigative 2. Communication 3. Analytical 4. ICT 5. Personal

B. Subject Specific Skills


1. Deep knowledge, 2. Estimation 3. Mathematical 4. Experimental/
understanding, Practical
application
5. Problem solving 6.Computational 7. Physics culture 8. Learning ability

Objectives ------------ Outcomes

Student-Centered
Teacher-Centered Outputs
Inputs Results
Assessments
Measurable
Observable

Content-Centered Performance &


Traditional Achievement

What else overlaps?

17
12/11/2016

Program Objectives (POs)


Statements: Describe what the program provides to the students to get them suitably
prepared (both knowledge and skills) to the career and professional accomplishments.
+ The services to community and nations

( )


Factors to be considered when forming POs
Consistency with the mission of the institution and department.
Check

Participation of stakeholders in the process of framing POs.


No. of POs should be manageable.
Needs of the stakeholders
POs should be Specific to the program (not too broad & not too narrow)
It should not be similar to the PLOs
35

Program Objectives (POs)

Program
objectives
can be Core
broadly Breadth competence
defined
based on
five counts
Preparation Professionalism

Learning
environment

36

18
12/11/2016

Examples
I. Core Competence:
To provide students with a solid foundation in mathematical, scientific and engineering
fundamentals required to solve engineering problems.
II. Breadth:
To train students with good scientific and engineering breadth as to comprehend,
analyze, design, and create novel products and solutions for the real life problems.
III. Preparation:
To prepare students to excel in competitive examinations / postgraduate
programs / advanced education or to succeed in industry/ technical profession
IV. Professionalism:
To inculcate in students professional and ethical attitude, effective communication
skills, teamwork skills, multidisciplinary approach, and an ability to relate engineering
issues to broader social context.
V. Learning Environment:
To provide students with an academic environment with awareness of excellence,
leadership, and the lifelong learning needed for a successful professional career

Remarks:

The POs are reviewed periodically based on feedback of


the programs various stakeholders

Achievement of POs
Evidence and documentation are
required to show the achievement of
POs as set by the Institution with the
help of the assessment and evaluation
processes that have been developed.

19
12/11/2016

Example: Physics Program


Goals The main goals of Physics program are :
To produce competent, productive physics graduates with
significant knowledge and skills in various physics field
Contribute to the science education of all students.

Physics Program will provide students with:


Objectives

Adequate breadth and depth of knowledge of physics which would


lead to a successful career in a physics related profession such as
engineering or education.
Essential knowledge and skills to enable them to suit graduate
programs in physics
Significant proficiency in the application of physics to problems of
science, society, and technology.
39

COURSE OBJECTIVES
A course objective describes what a faculty member will
cover in a course. They are generally less broad that goals
and more broad than student learning outcomes.

They are course or teacher


centered statements

What the course and /or the


teacher is providing to the
students and for what?!

20
12/11/2016

Course brief description


A brief description is necessary to overall picture of the course

Example: Laser Physics


This course is designed to provide students with the fundamentals of
laser oscillation, its properties and applications. It describes the
interaction of photon with matter and covers the essential laser
requirements, laser gain media, laser oscillations inside various
resonators, and their stability conditions. It also discusses the laser
beam characteristics, transformation, and mode structure. It enables
students to explore some of the laser types and the related aspects of
various technological applications that employ lasers and beam optics.

This course is designed to This course is designed to:


provide students with: 1. Provide students with the fundamental
concepts and principles of light matter
1. The fundamental concepts and
interactions
principles of light matter interactions
2. Develop skills of the students with essential
2. The essential concepts of laser
concepts of laser oscillations, its operational
oscillations, its operational
requirements and laser beam properties
requirements and laser beam
3. Acquaint students with the skills of
properties
formulations of laser rate equations in
3. The formulations of laser rate
various systems, laser threshold conditions
equations in various systems, laser
and some of the laser types and their
threshold conditions and some of the
characteristics.
laser types and their characteristics.
4. Equip the students with adequate analysis of
4. The adequate analysis of the
the continuous-wave and pulsed laser
continuous-wave and pulsed laser
operation using appropriate formalisms.
operation using appropriate
5. Enable the students to assess the optical
formalisms.
resonator stability and mode structure based
5. The criteria of assessment of optical
on certain criteria
resonator stability and mode
6. Provide the students with the basics of some
structure
laser applications
6. The basics of some laser applications

21
12/11/2016

Course objectives: General Physics


This course is designed to: General Physics Course is
Provide students with the basic physics of units designed to:
& measurements, dimensional analysis of
physical quantities, rotational motion, rigid body 1.Provide students with the basic
principles of Newtonian
kinematics, fluid flow, sound waves, electrical & mechanics, electric and magnetic
magnetic forces and basics of light optics fields, geometrical optics,
Acquaint students with sufficient knowledge and thermal and wave physics.
understanding of basic Physical principles behind
2.Develop students abilities in
various phenomena and scientific/Engineering solving physics problems related
applications to.
Train students mathematical ability in simple
derivation and manipulation of physical 3.Equip students with the
necessary skills of lab and hands
formulae. on activities related to general
Develop the students problem-solving skills in physics as well as their analysis,
related fields of physics reporting and presentation.
Prepare students to gain basic skills of lab
4. Initiate team work and group
measurements, recording, data analysis and work among students.
reporting

Formulation of outcomes
at program and course level
How to write LOs?
Questions of level and standard (with reference to the
NCAAA Qualifications Framework).
The language of Outcomes.

22
12/11/2016

Learning Outcomes
Learning Outcomes: are statements that

The specific intentions of a program or course, written in specific terms. They


describe what a student should know, understand, or be able to do at the end
of that program or course.

LOs describe significant and essential learning that learners have achieved,
and can reliably demonstrate at the end of a course or program.

LOs Identify what the learner will know and be able to do by the end of a
course or program

Learning Outcomes are challenging but it is impossible to have a


meaningful Higher Education area without their widespread and
consistent use
(Stephen Adams, 2008)

What is the difference between outcomes and outputs?

This distinction is important, especially in the development and review of


Student Learning Outcomes.

For example, while we produce a number of new graduates (the output), it


is critical that we have a measure of the quality of the graduates as defined
by the college or discipline (the outcome).

Effective Student Learning Outcomes describe, in measurable terms, these


quality characteristics by defining our expectations for students at the end of
the course or program.

23
12/11/2016

Examples
1. Our program graduated 30students in Spring 2011.

A. Output B. Outcome

2. Graduates of our program were highly demanded by


various sectors.

A. Output B. Outcome

Learning Outcomes
-
SMART
Writing SMART LOs also helps you
to think about and identify:

Elements of the assessment & evaluation plan and


measurement, namely: Key Performance Indicators (KPI)

48

24
12/11/2016

Program learning outcomes

They describe what a student should know,


understand, or be able to do at the end of that
program

49

Course learning outcomes

They describe what a student should know,


understand, or be able to do at the end of the
course

50

25
12/11/2016


knowledge Design/development Conduct investigations
of solutions of complex problems


Problem analysis
Modern tool usage
Considerations

Individual and team for Program
work The society aspects
LOs

Communication Environment and
sustainability

Ethics Project management &
finance Lifelong learning
51

Formulation of learning outcomes?



LOs should be integrated, developmental, transferable
/
Use discipline-specific competencies/standards as a basis
LOs should be written in a manner whereby the learning can be
assessed through the use of an assessment method.

)This link with assessment is crucial for quality assurance(.

LOs are linked to level descriptors (Linking LOs to level descriptors


ensures that we build progression into programs)

LOs linked by definition to the credit that is attributed to courses.

The words used imply that the learning is at a particular standard.

26
12/11/2016

Writing Learning
Outcomes is a Process
not an Event

Successful learning outcomes


require the integration of content
and meaningful assessment with
effective pedagogy.

How to get started


Identify each general topic of your course
Write down what each topic in the course is providing to the student
(those are list of objectives).
Derive the suitable learning outcomes
Consider the level at which they should be able to perform. (NQF +
Blooms taxonomy)
Focus on what you expect students to be able to demonstrate upon
completion of the module or programme.
Make sure you will be able to measure all objectives and LO
Rewrite each objective based en outcomes.

27
12/11/2016

CHALLENGES IN WRITING LEARNING OUTCOMES

be written in the future tense


identify important learning requirements
use clear language easily understandable to students , instructor
colleagues and external examiners

Include various domains (NQF domains)


- It is standard practice to use a phrase like
On successful completion of this program or module,
students should be able to
- Avoid complicated sentences. If necessary use one than one sentence to
ensure clarity.
- General recommendation: 5 8 learning outcomes per module.

Structure of The LO statement

BEHAVIOR CONDITION CRITERIA

Action verb to Circumstances Describing the


describe what under which limits or range of
an acceptable
participants will behavior will response. i.e.,
be able to do as a occur how well does the
consequence of a learner have to
learning activity perform?

28
12/11/2016

Good learning outcomes?


Active
Attractive

Visible

Good
Comprehensive
LO

Assessable

Appropriate
Attainable

Dr. Mohammed Fadhali

Some benefits of learning outcomes


Common language for education
Select content
Develop of instructional strategy and materials

Construct tests and other instruments for assessing and evaluating

Facilitate comparability across the various systems in different
countries.
Facilitate diversity: formal learning, informal learning, life long
learning, etc.
58

29
12/11/2016

Help to explain more clearly to students what is expected of them and thus help to
guide them in their studies motivation and sense of purpose

Help instructors to focus more clearly on what exactly they want students to
achieve in terms of knowledge and skills.

Help instructors to define the assessment criteria more effectively.

Help to provide guidance to employers about the knowledge & understanding and
skills possessed by graduates of programs.

A more transparent learning environment

Implications for LOs

LOs should be set in order to encourage deep learning

Assessment methods should require deep approaches to be deployed

Different LOs will suit/require different learning styles

LOs should be selected to suit a range of learning styles

Teaching should encourage students to explore their own learning styles


and address their weaknesses

Assessment methods should cover a range to suit all types of learner.

30
12/11/2016

Potential problems with Learning Outcomes

Could limit learning if learning outcomes written within a very


narrow framework lack of intellectual challenge to learners.

Learning outcomes should not be reductionist but rather expansive


and intended to promote the higher order thinking skills.

Danger of assessment-driven curriculum if learning outcomes too


confined.

Could give rise to confusion among students and staff if guidelines


not adhered to when drawing up learning outcomes, etc.

PROCESS OF FORMULATION OF
LEARNING OUTCOMES

31
12/11/2016

Blooms Taxonomy of Learning Outcomes

Blooms taxonomy (1956) is a very useful aid to


writing learning outcomes.
The taxonomy consists of a hierarchy of
increasingly complex processes which we want
our students to acquire.

3 Learning Domains
Cognitive, Mental
Skills (knowledge
and thinking)

Psychomotor Affective
Manual or Growth in feelings or
Physical Skills, emotional areas (Attitude)
How we do
How we feel

The cognitive (knowing or thinking) is composed of six successive levels


arranged in a hierarchy.

6. Evaluation

5. Synthesis

4.Analysis

3. Application

2. Comprehension

1. Knowledge

32
12/11/2016

Levels of Learning
Blooms Taxonomy Evaluating Compare
contrast,
&

critique, justify

Synthesizing Adapt, combine,


compare, contrast,
design, generate
Analyzing Correlate, diagram, distinguish,
outline, infer, discuss

Applying Determine, develop, compute,


utilize, conduct, use

Comprehending Classify,
explain, discuss, give examples,
summarize, describe

Knowing Define, list, reproduce, enumerate, identify

65

33
12/11/2016

1. Knowledge

Ability to recall or remember facts without necessarily


understanding them

Use action verbs like:

Arrange, collect, define, describe, duplicate, enumerate, examine, find,


identify, label, list, memorise, name, order, outline, present, quote, recall,
recognise, recollect, record, recount, relate, repeat, reproduce, show, state,
tabulate, tell.

Examples: Knowledge
Recall fundamental Physics laws : Newtons laws of motion work-energy
theorem- Coulombs law- etc.

List the units of measurements of some physical quantifies


Outline various form of energies and their transformations
Define fundamental physical quantities.
Identify the type of some mathematical equations.
Define various states of matter
State Snells law and lens maker formula of thin lenses

34
12/11/2016

2. Comprehension
ability to understand and interpret learned information

Use action verbs like:

Associate, change, clarify, classify, construct, contrast, convert,


decode, defend, describe, differentiate, discriminate, discuss,
distinguish, estimate, explain, express, extend, generalise, identify,
illustrate, indicate, infer, interpret, locate, predict, recognise,
report, restate, review, select, solve, translate.

Examples: Comprehension
Express kinetic and potential energy of a particle
Discuss the applications of continuity and Bernoulli's equations.
Classify the type of elastic moduli .
Convert between scales of units of measurements
Describe interference of light beams.
Relate the electric force, electric field and electric potential
Infer magnetic force of a moving charge, a carrying current wire and a
solenoid.
Explain Carnot cycle for an ideal engine

35
12/11/2016

3. Application
ability to use learned material in new situations, e.g. put
ideas and concepts to work in solving problems

Use action verbs like:

Apply, assess, calculate, change, choose, complete,


compute, construct, demonstrate, develop, discover,
dramatise, employ, examine, experiment, find, illustrate,
interpret, manipulate, modify, operate, organise,
practice, predict, prepare, produce, relate, schedule,
select, show, sketch, solve, transfer, use.

Examples: application

Use dimensional analysis for examining the validity of equations.


Apply Newtons laws for describing the motion
Solve problems related laser oscillation, threshold and beam
transformation
Drive formula for laser gain starting from the rate equation of
transitions.
Use energy and mass conservation to drive some physical formula
Measure some physical parameters in lab (Resistance, sound speed,
viscosity coefficient)
Draw graphs of experimentally measured quantities
Report results of lab experiments.

36
12/11/2016

4. Analysis
Ability to break down information into its components, e.g. look for
inter-relationships and ideas (understanding of organisational structure)

Use action verbs like:

Analyse, appraise, arrange, break down, calculate, categorise,


classify, compare, connect, contrast, criticise, debate, deduce,
determine, differentiate, discriminate, distinguish, divide,
examine, experiment, identify, illustrate, infer, inspect,
investigate, order, outline, point out, question, relate,
separate, sub-divide, test.

Examples: Analysis
Differentiate different types of laser modes
Distinguish time dependent and time independent Schrodinger
equations
Analyze experimental results of some physical quantities.
Compare experimental results with theoretical ones
Categorize various types laser applications
Calculate various physical quantities from their related formula
Investigate the behaviour of some physical parameters due to an
external effect
Estimate the values of some physical parameters based on
theoretical /experimental analysis

37
12/11/2016

5. Synthesis
Ability to put parts together
Use action verbs like:

Argue, arrange, assemble, categorise, collect, combine, compile,


compose, construct, create, design, develop, devise, establish,
explain, formulate, generalise, generate, integrate, invent,
make, manage, modify, organise, originate, plan, prepare,
propose, rearrange, reconstruct, relate, reorganise, revise,
rewrite, set up, summarise.

Examples: Synthesis

Recognize and formulate problems that are amenable to energy


management solutions.
Propose solutions of some energy alternatives
Assemble electronic components to build an electronic circuit.
Integrate concepts of energy and mass conservation in energy transfer
analysis.
Summaries the causes and effects of wave interference and diffraction.
Relate the sign of enthalpy changes to exothermic and endothermic
reactions.
Setup an experimental arrangement to measure a physical quantity

38
12/11/2016

6. Evaluation
Ability to judge value of material for a given purpose

Use action verbs like:

Appraise, ascertain, argue, assess, attach, choose, compare,


conclude, contrast, convince, criticise, decide, defend,
discriminate, explain, evaluate, interpret, judge, justify,
measure, predict, rate, recommend, relate, resolve, revise,
score, summarise, support, validate, value.

Examples: Evaluation
Assess the importance of key parameters in a physical effect.
Evaluate measured/ calculated experimental results
Appraise the role of team work in collecting data / conducting case studies.
Predict the effect of change in temperature on the position of equilibrium
Summarise the main contributions of Faraday to the field of
electromagnetic induction.

Recommend
Justify
Assess.

39
12/11/2016

Engineering Examples: for All


K: Identify and define a system, its behavior, and its
elements.
C: Explain the links between engineering theory and
practice.
A: Use prototypes and test articles in design
development.
A: Analyze the strengths and weaknesses of the design
team.
S: Formulate solutions to problems using creativity and
good decision making skills.
E: Appraise operational systems and recommend
improvements.

Affective Domain Krathwohl's domain

AFFECTIVE DOMAIN (Feeling) concerned with value issues :


involves attitudes.
Integration of beliefs, ideas and attitudes
5. Characterising

4. Organising Comparing, relating, synthesising values

3. Valuing Commitment to a value

2. Responding Active participation in own learning

1. Receiving Willingness to receive information

40
12/11/2016

Action verbs for affective domain


Accept, assist, attempt, challenge, combine, complete,
defend, demonstrate (a belief in), discuss, dispute, embrace,
follow, hold, integrate, order, organise, join, share, judge,
praise, question, relate, share, support, synthesise, value.

Examples of Learning Outcomes in Affective Domain


Accept the need for professional ethical standards.
follow the need for confidentiality in the professional client
relationship.
Display a willingness to communicate well with patients.
Relate to participants in an ethical and humane manner.
Resolve conflicting issues between personal beliefs and
ethical considerations.
Embrace a responsibility for the welfare of children taken
into care.
Participate in class discussions with colleagues and with
teachers.

41
12/11/2016

Recognize the ethical issues involved in using people in


scientific experiments.
Demonstrate the courage to act on principle despite
adversity.
Value the importance of hard work, intensity, and
attention to detail.
Commit to a personal program of lifelong learning and
professional development.

Psychomotor Domain Simpsons Psychomotor Domain

describe the ability to physically


manipulate a tool or instrument
6. ADAPTATION
Involves co-ordination of brain and
muscular activity 5. COMPLEX OVERT RESPONSE

usually focus on 4. MECHANISM


change and/or
development in 3. GUIDED RESPONSE
behavior 2. SET
and/or skills.
1. PERCEPTION

42
12/11/2016

Action verbs for PSYCHOMOTOR domain:

bend, grasp, handle, operate, perform, reach, relax,


shorten, stretch, differentiate (by touch), perform
(skilfully).

Examples
Recognize sounds that indicate malfunctioning equipment. (Level 1: Perception)
Show eagerness to assemble electronic components to complete a task. (Level 2:
Set)
Experiment with various ways to measure a given volume of a volatile chemical.
(Level 3: Guided response)
Demonstrate the ability to correctly execute a 60 degree banked turn in an aircraft
70 percent of the time (Level 4: Mechanism)
Dismantle and re-assemble various components of an automobile quickly with no
errors. (Level 5: Complex or Overt response)
Modify instruction to meet the needs of the learners. (Level 6: Adaptation)
Design a more efficient way to perform an assembly line task.(Level 7: Origination)

43
12/11/2016

Laboratory skills
Operate the range of instrumentation specified in the module safely and
efficiently in the chemistry laboratory.
Perform titrations accurately and safely in the laboratory.
Construct simple scientific sketches of geological features in the field.

Clinical Skills
Perform a comprehensive history and physical examination of patients in
the outpatient setting and the general medical wards, excluding critical care
settings.
Perform venipuncture and basic CPR.

Presentation skills
Deliver an effective presentation.
Demonstrate a range of graphic and CAD communication techniques.
Perform basic voice and movement tasks (theatre studies).

What is NQF?

44
12/11/2016

Standard 4 (NCAAA): Learning and Teaching


Student learning outcomes must be clearly specified, consistent with the
NQF and requirements for employment or professional practice.

Standards of learning must be assessed and verified through appropriate


processes and benchmarked against demanding and relevant external
reference points.

Teaching staff must be appropriately qualified and experienced for their


particular teaching responsibilities, use teaching strategies suitable for
different kinds of learning outcomes and participate in activities to improve
their teaching effectiveness.

Teaching quality and the effectiveness of programs must be evaluated


through student assessments and graduate and employer surveys with
evidence from these sources used as a basis for plans for improvement.

89


Important aspects for formulation of LOs

Qualifications Framework

Domains of
Levels Credits
Learning

Level 1. Associate Diploma 120-130 credit
Level 2. Diploma hours for a
Level 3. Bachelor
Bachelors degree
Level 4. Higher Diploma
Level 5. Master
Level 6. Doctor 90

45
12/11/2016

Domains of Learning

Psychomotor
Knowledge skills
-

Five
Learning
Domains:
Cognitive Comm. &IT
skills
NQF and numerical
skills


Interpersonal
skills and
responsibility


Dr. Mohammed Fadhali 91

1. Knowledge

Knowledge: the ability to recall, understand, and present information,


including:

Knowledge of specific facts and details

Knowledge of concepts, principles and theories

Answers may be memorized or closely paraphrased from assigned


material.

Knowledge of procedures; steps in a process.

46
12/11/2016

2. Cognitive Skills

Cognitive skills: the ability to.


Apply conceptual understanding of concepts, principles, and theories,

Apply procedures involved in critical thinking and creative problem solving,


both when asked to do so, and when faced with unanticipated new situations,

Investigate issues and problems in a field of study using a range of sources and
draw valid conclusions.

Ability to comprehend the meaning of material.

Answers must be in the students own words while still using terminology
appropriate to the course material.

3. Interpersonal Skills and Responsibility

Including the ability to:

Take responsibility for their own learning and continuing personal and
professional development,

Work effectively in groups and exercise leadership when appropriate,

Act responsibly in personal and professional relationships,

Act ethically and consistently with high moral standards in personal and
public forums.

47
12/11/2016

4. Communication, IT and Numerical Skills

Including the ability to:

Communicate effectively in oral and written form,


Use information and communications technology, and
Use basic mathematical and statistical techniques.

5. Psychomotor Skills

Psychomotor skills: manual dexterity

Extremely important in some fields of study. For example, very high


levels of psychomotor skills are required for a surgeon, an artist, or a
musician.

Psychomotor skills apply only to certain fields, and their nature


varies widely.

48
12/11/2016

NQF Learning Outcome domains and related Verbs


NQF Learning Domains Suggested Verbs
Knowledge list, name, record, define, label, outline, state, describe,
recall, memorize, reproduce, recognize, record, tell, write
estimate, explain, summarize, write, compare, contrast,
Cognitive Skills diagram, subdivide, differentiate, criticize, calculate,
analyze, compose, develop, create, prepare, reconstruct,
reorganize, summarize, explain, predict, justify, rate,
evaluate, plan, design, measure, judge, justify, interpret,
appraise
Interpersonal Skills & demonstrate, judge, choose, illustrate, modify, show, use,
Responsibility appraise, evaluate, justify, analyze, question, and write
Communication, demonstrate, calculate, illustrate, interpret, research,
Information question, operate, appraise, evaluate, assess, and criticize
Technology, Numerical
demonstrate, show, illustrate, perform, dramatize, employ,
Psychomotor manipulate, operate, prepare, produce, draw, diagram,
examine, construct, assemble, experiment, and reconstruct

97


.


.

.

98

49
12/11/2016

Important considerations
Avoid learning outcomes which are too broad in scope, such as Recall the
fundamental concepts of Structural, Mechanical and Electrical Engineering.
Avoid learning outcomes which are too narrow in scope, such as State the four
categories of bonds.
Avoid overloading your modules with too much (content: knowledge); consider
application, analysis, synthesis and evaluation.
The outcomes stated in your plan should be clear and simple. Avoid the use of
bundled or compound statements that join the elements of two or more
outcomes into one statement.
E.g., be able to analyze and interpret data to produce meaningful conclusions
and recommendations and explain statistics in writing is a bundled
statement.
This outcome really addresses two separate goals, one about analyzing and
interpreting data and another about writing.

Verbs and phrases that complicate measurability

Understand
An internal process that is indicated by demonstrated behaviors not recommended
for program or course SLOs
Appreciate; value
Internal processes that are indicated by demonstrated behaviors closely tied to
personal choice
Become familiar with
Focuses assessment on becoming familiar, not familiarity
Learn about, think about
Not observable; demonstrable through communication or other demonstration of
learning
Become aware of, gain an awareness of
Focuses assessment on becoming and/or gaining not actual awareness
Demonstrate the ability to
Focuses assessment on ability, not achievement or demonstration of a skill
Also introduce, cover, present..

50
12/11/2016

Examples
Improper Outcome
Students will name the three types of rock in order to
differentiate among them.

Proper Learning Outcome


Students will compare and contrast the characteristics of
the three types of rocks in order to differentiate among
them.

The students will be able to understand regression models


and time series models in economics.

The students will be able to interpret and develop regression


models and time series models in economics.

Students will know the differences in major contemporary


theories in the field of sociology

Students will be able to contrast major contemporary


theories in the field of sociology

51
12/11/2016

52
12/11/2016

Activity-1: Group discussion

Consider the NCAAA documents provided:


1. Write 3-4 learning objectives to your related course
2. Derive from that 3-5 learning outcomes

Tea Break
15 minutes

53
12/11/2016

Session (2) 11:00------12:30 pm

LO examples for Program and


course levels

Mapping Matrices

Generic outcomes
be able to
Plan and improve their own
performance

Research new areas of


knowledge using both library and
online resources

Show evidence of having worked


in groups and developed group
working skills

Plan and deliver an effective


presentation

54
12/11/2016

PROGRAM OUTCOMES
Narrow statements that describe what students are expected to know
and be able to do by the time of graduation.

Includes:
Knowledge:
Facts students know and concepts they understand
Skills:
Skills students use in managing and applying their knowledge such as
computation, experimentation, analysis, synthesis /design, evaluation,
communication, leadership, and teamwork.
Attitudes:
Attitudes that dictate the goals toward which their knowledge and skills
will be directed personal values, concerns, preferences, and biases.

Programme-level outcomes

Apply.quantitative science and engineering tools to analyze


problems (Engineering)

Able to identify central arguments, themes, perspectives, and


theoretical frameworks of secondary sources as well as their
assessment (Humanities)

Apply the principles, skills and knowledge of evidence-based


medicine (Medicine)

55
12/11/2016

You need to consider


Benchmarking

At least three similar cases

Example: Physics Program The student will be able to

Knowledge:
Demonstrate knowledge and comprehension of the basic and applied fields of
physics.
Problem Solving:
Develop independent problem solving skills.
Laboratory Work:
Conduct experimental activities using various technique & setups and analyze
results to make meaningful comparisons between experiment and theory.
Written Communications:
Acquire effective written communication skills via preparation of well-structured
lab. reports.
Oral Communications:
Communicate problem solution and give oral presentations in group discussion,
seminars and scientific meetings.
Professional Development:
Able to carry out professional activities in the area of applied technologies and
industry related to physics
112

56
12/11/2016

Program Student Learning Outcomes for MSE program Quoted

Content Knowledge
Apply knowledge of mathematics, science and engineering principles to
materials science and engineering.
Design and conduct materials science and engineering experiments and
analyze and interpret the data.
Critical Thinking
Design a MSE system, component or process to meet desired needs within
realistic economic, environmental, social, political, ethical, health and safety,
manufacturability and sustainability constraints.

Communication
Communicate technical data and design information effectively in speech and
in writing to other materials engineers.

Example of Programme Learning Outcomes [BSc(Ed)] Quoted


On successful completion of this program, students will be able to:
Recognise and apply the basic principles of classroom management
Identify the key characteristics of excellent teaching in science.
Develop comprehensive portfolios of lesson plans that are relevant to the
science curricula in schools.
Evaluate the various theories of teaching and learning and apply them to
assist in the creation of effective and inspiring science lessons.
Critically evaluate the effectiveness of their teaching of science in the
second-level school system.
Display a willingness to co-operate with members of the teaching staff in
their assigned school.
Foster an interest in science and a sense of enthusiasm for science subjects
in their pupils.
Synthesise the key components of lab. organisation and management and
perform lab. work in a safe and efficient manner.
Communicate effectively with the school community and with society at
large in the area of science education.
114

57
12/11/2016

Example of Program Learning Outcomes [Engineering] Quoted

On successful completion of this programme, students should be able to:


Identify, formulate, analyse and solve engineering problems. ??????
Derive and apply solutions from knowledge of sciences, engineering
sciences, technology and mathematics.
Design a system, component or process to meet specified needs and to
design and conduct experiments to analyse and interpret data.
Work effectively as an individual, in teams and in multi-disciplinary
settings together with the capacity to undertake lifelong learning.
Communicate effectively with the engineering community and with
society at large.

115

Example of Program Learning Outcomes [Computer Science ]


Quoted

On successful completion of this programme, students should be able


to:
Identify a full range of IT skills and display a mature computer literacy
Perform problem solving in academic and industrial environments.
Use, manipulate and create large computational systems.
Work effectively as a team member.
Organise and pursue a scientific or industrial research project.
Write theses and reports to a professional standard, equivalent in
presentational qualities to that of publishable papers.
Prepare and present seminars to a professional standard.
Perform independent and efficient time management.

116

58
12/11/2016

Examples Program Learning Outcomes [Medicine] Quoted

Graduates in medicine will be able to:


Carry out a consultation with a patient (history, examination ...)
Assess clinical presentations, order investigations, make differential diagnoses, and
negotiate a management plan
Provide immediate care of medical emergencies, including first aid & resuscitation
Prescribe drugs; carry out practical procedures (e.g. venepuncture)
Communicate effectively in a medical context;
Apply ethical and legal principles
Assess psychological and social aspects of a patient's illness
Apply the principles, skills and knowledge of evidence-based medicine
Use information and information technology effectively in a medical context
Apply scientific principles, method and knowledge to medical practice and
research/promote health, engage with population health issues and work effectively
in a health care system
117

Students should be able to: Quoted


Investigate the variety of human culture and demonstrate an
understanding of the ways in which cultures have changed;
Recognize and employ a wide range of humanistic, qualitative,
B.Sc. Humanities

quantitative, theoretical, or philosophical methods for recording and


explaining human experience;
Describe ways in which a given language reflects a way of thinking,
cultural heritage, larger set of cultural values, or aspects of society;
Identify and assess their own and others' values;
Identify the underlying premises in their own and others' arguments; and
Use appropriate technologies to conduct research on and communicate
about language, culture, and/or philosophy and to access, evaluate, and
manage information to prepare and present their work effectively.

59
12/11/2016

Exercise

In your subject group, complete the following task:

Medicine/medical sciences
Draft three common programme-level outcomes which in your view
would apply to all Bachelor-level medicine/medical sciences programs
Engineering/science
Draft three common programme-level outcomes which in your view
would apply to all Bachelor-level engineering/science programs

Humanities
Draft three common programme-level outcomes that should be acquired
by all Bachelor-level students of humanities, irrespective of their
specialism?
119

Examples course-level outcomes: Laser Physics


NQF Learning Domains and Course Learning Outcomes
1.0 Knowledge
1.1 Define the main requirements of laser operation
1.2 Describe various processes in laser matter interaction, various laser types and their applications
2.0 Cognitive Skills
2.1 Derive the Einstein relations, laser gain and other laser parameters from the laser rate equations
2.2 Explain the characteristics of light from pulsed and CW lasers
2.3 Discuss the stability of optical resonator and the output mode structure
2.3 Solve problems related to laser matter interaction, stability of laser cavity, laser oscillations and
laser beam characteristics
2.4 Discuss various laser applications in various fields
3.0 Interpersonal Skills & Responsibility
3.1 Show effective collaboration and bear individual responsibility during group work and/or
assignments
4.0 Communication, Information Technology, Numerical
4.1 Use internet resource to prepare a related study case on laser systems and their applications
5.0 Psychomotor
5.1 judge the suitable laser types for certain applications

60
12/11/2016

Examples Module-level outcomes in Medicine


Quoted

Upon completion of the module the student will be able to:

Explain the meaning of key concepts in parasite epidemiology

Describe techniques used in epidemiological surveys

Discuss the roles and effectiveness of chemotherapy and


chemoprophylaxis in control of parasites

Apply techniques used in epidemiological surveys

Critically analyze and interpret ecological and epidemiological data

(Module Parasite Epidemiology, School of Tropical


Medicine, University of Liverpool)

Examples Module-level outcomes in Engineering


Quoted

Upon completion of the module the student will be able to:

Demonstrate application of the key principles of DC circuit theory


including Kirchhoff's laws of current and voltage, and rules for current and
voltage division.

Analyze and solve simple AC series and parallel circuits using phasors
and complex numbers.

Explain the operation of ideal and non-ideal op-amp circuits and design
simple op-amp applications.

Comprehend the operation of simple semiconductor devices.


(Module Electrical and electronic principles, Electrical and
Electronic Engineering Major, University of Northumbria)

61
12/11/2016

Example Module in food technology


Quoted

Describe the general principles of mass balances in steady state systems.

Draw and use process flow diagrams with labels on flow streams for mass
balance problems.

Solve mass balance problems associated with food processing operations.

Design and solve mass balances for complex process flow systems,
including batch mixing problems, multiple stage flow problems, problems
with multiple inflows and outflows, recycle streams and multiple
components, and processes where chemical reactions take place.

Hartel and Foegeding (2004)

Example Research analysis


Quoted

On successful completion of this module students will be able to:

Select, carry out and interpret the results of appropriate statistical


methods for describing and analysing data sets, in the context of their
own research interests;
Make use of appropriate software packages to carry out statistical
analysis, and to interpret their output in terms relevant to the research
field;
Demonstrate an understanding of the concepts of sampling variability,
bias, sampling error and statistical significance, and their impact on the
interpretation of research findings;
Recognize a range of multivariate methods and their use and limitations
in a research context;
Examine critically their own and other researchers use of quantitative
methods of analysis.

62
12/11/2016

Module Title: Dental Surgery


Quoted

On successful completion of this module, students should be able to:

Summaries relevant information regarding the patients current condition to


generate a differential diagnosis
Formulate an appropriate treatment plan and justify the proposal giving due
consideration to patient expectations and limitations
Arrange appropriate tests and demonstrate the ability to interpret tests and
reports
Administer local anaesthetics safely and perform basic dento-alveolar surgical
procedures in a professional manner showing good clinical governance
Recognize, evaluate and manage medical and dental emergencies appropriately
Differentiate between patients that can/can not be safely treated by a GDP
Manage competing demands on time, including self-directed learning & critical
appraisal
Master the therapeutic and pharmacological management of patients with facial
pain and oro-facial disease

Exercise
In your subject or specialization group, write four student
learning outcomes; list the SLOs in order of progression.

63
12/11/2016

Mapping Matrices

127

Curriculum Mapping
A method of analyzing the relationship between the curriculum and
student learning outcomes.

A curriculum map graphically illustrates how a program's


courses/requirements introduce and reinforce the program's student
learning outcomes.

Useful Tools & Guides


Comprehensive Assessment How-to Guide: Curriculum Mapping/Curriculum Matrix -
University of Hawaii Manoa
Curriculum Mapping for Effective Assessment (PDF) -Michael A. Heel, Monroe
Community College
Creating a Curriculum Map, Samples, & Best Practices (DOC) -Rochester Institute of
Technology

64
12/11/2016


Mapping Matrices

Benefit
Benefit for Benefit for for
Programs Faculty Students

129

Curriculum Mapping
Connect LOs to
Course course activities
Maps
Projection Maps

Curriculum Connect Course


Program LOs to program
Map Maps objectives and
LOs

Connect Programs
Institution LOs to
Maps institutional
goal

130

65
12/11/2016

Learning outcomes and competences in study program


Examples

Program and/or Competences


course
LOs A B C D E F G H I J

LO 1
LO 2
LO 3
LO 4
131

Mapping PLOs to POs


The mapping could be ,one to many, many to one, and many to many parameters

Program Learning Outcomes


Program
Objectives a b c d e f g h

I
II
III
IV
V
132

66
12/11/2016

Mapping course LOs to program LOs


The mapping could be ,one to many, many to one, and many to many parameters

Program Learning Outcomes

Course Learning a b c d e f g h
Outcomes

I
II
III
IV
V

Allocation of Responsibilities for Learning Outcomes to Courses


Major Responsibility
x Minor Responsibility

In NCAAA
I (introduced) , P (Proficient) , A ( Advanced)

67
LO's
Examples: TPA model

Facts

Use of IT

statistics
behavior
Apply skills
Procedures

personal and
learning

and leadership

communication
Act responsibly-

Basic maths and


Oral and written
problem solving

Performance,
Concepts, theories

Group participation

Ethical standards of
Courses

Feeling, Carryout
Creative thinking and

Responsibility for own

professional situations
General Chem
Chem
Anal chem 101&102
Chem 223
Inst. Chem 323
Analysis
Org. chem Chem 201
& 202
Qual Chem 303
org.chem
Phys. Chem 212
Chem & Chem
311
Inorg. Chem 331
Chem & Chem
332
Chem Chem 479
seminar
Chem Chem 471
Project & chem.
472
Summer Chem 399
training
Knowledge

Cognitive Skills

Calculus Math 101

Psychomotor Skills
&102
Physics Phys 101
& 102
Computer ICS 101
Interpersonal Skills and Responsibility

Communication IT and Numerical Skills

prog.

English Engl
101&102
English Engl 214

Curriculum Mapping and Learning Outcomes


Professiona IAS 212
l ethcs

Human IAS 322


rights in
Islam
Practical IAS 101
Grammer

Professiona IAS 201


l Writing

Oral IAS 301


comm.
skills

136
135

Physical PE 101
education

68
12/11/2016
12/11/2016

Grouped based on competencies

137

Exercise: Outcome mapping

Could you please make mapping -


to:
LOs of your program to the
LOs of the course you teach
-
Allocation mapping of your
program LOs and the course
you teach.

138

69
12/11/2016

EFFECTIVE
TEACHING and LEARNING
STRATEGIES

Previous
workshop

Teaching & Learning in OBE OBE

Teaching is teaching if learners learn.


Learning is measured by its outcome.
In all teaching approaches.Focus on learning rather
than on teaching.
Subjects do not exist in isolation, should be linked together.
It is important that students learn how to learn, hence a
teacher should be innovative.

How then should teaching-learning be done in OBE?

70
12/11/2016

EFFECTIVE INSTRUCTIONAL ACTIVITIES


.
Be student centred )(
Build on previous learning )(
Consider useful & meaningful content that is consistent with sound
principles of learning.

Maintain students attention and interest
.
Encourage student participation, when appropriate )(
Lead students in the direction of the behaviour or learning outcomes specified
in the learning objective

Provide proper stimuli and reinforcement
Include appropriate terminology for the student
Be properly sequenced
Be easy to understand

141

Learner-Centered Assessment:
Implications for Classroom Practice

Clarifying learning outcomes at the course planning stage


Sharing learning goals with students throughout the semester
Asking appropriate and effective questions
Focusing oral and written feedback on the learning outcomes of
lessons and tasks

Encouraging students self-assessment against the learning


outcomes
Organizing individual student target-setting that builds on previous
achievement as well as aiming for the next level up

71
12/11/2016

Session ( 3 ) 1:00 2:30 pm

ASSESSMENT
of
LEARNING OUTCOMES

The Educational Cycle

2. Learning Opportunities

1. Learning objectives
& outcomes 3. Assessment

4. Using Results

72
12/11/2016

Students learn what they think theyll be assessed


on, not whats in the curriculum.

The trick is, then, to make sure the assessment tasks


mirror what you intended them to learn
Biggs, 2002, page 6

Purposes of Assessment: feedback,


diagnosis, motivation, guidance, learning
support, selection, grading, certification,
progression, professional recognition, etc.

Levels of Assessment

Assessment Assessment Assessment


of individuals of programs of institutions

Uses the individual Uses the institution


Uses the dept. or as the level of
student, and his/her program as the level
learning, as the analysis. Ideally,
of analysis. Ideally institution-wide
level of analysis. program goals and
e.g class room goals and objectives
objectives would would serve as a
assessment or a serve as the basis for
course assessment basis for the
the assessment. assessment.

146

73
12/11/2016

Create written
statements of
measureable SLOs
Steps of Assessment Cycle

Use results to
Choose the
improve student
evaluation tools
learning

Evaluate student Set standards for


performance, levels of
assemble data, and performance on
report results each SLO

Identify observable factors


that provide the basis for
Set benchmarks assessing which level of
performance has been
achieved

Assessment of academic programs


The systematic and ongoing method of


gathering, analyzing and using information
from various sources about a program and

measuring program outcomes in order to

improve student learning.

148

74
12/11/2016

Program assessments
Assessment modes and their relationship to the different
categories of LO, and to students learning styles.

Writing outcome-related assessment criteria

Linking criteria to feedback on assessment

Assessment mapping to ensure coverage of LOs.

149

Academic Program Assessment: Purposes

Effective program assessment helps


answer three questions: :
1. What are you trying to do?
2. How well are you doing it? .1
3. How (using the answers to 1 and 2) .2
can you improve?
.3
To improve
To inform
To prove
To support
150

75
12/11/2016

1. Identify Program Learning Outcomes


Top Down
Approach for
2. Design modules so that all Program designing new
Learning Outcomes are reflected in the
module Learning Outcomes programs

3. Assign credits to each

4. Design Teaching, Learning and


assessment strategies for each module
(module descriptions).

5. Check that Constructive Alignment exists


between module LOs, Teaching and Learning
Activities and Assessment

Characteristics of Effective Program Assessment



Systematic and comprehensive.

Built around the department mission statement.

Ongoing and cumulative.

Multi-faceted.

Faculty-designed and implemented

152

76
12/11/2016

Flow chart of assessment process

Program How/ How Summary Use of Plans/


Learning Where Assessed * of Results Results Schedule
outcomes Taught Going
Forward
1.

2.

3.

77
12/11/2016

Component of The Internal Assessment Process


Criteria Procedure

Specifies the process of


Determine the criteria
initiating, conducting, and
for self assessment.
implementing the assessment.

155

Example:
Criterion 1: Program Mission, Objectives and Outcomes

Intent: Each program must have a mission, objectives and intended learning
outcomes for graduates.
Outcomes include competency and tasks graduates are expected to perform after
completing the program.
A strategic plan must be in place to achieve the program objectives.
The extent to which these objectives are achieved through continuous assessment
and improvements must be demonstrated

Criterion 2: Curriculum Design and Organization

Intent: The curriculum must be designed and organized to achieve the programs
objectives and outcomes. Also course objectives must be in line with program
outcomes. Curriculum standards are specified in terms of credit hours of study.
156

78
12/11/2016

Standard Meeting Standard


Standard 1-1:. Document institution, college and program mission
The program must have documented statements.
measurable objectives that support State program objectives.
college and institution mission Describe how each objective is aligned with program, college
statements and institution mission statements.
Outline the main elements of the strategic plan to achieve the
program mission and objectives.
Provide for each objective how it was measured, when it was
measured and improvements identified and made
Standard 1-2 : Describe the means for assessing the extent to which graduates
are performing the stated program outcomes/learning objectives.
The program must have documented
Conducting a survey of graduating seniors every semester.
outcomes for graduating students. It must Conduct a survey of alumni every two years.
be demonstrated that the outcomes Conduct a survey of employers every two years.
support the program objectives and that Carefully designed questions asked during co-op and cap-
graduating students are capable of stone design projects presentations.
performing these outcomes. Outcome assessment exams
Standard 1-3: The results of programs Describe the actions taken based on the results of periodic
assessment and the extent to which they assessments.
are used to improve the program must be Describe major future program improvements plans based on
documented. recent assessments.

157

Assessment timeline
Program________________ College __________________
Assessment Assessment 1 Assessment 2 Assessment 3 Enter more as
needed
SLOs
Knowledge
#1
#2
Cognitive Skills
#3
#4
Interpersonal
#5
#6

79
12/11/2016

For both
Program or Course

Designing effective assessment


Student assessment mechanisms should be appropriate for the different
forms of learning sought.

Assessment practices should be clearly communicated to students at the


beginning of courses.

Appropriate, valid and reliable mechanisms should be used for verifying


standards of student achievement in relation to relevant internal and external
benchmarks.

The standard of work required for different grades should be consistent over
time, comparable in courses offered within a program and college and the
institution as a whole, and in comparison with other institutions.

80
12/11/2016

Grading should be assisted by the use of matrices or other means to ensure that the
planned range of domains of student learning outcomes are addressed.

Arrangements should be made within the institution for training of teaching staff in the
theory and practice of student assessment.

Policies and procedures of standards of student achievement (inadequate or


inconsistently assessed)

Effective procedures should be used to ensure that work submitted by students is


actually done by the students concerned.

Prompt feedback to students on their performance and results of assessments


accompanied by mechanisms for assistance if needed.

Assessments of student work should be conducted fairly and objectively.

Criteria and processes for academic appeals should be made known to students and
administered equitably.

Designing effective assessment

What are the outcomes to be assessed?


What are the capabilities/skills (implicit or explicit) in the outcomes?
Is the method of assessment chosen consonant with the outcomes and skills?
Is the method relatively efficient in terms of student time and staff time?
What alternative types of assessment are there? What are their advantages
and disadvantages?
Does the specific assessment task match the outcomes and skills?
Are the marking schemes or criteria appropriate?

81
12/11/2016

Assessment of Learning Outcomes, modules

How will I know if my students have achieved the desired learning


outcomes? How will I measure the extent to which they have achieved
these learning outcomes?

In other words how to match the method of assessment to the different kinds of
learning outcomes
e.g. a Learning outcome such as Demonstrate good presentation skills could be
assessed by the requirement that each student makes a presentation to their
peers.

When writing learning outcomes the verb is often a good clue to the assessment
technique.

How can we design our examination system so that it tests if learning outcomes
have been achieved?

1. Identify aims and objectives of


Bottom up module
approach for
existing modules 2. Write learning outcomes using
standard guidelines

For course 3. Develop a teaching and learning


or module strategy to enable students to
achieve learning outcomes

4. Design assessment to check if learning


outcomes have been achieved

5. Check for Constructive Alignment. If


necessary modify module content, teaching
and learning Strategies and assessment
according to findings

82
12/11/2016

General types of assessments


Formative & Summative Assessment

Formative Summative
assessment assessment

The gathering of
The gathering of information information at the
about student learning during conclusion of a course,
the progression of a course or program
program

Sometimes diagnostic is also considered 165

Summative Formative Diagnostic


assessment) assessment assessment

To provide feedback to
To pass or fail students to improve
students To look for
their learning predictable
To grade or rank To provide a profile of
students difficulties
what a student has To diagnose
To select for future learnt
courses strengths and
To help students to weaknesses
To predict success develop their skills of
in future courses self assessment
To motivate To motivate students -
students possibly through goal
setting

83
12/11/2016

Formative Assessment
Assessment FOR learning gives feedback to students and
teachers to help modify teaching and learning activities.
It is integrated into the T&L process and may not generate grades
Usually carried out at beginning or during a programme, e.g.
coursework which gives feedback to students.

Summative Assessment

summarises student learning at end of module or program Assessment


OF Learning.
Sums up achievement and generates a grade or mark.
Usually involves assessment using the traditional examination.
Only a sample of the Learning Outcomes are assessed cannot assess
all the Learning Outcomes.

FORMATIVE ASSESSMENT SUMMATIVE EVALUATION


Process-Oriented Product-Oriented
Reflective Prescriptive
Diagnostic Judgmental
Flexible Fixed
Absolute Comparative
Cooperative Competitive

84
12/11/2016

Nature of assessment



Indirect assessment
Direct assessment

gather evidence, gather reflection


based on student about the learning or
performance secondary evidence
of its existence
most classroom testing for
grades or evaluation of a student, alumni,
research paper on specific employer surveys
criteria
169

Direct Assessments
Focus on how students represent or demonstrate their
learning

Align with students learning and assessment experiences

Align with curricular-and co-curricular design verified through


mapping
Invite collaboration in design (faculty, students, tutors)

85
12/11/2016

Indirect Assessments
Focus group (representative of the population)
Interviews (representative of the population)
Surveys
Transcript analyses
Other sources of information that contribute to your inference making:
results, grades, participation rates or persistence in support services,
course-taking patterns, majors

Examples of Direct Methods


Samples of individual student work
Pre-test and post-test evaluations
Standardized tests
Performance on licensure exams
Blind scored essay tests
Internal or external juried review of student work
Case study/problems
Capstone papers, projects or presentations
Project or course imbedded assessment
Documented observation and analysis of student behavior/performance
Externally reviewed internship or practicum
Collections of work (portfolios) of individual students
Activity logs
Performances on national licensure
Interviews (including videotaped)

86
12/11/2016

Examples of Indirect Methods


Questionnaires and Surveys (including satisfaction surveys)
Students
Graduating Seniors Exit interviews of graduates and
Alumni focus groups graduate follow up
Employers studies
Retention and transfer studies
Syllabi and curriculum analysis Length of time to degree
Graduation and transfer rates
Transcript analysis Job placement rates

Locally Developed Surveys

Institutional level Program or department


alumni survey level
academic advising advisory board surveys
survey employer surveys
image survey customer surveys
student satisfaction program-specific surveys
survey graduating senior survey

87
12/11/2016

Types of Examinations or Tests


standardized exams
national test
state test
juried competitions
recitals
shows or exhibitions
locally developed exams
pre-post tests
course-embedded exam questions
comprehensive exam
qualifying exam

Some Options for Alternative Methods

E-Portfolios Critical incidents


Capstone projects (mid-point and end- Externally or internally juried review
point) of student projects
Performances, productions, creations Externally reviewed internship
Performance on a case study/problem
Visual representations (mind mapping,
charting, graphing) Performance on case study
accompanied with students analysis
Case studies
Locally developed tests
Disciplinary or professional practices
Pre-and post-tests
Agreed upon embedded assignments
Learning Logs or Journals
Writing to speaking to visual presentation
Videotaping over time
Team-based or collaborative projects
Simulationsvirtual labs, scenarios
Internships and service Projects that track decision making and actions
Oral examinations/questions Magic boxproblem solving over time

88
12/11/2016

Discussion

Identify both direct and indirect methods you might


use to assess several of your outcomes.

Discuss the pro and cons of various assessment


methods

To what extent has each Learning Outcome been achieved?

Not a question of yes or no to achievement of Learning Outcomes.


Rubric: A grading tool used to describe the criteria which are used in
grading the performance of students.
Rubric provides a clear guide as to how students work will be assessed.
[a rubric] explains to students the criteria against which their work will
be judged (the scoring rules).

Example
Organization of Thought (4-points): Work is clearly organized and
includes a diagram or step-by-step analysis.

criterion point value descriptor

89
12/11/2016

90
12/11/2016

Giving feedback to students


Feedback is one of the main purposes of assessments
Make the feedback quick, clear and focussed
Relate it to the assessment criteria and learning
outcomes.

Use rubrics or formal marking schemes to show


how well the requirements are met.

Steps in feedback:
Affirm what is done well
Clarify: ask questions about specific aspects
Make suggestions for improvement
Give guidance about what the student needs to do next
Using Rubrics to Provide: Feedback to Students

Activity: Discuss in groups

What assessment methods do you use?

What are the advantages and disadvantages of


these methods?
Do you consider Rubrics?!

91
12/11/2016

Constructive alignment
Learning Outcomes state what is to be achieved in fulfilment of the aims.

Learning activities should be organised so that students will be likely to achieve


those outcomes.

Assessment must be designed such that students are able to demonstrate that they
have met the learning outcomes.

Constructive alignment is just a fancy name for joining up the dots.

Some common problems with assessment

The assessment tasks do not match the stated learning outcomes


The marking criteria do not match the tasks or outcomes
The criteria are not known to students
Students do not understand the criteria
Overuse of one mode of assessment such as written examinations,
essays, MCQs
Assessment overload for students and staff
Insufficient time for students to do the assignments

92
12/11/2016

Too many assignments with the same deadline


Insufficient time for staff to mark the assignments or examinations
Absence of well defined criteria so consistency is difficult to achieve
Inadequate or superficial feedback provided to students
Variations in assessment demands of different modules

Examples of Changes
Increased attention to weaving experiences across the institution, a
program, or a department to improve student achievement
Changes in advising based on assessment results
Closer monitoring of student achievementtracking
Faculty and staff development to learn how to integrate experiences
that contribute to improved student learning
Changes in pedagogy and curricular and co-curricular design
Development of modules to assist learning; use of technology; self-
paced learning, supplemental learning

93
12/11/2016

Timeframe: Good practice


Assessment Practice Time frame

Identify student learning outcomes

Create curriculum map

Design assessment plan

Identify and pilot assessment methods

Begin to implement plan

The Black Holes of Assessment


Lack of.
Infrastructure
Commitment
Leadership
Resources
Acknowledgment
Feedback
Discussion
Action

94
12/11/2016

LINKING LEARNING OUTCOMES TO


ASSESSMENT

( ASSESSMENT MAPPING)

Matching assessment to learning outcomes

The measuring instrument should matches the thing to be measured

Example of Matching the Assessment to the Learning Outcome

Learning outcomes Assessment?


1. Demonstrate good a) Multiple choice questions
presentation skills. b) Prepare a 1000-word research
2. Formulate a product proposal
3. Identify an area for research c) Lab-based project
4. Identify certain parameters d) Make a presentation to peers
and/or processes

95
12/11/2016

Steps involved in linking Learning Outcomes, Teaching


and Learning Activities and Assessment
1. Clearly define the learning outcomes.
2. Select teaching and learning methods that If the learning
are likely to ensure that the learning outcomes are
outcomes are achieved. clearly written,
3. Choose a technique or techniques to the assessment is
assess the achievement of the learning quite easy to plan!
outcomes.
4. Assess the learning outcomes and check to
see how well they match with what was
intended

One of the big challenges is to move away from assessment based solely
on terminal exams not intelligence fair, forcing all kinds of learning to
fit into the paper and pencil test straight jacket.

LO alignment with NQF, teaching strategies, & assessment methods.


Template is designed for alignment
NQF Learning Domains Teaching Assessment
and Learning Outcomes Strategies Methods
1.0 Knowledge
1.1
1.2
1.3
2.0 Cognitive Skills
2.1
2.2
2.3
3.0 Interpersonal Skills & Responsibility
3.1
3.2
4.0 Communication, Information Technology, Numerical
4.1
4.2
5.0 Psychomotor
5.1
192

96
12/11/2016

Learning Outcomes Teaching and Assessment


Learning Activities
Cognitive Lectures End of module exam.
(Demonstrate: Multiple choice tests.
Knowledge, Tutorials Essays.
Comprehension, Reports on lab work and
Application, Analysis, Discussions research project.
Synthesis, Evaluation) Interviews/viva.
Laboratory work Practical assessment.
Poster display.
Affective Fieldwork.
(Integration of beliefs, Clinical work Clinical examination.
ideas and attitudes) Presentation.
Group work Portfolio.
Psychomotor Seminar Performance.
(Acquisition of physical Peer group Project work.
skills) presentation, Production of artefact
etc.

Type of Learning Examples of Types of Assessment How to Measure


Outcome
Objective Test items that require students to recall or recognize Accuracy correct vs number
KnowledgeRemember information: of errors
Fill-in the Blank Item Analysis (at the class
Students will be able to: Multiple Choice items with question such as, what is a, or level, are there items that had
recall which of the following is the definition of) higher error rates?
recognize Labeling diagrams Did some items result in the
Reciting (orally, musically, or in writing) same errors?)
CognitiveUnderstand Papers, oral/written exam questions, problems, class discussions, Scoring or performance
concept maps, homework assignments that require (oral or written). rubrics that identify critical
Students will be able to: Summarizing readings, films, speeches, etc. components of the work and
interpret Comparing and/or contrasting two or more theories, events, discriminates between
exemplify processes, etc. differing levels of proficiency in
classify Classifying or categorizing cases, elements, events, etc., using addressing the components
summarize established criteria
infer Paraphrasing documents or speeches
compare Finding or identifying examples or illustrations of a concept,
explain principle

Analyze Activities that require students to discriminate or select relevant Rubrics, scored by
Students will be able to: from irrelevant parts, determine how elements function together, or instructor/clinical staff,
differentiate determine bias, values or underlying intent in presented materials. external clients, employers,
organize These might include: internship supervisor, etc.
attribute Case studies, Critiques, Labs, Papers, Projects, Debates, Concept
Maps

97
12/11/2016

Assessing your assessment : Is it doing the job you want it to do? Is it


comprehensive?

Assessment Assessment Assessment Assessment


Task 1, e.g. Task 2 Task 3, e.g. Task 4
Written Exam e.g. Project Presentation e.g. Lab work
Learning
Outcome 1
Describe
Learning
Outcome 2
Investigate..
Learning
Outcome 3
Demonstrate.

Common ILOs Possible Assessment Tasks


Describe essay question, exam, oral presentation (peer assess.)

Explain assignment, essay question


exam, oral, letter-to-a-friend
Examples

Integrate project, assignment

Analyse case study, assignment

Apply project, case study, experiment

Solve problem case study, project, experiment

Design, create project, experiment

Reflect Reflective diary, portfolio, self-assessment

Communicate a range of oral, writing or listening tasks, e.g.


presentation, debate, role play, reporting,
assignment, prcis, paraphrasing,
answering questions etc.

98
12/11/2016

Assessment Matrix Can be Useful to Link the Where with the Outcomes

Learning Course Course Course Capstone


Outcome 1234 2345 3456

Application of Introduced Emphasized Used Assessed


theory

Skills and Introduced Used Assessed


knowledge

Communication Introduced Emphasized


skills

Assessment Map

99
12/11/2016

Assessment Map introductory level

Assessment Map intermediate level

100
12/11/2016

Discussion
Using the LOs which you developed earlier consider
how an appropriate assessment strategy might be
developed.

Selecting the Best


Assessment Methods

101
12/11/2016

Identified Based on

Relationship to assessment provide you with the


information you need
Reliability yields consistent responses over time
Validity appropriate for what you want to measure
Timeliness and cost preparation, response, and analysis
time; opportunity and tangible costs
Motivation provides value to student, respondents are
motivated to participate
other
results easy to understand and interpret
changes in results can be attributed to changes in the program

After Identifying the Potential List of Assessment


Methods You Need to

Select the best ones


try to identify at least two methods for assessing each outcome
Consider possible performance targets for the future
balance between stretch targets versus achievable targets

Develop assessment instruments


Surveys - exams - assignments - scoring rubrics - portfolios
Make sure they are reliable, valid, and cheap approaches
use external sources
seek help from internal sources (e.g., Academic Development and
Assessment Office)
do it yourself

The instrument may need to be modified based on assessment results

102
12/11/2016

Challenges and Pitfalls

One size does not fit all some methods work well for one program but
not others
Do not try to do the perfect assessment all at once take a continuous
improvement approach
Allow for ongoing feedback
Match the assessment method to the outcome and not vice-versa

Examples
Outcome 1: Graduates will be satisfied that their
undergraduate degree has prepared them to succeed in their
professional career
Graduates will be surveyed in the annual alumni survey on their preparedness
to succeed in their career
95% of the graduates surveyed in the annual alumni survey report that the
program enabled them to be very prepared or extremely prepared to
succeed in their career (next phase)
On-site internship supervisors each semester will rate interns from the program
on their skills necessary to succeed in the workplace
90% of on-site internship supervisors each semester rate interns from the
program as having the skills necessary to succeed in their career (next
phase)

103
12/11/2016

students in their capstone course will be administered a locally


developed, standardized exam regarding career preparedness
95% of student in their capstone course are able to successful
answer 90% of the questions regarding career preparedness on a
locally developed, standardized exam (next phase)
scores of graduates who have taken the state licensure exam within one
year of graduating from the program will be evaluated
85% of the graduates are able to pass the state licensure exam in
within one year of graduating from the program (next phase)

senior portfolios will be examined annually using a locally devised


rubric to show evidence of preparedness for success in related
professional careers on three key measures: communication,
leadership, and ethics
90% of senior portfolios examined annually using a locally
devised rubric will show evidence of preparedness for success in
related professional careers on three key measures:
communication, leadership, and ethics (next phase)
students will be observed performing basic technical lab skills
necessary for successful employment in a senior laboratory course
90% of students will be able to perform basic technical lab skills
necessary for successful employment in a senior laboratory
course (next phase)

104
12/11/2016

Effective utilization of results


Use evidence collected on student performance to inform program change.

Analyzing, reporting, and discussion on assessment results can take place in


faculty committees, meetings, etc.

Focus on results that show the greatest weakness and determine what can be
addressed now and what might need to be addressed in the future.

If changes are to be made to the program, include an action plan and/or


timeline.
If results from previous years relate to current results, bring them into the
discussion to bolster a longitudinal overview of program assessment.

The purpose of assessment is not to generate exemplary results, but rather to gauge
student progress and uncover any areas of improvement within the program

Example: Use of results


Results Use of Results

Students scoring lower than "proficient" showed


80% of students scored as
weakness in critical thinking. During a faculty
"proficient" or "highly
committee meeting, it was determined that
proficient" on core
students need additional research and writing
competencies outlined on the
support. A series of departmental writing groups
scoring rubric.
will be created for first-year students in the spring.

Feedback from exit interviews


suggested that students were Hired a full-time student advisor to strengthen
not receiving adequate existing advising support.
advising support.

105
12/11/2016

Why Making Assessment


Harder Than it Needs to Be?!

Exams
Try.
Instead of.
Aggregate the data across
Returning exams without students to see what they know,
discussion and with which concepts they
are struggling.

Process
Instead of. Try.
Trying to get everyone in your Identifying who in your
program to be enthusiastic about department might be interested
assessment in trying a few things.

106
12/11/2016

Grading
Try.
Instead of.
Using a rubric that allows you
to analyze students strengths
Giving papers points or letter and weaknesses individually
grades and collectively

Quizzes
Try.
Instead of.
Using quizzes as a Classroom
Giving points for daily or weekly Assessment Technique to find
quizzes out whether or not it makes
sense to move forward, or if you
need to reinforce concepts first.

Program Level Assessment


Try
Instead of.
Identifying the top five or six
Having a huge list of every single things you want students to
concept to be learned in the know or be able to do and map
program those outcomes to the courses in
the curriculum

Assessment
Try.
Instead of. Making the learning outcomes for
Conducting an assessment the course and program explicit to
program without telling the the students so they know what is
expected of them.
students about it

107
12/11/2016

Assessment
Try.
Instead of.
Focus on a few key learning
Collecting tons of data outcomes and collect, discuss and
analyze that data.

Assessment
Try.
Instead of. Take one outcome at a time
Attempting to implement your and figure out the best way to
entire assessment plan at once assess it -- then try another
one

Homework

Review your exams or assessments


Better yet have one of your colleagues review your exams or
assessments
Are your assessments in line with your student learning outcomes
(taxonomy)? Many will say yes. On review, most assessments are at fairly
low taxonomic levels.
Choose a cognitive level on Blooms taxonomy for each question that
best reflects the expectations of the questionsee what you find..

108
12/11/2016

6 Steps in Developing an
Assessment Plan

STEP ONE: Compare where you are now in your assessment


practices with where you want to be. Consider the following
questions:

1. What information on student learning do you normally collect?

2. How helpful has each been in helping you to understand students


backgrounds and the student learning process?

3. Given your newly designed course, how do the current assessments


relate to your newly specified learning goals?

4. Are there gaps between the information you now collect and what you
need to collect in order to improve student learning in your course and
to assess the degree to which your students have met the new learning
objectives?

109
12/11/2016

STEP TWO: Decide whether to keep, change, modify


and/or add new forms of assessment based on your
comparative analysis and new learning outcomes.
Keep in mind:

The need to assess learning at specific points in time (at the

moment) as well as over time

STEP THREE: Review new forms of assessment and consider

The amount of time it takes for preparation and review of these


assessments as well as the amount of student time to participate in
these assessments.

The purpose of each assessmentbegin with the end in mind.

Ensure that each learning outcome is assessable and that you have
tied assessment to each learning outcome

110
12/11/2016

STEP FOUR: Implement!

STEP FIVE: Analyze results and determine refinements for


next timecan use eyeball results or use a chart like this
one:

Assessing Assessment Type of Data What Do the What Will You Any
What? Method Collected Results Say? Do? Surprises?

Prior Student Student Quantitative 60% of the Challenge Thought


knowledge of background Survey Data students those who more
course probe and Open- have some know students
content ended pre- (possibly would be
responses knowledge. extra credit acquainted
40% have tasks), bring with subject
none others up to
speed (may
need tutorial
assistance)

STEP SIX: PULLING IT ALL TOGETHER. Taking this next


step, completing a matrix, is helpful for the instructor and
also provides a way to share your assessment experiences
with others.
Learning Assessment Results Interpretation Action to Take
Outcome Methods

111
12/11/2016

112
12/11/2016

113