Vous êtes sur la page 1sur 17

Internet Resources for Higher

Education Outcomes Assessment


http://www.assessmentcommons.org/

Formative vs Summative Assessment


The assessment of teaching and learning can be viewed as two
complementary and overlapping activities that aim to benefit both
the quality of student learning and the professional development of
the instructor. Assessing learning alone is not sufficient because the
ultimate success of students is also dependent upon their
motivation and commitment to learning. Similarly, assessing only
teaching behaviors and course activities is not sufficient because
qualities of the instructor may be appreciated by students but not
optimally helpful to their learning and growth. Done in tandem,
assessing teaching and learning can help instructors improve and
refine their teaching practices and help improve students learning
and performance.

Formative Assessment Summative Assessment


The goal of formative assessment is The goal of summative assessment is to
to gather feedback that can be used measure the level of success or
by the instructor and the students to proficiency that has been obtained at
guide improvements in the ongoing the end of an instructional unit, by
teaching and learning context. These comparing it against some standard or
are low stakes assessments for benchmark.
students and instructors. Examples:
Examples: Assigning a grade to a final exam
Asking students to submit one or two Critique of a Senior recital
sentences identifying the main point University Faculty Course Evaluations
of a lecture The outcome of a summative
Have students submit an outline for a assessment can be used formatively,
paper. however, when students or faculty take
Early course evaluations the results and use them to guide their
efforts and activities in subsequent
courses.

http://www.cmu.edu/teaching/assessment/howto/basics/formative-
summative.html
Grading vs. Assessment of Learning Outcomes:
Whats the difference?
There is often confusion over the difference between grades and
learning assessment, with some believing that they are totally
unrelated and others thinking they are one and the same. The truth
is, it depends. Grades are often based on more than learning
outcomes. Instructors grading criteria often include behaviors or
activities that are not measures of learning outcomes, such as
attendance, participation, improvement, or effort. Although these
may be correlated with learning outcomes, and can be valued
aspects of the course, typically they are not measures of learning
outcomes themselves.1
However, assessment of learning can and should rely on or relate to
grades, and so far as they do, grades can be a major source of data
for assessment. To use grades as the basis for learning outcomes,
grades would first have to be decomposed into the components that
are indicators of learning outcomes and those that are indicators of
other behaviors. Second, grades would have to be based on clearly
articulated criteria that are consistently applied. Third, separate
grades or subscores would have to be computed for the major
components of knowledge and skills so that evidence of students
specific areas of strength and weakness could be identified. For
example, although 30% of a class may receive a grade of B, the
group may all have shown a very highlevel of competence on one
skill set but only moderate achievement in another. This kind of
strength and weakness assessment provides feedback that is useful
to students because it can guide and focus their practice, to the
instructor, because it can reveal topics and skills that require further
instructional activities, and to the department, because it can guide
potential changes in curriculum to appropriately address areas of
strength and weakness.
This kind of analysis is not the same as producing sub scores for
different course activities, such as a score for homework, one for
exams, and another for projects. These are different methods of
assessment, and each of them may assess multiple skills and
abilities and may overlap with each other in terms of what
knowledge and skills they assess. To accurately assess learning
outcomes, each type of assessment (i.e., exam, project,
programming assignment, etc), would need to be analyzed in terms
of the different skills it addresses and scores across the various
types of assessment activity would have to be compiled and
assigned for each of the skills.
For example:
Items 1, 4, 5 and 9 on an exam and homework 2 might all deal with
the ability to identify the appropriate strategy to apply in a given
problem context. The combined score from those items would
comprise the identify solution strategy score.
Many instructors already have this information to some degree but
discard it when computing overall grades. Questions or problems on
exams or homework are individually scored already. To then turn
these scores into an assessment of student learning one would only
have to classifying the problem according to the skill (or learning
objective) it addresses, and then compute separate totals for each
different category.

Identify Identify Implement


Implement Total
solution solution solution
Skill/ solution Identify
strategy strategy strategy
objective strategy strategy
(Exam 1) (HW2) (Exam 2)
(HW 3) 35 pt
15pt 20pt 10 pt
10 pt

Std 1 10 15 6 9 25

Stdt 2 14 18 10 8 32

Std 3 14 17 5 5 31

Class Average 12.67 16.67 7 7.33 29.34

In this example, two skills have been scored separately on four


assessments: Exam 1, HW2, Exam2 and HW3. According to grade
assignment, Student 1 and Student 3 obtain the same grade but in
terms of their learning outcomes, it is clear that the students
learning outcomes are very different, with student 1 demonstrating
weakness in identification and implementation and student 3
demonstrating strength in identification and serious weakness in
implementation. Thus the grade alone does not identify for the
student or the instructor which component skills the student has
mastered. Furthermore, overall grades would not provide an
instructor with feedback on which skills the class overall found
difficult. The above representation would provide class level analysis
of strengths and weaknesses. This kind of feedback could be used
by individual instructors to target changes in instruction or by
departments for curriculum revision. By continually monitoring
learning outcomes the instructor could then track the impact of
instructional or curricular changes on specific learning outcomes.
This model of assessment puts the responsibility for the design,
implementation, and interpretation of the assessment in the hands
of the instructors or faculty members, since they alone have the
content expertise as well as the knowledge of the course learning
goals and assessment methods and materials. Furthermore, only
the course instructors, or the department faculty as a group, can
decide on the appropriate standards or criteria to classify a learning
outcome as Exceptional, Very good, Good, or Substandard.
However, when such an assessment of student learning is combined
with an explicit statement of the learning objectives and the
standards of performance, then external groups (e.g., college
deans, provost, external accreditation agencies, similar departments
in other institutions, employers, etc.) would be able to evaluate the
degree to which students, both individually and as group, have
achieved the desired learning outcomes.
1
If participation is scored according to the quality of the class contribution, then it can be
viewed as a learning outcome. For example, some instructors use a scoring system that
gives minimal points for providing a recallable fact from the reading and maximum points
for providing a meaningful integration and synthesis of ideas.
http://www.cmu.edu/teaching/assessment/howto/basics/grading-
assessment.html

The Assessment Triangle


Assessment is the process of reasoning from evidence. To design
assessments of student learning that will provide useful evidence
requires that we coordinate and align three key components:
cognition, which refers to a model of the thinking and learning of
students within the subject domain, observations, the tasks or
activities that students engage in that provide evidence of learning,
and interpretations, the process or methods for making sense of the
evidence.
Assessments are effective and useful only to the degree that these
three components are in synchrony. If we design tasks without
clearly thinking about if and how those tasks will require students to
demonstrate the targeted knowledge and skills, we may end up with
data that doesnt address our goals. Alternatively, if we design tasks
without thinking about how we will analyze the performance, we
may end up with data that dont allow us to detect students
strengths or weaknesses.
Align Assessments with Objectives
Assessments should provide us, the instructors, and the students
with evidence of how well the students have learned what we intend
them to learn. What we want students to learn and be able to do
should guide the choice and design of the assessment. There are
two major reasons for aligning assessments with learning
objectives. First, alignment increases the probability that we will
provide students with the opportunities to learn and practice the
knowledge and skills that will be required on the various
assessments we design. Second, when assessments and objectives
are aligned, good grades are more likely to translate into good
learning. When objectives and assessments are misaligned, many
students will focus their efforts on activities that will lead to good
grades on assessments, rather than focusing their efforts on
learning what we believe is important.
There are many different types of activities that can be used to
assess students proficiency on a given learning objective, and the
same activity can be used to assess different objectives. To ensure
more accurate assessment of student proficiencies, it is
recommended that you use different kinds of activities so that
students have multiple ways to practice and demonstrate their
knowledge and skills.
When deciding on what kind of assessment activities to use, it is
helpful to keep in mind the following questions:
What will the students work on the activity (multiple choice
answers, essays, project, presentation, etc) tell me about their level
of competence on the targeted learning objectives?
How will my assessment of their work help guide students practice
and improve the quality of their work?
How will the assessment outcomes for the class guide my teaching
practice?
This table presents examples of the kinds of activities that can
be used to assess different types of learning objectives, and the
ways that we can analyze or measure performance to produce
useful feedback for teaching and learning. The categorization of
learning objectives is taken from the revised Blooms Taxonomy.
Type of Learning
Objective Examples of Types of Assessment How

Remember Objective Test items that require students Accuracy


Students will be able to: to recall or recognize information: of errors
recall Fill-in the Blank Item Analy
recognize Multiple Choice items with question stems level, are t
such as, what is a, or which of the higher erro
following is the definition of) items resul
Labeling diagrams errors?)
Reciting (orally, musically, or in writing)

Understand Papers, oral/written exam questions, problems, Scoring or p


Students will be able to: class discussions, concept maps, homework that identify
interpret assignments that require (oral or written): of the work
exemplify Summarizing readings, films, speeches, between dif
classify etc. proficiency i
summarize Comparing and/or contrasting two or more components
infer theories, events, processes, etc.
compare Classifying or categorizing cases, elements,
explain events, etc., using established criteria
Paraphrasing documents or speeches
Finding or identifying examples or
illustrations of a concept, principle

Apply Activities that require students to use Accuracy sc


Students will be able to: procedures to solve or complete familiar or Rubrics, Prim
execute unfamiliar tasks; may also require students to
implement determine which procedure(s) are most
appropriate for a given task. Activities include:
Problem sets, performances, labs, Prototyping,
Simulations

Analyze Activities that require students to discriminate Rubrics, sc


Students will be able to: or select relevant from irrelevant parts, juries, exte
differentiate determine how elements function together, or employers,
organize determine bias, values or underlying intent in supervisor,
attribute presented materials. These might include:
Primary Tra
Case studies, Critiques, Labs, Papers, Projects,
Debates, Concept Maps,

Evaluate A range of activities that require students to Rubrics, sc


Students will be able to: test, monitor, judge or critique readings, juries, exte
check performances, or products against established employers,
critique criteria or standards. These activities might supervisor,
include:
Primary Tra
Journals, Diaries, Critiques, Problem Sets,
Product Reviews, Case Studies.
Create Research projects, musical compositions, Rubrics, sc
Students will be able to: performances, essays, business plans, website juries, exte
generate designs, prototyping, set designs employers,
plan supervisor,
produce Primary Tra

Glossary of Terms
Assessment for Accountability
The assessment of some unit, such as a department, program or
entire institution, which is used to satisfy some group of external
stakeholders. Stakeholders might include accreditation agencies,
state government, or trustees. Results are often compared across
similar units, such as other similar programs and are always
summative. An example of assessment for accountability would be
ABET accreditation in engineering schools, whereby ABET creates a
set of standards that must be met in order for an engineering
school to receive ABET accreditation status.
Assessment for Improvement
Assessment activities that are designed to feed the results directly,
and ideally, immediately, back into revising the course, program or
institution with the goal of improving student learning. Both
formative and summative assessment data can be used to guide
improvements.
Concept Maps
Concept maps are graphical representations that can be used to
reveal how students organize their knowledge about a concept or
process. They include concepts, usually represented in enclosed
circles or boxes, and relationships between concepts, indicated by a
line connecting two concepts. Example [
Direct Assessment of Learning
Direct assessment is when measures of learning are based on
student performance or demonstrates the learning itself. Scoring
performance on tests, term papers, or the execution of lab skills,
would all be examples of direct assessment of learning. Direct
assessment of learning can occur within a course (e.g., performance
on a series of tests) or could occur across courses or years
(comparing writing scores from sophomore to senior year).
Embedded Assessment
A means of gathering information about student learning that is
integrated into the teaching-learning process. Results can be used
to assess individual student performance or they can be aggregated
to provide information about the course or program. can be
formative or summative, quantitative or qualitative. Example: as
part of a course, expecting each senior to complete a research
paper that is graded for content and style, but is also assessed for
advanced ability to locate and evaluate Web-based information (as
part of a college-wide outcome to demonstrate information literacy).
External Assessment
Use of criteria (rubric) or an instrument developed by an individual
or organization external to the one being assessed. This kind of
assessment is usually summative, quantitative, and often high-
stakes, such as the SAT or GRE exams.
Formative Assessment
Formative assessment refers to the gathering of information or data
about student learning during a course or program that is used to
guide improvements in teaching and learning. Formative
assessment activities are usually low-stakes or no-stakes; they do
not contribute substantially to the final evaluation or grade of the
student or may not even be assessed at the individual student
level. For example, posing a question in class and asking for a
show of hands in support of different response options would be a
formative assessment at the class level. Observing how many
students responded incorrectly would be used to guide further
teaching.
High stakes Assessment
The decision to use the results of assessment to set a hurdle that
needs to be cleared for completing a program of study, receiving
certification, or moving to the next level. Most often, the
assessment so used is externally developed, based on set
standards, carried out in a secure testing situation, and
administered at a single point in time. Examples: at the secondary
school level, statewide exams required for graduation; in
postgraduate education, the bar exam.
Indirect Assessment of Learning
Indirect assessments use perceptions, reflections or secondary
evidence to make inferences about student learning. For example,
surveys of employers, students self-assessments, and admissions
to graduate schools are all indirect evidence of learning.
Individual Assessment
Uses the individual student, and his/her learning, as the level of
analysis. Can be quantitative or qualitative, formative or
summative, standards-based or value added, and used for
improvement. Most of the student assessment conducted in higher
education is focused on the individual. Student test scores,
improvement in writing during a course, or a students improvement
presentation skills over their undergraduate career are all examples
of individual assessment.
Institutional Assessment
Uses the institution as the level of analysis. The assessment can be
quantitative or qualitative, formative or summative, standards-
based or value added, and used for improvement or for
accountability. Ideally, institution-wide goals and objectives would
serve as a basis for the assessment. For example, to measure the
institutional goal of developing collaboration skills, an instructor and
peer assessment tool could be used to measure how well seniors
across the institution work in multi-cultural teams.
Local Assessment
Means and methods that are developed by an institution's faculty
based on their teaching approaches, students, and learning goals.
An example would be an English Departments construction and use
of a writing rubric to assess incoming freshmens writing samples,
which might then be used assign students to appropriate writing
courses, or might be compared to senior writing samples to get a
measure of value-added.
Program Assessment
Uses the department or program as the level of analysis. Can be
quantitative or qualitative, formative or summative, standards-
based or value added, and used for improvement or for
accountability. Ideally, program goals and objectives would serve
as a basis for the assessment. Example: How well can senior
engineering students apply engineering concepts and skills to solve
an engineering problem? This might be assessed through a
capstone project, by combining performance data from multiple
senior level courses, collecting ratings from internship employers,
etc. If a goal is to assess value added, some comparison of the
performance to newly declared majors would be included.
Qualitative Assessment
Collects data that does not lend itself to quantitative methods but
rather to interpretive criteria (see the first example under
"standards").
Quantitative Assessment
Collects data that can be analyzed using quantitative methods (see
"assessment for accountability" for an example).
Rubric
A rubric is a scoring tool that explicitly represents the performance
expectations for an assignment or piece of work. A rubric divides
the assigned work into component parts and provides clear
descriptions of the characteristics of the work associated with each
component, at varying levels of mastery. Rubrics can be used for a
wide array of assignments: papers, projects, oral presentations,
artistic performances, group projects, etc. Rubrics can be used as
scoring or grading guides, to provide formative feedback to support
and guide ongoing learning efforts, or both.
Standards
Standards refer to an established level of accomplishment that all
students are expected to meet or exceed. Standards do not imply
standardization of a program or of testing. Performance or learning
standards may be met through multiple pathways and
demonstrated in various ways. For example, instruction designed to
meet a standard for verbal foreign language competency may
include classroom conversations, one-on-one interactions with a TA,
or the use of computer software. Assessing competence may be
done by carrying on a conversation about daily activities or a
common scenario, such as eating in a restaurant, or using a
standardized test, using a rubric or grading key to score correct
grammar and comprehensible pronunciation.
Summative Assessment
The gathering of information at the conclusion of a course, program,
or undergraduate career to improve learning or to meet
accountability demands. When used for improvement, impacts the
next cohort of students taking the course or program. Examples:
examining student final exams in a course to see if certain specific
areas of the curriculum were understood less well than others;
analyzing senior projects for the ability to integrate across
disciplines.
Value Added
The increase in learning that occurs during a course, program, or
undergraduate education. Can either focus on the individual
student (how much better a student can write, for example, at the
end than at the beginning) or on a cohort of students (whether
senior papers demonstrate more sophisticated writing skills-in the
aggregate-than freshmen papers). To measure value-added a
baseline measurement is needed for comparison. The baseline
measure can be from the same sample of students (longitudinal
design) or from a different sample (cross-sectional).
Adapted from Assessment Glossary compiled by American Public
University System, 2005http://www.apus.edu/Learning-Outcomes-
Assessment/Resources/Glossary/Assessment-Glossary.htm
Glossary of Assessment Terms

Graduation RequirementsStudent ProcessFAQsData ReportsInitiatives


Resources
Assessment

The systematic collection, review, and use of information about


educational programs undertaken for the purpose of improving student
learning and development. (Palomba & Banta, 1999)

An ongoing process aimed at understanding and improving student


learning. It involves making our expectations explicit and public; setting
appropriate criteria and standards for learning quality; systematically
gathering, analyzing, and interpreting evidence to determine how well
performance matches those expectations and standards; and using the
resulting information to document, explain, and improve performance.
(Angelo, 1995)

Benchmarking

An actual measurement of group performance against an established


standard at defined points along the path toward the standard.
Subsequent measurements of group performance use the benchmarks
to measure progress toward achievement. (New Horizons for Learning)

Bloom's Taxonomy of Cognitive Objectives

Six levels arranged in order of increasing complexity (1=low, 6=high):

Knowledge: Recalling or remembering information without necessarily


understanding it. Includes behaviors such as describing, listing,
identifying, and labeling.
Comprehension: Understanding learned material and includes
behaviors such as explaining, discussing, and interpreting.
Application: The ability to put ideas and concepts to work in solving
problems. It includes behaviors such as demonstrating, showing, and
making use of information.
Analysis: Breaking down information into its component parts to see
interrelationships and ideas. Related behaviors include differentiating,
comparing, and categorizing.
Synthesis: The ability to put parts together to form something original. It
involves using creativity to compose or design something new.
Evaluation: Judging the value of evidence based on definite criteria.
Behaviors related to evaluation include: concluding, criticizing,
prioritizing, and recommending. (Bloom, 1956)
Classroom Assessment

The systematic and on-going study of what and how students are
learning in a particular classroom; often designed for individual faculty
who wish to improve their teaching of a specific course. Classroom
assessment differs from tests and other forms of student assessment in
that it is aimed at course improvement, rather than at assigning grades.
(National Teaching & Learning Forum)

Direct Assessment

Gathers evidence about student learning based on student performance


that demonstrates the learning itself. Can be value added, related to
standards, qualitative or quantitative, embedded or not, using local or
external criteria. Examples are written assignments, classroom
assignments, presentations, test results, projects, logs, portfolios, and
direct observations. (Leskes, 2002)

Embedded Assessment

A means of gathering information about student learning that is built into


and a natural part of the teaching-learning process. Often uses for
assessment purposes classroom assignments that are evaluated to
assign students a grade. Can assess individual student performance or
aggregate the information to provide information about the course or
program; can be formative or summative, quantitative or qualitative.
Example: as part of a course, expecting each senior to complete a
research paper that is graded for content and style, but is also assessed
for advanced ability to locate and evaluate Web-based information (as
part of a college-wide outcome to demonstrate information literacy).
(Leskes, 2002)

Evaluation

The use of assessment findings (evidence/data) to judge program


effectiveness; used as a basis for making decisions about program
changes or improvement. (Allen, Noel, Rienzi & McMillin, 2002)

Formative Assessment

The gathering of information about student learning-during the


progression of a course or program and usually repeatedly-to improve
the learning of those students. Example: reading the first lab reports of a
class to assess whether some or all students in the group need a lesson
on how to make them succinct and informative. (Leskes, 2002)

Indirect Assessment

Acquiring evidence about how students feel about learning and their
learning environment rather than actual demonstrations of outcome
achievement. Examples include surveys, questionnaires, interviews,
focus groups, and reflective essays. (Eder, 137)

Learning Outcomes

Operational statements describing specific student behaviors that


evidence the acquisition of desired knowledge, skills, abilities,
capacities, attitudes or dispositions. Learning outcomes can be usefully
thought of as behavioral criteria for determining whether students are
achieving the educational objectives of a program, and, ultimately,
whether overall program goals are being successfully met. Outcomes
are sometimes treated as synonymous with objectives, though
objectives are usually more general statements of what students are
expected to achieve in an academic program. (Allen, Noel, Rienzi &
McMillin, 2002)

Norm-Referenced Assessment

An assessment where student performance or performances are


compared to a larger group. Usually the larger group or "norm group" is
a national sample representing a wide and diverse cross-section of
students. Students, schools, districts, and even states are compared or
rank-ordered in relation to the norm group. The purpose of a norm-
referenced assessment is usually to sort students and not to measure
achievement towards some criterion of performance.

Performance Criteria

The standards by which student performance is evaluated. Performance


criteria help assessors maintain objectivity and provide students with
important information about expectations, giving them a target or goal to
strive for. (New Horizons for Learning)

Portfolio

A systematic and organized collection of a student's work that exhibits to


others the direct evidence of a student's efforts, achievements, and
progress over a period of time. The collection should involve the student
in selection of its contents, and should include information about the
performance criteria, the rubric or criteria for judging merit, and
evidence of student self-reflection or evaluation. It should include
representative work, providing a documentation of the learner's
performance and a basis for evaluation of the student's progress.
Portfolios may include a variety of demonstrations of learning and have
been gathered in the form of a physical collection of materials, videos,
CD-ROMs, reflective journals, etc. (New Horizons for Learning)

Qualitative Assessment

Collects data that does not lend itself to quantitative methods but rather
to interpretive criteria. (Leskes, 2002)

Rubric

Specific sets of criteria that clearly define for both student and teacher
what a range of acceptable and unacceptable performance looks like.
Criteria define descriptors of ability at each level of performance and
assign values to each level. Levels referred to are proficiency levels
which describe a continuum from excellent to unacceptable product.
(System for Adult Basic Education Support)

Standards

Sets a level of accomplishment all students are expected to meet or


exceed. Standards do not necessarily imply high quality learning;
sometimes the level is a lowest common denominator. Nor do they imply
complete standardization in a program; a common minimum level could
be achieved by multiple pathways and demonstrated in various ways.
(Leskes, 2002)

Summative Assessment

The gathering of information at the conclusion of a course, program, or


undergraduate career to improve learning or to meet accountability
demands. When used for improvement, impacts the next cohort of
students taking the course or program. Example: examining student
final exams in a course to see if certain specific areas of the curriculum
were understood less well than others. (Leskes, 2002)

Value Added

The increase in learning that occurs during a course, program, or


undergraduate education. Can either focus on the individual student
(how much better a student can write, for example, at the end than at
the beginning) or on a cohort of students (whether senior papers
demonstrate more sophisticated writing skills-in the aggregate-than
freshmen papers). Requires a baseline measurement for comparison.
(Leskes, 2002)

Sources

Allen, Mary; Noel, Richard, C.; Rienzi, Beth, M.; and McMillin, Daniel, J.
(2002). Outcomes Assessment Handbook . California State University,
Institute for Teaching and Learning, Long Beach, CA.
Angelo, Dr. Tom (1995). Reassessing (and defining) assessment. The
AAHE Bulletin, 48(2), 7-9.
Bloom, B.S. (1956). Taxonomy of educational objectives: the
classification of educational goals. Handbook I: Cognitive Domain.
White Plains, N.Y.: Longman.
DeMars, C. E., Cameron, L., & Erwin, T. D. (2003). Information literacy
as foundational: determining competence. JGE: The Journal of General
Education , 52(4), 253.
Eanes, R. [n. d.]. Rubrics
Eder, D. J. (2004). General education assessment within the disciplines.
JGE: The Journal of General Education , 53(2), 135.
Leskes, Andrea (2002). Beyond confusion: an assessment glossary.
Peer Review, 4(2/3).
McTighe, J., & Ferrara, S. (1998). Assessing learning in the classroom.
Washington D.C.: National Education Association.
National Center for Research on Evaluation, Standards & Student
Testing (CRESST). Glossary.
National Teaching & Learning Forum, Classroom Assessment
Techniques.
New Horizons for Learning. (2002). Glossary of Assessment Terms.
Palomba, C & Banta T. (1999). Assessment essentials: planning,
implementing, and improving assessment in higher education. San
Francisco Jossey Bass.
Smith, K., & Harm, T. (2003). Clarifying different types of portfolios.
Assessment & Evaluation in Higher Education , 28(6), 625.
System for Adult Basic Education Support. Glossary of Useful Terms .

http://www.apus.edu/community-scholars/learning-outcomes-
assessment/university-assessment/glossary.htm

Vous aimerez peut-être aussi