http://www.tedi.uq.edu.au/TEN/TEN_previous/TEN2_99/ten2_goosMoni.

html

Sharing the Load:

A Collaborative Approach to Developing and Implementing
Criterion-Referenced Assessment in Professional Degree Courses
________________________________________
Merrilyn Goos and Karen Monis
Graduate School of Education, the University of Queensland

Abstract
The new QUT Clinical Hypnosis Graduate Diploma is the first university course of
its kind in Queensland and only the second in Australasia. Geared towards busy
health professionals, it must be flexible in all aspects of delivery and
assessment. Combining the individual needs of these full fee paying students (who
may or may not be technologically literate) with national practice standards means
that sooner or later, either the anxiety of the students or the coordinator is
raised. This paper describes the different assessment procedures required by a
successful privately conducted professional practice course and a successful
graduate university academic course.
Introduction
This paper focuses on our approach to implementing criterion referenced
assessment, which we refer to as "sharing the load". Our particular goals have
been to develop coherent and specific sets of criteria and standards for
preservice teacher education subjects, and to offer students assessment tasks
which are both authentic and purposeful. We discuss five main themes:
1. collaboration between lecturers from different disciplines
2. collaboration between students on assessment tasks
3. the use of formal and informal peer assessment
4. the design of assessment tasks with multiple purposes and a range of
outcomes
5. criteria and standards drawn from professional education practice.

Background
As former secondary school teachers who are now university Education lecturers, we
find it satisfying to observe the move away from examination-based, norm-
referenced assessment of university subjects towards a greater variety of
assessment modes and the use of criteria and standards to judge the quality of
students' performance. This is an important area in which university practice has,
in general, lagged behind professional practice in other educational sectors; for
example, Queensland secondary schools have been required to use criteria and
standards based assessment since the mid 1980s (see Board of Secondary School
Studies, 1986-1988 for details). The knowledge and skills developed within
secondary schools over this time have been influential in shaping our own practice
as teachers and teacher educators, and have helped us to design assessment
programs which are consistent with the policies and guidelines recently adopted by
The University of Queensland (The University of Queensland Academic Board, nd.,
available at -
http://uqadminserver.jdstory.uq.edu.au/AcadBoardOffice/policy/assess_Pol&Guid.html
).
The aim of this paper is to illustrate some strategies for putting into practice
assessment principles endorsed by the university's Academic Board. These
principles include the following:
1. The primary focus of assessment is to encourage, direct and reinforce
learning.
2. The assessment methods employed should reflect the variety of subject and
course goals.
3. Well-constructed self assessment and peer assessment exercises have the
potential to provide valuable learning experiences and encourage lifelong
learning.
4. Staff should seek to lessen the threat posed by assessment. This may be
accomplished by giving students practice in the particular assessment methods
being employed.
5. Feedback is fundamental to the learning process and some form of feedback
should be offered on all items of assessment.
6. Each piece of assessment is to be accompanied by clear assessment criteria
which are effectively communicated to students and markers.
7. Marks and grades are awarded by reference to predetermined standards rather
than by reference to the performance of other students in the subject.
While these principles are equally relevant to school and university settings, a
direct transfer of school assessment methods to the university environment is
inappropriate, since differences in students' maturity, interests and needs need
to be taken into account. As our current students are adults undertaking a
postgraduate teacher education course, we believe it is important to offer
assessment tasks which mirror the authentic practices of the teaching profession.
In this paper we demonstrate how we have made connections between these practices
and the principles listed above, by referring to assessment tasks we have
developed for The University of Queensland's Postgraduate Diploma in Education
course.
The principle aim of the Diploma course is to provide preservice education for
intending secondary school teachers. Students undertake compulsory studies in
subjects dealing with professional issues and practices in education, and choose
two curriculum areas which represent their teaching interests. Our own curriculum
subjects prepare students to teach Mathematics and English across the broad range
of courses offered in the Junior and Senior secondary school. These subjects
introduce current theories of learning and teaching Mathematics and English, and
give attention to curriculum development, assessment and evaluation, social
justice issues, and the impact of technology on language/mathematics teaching and
learning.
To emphasise the link between our courses and professional practice, where
possible the teaching and learning approaches used in workshops and tutorials are
intended to model strategies that can be used in secondary classrooms. We
encourage collegiality, collaboration and cooperative learning practices in these
sessions as they are fundamental professional attributes.
In the remainder of the paper we explain how two lecturers from different
disciplines can work together to design effective assessment strategies, and we
describe some of the ways in which these strategies have enhanced students'
learning.

Assessment Strategies
Collaboration between lecturers from different disciplines
It may seem unusual for lecturers from different disciplines-Mathematics and
English-to collaborate in planning assessment programs. However, we have found
that a collaborative approach offers personal and practical rewards. First, the
sharing of ideas and expertise reduces the combined workload and prevents wasteful
duplication of effort and resources. When face to face interaction is not
possible, questions and information can easily be exchanged via email, and entire
documents can be sent back and forth for comment and amendment. In addition,
developing a coherent, consistent, cross-disciplinary approach to assessment is a
worthwhile venture in its own right, as it forces us to articulate and justify our
own values and principles in the process of reaching common ground. Students
should also benefit from receiving consistent messages about assessment in
different subjects, since this may reduce the perception that assessment is merely
a game which can be won by learning each subject lecturer's idiosyncratic rules.
One of the products of our collaboration is illustrated in Figure 1, which shows
how we have identified common themes in our subjects, and how assessment tasks in
Mathematics and English were planned to meet students' changing needs and concerns
in the different phases of the course. Assessment tasks have been designed to
complement each part of the course and to give appropriately timed feedback on the
development of students' knowledge and skills (The University of Queensland
Academic Board, nd.).
Because assessment tasks are closely tied to students' professional duties as
teachers, we recognise our responsibility to model task performance in workshops
and to provide exemplars of good practice. In addition, the close relationship
between assessment and learning is reinforced by using class time to provide
students with the skills they will need to complete assessment tasks. For example,
students are prepared to tackle the Mathematics textbook evaluation task in a
series of tutorials which deal with mathematics and language, analysis of
textbooks, and designing reading activities which compensate for a text's
weaknesses and capitalise on its strengths. In English, as an outcome of a
workshop on planning, the students were encouraged to develop a one page outline
of their proposed unit plan for formative assessment. In addition, since the
materials produced are intended to be useful resources for their future practice,
we believe that this approach is effective in eliciting students' genuine effort,
motivation and commitment (Maxwell, van Kraayenoord, Fields & Herschell, 1995).
Criteria and standards drawn from professional education practice
A further aspect of our collaboration has involved developing assessment criteria
which are grounded in professional educational practice, and realistic standards
for describing students' performance. We began with our course objectives, from
which we formulated general criteria to identify the qualities we expected
students to develop throughout the year. While the general criteria for
Mathematics and English were not identical, their underlying similarities are
clearly seen in the examples below. (There were four general criteria for
Mathematics, and six for English.)
Theme Phase Mathematics Assessment Tasks English Assessment Tasks
Teaching Mathematics/ English: models, strategies, classroom skills

Understanding Mathematics learning processes, language and literacy Phase 1:

Understanding curriculum and teaching (9 weeks)
Familiarisation with syllabus documents Review of an article from a
professional journal A set of lesson plans and accompanying resources
Practicum (5 weeks)
Curriculum development Phase 2: Consolidation and extension (4 weeks) Textbook
evaluation
Technology seminar Annotated resource file for a Junior syllabus topic
Practicum (7 weeks)
Issues in Mathematics/ English education Phase 3: Preparation for the first year
of teaching
(4 weeks) Annotated resource file for a Junior syllabus topic
Unit plan Seminar on an issue in English education
Statement of philosophy
Unit plan
Figure 1. Course structure and assessment tasks for Mathematics and English
curriculum studies

.
Mathematics Criteria English Criteria
1. Demonstrated understanding of concepts, contexts, and issues in mathematics
education 1. Demonstrated understanding of theories and principles of language
and learning in English education.
2. Application of theoretical ideas about teaching and learning to practical
problems and tasks. 2. Demonstrated understanding and application of a range of
approaches and strategies in English teaching relevant to classroom situations.
For each assessment task we then prepared detailed criteria and explicitly stated
standards, which mapped out in subject-specific terms the range of performance
levels we anticipated. Once again we shared ideas about task criteria, but these
were then individually adapted to incorporate the vocabulary and values of our
separate disciplines. For example, in both Mathematics and English curriculum
studies, students were required to work in pairs to present a seminar dealing with
an issue (English) or technology-based activity (Mathematics) of their choice. The
task-specific assessment criteria for Mathematics and English seminars are
provided in Figure 2, together with descriptions of the highest quality standard
of performance. The major difference between these two tasks was the intended
audience: although the purpose in both subjects was to simulate a professional
development seminar, in English the aim was to develop teachers' understanding of
the chosen issue, while in Mathematics the goal was for participants to try out an
activity designed for school students.
Figure 2 also shows how specific assessment tasks allow students to demonstrate
the qualities described by the general course criteria; for example, the first
task criterion, which refers to understanding of the issue or topic, is a specific
instance of the more general criterion dealing with understanding of theories and
principles (English), and concepts, contexts and issues (Mathematics).
MATHEMATICS ENGLISH
Criterion Top Standard Criterion Top Standard
Understanding of the use of technology in teaching mathematics
CC1 Seminar handout deals comprehensively with relevant aspects of implementing
the activity, giving balanced attention to mathematics, technology, and teaching
approach. Understanding of issue or topic selected
CC1, CC5 Seminar deals comprehensively with relevant aspects of issue/topic
giving balanced attention to different perspectives.
Relevance of selected technology based activity
CC3 Activity is imaginatively developed, selected, or adapted, and highly
appropriate for developing students' understanding of the mathematical topic.
Rationale clearly explains relevance and purpose of the activity. Relevance of
selected topic/ issue to English education
CC1, CC5 Implications for English teaching are highly relevant, imaginatively
developed, selected, or analysed, and highly appropriate for developing teachers'
understanding of the issue/topic.
Structure and organisation of seminar presentation
CC2 Logically structured; activity stimulates student interest and actively
engages them with the technology and the mathematics; resources are used with
flair and imagination; pacing makes best use of the time available. Structure
and organisation of seminar presentation
CC2, CC3 Seminar is sequenced to provide an engaging introduction, showed
logical development and variety, and a relevant conclusion. Resources were used
with imagination; pacing makes best use of the time available.
Quality of oral communication
CC4 Communicates with clarity; good use of variation; demonstrates well
developed questioning and explanation skills with individual students; uses
students' questions and comments to orchestrate whole class discussion of
activity. Quality of oral communication
CC6 Communicates with clarity; good use of variation; demonstrates well
developed questioning and explanation skills with individual students;
orchestrates class participation very effectively.
Quality of written communication
CC4 Writing is concise, well-structured and error-free. Format and structure of
handout make it easy to follow and practical to use. Quality of written
communication
CC6 Writing is concise, well-structured and error-free. Format and structure of
handout make it easy to follow and practical to use.
Figure 2. Task-specific criteria and top standard for Mathematics and English
seminars
Collaboration between students on assessment tasks
While we encourage students to collaborate informally on all assessment tasks as
well as during workshops, the previous section implies that we also see value in
designing formal assessment tasks in which students are required to work together
and prepare materials, ideas or activities for presentation to peers. The example
outlined above involves the use of professional development seminars. These
seminars have both an assessment and a professional focus, since their purposes
are not only to develop students' planning, communication and organisational
skills, but also to prepare them to work collegially in sharing ideas with fellow
teachers. To illustrate these benefits we describe some features of the Technology
Seminars presented by Mathematics students.
The students were required to prepare and present a technology based activity
designed to teach some aspect of a topic of their choice from the Senior (i.e.
Years 11 and 12) Mathematics Syllabuses. The activity was to make use of either
computer software or graphing calculators, and sufficient time was allowed for the
class to try the activity and offer feedback on its usefulness. To accompany their
oral presentation, students provided class members with a handout containing
information which would enable another teacher to implement the activity. Both the
written handout and oral presentation were assessed, as shown in Figure 2, and
students were assessed as a pair rather than individuals. Students had ample
opportunities to develop the requisite technology skills during workshops leading
up to the assessment task, and additional class time was set aside to allow them
to choose a partner, decide on a topic, visit the library to search for print or
Internet resources, try out activities, and consult with the lecturer. A full day
was set aside for the seminars, in order to provide a realistic simulation of a
teacher inservice course or conference. The level of expertise displayed by the
presenters, the intensity of the class's engagement with the activities, and the
stimulating discussion afterwards suggested that students found the task relevant
and interesting. Collaboration was also observed to extend beyond the pairs of
presenters, as students were encouraged to work together on the activities.
Indeed, the whole enterprise was collaborative in the sense that the tangible
product which resulted was a set of technology-based activities produced and
shared by all members of the class. The task also permitted students to share
their work with the wider professional community, as the next section describes.
The design of assessment tasks with multiple purposes and a range of outcomes Our
assessment programs are intended to be both process and outcome oriented; that is,
our goal is to offer authentic tasks which result in a useful product. This goal
is achieved most convincingly when students' work reaches a professional audience
beyond their course lecturers and peers. For example, one pair of Mathematics
students has presented their Technology Seminar at the Annual Conference of the
Queensland Association of Mathematics Teachers, and a further three pairs have
submitted articles based on their technology-based activities to the journal
published by this professional association. In addition, students have tendered
their unit plans (the final assessment task, see Figure 1) as evidence of their
competence when applying for teaching positions in secondary schools. This
evaluation of students' work by practising teachers reinforces the authenticity of
the original assessment tasks, and legitimates the criteria and standards we use
to judge the quality of students' performance.
The use of formal and informal peer assessment
The University of Queensland's assessment policy acknowledges the value of well-
constructed peer assessment exercises in providing valuable learning experiences
and lifelong learning (The University of Queensland Academic Board, nd). In both
our classes we include informal and formal peer assessment. Informal peer teaching
takes place throughout the year: in microteaching situations, for example, we ask
students to observe the teaching of their colleagues providing oral, and sometimes
written, feedback on their performance. Reflection after role playing activities
in workshops also allows the students to give constructive feedback in a
supportive atmosphere. The example of sharing the Annotated Resource File used in
the English curriculum subject demonstrates how more formally structured peer
assessment can be used to augment the lecturer's judgments.
English students were required to develop a collection of resources which could be
expanded during subsequent years of teaching (assessment focus). Each collection
consisted of ten to fifteen resources suitable for teaching in a junior secondary
English classroom. The aim of the task was to develop critical skills in selecting
and evaluating resources, and so students were asked to provide critical
evaluations of the resources focusing on language challenges they posed for
students with learning difficulties (professional focus). The resources file was
assessed by the lecturer and by peers. The peer assessment component of this task
focused on three goals, all of which have a professional component: first, to
share and swap resources, second, to provide peer feedback on resources, and
third, to provide experience in writing evaluative comments. The peer assessment
workshop lasted three hours and included the following steps:
1. A review of feedback techniques led by the lecturer. This introduction
addressed issues of how to write comments positively and provided a synthesis of
current research on effective feedback.
2. "One minute of fame" for students to share their topics. This enabled the
students to identify Resource Files that were relevant to their teaching interests
and needs for the forthcoming practicum.
3. Reading and evaluation. Students were required to read and evaluate a
minimum of two folders. Students used the same criteria as the lecturer for their
peer evaluations.
4. Whole group reflection.
Lively discussions were generated during the reading and evaluation phase and in
the reflections. The students found talking about their resource files and sharing
their collections to be the most important element of the workshop. They commented
that they felt validated in their judgment as teachers and supported in their
professional development. The students were also made aware of the amount of time
needed to generate written comments and the difficulties of writing constructive
comments on the performance of their friends This was a significant exercise in
making them sensitive to the issues they would face when performing similar tasks
with their own pupils. A very practical outcome of the task was the opportunity to
exchange professional materials that would be useful in their early years of
teaching.
So far we have presented a very positive picture of our implementation of
criteria-based assessment. However, as we refine our assessment programs and
implement them in our courses we are increasingly aware of issues that still need
to be worked through in using this form of assessment. In the next section we
highlight some of those issues.

Some Issues
The first issue concerns the continuing requirement to assign a single grade to
sum up performance across a wide range of tasks over the course of a full year.
Such a requirement seems at variance with the goal of providing detailed feedback
on the many qualities/criteria that characterise competent teaching performance.
Rich information is lost when achievement is reported in this way. We have shown
that the task of generating suitable criteria is one which can be tackled
collaboratively by lecturers within a broadly based discipline such as Education,
and we suggest that staff in other departments will benefit from this type of
professional development activity. A possible model could involve lecturers
working as a team to formulate generic criteria that apply to learning within
their discipline, followed by individuals identifying the unique knowledge and
abilities to be developed in specific subjects.
Turning to assessment strategies, we are aware that student collaboration on
assessment tasks may not meet with universal approval, since there are those who
protest that students forced to work in pairs or groups will not necessarily make
an equal contribution to the task. However, this assessment strategy has our full
support, since one of our central goals is for students to develop the
interpersonal skills needed to work in a collegial environment. Similar doubts may
be expressed about peer assessment, particularly on the question of students'
competence in making judgments about their fellow class members. We also
acknowledge that there are difficulties in deciding how to incorporate students'
evaluations into the final grade for the task.
Nevertheless, we maintain that this is a worthwhile approach if the right balance
can be struck between making the task as realistic as possible and avoiding
placing too much pressure on students who are still relatively inexperienced in
assessment.

Conclusion
In our discussion of five themes
• collaborative design of teaching and assessment programs;
• use of realistic criteria and standards;
• student collaboration to enhance learning;
• authentic assessment tasks; and
• peer feedback;
We have drawn on examples from our own practice to illustrate ways of implementing
The University of Queensland's new assessment policies. Using criteria and
standards allows us to map students' strengths and weaknesses in a wide variety of
activities relevant to their development as teachers, and provides valuable
feedback which shows them where and how they can improve on subsequent tasks. We
have also benefited from examining in detail our own expectations and assumptions
about learning, made explicit in the process of planning our assessment programs.
In this paper we have illustrated some of the assessment techniques that work for
us. However, it is important to remember that all assessment has social, economic
and political dimensions (Johnston, 1992), so that particular techniques are
merely an expression of underlying principles. Teachers working within different
academic disciplines must decide for themselves how best to apply those principles
to their own contexts.

References
Board of Secondary School Studies (1986-1988). Discussion papers on school based
assessment (Numbers 1-21). Brisbane: BSSS.
Johnston, P. (1992). Constructive evaluation of literacy. New York: Longman.
Maxwell, G., van Kraayenoord, C.E., Fields, C., & Herschell, P. (1995). Developing
an assessment program: A planning guide for schools (A Professional Development
Program). Brisbane: Department of Education, Queensland, Southern Vale School
Support Centre.
The University of Queensland Academic Board (nd). Policy and guidelines on
assessment: Advice to examiners from the assessment subcommittee. Available at
http://uqadminserver.jdstory.uq.edu.au/AcadBoardOffice/policy/assess_Pol&Guid.html

________________________________________
This site built on a Macintosh using Dreamweaver.. Last modified 30/8/99; 1:40:12
PM. Website queries, contact SusieGB@uq.edu.au.