Vous êtes sur la page 1sur 7

CRITERIONREFERENCED

ASSESSMENT
WHAT IS CRITERIONREFERENCED ASSESSMENT?
Assessment,generally,istheprocessofmakingafairandequitableevaluationaboutstudentlearning
outcomes.Inordertoachievethis,assessmenttasksneedtobedesignedinsuchaswayasto
demonstratetowhatextentstudentshavemetthelearninggoalsofaunitofstudy.Byspecifyingthe
criteriawhichistobemeasuredandthestandardofcompetencyorperformancetobeachieved,
judgementcanbemadeaboutstudentlearning.Thecriteriashouldbeclearlyalignedwiththeunitof
studyslearningoutcomes,whichwillbecomprisedofdisciplinespecificknowledgeandskillsaswellas
capabilitiespertainingtoemployabilityskillsandVUsgraduatecapabilities.Thestandardsofperformance
shouldberankedonadevelopmentalcontinuumandarelikelytorangefromhighlevelsynthesisand
applicationatoneendtoaminimalunderstandingattheother.
WHY USE CRITERIONREFERENCED ASSESSMENT?
CriterionreferencedassessmentisusedinmanyAustralianuniversities,includingVictoriaUniversity
because,asstatedabove,itisconsideredtobeafair,equitable,transparentandreliablewayinwhichto
measurestudentsacademicachievements.
Criterionreferencedassessment,asspecifiedinVUsStudentAssessmentPolicy1 fallswithinanapproach
thatpromotesassessmentthatgoesbeyondassessmentoflearning,measuringwhatstudentshave
learned(summative)andascribingagrade.VUspolicyalsopromotesassessmentforlearningwhichis
formativeandprovidesfeedbacktobothteachersandstudentsaboutstudentprogress.Additionally,VU
promotesassessmentaslearninginwhichstudentsareprovidedwithopportunitiestoreflectuponand
monitortheirprogressinordertoinformtheirfuturelearninggoalsandaspirations(alsoformative).
Otherassessmentapproachesincludenormativereferencedassessmentwhichcomparesstudents
performancetothatoftheirpeersandipsativereferencedassessmentwhichenablestudentstoreflecton
thedevelopmentoftheirpersonalvalues,expectations,andownlearninggoals(particularlyaptfor
reflectionjournalsinLearningintheWorkplaceandCommunity).

HOW TO DEVELOP CRITERIONREFERENCED ASSESSMENT?


Establishunitlearningobjectives/outcomes

Designlearningactivitiestoattainlearningoutcomes

Developassessmentstomeasurelearningoutcomes

Assessmentstoincludecriteria,standardsandmarks

VUsStudentAssessmentPolicy: http://wcf.vu.edu.au/GovernancePolicy/PDF/POA090212002.PDF

1. Establish unit learning objectives / outcomes


Assessmentsareestablishedwithinthecontextofthelearningoutcomesofaunitofstudy.Learning
outcomesdeterminewhatstudentsareexpectedtoachieveasaresultofstudyingaunit. Learning
outcomesshouldincludethedevelopmentofdisciplinespecificknowledge,skillsandabilitiesaswellasthe
moregenericgraduatecapabilitiesandemployabilityskills.Theseshouldbeexpressedasclearand
measurablestatementstothestudents.
Example:Asanoutcomeofthisunitstudentswillbeabletodevelopabusinessplanforoperatingawebdesign
business

2. Design learning activities to attain learning outcomes


Learningactivitiesarethemechanismsthroughwhichstudentsareabletodeveloptheskillsrequiredto
attainthelearninggoals.Thedesignoflearningactivitiesshouldincorporateopportunitiesforstudentsto
practiceandapplytheirlearning.Problembasedlearning,experientiallearning,learningintheworkplace
andcommunity,studentcentredlearning,andactionlearningaresomeofthevehiclesthroughwhich
studentscangainmasteryintheirunitofstudy.
Learningactivitiescanbeassessedprovidingarichopportunityforformativefeedbacktostudents.This
alignswiththeassessmentaslearningapproachinwhichtheassessmenttaskservesasadevelopmental
mechanismforstudentlearning.
Bloomstaxonomyoflearningmaybeausefulconstructtousewhendesigninglearningactivities.Bloom
maintainedthattherearethreedomainsoflearning:cognitive,affectiveandpsychomotorandefforts
shouldbemadetoenablestudentstodevelopskillsinallofthem.Forexample,inthecognitivedomain,
activitiescanbedesignedthatdevelopstudentsskillsonvaryingdegreesofcomplexityandhigherorder
thinking.Bloomscognitivelevelsincluded:knowledgeacquisition,comprehension,application,analysis
synthesisandevaluation.2

Example:Learningactivitiestodevelopabusinessplanwillinclude:
a.
Conductingacompetitoranalysis
b.
Undertakingmarketresearch
c.
Performingacostbenefitanalysis

3. Develop assessments to measure learning outcomes


Assessmenttasksshouldbedesignedtocollectivelycoverallthelearningoutcomesstatedintheunitof
study.Assessmentstasksmaybemappedagainstthelearningoutcomeswhichmaybevalidatedthrough
aformalvalidationprocess.Careshouldbetakentodesignassessmenttasksthatactuallymeasurewhat
theyportendtomeasure(validity)andhaveconsistencyinoutcomeirrespectiveofwhoandwheretheyare
(reliability)conducted.Attentionshouldalsobegiventothefairness,equity,transparency,andauthenticity
oftheassessmentmode.Arangeofauthenticassessmentmodesshouldbeemployed.

Example:Studentassessmentswillbecomprisedof:
a.
Preliminaryresearchandplan(formative)30%
b.
Finalbusinessplan(summative)
55%
c.
Oralpresentationofbusinessplan(summative)
15%
100%

Bloom,B.S.(Ed.).(19561964).TaxonomyofEducationalObjectives.NewYork:DavidMcKayCompanyInc.inUniversityof
Queensland,TeachingandEducationalDevelopmentInstitute(1996)Bloomstaxonomyofeducationalobjectives
http://www.tedi.uq.edu.au/downloads/Bloom.pdf

4. Assessments to include criteria, standards and marks


Assessmentshouldbecomprisedof:
a. asetofcriteriaforeachassessmenttask.Thesecriteriawillindicatetostudentswhatelementsof
thetaskwillbejudged.Ideallythecriteriawillbeclearlyrelatedtothegoalsoftheunit.Thecriteria
maybepresentedinaholistic,integratedwayratherthanasindividualisedelements.
b. asetofstandardsbywhicheachcriterionwillbemeasured.Eachstandardwilldescribealevelof

performancewhichisrankedalongacontinuum,rangingfromexemplaryandhighlyproficient
performancetopoorandunsatisfactoryperformance.
c. amarkingschemeforeachassessmenttaskwhichlooksatthestandardsreachedbythestudent

onthattask.Elementsofthetaskmaybeitemisedandamarkawardedforeachelement,or
alternately,themarkmightbeawardedtoamorecomprehensiveandholisticassessmenttasksuch
asareport.
d. agradingschemeinwhichthestudent'sperformanceoneachindividualassessmenttasksis

collatedintoasinglegradeforthatunit.3

Use of rubrics
Aneffectivemethodformeasuringmultiplecriteriacanbefoundintheuseofanassessmentrubric.Thisis
particularlyaptforacriterionreferencedlearningandassessmentframeworkbecauseitisunderpinnedby
thenotionofthedevelopmentofknowledge,skillsandabilitiesalongacontinuum.Anassessmentrubricis
essentiallyasetofcriteriaandperformancestandardslinkedtolearningobjectives.Itisoftenexpressedin
theformofamatrixwhichestablishestheperformancecriteriaalongtheYaxisandthestandardsof
performancealongtheXaxis.Thestandardsofperformanceareportrayedalongacontinuumdenoting
whichlevelhasbeenmet,rangingfromaminimalunderstandingandapplicationatoneendtoa
sophisticatedsynthesisandapplicationattheother.Thisscaleofincreasingcompetencecanserveas
valuableformativefeedbacktothestudent.Itsvaluecanbeoptimisedifstudentsaregiventherubricat
thesametimethatotherassessmentdetailsarepresented.

Example:Assessmentrubricforcompetitoranalysiselementofpreliminaryresearchandplanassessment
Criteria

VeryGood

Good

Fair

Indepthuseofanextensive
rangeofrelevantliterature.
Clearlydemonstratedanalysisof
ownresearchinrelationto
academicresearchandtheory.
Soundsynthesisofallmaterial.
Referencescitedcorrectly.
4
Extensiveresearchof
competitors.Comprehensive
investigationofcriticalfacetsof
competition.Analysisverywell
constructedandarticulated.

Someuseofasignificant
rangeofrelevantliterature.
Somelinkagesbetweenown
researchwiththatofother
academicsources.Some
analysisofmaterial.
Referencescitedcorrectly.
3
Goodresearchon
competitors.Solid
investigationofcriticalfacets
ofcompetition.Soundanalysis
andarticulation.

5marks
C.Constructofpreliminary
plan

5
Clearevidenceofsynthesisof
informationfromtheliterature
andcompetitoranalysisinthe
developmentthebusinessplan.
Comprehensiveandwellthought
outplan.

34
Evidenceoftheuseof
informationfromtheliterature
andcompetitoranalysisinthe
developmentthebusiness
plan.Satisfactorybusiness
plan.

Someuseofrelevant
literatureLittleevidence
oflinkagesoranalysisof
ownresearchwiththatof
otheracademicsources.
Most,butnotall,
referencescitedcorrectly.
2
Basicresearchon
competitorsand
investigationofcritical
facetsofcompletion.
Underdevelopedanalysis
andexpression.
12
Someevidenceoftheuse
ofinformationfromthe
literatureandcompetitor
analysisinthe
developmentthebusiness
plan.Businessplanhas
missingfacets.

6marks
Total15marks

45

AUseofrelevant
theory/literature

4marks
B.Analysisofcompetitors

23

Unsatisfactory
Nomeaningfuluseof
relevantliterature.
Inadequatecitingof
references.

01
Scantresearchand
investigation.Little
analysisand
unsatisfactoryexpression.

0
Littleornoevidenceof
theuseofinformation
fromtheliteratureand
competitoranalysisinthe
developmentthe
businessplan.Business
planhasnumerous
missingfacets.
01

TeachingandDevelopmentInstitute,UniversityofQueensland(2001)AssessmentforLearning
http://www.tedi.uq.edu.au/teaching/assessment

VALIDATION AND MODERATION


Validationisthequalityassuranceprocessthatdemonstratesalignmentbetween:
a.
b.
c.
d.

broadcourseobjectives
unitlearningoutcomes
unitlearningactivities
unitassessment

Unit learning
outcomes

Learning
activities
Assessment
Criteria
Performancelevel
Mark

Validationalsodemonstratesthatafair,equitable andtransparentassessmentapproachhasbeen
employedtomeasurestudentsuccess.
Validationistheprocesswherebyeducatorsevaluatetheirassessmentmethods,assessmentprocedures
andassessmentdecisions.Itmayinvolveinternalorexternalstakeholdersorboth.Validationformspartof
thereviewassessmentrequirementoftheTrainingPackageforAssessmentandWorkplaceTrainingand
auditrequirementsoftheAustralianQualityTrainingFramework(AQTF).Thegoalofvalidationistoensure
thatassessmentisvalid,reliableandfairandthatdecisionsaremadeonthebasisofsufficientand
appropriateevidence.
Apanelorgroupofpeerreviewersmaybeestablishedtoformallyreviewtheassessment.Thiswillinvolve
evaluatingandendorsingthefourprongedvalidationprocess:
a. Designingandaligningassessmenttaskswithlearningoutcomes
b. Establishingcriteriaandstandardsforassessmenttasksthatmeasurelevelsofperformanceon
selectedassessmentcriteria
c. Developingarangeofassessmentsallofwhicharereliable,valid,current,authentic,sufficientand
equitable
d. Moderatingassessmenttoensurethatconsistentjudgementsandgradesemanatefrom
transparentandfairassessmentcriteriaandstandards.Asamplingofassessedworkshouldresult
inconsistentjudgements.
Validationrecordingandreportingispartoftheformalvalidationprocess.

FURTHER READING
VUStudentAssessmentPolicy:http://wcf.vu.edu.au/GovernancePolicy/PDF/POA090212002.PDF
VULearningandTeachingPolicy: http://wcf.vu.edu.au/GovernancePolicy/PDF/POA040916001.PDF
GraduateCapabilities: http://wcf.vu.edu.au/GovernancePolicy/PDF/POA050510000.PDF
GraduateCapabilitiesResources:http://tls.vu.edu.au/portal/site/design/graduate_capabilities.aspx
AGuidetoDesigningLearningOutcomes:http://tls.vu.edu.au/portal/site/design/designing.aspx
Booth,RobynClayton,BerwynHouse,Robyn.(2002).Maximisingconfidenceinassessmentdecision
making:Resourcekitforassessors. http://www.ncver.edu.au/publications/780.html
Boud,David,UniversityofTechnology,Sydney.Assessmentfutures. assessmentfutures@uts.edu.au
CentreForTheStudyForHigherEducationForTheAustralianUniversitiesTeachingCommittee.(2002).

AssessingLearninginAustralianUniversities:Ideas,strategiesandresourcesforqualityinstudent
assessment.www.cshe.unimelb.edu.au/assessinglearning
Gillis, S., Bateman, A. & Clayton, B. (2009). A Guide to Developing Assessment Tools. National Quality
Council,Melbourne.
Gillis, S.,Bateman,A.& Clayton, B. (2009). ACode of Professional Practice:Validation and Moderation.
NationalQualityCouncil,Melbourne.
Gillis,S.,&Griffin,P.(2008).CompetencyAssessmentinJ.Athanasou(editor)AdultEducationand
Training,DavidBarlowPublishing,Sydney,pp.233256.
Gillis,S.,&Griffin,P.(2005).PrinciplesUnderpinningGradedAssessmentinVET:ACritiqueofPrevailing
Perceptions,InternationalJournalofTrainingResearch.3(1),5381.
Gillis,S.,&Griffin,P.(2007).CompetencyandPerformanceAssessment:Workbook. TheUniversityof
Melbourne,Melbourne.
Gillis,S.,&Griffin,P.(2004).UsingRubricstoRecogniseVaryingLevelsofPerformance. TrainingAgenda:
AJournalofVocationalEducationandTraining,12(2),2224.
Griffin,PatrickGillis,ShelleyCalvitto,Leanne.(2007).Standardsreferencedassessmentforvocational
educationandtraininginschools.AustralianJournalofEducation,v.51,no.1,pp.1938.
Nicholson,PatriciaGillis,Shelley&Dunning,Trisha.(2009).Theuseofscoringrubricstodetermine
clinicalperformanceintheoperatingsuite.http://www.education.unimelb.edu.au/arc/pdfs/20091546.pdf
Rust,Chris,OxfordBrookesUniversity,UK,TheAssessmentStandardsKnowledgeExchange.
http://www.business.brookes.ac.uk/learningandteaching/aske/
Tovey,M.D&Lawlor,D.R.(2008).TraininginAustralia:Design,Delivery,Evaluation,Management(3rd
ed).FrenchsForest,NSW:PearsonPrenticeHall.
UniversityofQueensland,TeachingandEducationalDevelopmentInstitute,(1996).Bloomstaxonomyof
educationalobjectives. http://www.tedi.uq.edu.au/downloads/Bloom.pdf
UniversityofQueensland,TeachingandEducationalDevelopmentInstitute,AssessmentCriteria,
StandardsandMarkingSchemehttp://www.tedi.uq.edu.au/downloads/Criteria.pdf
UniversityofQueensland,TeachingandEducationalDevelopmentInstitute,(2001).Assessmentfor
Learning http://www.tedi.uq.edu.au/teaching/assessment

ASSESSMENT DEFINITONS4
AssessmentAssessmentistheprocessofcollectingevidenceandmakingjudgementsonwhether
competencyhasbeenachieved
AssessmentcontextTheenvironmentinwhichtheassessmentwillbecarriedout.Thiswillinclude
physicalandoperationalfactors,theassessmentsystemwithinwhichassessmentiscarriedout,
opportunitiesforgatheringevidenceinanumberofsituations,thepurposeoftheassessment,whocarried
outtheassessmentandtheperiodoftimeduringwhichit
takesplace

Booth,Robyn,ClaytonBerwyn,House,Robyn,(2002)Maximisingconfidenceinassessmentdecisionmaking:Resourcekitfor
assessors http://www.ncver.edu.au/publications/780.html

AssessmentguidelinesOneoftheendorsedcomponentsofTrainingPackages,whichsetout
mechanismsforvalid,reliable,fairandflexibleassessmentofcompetencyagainstthecompetency
standards.TheguidelinesunderpinassessmentscarriedoutbyRegisteredTrainingOrganisationsunder
theAustralianQualityTrainingFramework(AQTF)
AssessmentplanAnassessmentplanisadocumentdevelopedbyanassessor,thatincludesthe
elementsorunitsofcompetencytobeassessed,whentheassessmentwilloccur,howtheassessmentwill
occur,theassessmentmethodstobeusedandthecriteriafortheassessmentdecision.
AssessmenttoolsAssessmenttoolsincorporateboththeinstrumentsandtheinstructionsforgathering
andinterpretingofevidence.Avarietyofassessmenttoolsshouldbeusedintheprocessofestablishing
competency.Evidencegathering/assessmenttoolsinclude:
specificinstructionsforcandidates
examplesofacceptableresponses
rulesofjudgementsinholisticcompetencyassessment
descriptionsoftypicalcompetentperformance
AuthenticityCompetentreferstotheconfidencethatanassessorhasthattheevidenceprovidedbya
learneraspartofassessmentisactuallythelearnersownwork
CompetencyThespecificationofknowledgeandskillandtheapplicationofthatknowledgeandskilltothe
standardsofperformancerequiredintheworkplace.
CompetencystandardsDefinethecompetenciesrequiredforeffectiveperformanceintheworkplace.
StandardsareexpressedinoutcometermsandhaveastandardformatcomprisingofUnittitle,Unit
descriptor,ElementsofCompetency,PerformanceCriteria,RangeofVariablesandEvidenceGuide.
EvidenceInformationgatheredwhich,whenmatchedagainsttheperformancecriteria,providesproofof
competency.Evidencecantakemanyformsandbegatheredfromanumberofsources.Directevidenceis
observationofperformanceunderrealorsimulatedworkconditions.Indirectevidenceisobservationof
performanceunderrealorsimulatedworkconditions.Indirectevidencecanbegatheredfromathird
person.Supplementarysourcesofevidencemayincludeanswerstooralorwrittenquestions,documented
informationaboutpastandcurrentachievements(portfolios,resumes),videosoraudiovisualrecordsor
priorperformance.
EvidenceGuidePartofaunitcompetency.Itspurposeistoguideassessmentoftheunitofcompetency
intheworkplaceand/oratrainingenvironment.TheEvidenceGuidespecifiesthecontextofassessment,
thecriticalaspectsofevidenceandtherequiredorunderpinningknowledgeandskills.TheEvidenceGuide
relatesdirectlytothePerformanceCriteriaandRangeofVariablesdefinedintheUnitofCompetency.
FairnessReferstothecontrolsinplacetoensurethatassessmentisavailabletoalleligiblepersons
regardlessofage,gender,ethnicity,disability,languagebarriers,andgeographiclocationandthat,inthe
courseofassessment,personsarenotdisadvantagedforanyreason.Thesecontrolsinclude:makingthe
assessmenttransparent(i.e.theassessmentprocessesandstandardsorotherassessmentcriteriaare
accessibleandunderstandable)ensuringassessmentmethodsareequitableadvisingpersonsbeing
assessedonthepurpose,methodsandproceduresforassessmentprovidingeffectivefeedbackduring
andafterassessmentandprovidingsuitableopportunitiesforchallengesandappealsandreassessment,
whennecessary
IntegratedassessmentAnapproachtoassessmentthatcoversmultipleelementsand/orunitsfrom
relevantcompetencystandards.Theintegratedapproachcombinesknowledge,understanding,problem
solving,technicalskills,attitudesandethicsintoassessmenttasks.
PerformancecriteriaEvaluatestatementswhichspecifywhatistobeassessedandtherequiredlevelof
performance.Theperformancecriteriaspecifytheactivities,skills,knowledgeandunderstandingthat
providesevidenceofcompetentperformanceforeachelementofcompetency.

QualityassuranceAplannedandsystematicprocessofensuringthattherequirementsoftheassessment
system,competencystandardsandanyothercriteriaareappliedinaconsistentmanner.Quality
assurancemechanismsorproceduresareanintegralpartofanassessmentsystem.
ReliabilityReferstotheextentinwhichconsistentoutcomesareachievedinassessmentregardlessof
whodoestheassessment,whenitisconductedandinwhatevercontextitisconducted.Thisusuallyis
achievedbyensuringclearandsufficientlydetailedprocedures,clearandunambiguousassessment
criteria,measurestoensureconsistentinterpretationofevidenceandsufficient,suitabletrainingfor
assessors.
ReportingofassessmentoutcomesThedifferentwaysinwhichtheoutcomesofassessmentprocesses
arereportedtothepersonbeingassessed,employersandotherappropriatepersonnelorstakeholders.
Assessmentoutcomesmaybereportedinavarietyofwaysincludinggraded,nongraded,statisticalor
descriptivereportingsystems.
ReviewofassessmentprocessPlannedandsystematicanalysisoftheassessmentsystem.
SimplicityReferstotheextenttowhichassessmentproceduresarestraightforwardandeasyto
understandandimplement
SufficiencyReferstohavingenoughsuitableevidencepresentedbyalearnertoensurethatanaccurate
decisioncanbemadeaboutachievementofcompetency.
TransparencyReferstotheextenttowhichallaspectsofanassessmentsystem(criteria,processes,
recordingarrangements,etc.)areopenandcleartoanyonewhotakesthetimetolookat,orthroughit.Itis
themeasureoftheclarityofanassessmentsystemtoallinvolved.
UnitofcompetencyDescribeadiscretejoborfunctionandiswritteninoutcometerms.Furtherdeveloped
throughElementsandPerformanceCriteria
ValidityAvalidassessmentassesseswhatitclaimstoassessevidencecollectedisrelevanttotheactivity
anddemonstratesthattheperformancecriteriahavebeenmet.
ValidationValidationinvolvesreviewing,comparingandevaluatingassessmentprocesses,toolsand
evidencecontributingtojudgementsmadebyarangeofassessorsagainstthesamestandards.The
processmayinvolvehavingbothtechnicalandassessmentspecialistsreviewtheassessmenttools,
proceduresandjudgementsforvalidity.Theprocessmaybeinternalwithstakeholderinvolvementor
externalvalidation

FURTHER INFORMATION
FriederikaKaider,CentreforInnovationinLearningandTeaching,99198390, friederika.kaider@vu.edu.au