Directions for Completing Item-Specific Training and Calibration

Log in to ONE.

Click to accept the Terms and Conditions.

Click to highlight the grade/test you need.

Then click

at the bottom of the screen.

Page 1 of 9

Prior to beginning item-specific training, read the document Tab

Information_Orientation to Training in the Program Specific Information
section of the Training tab. The document will guide you through each tab
in ONE and explain its contents, which you must reference during training
and scoring.

If you have already attended a WebEx training session for the

prompt/question to which you have been assigned, then you may
proceed to Interactive Training in the Training tab. Skip to page
5 of this document.

Page 2 of 9

If you did not attend a WebEx item-specific training, you must view
the video training that pertains to the item to which you have been
assigned. The videos are in Program Specific Information in the Training
tab. PASS will schedule you in ONE so that you can complete the video training on
one day and then complete the training and calibrations sets on a second day.
Go to the Training tab. Then click on Program Specific Information.

To locate the video you need, it is helpful if you know the Prompt/Cluster ID
or Prompt/Cluster Name of your assigned Written Composition prompt or
Reading question. The Prompt/Cluster ID is part of the description that
follows the document title. For example, the description of the video below
indicates the training is for question VH504666. The title also contains
information that assists in identifying the presentation you need. In the
example below, the title indicates the training is for English I (EI) Reading
Short Answer Connected (SAC).

1. You may need to scroll to the bottom of the list of documents that displays
and click on Next or Last to find the training video you need.

Page 3 of 9

2. Click on the document title to open the video training. After you have
opened the document, scroll to the first slide with the speaker icon in the
lower right-hand corner. In most cases, the icon will be on the first slide. In
a few cases, the speaker icon will not be on the first slide, so you will need
to go to the second slide to find the icon.

4. You will need to turn on your computers speakers to hear the

presentation. The presentation will automatically advance.

As part of the training, you may be directed to pause the video. To do so,
click the pause icon. To restart the presentation, click on the current slide.
The item-specific materials (e.g., rubric, anchor/benchmark
responses, annotations, etc.) you will need to reference during
the video training are located in Interactive Training. The video
narration may direct you to another ONE location. Disregard that
direction and go to Interactive Training.

Page 4 of 9

Click on Interactive Training.

The following window will open.

The training and calibration set numbers you will need to access as
part of your training are provided in the Training and Calibration
Set Assignments document in Program Specific Information.
Write down the numbers.

Click on Program Specific Information. Locate the document,

Training and Calibration Set Assignments. Open the document.
Page 5 of 9

The document provides a list of the training sets and calibration sets you need to
access and complete. Write down the numbers. It is important that you work
through the training sets in the order in which they are listed. You must complete
the feedback sets before taking the practice sets.
Select the first feedback set for the item on which you are training to score. See the
example below.
In the example below, notice that the rater has correctly gone to the Feedback Sets
section. After the rater referenced the Training and Calibration Set Assignments
document, he/she knew the Set ID # for Training Set 1. In this case, that number was
5745. Therefore, the rater looked down the Set ID column and found 5745. (In this
example, the set was the first set on the list. You will have to scroll down the Set ID
column to find the set you need.) To open the training set, click on the set ID number.

Page 6 of 9

The window that opens will display the tabs shown below.

Read through the information in the Prompt, Scoring Guide, Scoring Notes,
and Benchmark tabs before you choose the response tab to begin scoring the
feedback set(s). The information in the Scoring Notes tab may be printed, but you
must destroy the information at the end of scoring.
Remember that after you enter your score for each response, the actual score and
an annotation will be provided in a pop-up window.

To go to the next training set, click on the Training tab.

Go back to Interactive Training.
When you have completed the feedback training set(s), scroll down the Training
window until you see a blue bar that says Practice Sets:

Complete the practice set(s) associated with the item you are training to score.
Select the Summary link in the upper- right corner to view your results. Click on
the response number to see an annotation for the response.
Instructions while you are scoring in the Response tab:
1. Read the response in its entirety before you assign a score. Use the scroll bar to
review the entire response. The score-point buttons will remain inactive until you
scroll to the bottom of the response.
2. Click the button associated with the score point you wish to assign to the
response. Upon clicking a score button, the selected score becomes highlighted
and the Submit Score button becomes active.
3. Click the Submit Score button to submit the score for the response. Upon
submitting the score, the Scoring Results window displays your results.
4. Continue scoring each response in the feedback set until all the responses have
been scored.
5. Upon submitting the score for the last response, the Scoring Results window
displays the Summary button instead of the OK button.
6. Click the Summary button to view scoring results for the set.
7. Click the Go to Home button to return to the Online Network for Evaluation
Home page. Reenter the Training tab to access the next assigned set.
Page 7 of 9

Calibration
Following the last feedback set, select the calibration link and score the
exam set of 15 responses. If you make a passing score on the first calibration
set, you do not have to take the second set. To qualify to score this project,
you must achieve 80% exact agreement on one of the two sets. After you
complete the calibration set, you will be logged out of ONE. PASS will contact
you about your next steps.

Viewing scanned responses

During production scoring, the following buttons are available to you for viewing
scanned hand-written responses. These buttons may not be available during
training.

Note: Handwriting is not a basis for a scoring decision. Do not allow handwriting to
influence the score you assign. Be sure to read the Avoiding Personal Bias in
Page 8 of 9

Scoring document in the Program Specific Information section under the

Training tab.

Page 9 of 9