Basic Pipe Lining

CSCE 430/830 Computer Architecture Basic Pipelining & Performance
Adopted from Professor David Patterson

Electrical Engineering and Computer Sciences University of California, Berkeley
Outline
MIPS An ISA for Pipelining 5 stage pipelining Structural and Data Hazards Forwarding Branch Schemes Exceptions and Interrupts Conclusion
2/12/2012
CSCE 430/830, Basic Pipelining & Performance
A "Typical" RISC ISA

32-bit fixed format instruction (3 formats) 32 32-bit GPR (R0 contains zero, DP take pair) 3-address, reg-reg arithmetic instruction Single address mode for load/store: base + displacement
no indirection
Simple branch conditions Delayed branch

see: SPARC, MIPS, HP PA-Risc, DEC Alpha, IBM PowerPC, CDC 6600, CDC 7600, Cray-1, Cray-2, Cray-3
2/12/2012
Example: MIPS (- MIPS)

Register-Register
31 26 25 21 20 16 15 11 10 6 5 0
Op
31
Rs1
26 25 21 20
Rs2
Rd
Opx
0
Register-Immediate
16 15
Op Branch
31
Rs1
26 25 21 20
Rd
16 15
immediate
Op Jump / Call
31
Rs1
Rs2/Opx
immediate
26 25
Op
target
2/12/2012
Datapath vs Control
Datapath Controller
signals
Control Points
Datapath: Storage, FU, interconnect sufficient to perform the desired functions

Inputs are Control Points Outputs are signals
Controller: State machine to orchestrate operation on the data path

Based on desired function and signals 2/12/2012 CSCE 430/830, Basic Pipelining & Performance 5
Approaching an ISA
Instruction Set Architecture
Defines set of operations, instruction format, hardware supported data types, named storage, addressing modes, sequencing
Meaning of each instruction is described by RTL (Register Transfer Language) on architected registers and memory Given technology constraints assemble adequate datapath
Architected storage mapped to actual storage Function units to do all the required operations Possible additional storage (eg. MAR, MBR, ) Interconnect to move information among regs and FUs
Map each instruction to sequence of RTLs Collate sequences into symbolic controller state transition diagram (STD) Lower symbolic STD to control points Implement controller
2/12/2012 CSCE 430/830, Basic Pipelining & Performance 6
5 Steps of MIPS Datapath

Figure A.2, Page A-8 Instruction Fetch
Next PC
Instr. Decode Reg. Fetch
Execute Addr. Calc
Memory Access
MUX
Write Back
Adder
Next SEQ PC
Zero?
RS1
4
Address
PC <= PC + 4
2/12/2012
MUX MUX
IR <= mem[PC];
Reg[IRrd] <= Reg[IRrs1] opIRop Reg[IRrs2]
Memory
Reg File
RS2
Inst
ALU
Data Memory
RD
L M D
MUX
Imm
Sign Extend
WB Data

Next PC

Next SEQ PC
Execute Addr. Calc

Next SEQ PC
Memory Access
MUX
Write Back
Adder
RS1
4
Address
PC <= PC + 4
2/12/2012
Zero?
MUX MUX
MEM/WB
Imm
Sign Extend
A <= Reg[IRrs1]; B <= Reg[IRrs2] rslt <= A opIRop B WB <= result Reg[IRrd] <= WB
RD
RD
RD
WB Data
IR <= mem[PC];
Memory
EX/MEM
Reg File
RS2
ID/EX
IF/ID
ALU
RD
Data Memory
MUX
Inst. Set Processor Controller

IR <= mem[PC]; PC <= PC + 4 Ifetch
JSR br
A <= Reg[IRrs1];
opFetch-DCD
JR jmp
B <= Reg[IRrs2]
ST RI LD
r <= A + IRim
RR
PC <= IRjaddr r <= A opIRop B
if bop(A,b) PC <= PC+IRim
r <= A opIRop IRim
WB <= r
WB <= r
WB <= Mem[r]
Reg[IRrd] <= WB
Reg[IRrd] <= WB
Reg[IRrd] <= WB

Next PC

Next SEQ PC
Execute Addr. Calc

Next SEQ PC
Memory Access
MUX
Write Back
Adder
RS1
4
Address local 2/12/2012
Zero?
MUX MUX
MEM/WB
Imm
Sign Extend
RD
RD
RD
Data stationary control

decode forCSCE 430/830, Basic Pipelining & Performance each instruction phase / pipeline stage
10
WB Data
Memory
EX/MEM
Reg File
RS2
ID/EX
IF/ID
ALU
Data Memory
MUX
Visualizing Pipelining
Figure A.2, Page A-8 Time (clock cycles)
Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7
ALU
I n s t r. O r d e r
ALU
Ifetch
Reg
DMem
Reg
Ifetch
Reg
DMem
Reg
ALU
Ifetch
Reg
DMem
Reg
ALU
Ifetch
Reg
DMem
Reg
2/12/2012
11
Instruction-Level Parallelism
Review of Pipelining (the laundry analogy)
2/12/2012
12
Pipelining is not quite that easy!
Limits to pipelining: Hazards prevent next instruction from executing during its designated clock cycle
Structural hazards: HW cannot support this combination of instructions (single person to fold and put clothes away) Data hazards: Instruction depends on result of prior instruction still in the pipeline (missing sock) Control hazards: Caused by delay between the fetching of instructions and decisions about changes in control flow (branches and jumps).
2/12/2012
13
One Memory Port/Structural Hazards

Figure A.4, Page A-14
Time (clock cycles)

ALU
I Load Ifetch n s Instr 1 t r. O r d e r
ALU
Reg
DMem
Reg
ALU
Ifetch
Reg
DMem
Reg
Instr 2 Instr 3 Instr 4

2/12/2012
Ifetch
Reg
DMem
Reg
ALU
Ifetch
Reg
DMem
Reg
ALU
Ifetch
Reg
DMem
Reg
14
One Memory Port/Structural Hazards

(Similar to Figure A.5, Page A-15)
Time (clock cycles)

ALU
I Load Ifetch n s Instr 1 t r. O r d e r
ALU
Reg
DMem
Reg
ALU
Ifetch
Reg
DMem
Reg
Instr 2 Stall Instr 3
Ifetch
Reg
DMem
Reg
Bubble
Bubble Bubble
Bubble
ALU
Bubble
Ifetch
Reg
DMem
Reg
How do you bubble the pipe?

Speed Up Equation for Pipelining
CPIpipelined ! Ideal CPI Average Stall cycles per Inst Cycle Timeunpipelined Ideal CPI v Pipeline depth Speedup ! v Ideal CPI Pipeline stall CPI Cycle Timepipelined
For simple RISC pipeline, CPI = 1:

Cycle Timeunpipelined Pipeline depth Speedup ! v 1 Pipeline stall CPI Cycle Timepipelined
Example: Dual-port vs. Single-port

Machine A: Dual ported memory (Harvard Architecture) Machine B: Single ported memory, but its pipelined implementation has a 1.05 times faster clock rate Ideal CPI = 1 for both Loads are 40% of instructions executed
SpeedUpA = Pipeline Depth/(1 + 0) x (clockunpipe/clockpipe) = Pipeline Depth SpeedUpB = Pipeline Depth/(1 + 0.4 x 1) x (clockunpipe/(clockunpipe / 1.05) = (Pipeline Depth/1.4) x 1.05 = 0.75 x Pipeline Depth SpeedUpA / SpeedUpB = Pipeline Depth/(0.75 x Pipeline Depth) = 1.33
Machine A is 1.33 times faster

Data Hazard on R1
Time (clock cycles)

IF ID/RF EX MEM
DMem
WB
Reg
add r1,r2,r3 sub r4,r1,r3 and r6,r1,r7 or
Ifetch
Reg
ALU
ALU
Ifetch
Reg
DMem
Reg
ALU
Ifetch
Reg
DMem
Reg
ALU
r8,r1,r9
Ifetch
Reg
DMem
Reg
ALU
xor r10,r1,r11
2/12/2012
Ifetch
Reg
DMem
Reg
18
Three Generic Data Hazards

Read After Write (RAW) InstrJ tries to read operand before InstrI writes it I: add r1,r2,r3 J: sub r4,r1,r3 Caused by a Dependence (in compiler nomenclature). This hazard results from an actual need for communication.
2/12/2012
19

Write After Read (WAR) InstrJ writes operand before InstrI reads it I: sub r4,r1,r3 J: add r1,r2,r3 K: mul r6,r1,r7 Called an anti-dependence by compiler writers. This results from reuse of the name r1. Can it happen in MIPS 5 stage pipeline? All instructions take 5 stages, and Reads are always in stage 2, and Writes are always in stage 5

Write After Write (WAW) InstrJ writes operand before InstrI writes it. I: sub r1,r4,r3 J: add r1,r2,r3 K: mul r6,r1,r7 Called an output dependence by compiler writers This also results from the reuse of name r1. Cant happen in MIPS 5 stage pipeline because: All instructions take 5 stages, and Writes are always in stage 5 Will see WAR and WAW in more complicated pipes
Forwarding to Avoid Data Hazard

Figure A.7, Page A-19 Time (clock cycles) I n s t r. O r d e r
add r1,r2,r3 Ifetch sub r4,r1,r3 and r6,r1,r7 or r8,r1,r9
ALU
Reg
DMem
Reg
ALU
Ifetch
Reg
DMem
Reg
ALU
Ifetch
Reg
DMem
Reg
ALU
Ifetch
Reg
DMem
Reg
ALU
xor r10,r1,r11
2/12/2012
Ifetch
Reg
DMem
Reg
22
HW Change for Forwarding

NextPC
mux
Immediate
2/12/2012
Registers
MEM/WR
EX/MEM
ALU
What circuit detects and resolves this hazard?

CSCE 430/830, Basic Pipelining & Performance 23
ID/EX
Data Memory
mux
mux
Forwarding to Avoid LW-SW Data Hazard

Time (clock cycles) I n s t r. O r d e r
add r1,r2,r3 Ifetch lw r4, 0(r1) sw r4,12(r1) or r8,r6,r9
ALU
Reg
DMem
Reg
ALU
Ifetch
Reg
DMem
Reg
ALU
Ifetch
Reg
DMem
Reg
ALU
Ifetch
Reg
DMem
Reg
ALU
xor r10,r9,r11
2/12/2012
Ifetch
Reg
DMem
Reg
Any hazard that cannot be avoided with forwarding?

Data Hazard Even with Forwarding

Time (clock cycles)

ALU
lw r1, 0(r2)Ifetch sub r4,r1,r6 and r6,r1,r7 or r8,r1,r9
Reg
DMem
Reg
ALU
Ifetch
Reg
DMem
Reg
ALU
Ifetch
Reg
DMem
Reg
ALU
Ifetch
Reg
DMem
Reg
2/12/2012
25
Data Hazard Even with Forwarding

(Similar to Figure A.10, Page A-21)
Time (clock cycles) I n s t r. O r d e r
lw r1, 0(r2) sub r4,r1,r6 and r6,r1,r7 or r8,r1,r9

2/12/2012
ALU
Ifetch
Reg
DMem
Reg
How is this detected?

DMem Reg
Ifetch
Reg
Bubble
ALU
Ifetch
Bubble
ALU
Reg
DMem
Reg
Bubble
Ifetch
Reg
ALU
DMem
26
Software Scheduling to Avoid Load Hazards

Try producing fast code for a = b + c; d = e f; assuming a, b, c, d ,e, and f in memory.
Slow code: LW LW ADD SW LW LW SUB SW Rb,b Rc,c Ra,Rb,Rc a,Ra Re,e Rf,f Rd,Re,Rf d,Rd Fast code: LW LW LW ADD LW SW SUB SW Rb,b Rc,c Re,e Ra,Rb,Rc Rf,f a,Ra Rd,Re,Rf d,Rd
Compiler optimizes for performance. Hardware checks for safety.

Outline
Review Quantify and summarize performance
Ratios, Geometric Mean, Multiplicative Standard Deviation
F&P: Benchmarks age, disks fail,1 point fail danger MIPS An ISA for Pipelining 5 stage pipelining Structural and Data Hazards Forwarding Branch Schemes Exceptions and Interrupts Conclusion
2/12/2012
Control Hazard on Branches Three Stage Stall

ALU
10: beq r1,r3,36 14: and r2,r3,r5 18: or r6,r1,r7
Ifetch
Reg
DMem
Reg
ALU
Ifetch
Reg
DMem
Reg
ALU
Ifetch
Reg
DMem
Reg
ALU
22: add r8,r1,r9 36: xor r10,r1,r11
Ifetch
Reg
DMem
Reg
ALU
Ifetch
Reg
DMem
Reg
What do you do with the 3 instructions in between? How do you do it? Where is the commit?
Branch Stall Impact

If CPI = 1, 30% branch, Stall 3 cycles => new CPI = 1.9! Two part solution:
Determine branch outcome sooner, AND Compute taken branch (target) address earlier
MIPS branch tests if register = 0 or { 0 MIPS Solution:

Move Zero test to ID/RF stage Adder to calculate new PC in ID/RF stage 1 clock cycle penalty for branch versus 3
2/12/2012
30
Pipelined MIPS Datapath

Figure A.24, page A-38 Instruction Fetch
Next PC

Next SEQ PC
Execute Addr. Calc
Memory Access
Write Back
MUX
Adder
Adder
4
Address
2/12/2012
Zero?
RS1
MEM/WB
Imm
Sign Extend
RD
RD
RD
Interplay of instruction set design and cycle time.

WB Data
Memory
EX/MEM
RS2
Reg File
ID/EX
ALU
IF/ID
Data Memory
MUX
MUX
Four Branch Hazard Alternatives

#1: Stall until branch direction is clear #2: Predict Branch Not Taken
Execute successor instructions in sequence Squash instructions in pipeline if branch actually taken Advantage of late pipeline state update 47% MIPS branches not taken on average PC+4 already calculated, so use it to get next instruction
#3: Predict Branch Taken

53% MIPS branches taken on average But havent calculated branch target address in MIPS MIPS still incurs 1 cycle branch penalty Other machines: branch target known before outcome
2/12/2012
32
Four Branch Hazard Alternatives
#4: Delayed Branch

Define branch to take place AFTER a following instruction branch instruction sequential successor1 sequential successor2 ........ sequential successorn branch target if taken
Branch delay of length n
1 slot delay allows proper decision and branch target address in 5 stage pipeline MIPS uses this
2/12/2012
33
Scheduling Branch Delay Slots (Fig A.14)

A. From before branch add $1,$2,$3 if $2=0 then delay slot B. From branch target sub $4,$5,$6 add $1,$2,$3 if $1=0 then delay slot becomes C. From fall through add $1,$2,$3 if $1=0 then delay slot sub $4,$5,$6 becomes add $1,$2,$3 if $1=0 then sub $4,$5,$6
becomes if $2=0 then add $1,$2,$3
add $1,$2,$3 if $1=0 then sub $4,$5,$6
A is the best choice, fills delay slot In B, the sub instruction may need to be copied, increasing IC In B and C, must be okay to execute sub when branch fails
Delayed Branch
Compiler effectiveness for single branch delay slot:
Fills about 60% of branch delay slots About 80% of instructions executed in branch delay slots useful in computation About 50% (60% x 80%) of slots usefully filled
Delayed Branch downside: As processor go to deeper pipelines and multiple issue, the branch delay grows and need more than one delay slot
Delayed branching has lost popularity compared to more expensive but more flexible dynamic approaches Growth in available transistors has made dynamic approaches relatively cheaper
2/12/2012
35
Evaluating Branch Alternatives

Pipeline speedup = Pipeline depth 1 +Branch frequencyvBranch penalty
Assume 4% unconditional branch, 6% conditional branchuntaken, 10% conditional branch-taken Scheduling Branch CPI speedup v. speedup v. scheme penalty unpipelined stall Stall pipeline 3 1.60 3.1 1.0 Predict taken 1 1.20 4.2 1.33 Predict not taken 1 1.14 4.4 1.40 1.45 Delayed branch 0.5 1.10 4.5
2/12/2012
36
More branch evaluations

Suppose the branch frequencies (as percentage of all instructions) of 15% cond. Branches, 1% jumps and calls, and 60% cond. Branches are taken. Consider a 4-stage pipeline where branch is resolved at the end of the 2nd cycle for uncond. Branches and at the end of the 3rd cycle for cond. Branches. How much faster would the machine be without any branch hazards, ignoring other pipeline stalls? Pipeline speedupideal = Pipeline depth/(1+Pipeline stalls) = 4/(1+0) = 4 Pipeline stallsreal = (1x1%) + (2x9%) + (1x6%) = 0.24 Pipeline speedupreal = 4/(1+0.24) = 3.23 Pipeline speedupwithout control
2/12/2012
hazards
= 4/3.23 = 1.24
37
More branch question

A reduced hardware implementation of the classic 5-stage RISC pipeline might use the EX stage hardware to perform a branch instruction comparison and then not actually deliver the branch target PC to the IF stage until the clock cycle in which the branch reaches the MEM stage. Control hazard stalls can be reduced by resolving branch instructions in ID, but improving performance in one aspect may reduce performance in other circumstances. How does determining branch outcome in the ID stage have the potential to increase data hazard stall cycles?
2/12/2012
38
Problems with Pipelining

Exception: An unusual event happens to an instruction during its execution
Examples: divide by zero, undefined opcode
Interrupt: Hardware signal to switch the processor to a new instruction stream

Example: a sound card interrupts when it needs more audio output samples (an audio click happens if it is left waiting)
Problem: It must appear that the exception or interrupt must appear between 2 instructions (Ii and Ii+1)
The effect of all instructions up to and including Ii is totally complete No effect of any instruction after Ii can take place
The interrupt (exception) handler either aborts program or restarts at instruction Ii+1
Precise Exceptions in Static Pipelines
Key observation: architected state only change in memory and register write stages.
And In Conclusion: Control and Pipelining

Quantify and summarize performance
Ratios, Geometric Mean, Multiplicative Standard Deviation
F&P: Benchmarks age, disks fail,1 point fail danger Next time: Read Appendix A, record bugs online! Control VIA State Machines and Microprogramming Just overlap tasks; easy if tasks are independent Speed Up e Pipeline Depth; if ideal CPI is 1, then:
Cycle Timeunpipelined Pipeline depth Speedup ! v 1 Pipeline stall CPI Cycle Timepipelined
Hazards limit performance on computers:
Structural: need more HW resources Data (RAW,WAR,WAW): need forwarding, compiler scheduling Control: delayed branch, prediction
Exceptions, Interrupts add complexity Next time: Read Appendix C, record bugs online!

Basic Pipe Lining

Transféré par

Informations du document

Description originale:

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Basic Pipe Lining

Transféré par

Droits d'auteur :

Formats disponibles

CSCE 430/830 Computer Architecture Basic Pipelining & Performance

Adopted from Professor David Patterson

CSCE 430/830, Basic Pipelining & Performance

A "Typical" RISC ISA

Simple branch conditions Delayed branch

CSCE 430/830, Basic Pipelining & Performance

Example: MIPS (- MIPS)

CSCE 430/830, Basic Pipelining & Performance

Datapath: Storage, FU, interconnect sufficient to perform the desired functions

Controller: State machine to orchestrate operation on the data path

5 Steps of MIPS Datapath

Instr. Decode Reg. Fetch

Execute Addr. Calc

Reg[IRrd] <= Reg[IRrs1] opIRop Reg[IRrs2]

CSCE 430/830, Basic Pipelining & Performance

5 Steps of MIPS Datapath

Instr. Decode Reg. Fetch

Execute Addr. Calc

CSCE 430/830, Basic Pipelining & Performance

Inst. Set Processor Controller

if bop(A,b) PC <= PC+IRim

r <= A opIRop IRim

CSCE 430/830, Basic Pipelining & Performance

5 Steps of MIPS Datapath

Instr. Decode Reg. Fetch

Execute Addr. Calc

Data stationary control

CSCE 430/830, Basic Pipelining & Performance

CSCE 430/830, Basic Pipelining & Performance

Pipelining is not quite that easy!

CSCE 430/830, Basic Pipelining & Performance

One Memory Port/Structural Hazards

Time (clock cycles)

I Load Ifetch n s Instr 1 t r. O r d e r

Instr 2 Instr 3 Instr 4

CSCE 430/830, Basic Pipelining & Performance

One Memory Port/Structural Hazards

Time (clock cycles)

I Load Ifetch n s Instr 1 t r. O r d e r

Instr 2 Stall Instr 3

How do you bubble the pipe?

Speed Up Equation for Pipelining

For simple RISC pipeline, CPI = 1:

Example: Dual-port vs. Single-port

Machine A is 1.33 times faster

Time (clock cycles)

add r1,r2,r3 sub r4,r1,r3 and r6,r1,r7 or

CSCE 430/830, Basic Pipelining & Performance

Three Generic Data Hazards

CSCE 430/830, Basic Pipelining & Performance

Three Generic Data Hazards

Three Generic Data Hazards

Forwarding to Avoid Data Hazard

add r1,r2,r3 Ifetch sub r4,r1,r3 and r6,r1,r7 or r8,r1,r9

CSCE 430/830, Basic Pipelining & Performance

HW Change for Forwarding

What circuit detects and resolves this hazard?

Forwarding to Avoid LW-SW Data Hazard

Time (clock cycles) I n s t r. O r d e r

add r1,r2,r3 Ifetch lw r4, 0(r1) sw r4,12(r1) or r8,r6,r9

Any hazard that cannot be avoided with forwarding?

Data Hazard Even with Forwarding

Time (clock cycles)