Computer Interface Technologies

PCI Express

Professor Dr.-Ing. Sven Simon

Institute for Parallel and Distributed Systems
University of Stuttgart

S. Simon 1
 Overview (1/2)
• computer evolution
• history of the IO Bus
• PCI-SIG
• PCI-Express
• Lanes
• transfer rates
Logo
• PCI Express for Graphics reference: www.pcisig.com

• compatibility
• advantages
• PCI-Express usage

S. Simon 2
 Overview (2/2)
• Design and components
• OSI-Modell
• Overview
• Comparisons
• Physical Layers
• Phases in detail
• Transactions
• Concept
• Example

S. Simon 3
 Rapid computer evolution
• data medium size from kB to TB

reference: http://wikipedia.de

• network transfer rates from 10 Mbit/s to 10 Gbit/s

• high amount of data in graphical applications

reference: http://verkaufsbilder.pc-interaktiv.de/

reference: http://www.osxbook.com/

S. Simon 4
 Rapid computer evolution
• raising processorspeed

reference: bw.fh-deggendorf.de
• multicoreprocessing
• I/O-bus getting more and more a bottleneck

S. Simon 5
 History of the IO Bus
• ISA (Industry Standard Architecture)
• developed in 1979
• first use with 8086 and 8088 Intel
processor
• 16,66 MByte/s
3xVLB, 3xISA 16Bit, 1xISA 8Bit
• VLB (Vesa Local Bus) reference: http://dic.academic.ru

• developed in 1992
• based on Intel 80486 CPUs
• 276 MByte/s
• AGP (Accelerated Graphics Port)
• directly connected to North-
Bridge
• independent from PCI
• 266 MByte/s - 2,1 GByte/s
reference: http://ist94.wikispaces.com

S. Simon 6
 History of the IO Bus
• PCI (Peripheral Component Interconnect)
• displaced VLB
• PCI 1.0 (1991) – PCI 3.0 (2004)
• 133 MByte/s - 532 MByte/s
• PCI-X (PCI-Extended)
• high performance 64 bit designed for
servers 1xPCI, 2xPCIe 1x, 1xPCIe 15x
reference: weblearn.hs-bremen.de

• PCI-X 1.0 (1998) - PCI-X 2.0 (2003)

• 533 MByte/s - 4,3 GByte/s

PCI-X Card
reference: wikipedia.de

S. Simon 7
 History of the IO Bus
• PCIe/PCI-E (PCI-Express)
• formerly known as 3GIO
• PCIe 1.0 (2002) – PCI3.0 (2009?)
• 250MByte/s – 32GByte/s
• more lanes -> higher datarate
• 8B10B-coding for clock recovery
• 8B10B code = 20% overhead

PCIe Datarates
reference: wikipedia.de

S. Simon 8
 PCI-SIG

Logo
reference:
www.pcisig.com

• „Peripheral Component Interconnect Special Interest Group“

• established in 1992
• challenge: administration and further development of the PCI-
standards
• companies and organisations can become a member
• more then 900 members
• reputated companies like Intel, AMD, Microsoft,
Dell, HP, and so on

reference: www.pcisig.com

S. Simon 9
 PCI
• parallel bus-system
• used for network cards, sound cards,
modems, USB-Ports,
TV tuner cards and disk controllers
• connected components
have to share the bandwith

reference: weblearn.hs-bremen.de

reference: http://dell.com

S. Simon 10
 PCI-Express
• high-speed serial bus-system
• data transfer with lanes
• point to point connection
• communication through switches
• an important characteristic
of the PCI-Express-Standard
is the softwarecompatibility to PCI
• number of required conductors on cards or on the mainboard is
intensively reduced reference: weblearn.hs-bremen.de

reference: dell.com

S. Simon 11
 Lanes
• each lane with two unidirectional pairs of lines
• full-duplex
• 250 MByte/s in each direction
• coupling of several lanes
can be increase speed,
e.g. 2 lanes = 500MByte/s

reference: arstechnica.com

S. Simon 12
 PCI Express for Graphics
• PEG (PCI Express for Graphics)
• standard PCI-Express-slot max. 25 Watt
• PCIe-x16 slot
• up to 75 Watt
• direct connection to power supply
• 6 pole additionally 75 Watt
• 8 pole additionally 150 Watt

8 pole power connector

reference: ht4u.net

power consumption of grafic cards

reference: www.tecchannel.de

S. Simon 13
 Hardware
• easy layout
• serpentine routing uncomplicated, only aline pair of a lane have
to be exactly layouted
meander
• pair of lane may not exceed 50,8 cm reference: www.s-t-e.de

• due to the lower lines, less layers are required

• space- and cost saving

conductor board
reference: www.leiterplatten-print.de

S. Simon 14
 PCI/PCI-Express compatibility
• software-compatibility to PCI-standard
• Neither operating system, nor the application has to be
adapted
• configurationspace (initialising- and configurationdata)
• PCI 256 Byte
• PCI-Express 4096 Byte
• PCI and PCI-Express hardware
not compatible

reference: www.ht4u.net reference: www.scantec.de

S. Simon 15
 PCI-Express Kompatibilität
Cars/Slot PCIe x1 PCIe x4 PCIe x8 PCIe x16
PCI Express x1 Ok Ok Ok Ok
PCI Express x4 - Ok * *
PCI Express x8 - - Ok *
PCI Express x16 - - - Ok
Ok = compatible / - = incompatible / * = not regulated, but possible

PCIe-slots on a mainboard
reference: zone.ni.com

S. Simon 16
 PCI-Express-bridge
• in the transitional period PCI-Express-bridges are needed for
reducing the costs
• AGP to PCIe for using available graphic chips on the PCIe
architecture
• PCIe to AGP e.g. gor using new graphic chips with the AGP-
slot

reference: www.scantec.de

S. Simon 17
 Advantage PCI-Express
• software compatibility
• scalability
• higher datarates
• lower costs due to serial bus system
• universal use
• I/O- connections
• chip-to-chip-interconnect
• hot plug capable
• quality of service: virtual channels with priorities
• realtime applications
• plug & play
• direct communication between
PCIe components due to switches
reference: www.pcwissen.eu

S. Simon 18
 PCI-Express usage
• Mini-PCI-Express
• onboard extensions for notebooks and PCs
• higher bandwith
• lower dimensions

reference: www.wikipedia.de

S. Simon 19
 PCI-Express usage
• ExpressCard (NewCard)
• PCMCIA is being replaced
• with the high bandwith new range of applications
(dockingstation etc.)
• tasks:
network,
soundcards,
measuringcards,
SATA-, USB-
and Firewire-adapter

Quelle: www.wikipedia.de
reference: www.wikipedia.de

S. Simon 20
 PCI-Express usage
• Solid State Drive (SSD)
• up to Terabyte region
• transferrate up to 750 MByte/s
• by comparison S-ATA up to 300 MByte/s

• high-performance graphic cards reference: img.boot.lv

• Multi-GPU technology by using

SLI (Scalable Link Interface)

reference: www.notebookjournal.de

S. Simon 21
 Overview
• PCI-Express
• ….
• PCI-Express useage
• Design and components
• OSI-Modell
• Overview
• Comparisons
• Physical Layers
• Phases in detail
• Transactions
• Concept
• Example

S. Simon 22
 Design and components (1/2)
• Schematic outline
CPU
• Simple tree like structure
PCIe
• responsible for system End- PCIe
configuration, enumeration of point
PCIe resources, manages Root
Complex Memory
interrupts and errors
(Hub)

PCIe PCIe

PCIe to
PCI
Bridge PCIe
Switch
PCIe
PCIe
End-
…
…

point
Legacy PCIe
End- PCIe
PCI, PCI-X
point
PCIe
Legacy End-
End- point
point

Based on Introduction to PCI Express - Figure 5.3

Jürgen Räuchle, Alexander Dridiger 23
23
 Design and components (2/2)
• Root Complex  Graphics & Memory Controller Hub (Northbridge),
+ I/O Controller Hub (Southbridge)
• Endpoint  Component capable of requesting and completing
PCIe Transactions for itself
• PCIe Endpoint („native Endpoint“)
• Legacy Endpoint
• implements special features considered by the PCIe Base
Specification for backward compatibility but are restricted
from using some of the advanced features of PCI Express
• Switch  Link coordination (connections between Lanes)
• PCIe to PCI Bridge  1-to-n-association between PCIe Port and
PCI/PCI-X Interfaces

Jürgen Räuchle, Alexander Dridiger 24

 Overview
• Design and components
• OSI-Model
• Overview
• Comparisons
• Physical Layers
• Phases in detail
• Transactions
• Concept
• Example

S. Simon 25
 OSI-Model (1/3) - PCIe
OSI-Layers PCIe-Layers
Applications
• Very similar layer
Application architecture
• Functionally identical
Presentation Software/OS Layer „Physical Layer“ and “Data
Link Layer“
Session
• „Transaction Layer“
combines transport and
network
Transport • „Application Layer“ implied
by the operating system,
Transport drivers and general
Transaction Layer application software.
Network

Data Link Data Link Layer

Physical Physical Layer

Based on Introduction to PCI Express - Figure 5.4

Jürgen Räuchle, Alexander Dridiger 26
26
 OSI-Model (2/3) - PCI
• Hard map; PCI has no layered architecture
• „Classic“ Bus-System
• Transport is realized by Full Bus Mastering (Master - Target)
• Immediate control over the bus by the device
• Immediate initialization of a transaction (device  Bridge 
CPU)
• Transactions done in phases

Jürgen Räuchle, Alexander Dridiger 27

 OSI-Model (3/3) - PCI
Applications
• „Application
Application Layer“ identical
with PCIe
Presentation Software/OS Layer

Session

Transport

Transport

Network

The „Device“
Data Link

Physical

Based on Introduction to PCI Express - Figure 5.4

Jürgen Räuchle, Alexander Dridiger 28
28
 Overview
• Design and components
• OSI-Model
• Overview
• Comparisons
• Physical Layers
• Phases in detail
• Transactions
• Concept
• Example

S. Simon 29
 Physical Layers - PCIe (1/3)
• Located exactly where data packages are sent and received
• Works with the hardware components of PCIe-Links (Mechanical
Layer); initializes Links
• Prepares sending data per link or passing them to the Data Link
Layer
• 8-bit/10-bit encoding
• Serial/Parallel conversion
• 2-Sub-Blocks:

Physical Layer

Logic

Electric

Based on Introduction to PCI Express - Figure 8.2

Jürgen Räuchle, Alexander Dridiger 30
30
 Physical Layers - PCIe (2/3)
• Physical Layer in detail:
• Differential-CML (electronic driver technology) can be used as
transport technology
• Electric Block functions as the delivery mechanism
• Contains TX- and RX-buffers to transform messages in
electrical signals and back
• Works in various phases
• Logical Block as decision maker with three operations
• Data Scrambling
• 8-bit/10-bit (De-)Encoding
• Packet Framing

Jürgen Räuchle, Alexander Dridiger 31

31
 Phases in detail (1/2)
• Phases of the Electrical Block:
• Serial/Parallel Conversion
• Clock Extraction
• Lane-to-Lane De-Skew
• Differential Signaling
• Phase Locked Loop (PLL) Circuit
• AC Coupling
• De-Emphasis

Jürgen Räuchle, Alexander Dridiger 32

32
 Phases in detail (2/2)
• Phases of the Logical Block:
• Data Scrambling
possibility of electrical resonances on the link is reduced
(caused by repeated data patterns at the system’s preferred
frequency); can optionally be turned off
linear feedback shift register
• 8-Bit/10-Bit Encoding

• Packet Framing

START
Scrambled/Encoded Transaction Layer Paket END
TLP

OR
START
Scrambled/Encoded Data Link Layer Paket END
DLLP

Jürgen Räuchle, Alexander Dridiger 33

33
 Physical Layers - PCI (3/3)
• „Physical components“ are the CPU and the pins

reference: weblearn.hs-bremen.de

Jürgen Räuchle, Alexander Dridiger 34

34
 Index
1. Introduction of PCI Express Protocol

2. The Three Layers Architecture of PCIe

1. Transaction Layer

2. Data link Layer

3. Physical Layer

3. Software Layer

4. Types of Transaction

5. Packets

6. Transaction Layer Packet

S. Simon 35
 Index
10. Error handling

11. Link Training and Initialization

1. Training States

2. Link States

3. Other States

12. Flow control

13. Symbols

14. Summary

S. Simon 36
 Index
1. Introduction of PCI Express Protocol

2. The Three Layers Architecture of PCIe

1. Transaction Layer

2. Data link Layer

3. Physical Layer

3. Software Layer

4. Types of Transaction

5. Packets

6. Transaction Layer Packet

S. Simon 37
 Index
1. Introduction of PCI Express Protocol

2. The Three Layers Architecture of PCIe

1. Transaction Layer

2. Data link Layer

3. Physical Layer

3. Software Layer

4. Types of Transaction

5. Packets

6. Transaction Layer Packet

S. Simon 38
 The Three Layers Architecture of PCIe
• Transaction layer is for beginning the
process of turning request or data from
the device’s core into a PCIe packet

• Secondly comes the Data Link layer,

which is responsible for making sure that
the packet is received properly

• Physical layer is responsible for the actual

transmitting and receiving of the packet
through the PCIe Link

S. Simon 39
 Index
1. Introduction of PCI Express Protocol

2. The Three Layers Architecture of PCIe

1. Transaction Layer

2. Data link Layer

3. Physical Layer

3. Software Layer

4. Types of Transaction

5. Packets

6. Transaction Layer Packet

S. Simon 40
 Transaction Layer
Transaction Layer:

• Is for creating the PCI Express Request and completion

packet

• Transmits the outgoing packet

• It receives -

• Incoming transaction,

• Request data from the device

• Packets from the Data-Link layer

• Uses TLPs to communicate request and completion data

• It contains a header for determining the type of packet,

such as: Read/Write, Memory or Configuration

S. Simon 41
 Index
1. Introduction of PCI Express Protocol

2. The Three Layers Architecture of PCIe

1. Transaction Layer

2. Data link Layer

3. Physical Layer

3. Software Layer

4. Types of Transaction

5. Packets

6. Transaction Layer Packet

S. Simon 42
 Data-Link Layer
• It checks the correctness of the data

• Receives TLPs from the Transaction Layer and

from the Physical Layer

• Gives the TLPs a sequence number

• Forwards the data packets only if

they are correct

• “ACK” vs. “NAK”

S. Simon 43
 Index
1. Introduction of PCI Express Protocol

2. The Three Layers Architecture of PCIe

1. Transaction Layer

2. Data link Layer

3. Physical Layer

3. Software Layer

4. Types of Transaction

5. Packets

6. Transaction Layer Packet

S. Simon 44
 Physical Layer
• sends and receives the data which is sent with
PCIe link

• It interacts with physical PCIe link such as

wires, cable, optical fiber ...

• It contains all circuitry for interface:

• Input, output buffers

• Parallel <> serial converters

• Build out of two parts:

• Logical Physical Layer

• Electrical Physical Layer

S. Simon 45
 Index
1. Introduction of PCI Express Protocol

2. The Three Layers Architecture of PCIe

1. Transaction Layer

2. Data link Layer

3. Physical Layer

3. Software Layer

4. Types of Transaction

5. Packets

6. Transaction Layer Packet

S. Simon 46
 Software Layer
• Has a PCI-compatible configuration mechanism

• Compatibility with Applications and operating

system

• No dependency on operating system

• It provides / receives the information which are

required by the PCIe device to generate TLPs,
such as packet type

• It contains the Device-Core Logic by reusing PCI

core logic design

v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide

S. Simon 47
 Software Layer
• Containing the enhanced
mechanism that increase the
size of available configuration
space

• This prinzip was marketed by

Intel with the name Plug-And-
Play

v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide

S. Simon 48
 Index
1. Introduction of PCI Express Protocol

2. The Three Layers Architecture of PCIe

1. Transaction Layer

2. Data link Layer

3. Physical Layer

3. Software Layer

4. Types of Transaction

5. Packets

6. Transaction Layer Packet

S. Simon 49
 Types of Transaction
• Memory Transaction:

• For transferring the memory space data

• Such like: Memory Read Request, Memory Read Completion, and

Memory Write Request

• it use two different kinds of address format, 32-bit or 64-bit

• I/O Transactions:

• Targeting the I/O space transfer data to or from I/O mapped location

• PCIe uses this for compatibility with existing devices

S. Simon 50
 Types of Transaction
• Configuration Transaction:

• Used for device configuration and setup

• access the config registers of PCIe devices

• Message Transaction:

• For communication with miscellaneous messages between the PCIe

devices

• Example: Used for error signaling

S. Simon 51
 Index
1. Introduction of PCI Express Protocol

2. The Three Layers Architecture of PCIe

1. Transaction Layer

2. Data link Layer

3. Physical Layer

3. Software Layer

4. Types of Transaction

5. Packets

6. Transaction Layer Packet

S. Simon 52
 Packets
• Transaction Layer Packet (TLP)
• Are the means for communicating between
PCI Express devices
• It's build out of Header, Optional
Data Payload, Optional TLP digest
• Generated by Transaction Layer with information
received from its device core
• Data Link Layer Packet (DLLP)
• It´s responsible for the integrity of TLPs
movement, link initialization and power
managment
• Used for passing messages and status
between the Transaction Layer and
the Physical Layer
• Used only for the Local Traffic

S. Simon 53
 Packets

v.g.l: Ravi Budruk , PCI Express System Architecture

S. Simon 54
 Index
1. Introduction of PCI Express Protocol

2. The Three Layers Architecture of PCIe

1. Transaction Layer

2. Data link Layer

3. Physical Layer

3. Software Layer

4. Types of Transaction

5. Packets

6. Transaction Layer Packet

S. Simon 55
 Transaction Layer Packet
• Are the means for communicating
between PCI Express devices

• It's build out of

• Header

• basic identifying information

• Optional Data Payload

• Reciever ID

• Optional TLP digest

• For ECRC Use

• Transaction Layer generate a TLP

v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's

with information received from its Guide

device core
S. Simon 56
 Transaction Layer Packet: Header

v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's

Guide

• It contains basic identifying information for the transaction

• It can be three or four DWords in length

TLP

S. Simon 57
 Transaction Layer Packet: Header

v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's

Guide

• Fmt contains information about header size

• Tells if the TLP contain data payload

• 00b 3DW, no data TLP

• 01b 4DW header, no data

• 10b 3DW header, with data

• 11b 4DW header, with data

S. Simon 58
 Transaction Layer Packet: Header

v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's

Guide

• Type has a size of 5 bits

• Determines type of transaction

• Its used with Fmt Field to specify TLP

• Header size

• Also if data-payload is included

S. Simon 59
 Transaction Layer Packet: Header

v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's

Guide

• TC determines the relative priority of the Transaction

• Traffic classes are :

• 000b = traffic class 0 TLP

…

111b = traffic class 7

S. Simon 60
 Transaction Layer Packet: Header

v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's

Guide

• It determines if the optional DW TLP Digest field is included in the TLP

TLP

S. Simon 61
 Transaction Layer Packet: Header

v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's

Guide

• If it is set to =1 then the data accompanying this data

should be considered invalid and the transaction

is being allowed to complete normally TLP

S. Simon 62
 Transaction Layer Packet: Header

v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide

• Indicating two attributes (modes) for the transaction

• Bit 5 ,the Relaxed ordering

If set to =1 then PCI-X relaxed ordering is

enabled for this TLP

TLP
• Bit 4 ,the No Snoop

If set to =1 then requester indicate that no host

cache coherency issues exist

S. Simon 63
 Transaction Layer Packet: Header

v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide

• Length represent the size of the data payload that is

transferred in this TLP
10
• Maximum size is 10 bits 2 = 1024 DW(4kb)

• 00 0000 0001b = 1DW TLP

….

11 1111 1111b = 1023 DW

00 0000 0000b = 1024 DW

S. Simon 64
 Transaction Layer Packet: Header

v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's

Guide

• The field that marked with R are reserved

and have to be =0

TLP

S. Simon 65
 Transaction Layer Packet: Header

v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide

v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide

S. Simon 66
 Transaction Layer Packet
• CRC, ECRC and LCRC are for providing a method for PCIe
devices to verify the contents of received packets.

• The incoming packet contains the calculation results. The

attached result are compared to the calculation results of the
receiver for checking the packet.

S. Simon 67
 Transaction Layer Packet: Procedure
• Through the flowing of the transaction into the layers, each layer adds
on the specific information.

• Transaction layer make a header and adds the data also a optional
ECRC. The data link layer generate the sequence number and LCRC.
The physical layer frames it for transmission to the other device.

• After it received the complete process occurs reversal.

S. Simon 68
 Transaction Layer Packet: Procedure

S. Simon 69
 Index
10. Error Handling

11. Link Training and Initialization

1. Training States

2. Link States

3. Other States

12. Flow Control

13. Symbols

14. Summary

S. Simon 70
 Error Handling
PCI express defines two error reporting mechanisms:

-Baseline

-Advanced Error Reporting

Baseline:

Baseline defines the minimum error reporting capabilities required by all PCI
express devices

Advanced Error Reporting:

Allows a device to identify each uncorrectable error as either fatal or
nonfatal. This is accomplished via the Uncorrectable Error Severity register.

S. Simon 71
 Error Handling

Classification to correctable and

uncorrectable error types.

Correctable Errors can be recovered by

the protocol without loss of information.

Uncorrectable Errors impact the

functionality of the interface.

v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide

S. Simon 72
 Error Handling

Classification of uncorrectable errors

to fatal and nonfatal errors.

Fatal errors render a link unreliable

and may require a link reset

to return to a reliable state.

Nonfatal errors render (only) a

transaction unreliable.

v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide

S. Simon 73
 Index
10. Error handling

11. Link Training and Initialization

1. Training States

2. Link States

3. Other States

12. Flow control

13. Symbols

14. Summary

S. Simon 74
 Link Initialization

Training states:
Detect

Detects devices and has two substates

(Quiet and Active). Exits to polling if a
device is detected.

Polling

Used to establish different settings.

v.g.l: Ravi Budruk , PCI Express System Architecture

S. Simon 75
 Link Initialization
Training states:
Polling

Polling uses two ordered

training sets (TS1, TS2) to
exchange the necessary
information.

v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide

S. Simon 76
 Link Initialization
Training states:
Polling

Is used for:

-Link data rate exchange

-Clock synchronization

-Checking lane polarity

-Lane numbering configuration

-Forcing resets

-Link Disabling

-Test modes

-Data scrambling. v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide

S. Simon 77
 Link Initialization

Training states:
Configuration

Establishes link width and lane ordering.

Exits to disabled state if one device has

not enough lanes (downward
compatibility)

Recovery

used to re-establish the connection fast.

v.g.l: Ravi Budruk , PCI Express System Architecture

S. Simon 78
 Index
10. Error handling

11. Link Training and Initialization

1. Training States

2. Link States

3. Other States

12. Flow control

13. Symbols

14. Summary

S. Simon 79
 Link Initialization

Link states
L0

Full on

L0s

Standby mode. Needs just several

microseconds to return to L0.

Part of the Active State Power

Management (ASPM)

S. Simon 80
 Link Initialization

Link States
L1

Idle state with phase lock loop circuit

turned off. Also part of ASPM.

L2

auxiliary idle state, with Internal phase

lock loop, external clock source and main
power turned off

(L2/L3 ready: preparatory to L3: power off)

S. Simon 81
 Index
10. Error handling

11. Link Training and Initialization

1. Training States

2. Link States

3. Other States

12. Flow control

13. Symbols

14. Summary

S. Simon 82
 Link Initialization

Other States
Disabled

Used to transition lanes to electric idle

Hot Reset

Used to reset a link without requiring a

full system reset

(External) Loopback

only for lab and debug environment

v.g.l: Ravi Budruk , PCI Express System Architecture

S. Simon 83
 Index
10. Error handling

11. Link Training and Initialization

1. Training States

2. Link States

3. Other States

12. Flow control

13. Symbols

14. Summary

S. Simon 84
 Flow Control
Flow Control guarantees that transmitters never send Transaction Layer
Packets (TLPs) that the receiver can't accept. This is made possible by Flow
Control Buffers that report their available buffer space to the opposite end of
the link. This Buffers are organized in virtual channels.

Flow Control

Flow Control

S. Simon 85
 Flow Control

Flow Control discerns three transactions:

•Posted (posted request header, posted request data)
• Memory Writes and Messages
•Non-Posted (non-posted request header, non-posted request data)
• Memory reads, Configuration Reads and Writes, and I/0 Reads
and Writes
•Completions (completion header, completion data)
• Read Completions and Write Completions

Each virtual channel has independent flow control and thus maintains
independent flow control pools for these types.

S. Simon 86
 Index
10. Error handling

11. Link Training and Initialization

1. Training States

2. Link States

3. Other States

12. Flow control

13. Symbols

14. Summary

S. Simon 87
 Symbols
8b/10b Data Symbols

The 8b/10b encoding achieves DC-balance. Also it embeds a clock into the
code.

A byte value represented by bits HGFEDCBA is broken into two separate bit
streams, mainly HGF and EDCBA .

H G F
0 0 1
H G F E D C B A
25h D for Data D5.1
0 0 1 0 0 1 0 1
E D C B A
0 0 1 0 1
a b c d e f
D5.x 1 0 1 0 0 1
Nach Tabelle
a b c d e f g h i j
1 0 1 0 0 1 1 0 0 1
Dx.1 g h i j
1 0 0 1

S. Simon 88
 Symbols
5b/6b
input RD = −1 RD = +1 input RD = −1 RD = +1

EDCBA abcdei EDCBA abcdei

D.00 0 100111 011000 D.16 10000 011011 100100
D.01 1 011101 100010 D.17 10001 100011
D.02 10 101101 010010 D.18 10010 010011
D.03 11 110001 D.19 10011 110010
D.04 100 110101 001010 D.20 10100 001011
D.05 101 101001 D.21 10101 101010
D.06 110 011001 D.22 10110 011010
D.07 111 111000 000111 D.23 10111 111010 000101
D.08 1000 111001 000110 D.24 11000 110011 001100
D.09 1001 100101 D.25 11001 100110
D.10 1010 010101 D.26 11010 010110
D.11 1011 110100 D.27 11011 110110 001001
D.12 1100 001101 D.28 11100 001110
D.13 1101 101100 D.29 11101 101110 010001
D.14 1110 011100 D.30 11110 011110 100001
D.15 1111 010111 101000 D.31 11111 101011 010100

K.28 11100 001111 110000

S. Simon 89
 Symbols

3b/4b

input RD = −1 RD = +1 input RD = −1 RD = +1

HGF fghj HGF fghj

D.x.0 0 1011 0100 K.x.0 0 1011 0100
D.x.1 1 1001 K.x.1 1 0110 1001
D.x.2 10 0101 K.x.2 1 1010 0101
D.x.3 11 1100 0011 K.x.3 11 1100 0011
D.x.4 100 1101 0010 K.x.4 100 1101 0010
D.x.5 101 1010 K.x.5 1 0101 1010
D.x.6 110 0110 K.x.6 1 1001 0110

D.x.P7 111 1110 0001

D.x.A7 111 0111 1000 K.x.7 111 0111 1000

S. Simon 90
 Symbols
8b/10b Special Symbols
Special symbols are
coded according to the
same process except
that the prefix K is used
instead of D. There are
only 12 special symbols.

v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide

S. Simon 91
 Summary
• PCIe is a layered approach of device design, which contains
Transaction, Data-Link and Physical Layer
• TLPs used for transporting the data through the layers between the
PCIe devices
• DLLPs used for the local communication between the Data-Link
layers
• PCIe devices have a complete software compatibility
• Error Handling happens on all layers and is reported by messages
• Link Training and Initialization is based on a state machine
• Flow Control is an advanced approach for link monitoring

S. Simon 92
 bibliography
[1] Intel Press, Introduction to PCI Express: A Hardware and Software
Developer's Guide

[2] Ravi Budruk, PCI Express System Architecture

[3] Stefan Tauschek, Grundlagen PCI Express

S. Simon 93