Vous êtes sur la page 1sur 123

1

Attention
The material contained in this presentation is the property of Avnet Electronics Marketing. Use of this material in its whole or in part is restricted to Avnets X-Fest program and Avnet employees. Any use by non-Avnet employees outside of the X-Fest program is prohibited. For additional information, please contact Jim Beneke at Avnet (jim.beneke@avnet.com).

Copyright 2009. Avnet, Inc. All rights reserved.

Designing with the Spartan-6 Gigabit Transceivers and the PCIe Endpoint Block

Objectives
Introduce engineers to the Xilinx Spartan-6 RocketIO GTP transceiver and integrated block for PCIe Provide an overview of the GTP and PCIe Endpoint Block design and verification tools At the end of the presentation, engineers will learn Basic understanding of the serial communications The basic architecture of the Spartan-6 GTP and PCIe Endpoint Block What Xilinx tools to use to design with the Spartan-6 GTP and PCIe Endpoint Block

Copyright 2009. Avnet, Inc. All rights reserved.

Agenda
Spartan-6 LXT Platform Family Introduction to Serial Communications Introduction to Spartan-6 GTP Transceivers Designing with Spartan-6 GTP Transceivers Xilinx PCIe solutions Introduction to Spartan-6 PCIe Endpoint Block Designing with Spartan-6 PCIe Endpoint Block PCIe Compliance Testing Wrap-Up

Copyright 2009. Avnet, Inc. All rights reserved.

Agenda
Spartan-6 LXT Platform Family Introduction to Serial Communications Introduction to Spartan-6 GTP Transceivers Designing with Spartan-6 GTP Transceivers Xilinx PCIe solutions Introduction to Spartan-6 PCIe Endpoint Block Designing with Spartan-6 PCIe Endpoint Block PCIe Compliance Testing Wrap-Up

Copyright 2009. Avnet, Inc. All rights reserved.

Spartan-6 Platform Family


Spartan-6 is the 3rd generation of Advanced Silicon Modular Block (ASMBL) architecture Virtex-4, 1st generation Virtex-5, 2nd generation Spartan-6 sub-families LX : High-performance logic and parallel IO LXT: Logic-oriented with serial capabilities Users can choose the best mix of resources to optimize cost and performance LXT Platform LX Platform

Copyright 2009. Avnet, Inc. All rights reserved.

Spartan-6 LXT Platform Family Embedded Features


High-speed serial GTP transceivers Up to 3.125Gbps serial data rate Depending on the package, LXT devices support 2-8 GTP transceivers

PCI Express endpoint block One PCI Express Endpoint Block PCI Express Gen1 at 2.5Gbps

Memory Controller Up to 4 memory controllers Support for DDR, DDR2, DDR3, and Mobile DDR memories DDR2 and DDR3 at 800Mbps DDR and Mobile DDR at 400Mbps Higher performance and lower power than a soft memory controller

MCB

DDR DDR2 DDR3 LPDDR

Spartan-6

Copyright 2009. Avnet, Inc. All rights reserved.

Spartan-6 LXT Platform Family


LX25T Logic Cells Slices FPGA Flip-Flops 18k BRAM Blocks BRAM Kbits Clock Managers (DCM/PLL) DSP48A Multipliers Hardened PCI Express Lanes High Speed External Memory Ports Package CSG324 FG484 CSG484 FG676 FG900 SerDes Lanes Size 15x15mm 23x23mm 19x19mm 27x27mm 31x31mm
x,y

LX45T 44K 6,822 54K 116 2.1K 8/4 58 1 2 4 190,4 296,4 296,4

LX75T 75K 11,662 93K 172 3.1K 12/6 132 1 4 8

LX100T 101K 15,822 126K 268 4.8K 12/6 180 1 4 8

LX150T 148K 23,038 184K 268 4.8K 12/6 180 1 4 8

24K 3,750 30K 52 936 4/2 38 1 2 2 190,2 250,2

268,4 292,4 348,8

296,4 296,4 376,8 498,8

296,4 296,4 396,8 540,8

x = SelectIO, y = GTP Transceivers

Copyright 2009. Avnet, Inc. All rights reserved.

Agenda
Spartan-6 LXT Platform Family Introduction to Serial Communications Introduction to Spartan-6 GTP Transceivers Designing with Spartan-6 GTP Transceivers Xilinx PCIe solutions Introduction to Spartan-6 PCIe Endpoint Block Designing with Spartan-6 PCIe Endpoint Block PCIe Compliance Testing Wrap-Up

Copyright 2009. Avnet, Inc. All rights reserved.

Serial Signaling
Serial signaling is the preferred choice in all markets such as telecom, datacom, computing, and storage Supports very high multi-gigabit data rates Avoids clock/data skew by using embedded clock Reduces EMI & power consumption It is the only IO technology that meets today's high-speed requirements It is faster than parallel signaling Parallel I/O data rates are inherently limited due to unavoidable skew between clock lines and multiple data lines

Device A Parallel Data

Device B Parallel Data

User Logic

SerDes

Multi-Gbps

SerDes

User Logic

Copyright 2009. Avnet, Inc. All rights reserved.

Clock and Data Recovery (CDR)


A PLL is used to generate the receive sampling clock (RCLK) PLL first locks to input reference clock (frequency locking) PLL then looks for transitions on RX serial input to lock to the frequency and phase of the incoming data stream (phase locking)
SERDES RXP 2.5Gbps RXN
PLL

10

D RCLK

RXDATA

Reference Clock @ 125MHz

RXP/N RCLK

RXDATA

Copyright 2009. Avnet, Inc. All rights reserved.

Byte Alignment
Byte alignment is necessary to convert the serial data stream to parallel data Receiver looks for and aligns to a pre-defined alignment pattern
Transmit Order

11

Pre-defined Alignment Pattern

Copyright 2009. Avnet, Inc. All rights reserved.

Misalignment
If the byte alignment pattern is not unique, misalignment can happen Special encoding such as 8B/10B is often used to ensure the alignment pattern does not occur in regular data sequence
Transmit Order Intended Sequence

12

Pre-defined Alignment Pattern

Copyright 2009. Avnet, Inc. All rights reserved.

8B/10B Data Encoding


8B/10B takes an 8-bit input and encodes it to a 10-bit output 256 data patterns are defined (data 0x00 0xFF) 12 special characters (K-characters) are defined for control K28.1 and K28.5 K-characters are commonly used for alignment (Comma Characters)
S pecial C ode N am e K 28.0 K 28.1 K 28.2 K 28.3 K 28.4 K 28.5 K 28.6 K 28.7 K 23.7 K 27.7 K 29.7 K 30.7 B its HGF EDCBA 000 11100 001 11100 010 11100 011 11100 100 11100 101 11100 110 11100 111 11100 111 10111 111 11011 111 11101 111 11110 C urrent R D abcdei fghj 001111 0100 001111 1001 001111 0101 001111 0011 001111 0010 001111 1010 001111 0110 001111 1000 111010 1000 110110 1000 101110 1000 011110 1000 C urrent R D + abcdei fghj 110000 1011 110000 0110 110000 1010 110000 1100 110000 1101 110000 0101 110000 1001 110000 0111 000101 0111 001001 0111 010001 0111 100001 0111

13

H ow to read the 8B /10B code nam e: 101 11100

K 28.5

Copyright 2009. Avnet, Inc. All rights reserved.

8B/10B Characteristics
General 8B/10B characteristics No more than 5 consecutive 1s or 0s in an encoded data or control character Helps CDR PLL not to drift from the center of the eye K-Character K character is independent from data character Byte alignment logic will not incorrectly align to data character Each data/K character is encoded into a Plus or Minus symbol Transmitter will send either Plus or Minus symbol to maintain a DC balanced line (this concept is referred to as Running Disparity) DC balanced is needed to avoid Inter-Symbol Interference (ISI) Plus and Minus symbols are complement of each other for easier symbol detection at the receiver

14

Copyright 2009. Avnet, Inc. All rights reserved.

Byte Alignment with Comma


Use K28.5 as Comma-Character Since K-characters are unique, miss-alignment is not possible Other methods of non-8B/10B byte alignment are also possible SONET aligns to a periodically repeated A1A2 sequence
Transmit Order

15

K28.5

D31.7

D30.2

D1.7

K28.5

D31.7

Pre-defined Alignment Pattern

Copyright 2009. Avnet, Inc. All rights reserved.

D30.2

Data Enable Indication


To avoid long streaks of 1s and 0s, data needs to be sent continuously When there are no data available, a pre-defined idle sequence is transmitted until data is available Idle sequence needs to be stripped out by the receiver before passing the data to the user logic
16 bit K28.5 D21.4 Data Data Data Data Data Data K28.5 K28.5 D21.4 D21.4 Data Data

16

Pre-defined Idle Sequence

EN DATA [15:0]

Idle

Data

Data

Data

Idle

Idle

Data

The EN signal is called RXCHARISK[0:3] for Spartan-6 GTP transceiver

Copyright 2009. Avnet, Inc. All rights reserved.

Review Questions
What hard IP cores are integrated into the Spartan-6 LXT family? GTP transceivers, PCIe Block, and Memory Controllers How is byte alignment implemented in serial data transfer? Using an alignment character Why is data encoding such as 8B/10B needed in serial signaling? 8B/10B provides sufficient number of transitions in the serial data It also provides unique alignment characters

17

Copyright 2009. Avnet, Inc. All rights reserved.

Agenda
Spartan-6 LXT Platform Family Introduction to Serial Communications Introduction to Spartan-6 GTP Transceivers Designing with Spartan-6 GTP Transceivers Xilinx PCIe solutions Introduction to Spartan-6 PCIe Endpoint Block Designing with Spartan-6 PCIe Endpoint Block PCIe Compliance Testing Wrap-Up

18

Copyright 2009. Avnet, Inc. All rights reserved.

Spartan-6 RocketIO GTP Transceivers


The Spartan-6 RocketIO GTP transceiver is a highly configurable and power-efficient transceiver Available in Spartan-6 LXT family Current Mode Logic (CML) serial drivers/buffers with configurable termination and voltage swing Programmable TX pre-emphasis and RX equalization for optimized signal integrity Line rates from 614Mbps to 3.125Gbps (-2 speed grade devices support up to 2.7Gbps) 614Mbps to 810Mbps 1.22Gbps to 1.62Gbps 2.45Gbps to 3.125Gbps Built-in PCS features, such as 8B/10B encoding, comma alignment, channel bonding, and clock correction PRBS pattern generation and checker

19

Copyright 2009. Avnet, Inc. All rights reserved.

Spartan-6 GTP Migration from Virtex-5


65nm 45nm 40nm Virtex-6 GTH 10G+ Virtex-6 GTH is a New Design

20

Virtex-5 GTX 6.5G

Virtex-6 GTX 6.5G

Virtex-6 GTX is a port from Virtex-5

Virtex-5 GTP 3.75G

Spartan-6 GTP 3.125G

Spartan-6 GTP is a port from Virtex-5

Copyright 2009. Avnet, Inc. All rights reserved.

Spartan-6 GTP Tile


A single GTP_DUAL tile consists of two GTP transceivers. The tile has 2 PLLs that can be configured in one of two modes of operations The TX and RX sides of each transceiver have one independent PLL,for a total of two active PLLs for each GTP_DUAL tile The TX and RX sides of both transceivers share one PLL and the second PLL should be powered down for power savings

21

. .

PLL0

TX Clock Divider

TX PM A TX PCS

REFCLK Distribution

GTP0 GTP1

RX Clock Divider

RX PM A RX PCS

TX Clock Divider

TX PM A TX PCS

. .

PLL1

RX Clock Divider

RX PM A RX PCS

Copyright 2009. Avnet, Inc. All rights reserved.

Spartan-6 GTP PLL


PLL LOCKED

22

Lock Indicator

PLL CLKIN

/M

Phase Frequency Detector

Charge Pump

Loop Filter

VCO

PLL CLKOUT

/ N2

/ N1

The PLL has a nominal operating range of 1.2GHz to 1.62GHz The TX and RX clock dividers can divide the PLL clock by 1, 2, or 4 PLL Clock = PLL CLKIN * N1 * N2 / M Line Rate = PLL Clock * 2 / (TX or RX Clock Divider) TX/RX Clock Divider 1 2 4 Supported Line Rate 2.45Gbps 3.25Gbps 1.22Gbps 1.62Gbps 0.61Gbps 0.81Gbps

Copyright 2009. Avnet, Inc. All rights reserved.

Spartan-6 GTP Transmitter Overview


GTP transmitter features 614 Mbps to 3.125 Gbps operation TX Pre-emphasis PRBS Pattern generator 1, 2 or 4-byte fabric interface Internal data path is 1-byte wide
PCS Parallel Clock Domain

23

Physical Media Attachment (PMA) Differential drivers Programmable pre-emphasis Physical Coding Sub-layer (PCS) Support for multiple coding standards Channel bonding and clock correction
PMA Parallel Clock Domain
Polarity Control TX Pre-emp PLL

8B/10B Encoder FPGA TX I/F TX PIPE Control PRBS Generator

Phase Adjust FIFO

SATA OOB

PCIe Beacon

TX PCS

TX PMA

Copyright 2009. Avnet, Inc. All rights reserved.

PISO

Spartan-6 GTP Receiver Overview


GTP transmitter features 614 Mbps to 3.125 Gbps operation RX linear equalization PRBS Pattern checker 1, 2 or 4-byte fabric interface Internal data path is 1-byte wide Physical Media Attachment (PMA) Differential receiver input Linear Equalization Clock and Data Recovery (CDR)

24

Physical Coding Sub-layer (PCS) Support for multiple coding standards Channel bonding and clock correction
PMA Parallel Clock Domain PCS Parallel Clock Domain

RX Polarity

RX CDR

SIPO

RX EQ RX OOB

8B/10B Decoder

Comma Detect & Align

Elastic Buffer

FPGA RX Interface

PLL

Loss of Sync PRBS Checker RX Status Control

RX PMA

RX PCS

Copyright 2009. Avnet, Inc. All rights reserved.

Spartan-6 GTP PRBS Pattern Generator/Checker


Pseudo-random bit sequences (PRBS) are commonly used to test the signal integrity of high-speed links These sequences appear random but have specific properties that can be used to measure the quality of a link The Spartan-6 GTP transceiver pattern generator block can generate several industry-standard PRBS patterns listed below
Name PRBS-7 PRBS-15 PRBS-23 Polynomial 1+ x6+ x7 1+ x14 + x15 1+ x18 + x23 Length of Sequence 27 1 bits 215 1 bits 223 1 bits Descriptions Used to test channels with 8B/10B coding. PRBS-15 is often used for jitter measurement PRBS-23 is often used for non-8B/10B encoding scheme. One of the recommended test patterns in the SONET specification. PRBS-31 is often used for non-8B/10B encoding scheme. A recommended PRBS test pattern for 10 Gigabit Ethernet.

25

PRBS-31

1+ x28 + x31

231 1 bits

Copyright 2009. Avnet, Inc. All rights reserved.

Spartan-6 GTP PRBS Pattern Generator/Checker


In addition to PRBS patterns, the GTP transceiver supports 20 UI (or 16 UI) and 2 UI square wave test patterns and PCIe compliant pattern generation
PCI Express Compliance Pattern

26

Symbol Disparity Pattern

K28.5 0

D21.5 1

K28.5 1

D10.2 0

0011111010 1010101010 1100000101 0101010101

PCI Express 20 UI Square Wave

20 UI

Copyright 2009. Avnet, Inc. All rights reserved.

Spartan-6 GTP Transceiver Protocol Support

27

Protocols Supported
PCI Express (Rev 1.0a) XAUI 802.3ae D5p0 DisplayPort EPON GPON Gigabit Ethernet SATA Gen 1 CPRI (V3.0) OBSAI RP3 (V4.0) Serial RapidIO Aurora

Data Rates
2.5 Gb/s 3.125 Gb/s 1.62 Gb/s, 2.7 Gb/s 1.25 Gb/s 622 Mb/s, 1.25 Gb/s, 2.5 Gb/s 1.25 Gb/s 1.5 Gb/s 614.4 Mb/s, 1228.8 Mb/s, 2457.6 Mb/s, 3072.0 Mb/s 768 Mb/s, 1536 Mb/s, 3072 Mb/s 1.25 Gb/s, 2.5 Gb/s, 3.125 Gb/s 614 Mb/s - 3.2 Gb/s

Lanes Supported
1, 4 4 1, 2, 4 1 1 1 1 1 1 1, 2, 4 1, 2, 4

Copyright 2009. Avnet, Inc. All rights reserved.

Spartan-6 GTP Clocking


For each PLL within a Spartan-6 GTP tile, seven clock sources can be used as the reference clock input
Two reference clock pin pairs from external pins (GTP dedicated clock inputs) CLK00, CLK10 for PLL0 CLK01, CLK11 for PLL1 One reference clock pin pair from the east or west GTP tile CLKINEAST0 or CLKINWEST0 for PLL0 CLKINEAST1 or CLKINWEST1 for PLL1 Two reference clock pin pairs from the PLL of the FPGA PLLCLK00, PLLCLK10 for PLL0 PLLCLK01, PLLCLK11 for PLL1 Two reference clock signals from the FPGA global clock inputs (for testing only) GCLK00, GCLK10 for PLL0 GCLK01, GCLK11 for PLL1

28

REFSELDYPLL0[2:0] CLK00 GCLK00 PLLCLK00 CLKINEAST0 CLK10 GCLK10 PLLCLK10 CLKINWEST0 REFSELDYPLL1[2:0] CLK01 GCLK01 PLLCLK01 CLKINEAST1 CLK11 GCLK11 PLLCLK11 CLKINWEST1 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7

PLL0

PLL1

CLK00 and CLK01 are default clock sources for PLL0 and PLL1 respectively.

Copyright 2009. Avnet, Inc. All rights reserved.

GTP Reference Clock Oscillator


A high-quality crystal oscillator is essential for good performance When using one of the recommended oscillators, the manufacturers power supply design guide must be followed Spartan-6 device characterization is based on the same recommended oscillators

29

When considering alternate clock sources, the alternate oscillators must meet or exceed the specifications of the recommended oscillators Depending on the application and its performance goals, it is possible to stray from the clock source specifications, but in that case the specified performance of the GTP is not guaranteed
0.1uF

Z0
LVDS Clock Source IBUFDS

Z0
0.1uF

Copyright 2009. Avnet, Inc. All rights reserved.

Reference Clock Source Example


The Maxim MAX3674 programmable clock source meets or exceeds the specifications of the Xilinx recommended oscillators Dual programmable differential LVPECL outputs Output frequency range of 21.25MHz to 1360MHz Crystal input frequency range of 15MHz to 20MHz Total period jitter of 18ps (typical)

30

Copyright 2009. Avnet, Inc. All rights reserved.

Reference Clock Source Example


The Texas Instruments CDCM61002 programmable clock source meets or exceeds the specifications of the Xilinx recommended oscillators Two programmable differential LVPECL/LVDS outputs Output frequency range of 43.75MHz to 683.264MHz Crystal input frequency range of 21.875MHz to 28.47MHz Total period jitter of 27ps (typical)

31

Copyright 2009. Avnet, Inc. All rights reserved.

Application Example - 10G Packet Processing


DDR3 DDR3 SDRAM SDRAM

32

Memory Memory Controller Controller

DSL DSL Chipset Chipset

Level 2 Level 2 Core Core

Packet Packet Processor Processor

10G 10G MAC MAC

GTP GTP XAUI XAUI GTP GTP GTP GTP

MicroBlaze MicroBlaze

National DS64BR401 (Use this device for traces > 14)

4x 3.125 Gbps

GTP features used 8B10B encoder/decoder Channel bonding Clock correction XAUI

Hard memory controller MicroBlaze soft processor

Copyright 2009. Avnet, Inc. All rights reserved.

Backplane

UL2

UTOPIA UTOPIA

GTP GTP

Agenda
Spartan-6 LXT Platform Family Introduction to Serial Communications Introduction to Spartan-6 GTP Transceivers Designing with Spartan-6 GTP Transceivers Xilinx PCIe solutions Introduction to Spartan-6 PCIe Endpoint Block Designing with Spartan-6 PCIe Endpoint Block PCIe Compliance Testing Wrap-Up

33

Copyright 2009. Avnet, Inc. All rights reserved.

Spartan-6 GTP Configuration Overview


There are two ways to configure a Spartan-6 GTP Static configuration - The transceiver is configured using a combination of GTP port tie-offs and attribute settings at design time to support a specific protocol Dynamic configuration - The transceiver is configured by driving the GTP ports and using the Dynamic Reconfiguration Port (DRP) to modify the run-time configuration of the GTP GTP configuration can be complex because of the large number of possible settings Xilinx provides a RocketIO wizard to help manage the configuration process of the Spartan-6 GTP The wizard is highly recommended for any design that utilizes the Spartan-6 RocketIO GTP

34

Copyright 2009. Avnet, Inc. All rights reserved.

RocketIO GTP Coregen Wizard


The RocketIO Wizard can be used to create a design for a pre-defined (Gigabit Ethernet, CPRI, XAUI, etc.) or a custom protocol Each Coregen wrapper includes example design, test bench; and both implementation and simulation scripts
3 1 Click on Spartan-6 Click on Spartan-6 FPGA RocketIO GTP FPGA RocketIO GTP Transceiver Wizard. Transceiver Wizard. Select the number of GTP Select the number of GTP tiles used in the design. tiles used in the design. 4

35

For each GTP in a tile, select the For each GTP in a tile, select the GTP reference clock source. GTP reference clock source.

2 Click on Customize to Click on Customize to configure the Spartan-6 configure the Spartan-6 GTP(s). GTP(s).

5 Follow the wizard to customize the Follow the wizard to customize the remainder of the GTP parameters. remainder of the GTP parameters.
Copyright 2009. Avnet, Inc. All rights reserved.

Review Questions
What are the min and max line rates of the Spartan-6 GTP transceiver? 614Mbps and 3.125Gbps For each PLL within a Spartan-6 GTP tile, how many clock sources can be used as the reference clock input and what are they? 7 clocks: 2 differential clocks from the FPGA pins, 1 clock from the East or West GTP tile, 2 from the internal PLL, and 2 from the FPGA global clock pins What Xilinx tool can you use to control and monitor the operation of the Spartan-6 GTP in real-time? Serial IO Toolkit

36

Copyright 2009. Avnet, Inc. All rights reserved.

Agenda
Spartan-6 LXT Platform Family Introduction to Serial Communications Introduction to Spartan-6 GTP Transceivers Designing with Spartan-6 GTP Transceivers Xilinx PCIe solutions Introduction to Spartan-6 PCIe Endpoint Block Designing with Spartan-6 PCIe Endpoint Block PCIe Compliance Testing Wrap-Up

37

Copyright 2009. Avnet, Inc. All rights reserved.

PCI Express Standard


The PCI Express standard is a next-generation evolution of the older PCI and PCI-X parallel bus standards It is a high-performance, general-purpose interconnect architecture, designed for a wide range of computing and communications platforms It is a packet-based, point-to-point serial interface that is backward compatible with PCI and PCI-X configurations, device drivers, and application software The effective bandwidth is 80% of the raw bandwidth due to 8B/10B encoding
Link x1 x2 x4 x8 x12 x16 x32 PCI Express Bandwidth Raw Bandwidth Per Effective Bandwidth Per Direction Direction 2.5Gbps 2Gbps 5Gbps 4Gbps 10Gbps 8Gbps 20Gbps 16Gbps 30Gbps 24Gbps 40Gbps 80Gbps 32Gbps 64Gbps

38

Copyright 2009. Avnet, Inc. All rights reserved.

PCI Express Use Model


Root Complex A Root Complex (RC) denotes the root of an I/O hierarchy that connects the CPU and memory subsystem to the I/O Comparable to the PCI North Bridge Switch Logical assembly of multiple virtual PCIe-PCIe bridge devices Comparable to the PCI South Bridge Endpoint Previously known as the Peripheral

39

CPU

PCIe to PCI Bridge

Root Complex

Memory

Upstream Port

Root Port

Legacy PCIe Endpoint


End-Point

Switch

PCIe Endpoint
Downstream Port

PCIe Endpoint

PCIe Endpoint

At a minimum the Host Bridge, Memory Arbiter, and Root Port are required to create a Root Complex

Copyright 2009. Avnet, Inc. All rights reserved.

PCIe v1.1
PCI Express v1.1 (Gen 1) 2.5Gbps per lane PCI Express specifications Obtained from PCI-SIG (PCI Special Interest Group) http://www.pcisig.com/specifications/pciexpress/specifications Base Specification for protocol Card Electromechanical Specification (CEM) for electrical characteristics PCIe 1.1 specifications defines 100 Ohms differential trace impedance

40

Copyright 2009. Avnet, Inc. All rights reserved.

Xilinx PCIe Solutions

41

FPGA Device Spartan-6 Spartan-6 Virtex-6 Virtex-6 Virtex-5 LXT/SXT/FXT/TXT Virtex-5 FXT/TXT Virtex-4 FX Spartan-3

Lanes 1 2 and 4 1, 2, 4, and 8 4 and 8 1, 2, 4, and 8 1, 2, 4, and 8 1, 4, and 8 1

Gen 1 / Gen 2 Gen 1 Gen 1 Gen 1 / Gen 2 Gen 2 Gen 1 Gen 1 / Gen 2 Gen 1 Gen 1

IP Integrated Block Soft IP (Alliance Partner) Integrated Block Soft IP (Alliance Partner) Integrated Block Soft IP (Alliance Partner) Soft IP (DO-DI-PCIEXP) Soft IP (DO-DI-PCIE-PIPE)

Alliance program members: Northwest Logic and PLDA

Copyright 2009. Avnet, Inc. All rights reserved.

Xilinx Spartan-6 PCI Express IP Roadmap

42

Beta 2 (March 09)

ISE 11.3 (Sept 09)

End-point Wrapper x1 Gen 1 Delivered through Coregen Simulation and implementation Verilog only top level Full documentation

End-point Wrapper x1 Gen 1 All package/device combinations Delivered through Coregen Source code RTL wrapper Simulation and implementation Verilog/VHDL Full documentation

Alliance Partner x4 Gen 1

Copyright 2009. Avnet, Inc. All rights reserved.

Avnet Spartan-6 LX150T PCIe x4 Board


Board Features Spartan-6 LX150T FPGA (Xilinx) PCIe x1 and x4 support SFP & SATA Connectors (Tyco) 128MB DDR3 SDRAM (Micron) 32MB Parallel Flash (Spansion) 10/100/1000 Ethernet PHY (NSC) USB 2.0 PHY (NXP) USB Bridge (Silicon Labs) LVDS Clock Generator (TI) Temp Sensor & RTC (Maxim) Platform Flash (Xilinx) LCD Panel Interface Dual FMC LPC Slots SD Card Slot $995 Target Resale Available ~Q4 2009
Copyright 2009. Avnet, Inc. All rights reserved.
Connector GTP Interfaces PCI-Express x4

43

FMC LPC Slot

SFP Connector Connector

SATA Connector

FMC LPC Slot

Communication Ports 10/100/1000 PHY USB-RS232 Bridge Spartan-6 LX150T FGG676

JTAG Port

Platform Flash

USB 2.0 PHY

Memory Interfaces 128MB DDR3 SDRAM

RS232 Port

Miscellaneous I/O Push and DIP Switches User LEDs

SD Card Connector 32MB Parallel Flash

Clock Sources Programmable LVDS Clock Source LVTTL OSC @ 100 MHz

Power Supply 0.75V, 1.2V, 1.5V, 1.8V, 2.5V, and 3.3V Regulators

ALI Connector Temp Sensor and Real-Time Clock

Agenda
Spartan-6 LXT Platform Family Introduction to Serial Communications Introduction to Spartan-6 GTP Transceivers Designing with Spartan-6 GTP Transceivers Xilinx PCIe solutions Introduction to Spartan-6 PCIe Endpoint Block Designing with Spartan-6 PCIe Endpoint Block PCIe Compliance Testing Wrap-Up

44

Copyright 2009. Avnet, Inc. All rights reserved.

Xilinx Spartan-6 PCI Express Endpoint Features


The Spartan-6 PCIe Endpoint block contains the functionality defined in the specifications maintained by the PCI-SIG Available in all Spartan-6 LXT devices Compliant with PCI Express base specification, revision 1.1 (2.5Gbps) Compliant PCI-SIG Endpoint requires minimal fabric logic (< 100 LUTs) RocketIO GTP transceivers can implement a fully compliant PCIe PHY Cut-through mode on transmit path for reduced latency Maximum payload size of 128/256/512 bytes supported Up to 6 x 32 bit or 3 x 64 bit BARs BARs configurable for memory or I/O

45

Copyright 2009. Avnet, Inc. All rights reserved.

Spartan-6 PCI Express Endpoint Block Diagram


Block RAM Interface PCIe Block

46

Transaction Layer Interface

Transaction Layer Module

Data Link Layer Module

Physical Layer Module

RocketIO GTP

Configuration Management Interface

Configuration and Capabilities Module

Debug Port

Clock and Reset Interface

Copyright 2009. Avnet, Inc. All rights reserved.

Spartan-6 PCIe Clocking


The Integrated PCIe Block core requires a 125 or 250MHz system clock input The clock frequency used must match the clock frequency selection in the CORE Generator GUI In a typical PCI Express solution, the PCI Express reference clock is a Spread Spectrum Clock (SSC), provided at 100MHz

47

PCIe 100MHz Clock

CLKP CLKN External PLL

125 or 250MHz

Spartan-6 PCIe Block

Copyright 2009. Avnet, Inc. All rights reserved.

Spartan-6 PCIe Clocking Examples

48

Embedded System Using 125/250MHz System Clock

Open System Add-In Card Using 125/250MHz System Clock CLK @ 125/ 250MHz External PLL

PCIe Switch or Root Complex Device REFCLK @ 100MHz PCIe Clock OSC

PCIe Link

Spartan-6 Endpoint Device CLK @ 125/ 250MHz

Spartan-6 Endpoint Device PCIe Link

100MHz External PLL

100MHz PCI Express Clock (SSC)

PCIe Edge Connector

Copyright 2009. Avnet, Inc. All rights reserved.

PCIe Clocking on Avnet S6LX150T Board

49

PCIe_RefClk_100_P CLK PCIe_RefClk_100_N nCLK

QB0 nQB0 QA0 ICS874003-05 nQA0

MGTREFCLK0P MGTREFCLK0N MGTREFCLK1P MGTREFCLK1N

3.3V

PB Switch
ON OFF 3 2 1

MR F_SEL0 F_SEL1 F_SEL2

The QA0/nQA0 and QB0/nQB0 outputs can be set to 125MHz or 250MHz using the F_SEL[2:0] inputs of the ICS874003-05 device.

Copyright 2009. Avnet, Inc. All rights reserved.

Agenda
Spartan-6 LXT Platform Family Introduction to Serial Communications Introduction to Spartan-6 GTP Transceivers Designing with Spartan-6 GTP Transceivers Xilinx PCIe solutions Introduction to Spartan-6 PCIe Endpoint Block Designing with Spartan-6 PCIe Endpoint Block PCIe Compliance Testing Wrap-Up

50

Copyright 2009. Avnet, Inc. All rights reserved.

Designing with Spartan-6 PCIe Endpoint Block


Typical design flow uses the Core Generator The Core Generator wizard configures the required blocks such as GTP, BRAM, Clock, and Reset It outputs a Programmed Input Output (PIO) example design The Core Generator example simulation design consists of the following discrete parts
Example Design PIO A completer application for PCI Express Simulation/Design Files Design Files - PIO example design Testbench - The Root Port Bus Functional Model (BFM), a test bench that generates, consumes, and checks PCI Express bus traffic

51

The example design has been tested and verified with Mentor Graphics ModelSim, NC-Sim, and Synopsys VCS simulators Aldec simulator can also be used (see the Appendix for more information) ISIM support (possibly in 11.4)

Copyright 2009. Avnet, Inc. All rights reserved.

Xilinx Spartan-6 PCI Express Endpoint Configuration


The Transceiver, Memory, Clock and the Reset interfaces are automatically connected in the CORE Generator wrappers These interfaces are not visible outside of the wrappers User application must be implemented in the FPGA fabric and interfaced to the PCIe block using the Transaction Layer Interface
Block RAM Interface

52

User connection
Transaction Layer Interface Transaction Layer Data Link Layer Physical Layer Transceiver Interface

Configuration Management Interface

Configuration and Capabilities Module PCIe Block

Automatically connected by the CORE Generator

Optional connection

Debug Port

Clock and Reset Interface

Copyright 2009. Avnet, Inc. All rights reserved.

PCI Express Wizard Endpoint Block Wrapper Output

53

Block RAM

Block RAM

PCIe Endpoint Block Wrapper

User Interface

Transaction Layer

Data Link Layer

Physical Layer

GTP Tile

Serial Interface

Configuration Management Interface

Configuration and Capabilities Module

PCIe Block

Clock Module

Reset Module

Debug Port
Copyright 2009. Avnet, Inc. All rights reserved.

Core Generator Deliverables


Parameterized hard IP core (RTL wrapper source code) Programmed Input Output (PIO) example design Customer simulation demonstration test bench

54

Verilog HDL simulation flow (VHDL is scheduled for 11.4)


Customer implementation demonstration Example UCF Complete implementation scripts delivered for PIO design

Copyright 2009. Avnet, Inc. All rights reserved.

Review Questions
What is the minimum fabric logic needed to implement a PCIe x1 Endpoint using the Spartan-6 integrated PCIe block? 100 LUTs In addition to the PCIe hard block, what other blocks are automatically configured by the PCIe Coregen Wizard? GTP transceiver, BRAM, Clock, and Reset What is the procedure for assigning GTP pins to the PCIe lane? Use Coregen to generate a UCF

55

Copyright 2009. Avnet, Inc. All rights reserved.

Agenda Spartan-6 LXT Platform Family Introduction to Serial Communications Introduction to Spartan-6 GTP Transceivers Designing with Spartan-6 GTP Transceivers Xilinx PCIe solutions Introduction to Spartan-6 PCIe Endpoint Block Designing with Spartan-6 PCIe Endpoint Block PCIe Compliance Testing Wrap-Up

56

Copyright 2009. Avnet, Inc. All rights reserved.

Xilinx Spartan-6 PCIe PCI-SIG Compliance Testing


Spartan-6 PCIe Endpoint block has passed the following tests PCI-SIG compliance test 3 SIG Gold suites (Electrical, Configuration and Protocol) Interoperability x1 Endpoint configuration Reference board used for PCI-SIG compliance Xilinx SP605 evaluation board

57

Copyright 2009. Avnet, Inc. All rights reserved.

Agenda Spartan-6 LXT Platform Family Introduction to Serial Communications Introduction to Spartan-6 GTP Transceivers Designing with Spartan-6 GTP Transceivers Xilinx PCIe solutions Introduction to Spartan-6 PCIe Endpoint Block Designing with Spartan-6 PCIe Endpoint Block PCIe Compliance Testing Wrap-Up

58

Copyright 2009. Avnet, Inc. All rights reserved.

Key Takeaways
The Spartan-6 RocketIO GTP transceiver is a highly configurable and power-efficient transceiver Xilinx Coregen provides a RocketIO wizard to help manage the configuration process of the Spartan-6 GTP transceiver The Wizard can be used to create a design for a pre-defined (Ethernet, XAUI, etc.) or a custom protocol Spartan-6 integrated PCIe block is fully compliant with the PCI Express Base 1.1 Specification Compliant PCI-SIG Endpoint requires minimal fabric logic (< 100 LUTs) RocketIO GTP transceiver implement a fully compliant PCIe PHY Xilinx Coregen can be used to generate a PIO example design Complete implementation script and UCF delivered for PIO design Verilog HDL simulation support (VHDL is scheduled for 11.4)

59

Copyright 2009. Avnet, Inc. All rights reserved.

Spartan-6 PCIe Demo


S6 PCIe Demo Description GTP transceiver running @ 2.5Gbps (PCIe Gen 1) Memory Controller Block in DDR3 mode Packet based DMA Design Uses Xilinx SP605 evaluation kit Other Equipment used Software Application / GUI / Device Drivers PC with Windows XP / Vista OS

60

Copyright 2009. Avnet, Inc. All rights reserved.

Closing Comments
Please go to the following URL to download the X-Fest 2009 course presentations http://em.avnet.com/xfsupport2010 Visit the X-Fest 2009 forum for latest discussions on various courses http://community.em.avnet.com/t5/XFEST-2009/ct-p/XFEST_2009

61

Please Visit the Demo Area Thank You

Copyright 2009. Avnet, Inc. All rights reserved.

Appendix

Agenda
Serial communications definitions and terminologies Spartan-6 PCIe Block overview Debugging PCIe Designs

63

Copyright 2009. Avnet, Inc. All rights reserved.

Eye Diagram
An eye diagram corresponds to a superposition of a substantial number of bits of a particular stream of data after this data has propagated through the designed subsystem One of the most accepted standards for qualifying design performance Vertical thickness of the lines indicate the magnitude of the AC voltage noise Horizontal thickness of the lines indicate jitter

64

Copyright 2009. Avnet, Inc. All rights reserved.

Eye Mask

65

Every serial communications standard has its own set of eye mask parameters Defined as part of the electrical specifications for each standard Usually an eye mask for signal at output of the transmitter and input to the receiver is defined Eye Mask is the size of the opening in the center of the eye diagram Eye Mask indicates the amount of voltage and timing margin available to sample the signal

Eye mask without violations


Copyright 2009. Avnet, Inc. All rights reserved.

Unit Interval (UI)


The Unit Interval (UI) is the unit of time corresponding to one bit period (the time it takes to send one bit) The number of 1s and 0s must be equal to yield DC balance # of Bits without transition (Run Length)

66

Bit time or Unit Interval (UI)

Copyright 2009. Avnet, Inc. All rights reserved.

Miscellaneous Terms and Definitions


Jitter Delay between the actual and expected transition of a signal Jitter Tolerance The peak-to-peak amplitude of sinusoidal jitter applied to the input signal that causes an acceptable loss of information at the output Differential Signaling A signaling scheme which uses two complementary (P and N) signals to transmit data Offers improved data rates with reduced signal swing Bit Error Rate The number of errors detected at a receiver in a given length of time

67

Copyright 2009. Avnet, Inc. All rights reserved.

Pre-emphasis
Pre-emphasis is the boosting of the magnitude of high frequency spectral components in the transmitted signal to improve the eye mask of the received signal

68

Copyright 2009. Avnet, Inc. All rights reserved.

Agenda
Serial communications definitions and terminologies Spartan-6 PCIe Block overview Debugging PCIe Designs

69

Copyright 2009. Avnet, Inc. All rights reserved.

PCI Express Configuration Space


The PCI Express configuration space consists of three primary sections Legacy PCI v3.0 type 0 configuration header Legacy extended capability items PCIe capability item Power management Message signaled interrupt PCIe extended capabilities Device serial number Virtual channel
31 24 23 8 7 0 Device ID Vendor ID Status Command Class Code Rev ID BIST Header Lat Timer Cache Ln Base Address Register 0 Data 16 15 Address 000h 004h 008h 00Ch 010h

70

Copyright 2009. Avnet, Inc. All rights reserved.

PCI Express Configuration Space


PCI Express configuration space consists of 4096 bytes First 256 bytes match the PCI configuration space Some configuration data is simply read by the Root Complex Vendor and Device ID Memory blocks and size requirements Certain capabilities of the subsystem such as the device type and power management Some configuration data is assigned by the Root Complex Base Address Register (BAR) Specific address for each requested memory block Six maximum BARs per subsystem Transaction ID consisting of Bus and Device Number Consistent with the physical slot

71

Copyright 2009. Avnet, Inc. All rights reserved.

PCI Express Communication Methodology


The communication between two PCIe devices is referred to as a transaction Transactions are packet-based Each PCIe device can be a Requester and/or a Completer Requester initiates a transaction Completer responds to a request Both the Root Complex and the Endpoint device can function as a Completer as well as a Requester
Root Complex Packet Based Transactions Packet Based Transactions Legacy PCIe Endpoint

72

Switch

PCIe Endpoint

Copyright 2009. Avnet, Inc. All rights reserved.

PCI Express Protocol Support


The PCI Express protocol supports four types of transactions Memory (read and write) I/O (read and write) Configuration (read and write) Message (communication information outside of the Memory, I/O, and Configuration spaces such as interrupt signaling, error signaling, etc.) Transactions are divided into three categories Posted transactions Non-posted transactions Completion transactions

73

Copyright 2009. Avnet, Inc. All rights reserved.

Posted and Non-Posted Transactions


Memory writes and message transactions are posted transactions The requester sends a packet, but the receiver does not return a completion Non-posted transactions (memory reads, I/O reads and writes, and configuration reads and writes) require a response and are implemented as split transactions Completion packets can be directed to the correct originator because each packet has a unique identifier

74

Copyright 2009. Avnet, Inc. All rights reserved.

Transaction Layer Module

75

The Transaction Layer Module (TLM) is the upper layer in the architecture This module takes Transaction Layer Packets (TLPs) presented by user logic at the Transaction Layer interface and schedules them for transmission over the link The Transaction Layer module also advises the user application when TLPs are received TLPs can both make requests and complete requests from another device
Block RAM Interface

4 bytes, 62.5MHz Interface

Transaction Layer Interface

Transaction Layer Module

Data Link Layer Module

Copyright 2009. Avnet, Inc. All rights reserved.

Transaction Layer Module - Packet


A Transaction Layer Packet (TLP) is composed of a header, data payload (for most packets), and optional end-to-end CRC (ECRC) Packets must be formed by the user in accordance with the PCI Express specification Packets must be decoded by the user properly
Criteria Size Notes 1 DWORD for address 2 DWORD for address No Payload 512 bytes in Spartan-6 PCIe Block Passed by TL to User Still exists, but trimmed by TL

76

32-bit 3 DWORDS Machine Header 64-bit 4 DWORDS Machine Read 0 Request Payload Write or Read Max Payload Size Completion (MPS) or less Included 1 DWORD ECRC (Digest) Trimmed 0

Copyright 2009. Avnet, Inc. All rights reserved.

Data Link Layer Module


The Data Link Layer Module (DLLM) resides between the Transaction Layer and the Physical Layer modules Its primary responsibility is to provide a reliable mechanism for the exchange of TLPs between two components on a link Data Link Layer provides data exchange (TLPs), error detection and recovery, initialization services and the generation and consumption of Data Link Layer Packets (DLLPs) The transmission portion of the DLL accepts TLPs from the Transaction Layer and generates the appropriate TLP sequence number and Link CRC (LCRC), then passes the packet to the Physical Layer Data Link Layer Module also places a copy of the packet in a retry buffer, making it available if the packet needs to be resent
Block RAM Interface

77

Transaction Layer Interface

Transaction Layer Module

Data Link Layer Module

Physical Layer Module

Copyright 2009. Avnet, Inc. All rights reserved.

Data Link Layer Module Data Link Layer Packets


The DLL also generates and consumes special packets called Data Link Layer Packets (DLLPs) that do not pass to the Transaction Layer Types of DLLPs include acknowledgment (ACK/NAK), flow control, and power management The reception portion of the DLL checks the integrity of received TLPs It also orders retransmission when the received TLP is found to be corrupted The transmission portion controls the order of release of the different types of packets A prioritizer is included to sort the different sources of transmission into order of priority and schedule them for transmission according to the priority order recommended in the PCIe Base Specification (Rev 1.1)

78

Copyright 2009. Avnet, Inc. All rights reserved.

Physical Layer Module


The Physical Layer (PL) module carries out the following functions defined for the PL of a PCIe device Packet framing and de-framing (Start of Frame and End of Frame) Byte striping and un-striping; that is, distributing Tx packets over the associated PL lanes and reassembling Rx packets received over the different PL lanes Link initialization and training Scrambling, de-scrambling, and 8B/10B encoding and decoding of data are also performed by the Physical Layer Module
Block RAM Interface

79

Transaction Layer Interface

Transaction Layer Module

Data Link Layer Module

Physical Layer Module

Transceiver Interface

Copyright 2009. Avnet, Inc. All rights reserved.

PCI Express Packet Summary

80

A TLP is composed of a header, data payload (for most packets), and optional end-to-end CRC (ECRC) The transmission portion of the DLL accepts TLPs from the Transaction Layer and generates the appropriate TLP sequence number and Link CRC (LCRC) The Physical Layer appends the Start and End to the packet
Sequence Number

Start

Header

Data Payload

ECRC

LCRC

End

Presented to Transaction Layer Appended by Data Link Layer Appended by Physical Layer

Copyright 2009. Avnet, Inc. All rights reserved.

Configuration and Capabilities Module


The Configuration and Capabilities module provides the repository for the different registers within the Configuration Space It implements the legacy PCI configuration header defined in both the PCI Express Base Specification and the earlier PCI bus specifications It also implements the extended configuration space supported by PCI Express systems that contain Power management Message signaled interrupts Error reporting
Transaction Layer Interface Transaction Layer Module Data Link Layer Module Physical Layer Module

81

Configuration Management Interface

Configuration and Capabilities Module

Copyright 2009. Avnet, Inc. All rights reserved.

Block RAM Interface


The PCIe block buffers are implemented using block RAMs The sizes of the buffers can vary based on the applications needs There are two options for configuring these buffers in PCIe Wizard Minimize Block RAM usage Maximize performance

82

Transmit Block RAM

Receive Block RAM

Retry Block RAM

Transaction Layer Module

Data Link Layer Module

Copyright 2009. Avnet, Inc. All rights reserved.

Agenda
Serial communications definitions and terminologies Spartan-6 PCIe Block overview Debugging PCIe Designs

83

Copyright 2009. Avnet, Inc. All rights reserved.

PCIe Debug Overview


Debug using PCIe configuration space registers Why are these registers useful PCI Express capability structure How to read the capability structure linked-list De-scrambling What it is How it works Why it might be useful Debug Ports Overview of debug options included in Spartan-6 and Virtex-6 to help customers debug PCIe designs

84

Copyright 2009. Avnet, Inc. All rights reserved.

PCI Express Configuration Space Review


The PCIe configuration space can be divided into three sections PCI Express extended capabilities (0x100 0xfff) Legacy extended capabilities list (0x40 0xff) PCI 3.0 compatible configuration space (0x00 0x3f)

85

Copyright 2009. Avnet, Inc. All rights reserved.

Where Does the Capability List Start?


The value contained in configuration space address 0x34 is the starting point and always points to the very first item in the linked-list structure Note that since PCIe requires the PCIe capability structure, this register will always contain a value

86

Copyright 2009. Avnet, Inc. All rights reserved.

Structure of the Extended Capabilities Linked-lists


There are two separate linked lists of capability structures Legacy PCI starts with the capabilities pointer at 0x34 PCIe Capabilities always starts at 0x100 Next Ptr = 0 terminates both lists Each capability structure must contain two items ID which identifies the type of structure Pointer to next item in the list Size of the capability structure is implied by the ID

87

ID

PCI Express Extended Capabilities


0x100 ID Next Ptr

ID

ID

Next Ptr

PCI Extended Capabilities

ID

Next Ptr

0x40

ID

Next Ptr

0x34

Capabilities Pointer

Copyright 2009. Avnet, Inc. All rights reserved.

PCI Express Capability Structure


Lots of useful debug info in this structure Device control register Max payload, max read request Device status register Errors detected Link control register Read Completion Boundary (RCB) Link status register Negotiated link width Current link speed For exhaustive discussion see Chapter 7.8 of the PCI Express Base Specification Rev 2.0
31 15 7

88

PCI Express Capabilities Register

Next Cap Pointer

0 Byte PCI Express Offset 00h

Cap ID

Device Capabilities Device Status Link Capabilities Link Status Slot Capabilities Slot Status Root Capabilities Root Status Device Capabilities 2 Device Status 2 Link Capabilities 2 Link Status 2 Slot Capabilities 2 Slot Status 2 Slot Control 2 Link Control 2 Device Control 2 Root Control Slot Control Link Control Device Control

04h 08h 0Ch 10h 14h 18h 1Ch 20h 24h 28h 2Ch 30h 34h 38h

Copyright 2009. Avnet, Inc. All rights reserved.

Customers Common PCIe Issues


Example: Customer is seeing very low bandwidth Possible reasons Link partner lane width capability is less than expected Many x8 connectors on motherboards only route 4 lanes Width down-trained due to SI issues on upper lanes Gen 2 speed change did not occur at link-training due to SI Customer is not using bus-mastering DMA Check the link capability and link status registers of both devices on the link Check the link capability register of both devices Check negotiated link width and current link speed fields in the link status register

89

Copyright 2009. Avnet, Inc. All rights reserved.

Reading the Link Capability/Status Register to Verify Negotiated Link Width and Speed
Link capability register (0Ch) Where we declare our capabilities Supported link speed Max link width Etc. Link status register (12h) Whats currently happening on the link Current link speed Negotiated link width Slot clock configuration Etc.
31 15 7

90

PCI Express Capabilities Register

Next Cap Pointer

PCI Express Cap ID

0 Byte Offset 00h 04h

Device Capabilities Device Status Link Capabilities Link Status Slot Capabilities Slot Status Root Capabilities Root Status Device Capabilities 2 Device Status 2 Link Capabilities 2 Link Status 2 Slot Capabilities 2 Slot Status 2 Slot Control 2 Link Control 2 Device Control 2 Root Control Slot Control Link Control Device Control

08h 0Ch 10h 14h 18h 1Ch 20h 24h 28h 2Ch 30h 34h 38h

Also check the same register in the Link Partner to verify Link Partner capabilities
Copyright 2009. Avnet, Inc. All rights reserved.

Reading the Link Status Register to Verify Negotiated Link Width and Speed
31 15 7

91

PCI Express Capabilities Register

Next Cap Pointer

0 Byte Offset PCI Express 00h

Cap ID

Device Capabilities Device Status Link Capabilities Link Status Link Control Device Control

04h 08h 0Ch 10h

Also check the same register in the Link Partner to verify Link Partner capabilities
Copyright 2009. Avnet, Inc. All rights reserved.

Determining the Max Payload Size


Device capability register Where we declare the maximum payload size of which we are capable
31 15 7

92

PCI Express Capabilities Register

Next Cap Pointer

PCI Express Cap ID

0 Byte Offset 00h 04h

Device Capabilities Device Status Link Capabilities Link Status Slot Capabilities Slot Status Root Capabilities Root Status Root Control Device Capabilities 2 Device Status 2 Link Capabilities 2 Link Status 2 Slot Capabilities 2 Slot Status 2 Slot Control 2 Link Control 2 Device Control 2 Slot Control Link Control Device Control

08h 0Ch 10h 14h 18h 1Ch 20h 24h 28h 2Ch 30h 34h 38h

Device control register Where the system software sets all of the devices to a common maximum payload size

Copyright 2009. Avnet, Inc. All rights reserved.

Debugging Using PCIe Registers


Use PCITREE/HWDirect (NT) or LSPCI (Linux) to read registers in PCIe Endpoint Most of these register values are outputs to user via buses on the CFG port 15 6 5 4 Device control register = cfg_dcommand[15:0] RsvdZ Device status register = cfg_dstatus[15:0]
Transaction Pending AUX Power Detected Unsupported Request Detected Fatal Error Detected

93

3 2 1 0

Example: Use for ChipScope Triggers Non-Fatal Error Detected Correctable Error Detected Imagine system is blue screening Usually, FPGA is transmitting a fatal error message Triggering on device status bit 2 gives view of situation when message transmitted cfg_dstatus<2> = fatal error detected At this trigger point, you may want to view status of MGT ports or other PCIe block ports to see if you can determine reason for failure
Copyright 2009. Avnet, Inc. All rights reserved.

Using PCItree

94

Selected PCIe Device

Configuration Registers

Copyright 2009. Avnet, Inc. All rights reserved.

Using HWDirect
PCItree cannot read extended capability space HWDirect can read extended space Low-cost Windows shareware program (~$38) www.eprotek.com On Linux, use LSPCI with xxxx switch: e.g.: /sbin/lspci -xxxx

95

Copyright 2009. Avnet, Inc. All rights reserved.

Debug Using Configuration Space Review


PCI Configuration Space is useful for certain debug situations Bandwidth issues speed and lane width Error logging/reporting Verifying capabilities of device Checking if system software enabled certain capabilities

96

Knowing how to read the linked-list is helpful Especially when checking the link partner capabilities list

Configuration port outputs of the PCIe core can be used as ChipScope or link analyzer triggers

Copyright 2009. Avnet, Inc. All rights reserved.

Spartan-6 Debug Ports


Available to help user understand causes of errors Fatal, non-fatal, correctable User knows an error is generated based on the cfg_dstatus bus User does not know why Debug ports will tell user why an error occurs

97

Copyright 2009. Avnet, Inc. All rights reserved.

Using the Spartan-6 Debug Ports


Debug ports are included in the Coregen PCIe wrapper file S6_pcie_v1_1.v Debug ports are not pulled up to user level User would need to add ports to block wrapper file or add logic directly in wrapper
wire wire wire wire wire wire wire wire wire wire wire wire wire wire wire wire wire wire wire wire wire wire wire dbg_bad_dllp_status; dbg_bad_tlp_lcrc; dbg_bad_tlp_seq_num; dbg_bad_tlp_status; dbg_dl_protocol_status; dbg_fc_protocol_err_status; dbg_mlfrmd_length; dbg_mlfrmd_mps; dbg_mlfrmd_tcvc; dbg_mlfrmd_tlp_status; dbg_mlfrmd_unrec_type; dbg_poistlpstatus; dbg_rcvr_overflow_status; dbg_reg_detected_correctable; dbg_reg_detected_fatal; dbg_reg_detected_non_fatal; dbg_reg_detected_unsupported; dbg_rply_rollover_status; dbg_rply_timeout_status; dbg_ur_no_bar_hit; dbg_ur_pois_cfg_wr; dbg_ur_status; dbg_ur_unsup_msg; .DBGBADDLLPSTATUS .DBGBADTLPLCRC .DBGBADTLPSEQNUM .DBGBADTLPSTATUS .DBGDLPROTOCOLSTATUS .DBGFCPROTOCOLERRSTATUS .DBGMLFRMDLENGTH .DBGMLFRMDMPS .DBGMLFRMDTCVC .DBGMLFRMDTLPSTATUS .DBGMLFRMDUNRECTYPE .DBGPOISTLPSTATUS .DBGRCVROVERFLOWSTATUS .DBGREGDETECTEDCORRECTABLE .DBGREGDETECTEDFATAL .DBGREGDETECTEDNONFATAL .DBGREGDETECTEDUNSUPPORTED .DBGRPLYROLLOVERSTATUS .DBGRPLYTIMEOUTSTATUS .DBGURNOBARHIT .DBGURPOISCFGWR .DBGURSTATUS .DBGURUNSUPMSG ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( dbg_bad_dllp_status ), dbg_bad_tlp_lcrc ), dbg_bad_tlp_seq_num ), dbg_bad_tlp_status ), dbg_dl_protocol_status ), dbg_fc_protocol_err_status ), dbg_mlfrmd_length ), dbg_mlfrmd_mps ), dbg_mlfrmd_tcvc ), dbg_mlfrmd_tlp_status ), dbg_mlfrmd_unrec_type ), dbg_poistlpstatus ), dbg_rcvr_overflow_status ), dbg_reg_detected_correctable), dbg_reg_detected_fatal ), dbg_reg_detected_non_fatal ), dbg_reg_detected_unsupported), dbg_rply_rollover_status ), dbg_rply_timeout_status ), dbg_ur_no_bar_hit ), dbg_ur_pois_cfg_wr ), dbg_ur_status ), dbg_ur_unsup_msg ),

98

Copyright 2009. Avnet, Inc. All rights reserved.

Debugging Conclusion
Standard PCIe register set is helpful in debugging problems Virtex-6 and Spartan-6 user guides contain new debug chapters Debug ports available to assist in analyzing problems

99

Copyright 2009. Avnet, Inc. All rights reserved.

De-scrambling Design
All TLPs, DLLPs, and Idles are scrambled Required by the PCIe specification to reduce EMI noise De-scrambling design places a descrambler on both the TX and RX path Descrambler taps off TX and RX data path Produces data in legible format Packet decoder also included to provide trigger signals for ChipScope Allows user to view traffic at the MGTs using ChipScope Pro Also works in simulation The De-scrambling design will be available as an Answer Record

100

Copyright 2009. Avnet, Inc. All rights reserved.

De-scrambler Block Diagram


FPGA ChipScope ILA Block RAM Interface TX De-scrambler

101

Transaction Layer Module

Data Link Layer Module

Physical Layer Module

GTP/GTX Transceivers

PCIe Block

RX De-scrambler

ChipScope ILA

Copyright 2009. Avnet, Inc. All rights reserved.

How is De-scrambling Useful?


Many customers do not have link analyzer This is not a replacement for a link analyzer (better than nothing at all) Verify ACKs/NAKs being returned as expected Excessive NAKs or no ACK/NAKs at all could cause problems Verify flow control is progressing Sometimes, there seems to be a stall on the link and customers assume its the PCIe blocks fault Being able to verify flow control credits are being returned and the ACK/NAK status may be insightful Verify integrity of packet Many cases have come up where customer believes block is manipulating packets Usually, this turns out to be a software issue Using this design will allow customer to verify packet integrity right before it goes into MGTs or as it comes out of MGTs
Copyright 2009. Avnet, Inc. All rights reserved.

102

Agenda
Serial communications definitions and terminologies Spartan-6 PCIe Block overview Debugging PCIe Designs Using the Aldec Simulator

103

Copyright 2009. Avnet, Inc. All rights reserved.

Table of Contents
What ALDEC Simulators Can Do? ALDEC XILINX Support Timeline ALDEC and XILINX Starting Active-HDL from ISE Xilinx Tools in Active-HDL Flow Starting Coregen from Active-HDL

104

Copyright 2009. Avnet, Inc. All rights reserved.

What ALDEC Simulators Can Do?


Simulate Xilinx ISE designs using any combination of source languages: VHDL, Verilog, System Verilog, SystemC, EDIF, etc. Include pre-compiled Xilinx libraries (including SecureIP) Support latest Xilinx families: Spartan-6 and Virtex-6 Support all popular IP Cores (PCIe, RocketIO Transceivers, SERDES, EMAC/TEMAC, etc.) Support SmartModels/SWIFT Can handle DSP co-simulation (MATLAB and Simulink interfaces ) Simulate structural MPU models from EDK Support all legacy Xilinx families (all versions of Virtex, Spartan, Coolrunner, XC4000, XC9500)

105

Copyright 2009. Avnet, Inc. All rights reserved.

Aldec Active-HDL 8.2


Common-Kernel Mixed Language Simulator Languages: VHDL, Verilog, SystemVerilog (Design & Assertions), SystemC & EDIF HDL Design Tools: Design Entry, Design Creation, Code2Graphics, Block and State Diagram, Waveform Editor, stimulus generation, code auto-complete and language templates, scripting, legacy design support. Design Flow Manager: use popular third-party tools throughout the design flow within the same FPGA environment. Debugging: Code execution tracing, Waveform/Compare, Memory Viewer, Xtrace, Advanced Dataflow and Profiler. Coverage: Code Coverage, Toggle & Functional Coverage. Additional Interfaces: MATLAB and Simulink cosimulation interfaces, Zuken CADSTAR PCB Design interface Assertion and Coverage(OPTION) SystemVerilog, PSL & OVA support; dedicated Assertion Viewer, assertion coverage, assertion breakpoints.

106

Users who need Linux-based simulation tools or are not interested in graphical entry may want to try Riviera-PRO line of high-performance simulators.

Copyright 2009. Avnet, Inc. All rights reserved.

ALDEC XILINX Support Timeline

107

Aldec and Xilinx sign Cooperation Agreement First Aldec simulators with SecureIP support
(Active-HDL 8.1sp2 and Riviera-PRO 2009.02)

Over 30 mutual customers confirm working SecureIP

Q3

Q4

Q1

Q2

Q3

Q4

Q1

Q2

2008

2009

2010

Second release of Aldec simulators with SecureIP for


Spartan-6 and Virtex-6 (Active-HDL 8.2 and Riviera-PRO 2009.06SR1)

ISE Design Suite 12.1 with SecureIP sources


supporting Aldec keys and Library Compiler support for Aldec
Copyright 2009. Avnet, Inc. All rights reserved.

ALDEC and XILINX


Aldec was the creator of design entry, project management and gatelevel simulation tools in Xilinx Foundation Classic suites (our tools can import Foundation schematic designs and turn them into HDL designs) Aldec tools support design flows including all versions of Xilinx tools, starting from Xilinx Alliance/Foundation Classic up to Xilinx ISE 11.2 Xilinx precompiled HDL libraries and SecureIP are currently available directly from Aldec Xilinx library sources (including SecureIP) will compile for Aldec simulators using compxlib in ISE Design Suite 12.1

108

Copyright 2009. Avnet, Inc. All rights reserved.

Starting Active-HDL from ISE


With very little effort, it is possible to start Active-HDL simulator from Xilinx ISE Project Navigator to run behavioral and timing simulations The setup procedure is described in the application note Starting Active-HDL as the Default Simulator in Xilinx ISE available on ALDEC website (follow the link below or do the search on the title of the document) The application note provides detailed, easy to follow instructions and the link to multimedia presentation demonstrating the use of the interface http://support.aldec.com/KnowledgeBase/Article.aspx?aid=000771

109

Copyright 2009. Avnet, Inc. All rights reserved.

Xilinx Tools in Active-HDL Flow


Active-HDL is equipped with Design Flow Manager that, when configured properly, allows quick start of Xilinx or third party synthesis tools and ISE implementation engine Options required to start each application can be configured in userfriendly GUI windows Simulation files are imported back to Active-HDL Applications like Coregen, Constraints Editor, STA, ChipScope Pro can be stared from the Flow Detailed instructions are available in the Using Active-HDL with Xilinx ISE application note (link listed below) http://support.aldec.com/KnowledgeBase/Article.aspx?aid=000640

110

Copyright 2009. Avnet, Inc. All rights reserved.

111

STARTING COREGEN FROM ACTIVE-HDL

Copyright 2009. Avnet, Inc. All rights reserved.

Flow Enabling
To access Coregen in Xilinx ISE 11.2 from Active-HDL, user has to enable Design Flow Manager in Preferences window accessible from the Tools menu in Active-HDL

112

Copyright 2009. Avnet, Inc. All rights reserved.

Flow Tools Configuration


After changes of installed tool versions users should update locations of synthesis and implementation tools in the Tools | Preferences window, Environment | Flows | Integrated Tools category (both version and location of each tool can be modified)

113

Copyright 2009. Avnet, Inc. All rights reserved.

Tool/Version Selection
When Active-HDL installation is up-to-date, the latest Xilinx tool versions should be visible in flow configuration

114

Copyright 2009. Avnet, Inc. All rights reserved.

Showing the Flow Manager


Once enabled, Design Flow Manager can be displayed in Active-HDL using main toolbar button: Displayed Design Flow Manager looks like this:

115

Copyright 2009. Avnet, Inc. All rights reserved.

Flow Modifications
If quick change of tool version or default Xilinx family is needed, it is possible in Flow Configuration Settings window displayed after clicking Flow Settings button

116

Copyright 2009. Avnet, Inc. All rights reserved.

Starting Coregen
To start Coregen from the Flow, user should click Tools button in the Design Flow Manager and select CoreGen & Architecture Wizard button in the popup window (see the illustration below)

117

Copyright 2009. Avnet, Inc. All rights reserved.

Coregen Interface Window


Active-HDL provides interface window that allows starting Coregen and managing generated cores Users should adjust basic options before clicking Run CORE Generator button (see illustration below)

118

Copyright 2009. Avnet, Inc. All rights reserved.

Selecting GTP in Coregen


Once Xilinx CORE Generator window shows, user should follow the typical steps to generate Spartan-6 GTP core Be sure to select correct chip and save log before closing this window

119

Copyright 2009. Avnet, Inc. All rights reserved.

Importing Files to Active-HDL


After Xilinx CORE Generator window closing, generated IP will show up in the Active-HDL interface window Users should select the IP, then click Add IP Core File to transfer files needed for simulation back to Active-HDL

120

Copyright 2009. Avnet, Inc. All rights reserved.

Other Options for PCIe Simulation


HDL models of PCIe are accurate, but slow For increased simulation speed, Bus Functional Models (BFM) of PCIe are available as Verification IP Aldec simulators are officially supported by many respectable IP vendors, including providers of PCIe cores: Northwest Logic with its PCI Express Verification Suite nSys with its PCI Express nVS (golden standard in PCIe verification) To get more info, visit IP Products page in the Products section of our website http://www.aldec.com/products/ipcores

121

Copyright 2009. Avnet, Inc. All rights reserved.

Simulation
No matter if you are using VHDL, Verilog or both: our simulators will be able to handle your design If you need to get result quickly you have speed Expect performance matching leading competitors, but in easier to use package If you have to investigate some design issues you have extensive debugging capabilities

122

Happy Simulating!

Copyright 2009. Avnet, Inc. All rights reserved.