Documentation

Hardware and PCI Overview FAQs

Please send any updates, additions, or questions on the developer forums at http://forum.sun.com 10. Hardware related questions
10.1 What are the power consumption and dissipation of the PCI cards/slots? 10.2 Will Sun make a bridge chip, like with Sbus SLIC chip available so that it`s easier to interface with PCI? 10.3 How many bytes are in a burst? Does it go by bytes or data phase? 10.4 What values should I use in the cache line register and latency timer register? 10.5 What form factor should I use for my PCI card? 10.6 Is there a approved set of PCI-PCI bridge chips for Sun developers? 10.7 What size mounting brackets should I use ? PCI 2.1 specs talks about different sizes for EISA, MC ..etc. 10.8 Is it possible to program current Sun PCI bridge chip to enter a 32-byte burst mode rather than 64 on the write cycle? Can it enter non-burst mode on the read cycle? Do any of the configuration register settings affect these bursts? 10.9 If target device A has to send a DISCONNECT message after accepting 32 bytes of the 64-byte burst phase during the write cycle (due to its receive buffer size constraints), what happens to the other 32-bytes that is waiting to be transferred? Will it wait for the device to come back and re-acquire the bus? 10.10 What value does the OBP put into the PCI device command register at boot time? 10.11 Does current Sun PCI bridge chip implement retry-counter mechanism while trying to establish connection to a target device that is issuing retry ? If so, how many times will the current Sun PCI bridge chip retry before giving up on this target ? In other words, what is the Retry counter limit on current Sun PCI bridge chip ? ( I read that this is PCI recommended but not a requirement.)< 10.12 What is the cache line size for the UltraSPARC PCI machines? 10.13 Concerning the use of PCI Memory Write (MW) and Memory Write and Invalidate (MWI), what is the system preference and implication if misused? 10.14 What are the sizes and numbers of PCI write-posted buffers found in the PCI bridge device? Can a PCI initiator overflow these buffers? If so, at what PCI transfer rate? 10.15 Concerning the use of PCI MR, MRL, and MRM, what is the system preference and what are the implications if misused? 10.16 Does the PCI bridge device use prefetch for PCI MRM and MRL? Is prefetch a dynamic or static assignment to address regions or pages? Does the bridge device have

any priority requester capability, that is, favor a PCI requester or host memory region? 10.17 How many cache lines does the bridge device prefetch for MRM or MRL? 10.18 How many cache lines need to be read to achieve adequate system and PCI performance, (and not negatively impact system or PCI prefetch)? 10.19 Is the use of 32 bit PCI addressing a performance limitation in this system? Does using 64 bit PCI addressing increase performance? 10.20 How are prefetch buffers in the bridge affected by writes, interrupts, or reads? 10.21 How many interrupt lines are provided per PCI expansion card slot? Are the INTA# through INTD# interrupt lines shared?

11. PCI Overview

11.1 11.2 11.3 11.4 11.5 What is PCI ? What are some of the key features of the PCI bus? Do PCI drivers require a configuration file? Does Sun Support PCI 2.2 specification? Does Sun Support PCI-X specification.

Questions and Answers 10. Hardware related questions

10.1 What are the power consumption and dissipation of the PCI cards/slots? The Spec States that the Maximum Power a PCI slot can draw is 25 Watts The system must supply the following: 5V +/-5% 5A Max. per connector 3.3V +/-.3V &.6A Max. per connector 12V +/-5% 500ma per connector -12V +/-10% 100ma per connector. 10.2 Will Sun make a bridge chip, like with Sbus SLIC chip available so that it's easier to interface with PCI ? No. There are several of these available on the market. The PCI-SIG group should keep a list of these. 10.3 How many bytes in a burst? Does it go by bytes or data phase? The number of bytes can be any size. The answer depends on the PCI device and the PCI bridge. PCI supports variable size bursts. Existing Sun systems will disconnect at 64 byte boundaries. Chips designed and supported by Sun will burst at 64 bytes. Bursting for the device is controlled by the latency timer and cache line size registers in the devices configuration header block.

10.4 What values should I use in the cache line register and latency timer register? The setting of the cache line size and latency timer registers should be handled by the OpenBoot PROM (OBP). 10.5 What form factor should I use for my PCI card ? Standard cards (not PMC, ie. mezzanine), the shorter the better, should be used. Sun will most likely continue to design systems to handle the PCI long card (standard) form factor in at least one of the slots. These cards measure 12.283" by 4.2". However, the industry trend to date has been to use the short card, which measures max 4.2" x 6.875". The term max is used because the specification allows a variable length short card down to a height of 1.42". 10.6 Is there a approved set of PCI-PCI bridge chips for Sun developers? There is a generic PCI to PCI bridge driver in the Solaris environment, and most of the available PCI to PCI bridge chips are likely to work as well. 10.7 What size mounting brackets should I use ? PCI 2.1 specs talks about different sizes for EISA, MC ..etc. If the developer plans to sell the card to someone other than Sun, then the specification says that both EISA and MC mounting brackets must be supplied. Sun systems accepts ISA type brackets. 10.8 Is it possible to program current Sun PCI bridge chip to enter a 32-byte burst mode rather than 64 on the write cycle ? Can it enter non-burst mode on the read cycle ? Do any of the configuration register settings affect these bursts? The current Sun PCI bridge chip cannot be programmed to change the attempted burst size on a PIO transaction to PCI. The current Sun PCI bridge chip will always try to complete an entire UPA transaction in a single burst. Using different UPA transactions (which translates to using different ld and st commands) can control some of the bursting behavior, but a 32-byte UPA transaction doesn't exist. However, if the target disconnects after 32-bytes, then effectively, each 64-byte transaction will turn into two 32-byte bursts. 10.9 If target device A has to send a DISCONNECT message after accepting 32 bytes of the 64-byte burst phase during the write cycle (due to its receive buffer size constraints), what happens to the other 32-bytes that is waiting to be transferred? Will it wait for the device to come back and re-acquire the bus? If any current Sun PCI bridge chip receives a disconnect before it has completed the

entire data transfer for a PIO transaction, the chip will re-issue the transaction, with the address appropriately incremented. The target is not required to take any action. If disconnected again, it will keep re-issuing the transaction until it is complete. In the worst case, trying to do a 64-byte burst to a device that only accepts words, the bridge chip will end up doing 16 separate PCI transactions to satisfy the single 64-byte UPA transaction. Back to Top 10.10 What value does the OBP put into the PCI device command register at boot time? The serial PROM on the card (if present) loads the configuration registers with default values as programmed by the card manufacturer. The PCI bus bindings (IEEE-1275) document states that, during the probe-pci operation at boot time, "Disable fixedaddress response by clearing the Bus Master, Memory space and I/O space bits in the Command register." At boot time, the PCI nexus driver will load the command register with default values. If the device driver does not agree with these values, it can change them to enable proper operation modes. 10.11 Does Sun PCI bridge chip implement a retry-counter mechanism while trying to establish connection to a target device that is issuing retry ? If so, how many times will the current Sun PCI bridge chips retry before giving up on this target ? In other words, what is the retry counter limit on current Sun PCI bridge chips ? ( I read that this is PCI recommended but not a requirement.) The retry counter limit is 16K (16384) and is enabled by default. The counter can be turned off using a bit in the PCI Diag register in the PCI Bus Module (PBM) block. If the retry limit is enabled, and it is reached, the system behavior depends on whether the PIO was a write or a read. For PIO reads, a timeout (P_RTO) response on the UPA bus is returned. For PIO writes, the error is logged into the PCI AFSR/AFAR, and an interrupt is generated if enabled. While a PIO is being retried on the PCI bus, DMA traffic will still be accepted and processed. However, since PIOs are handled one at a time, in order, no other PIO will get through until the first one finishes or times out. 10.12 What is the cache line size for the UltraSPARC PCI machines? The cache line size is 64 bytes. This is also the value written into each PCI device's cache line size register, if supported. If a value of 64 is not supported, 0 is written into the register. Please refer to the PCI bus bindings (IEEE-1275) for information. 10.13 Concerning the use of PCI Memory Write (MW) and Memory Write and Invalidate (MWI) commands, what is the system preference and implication if

misused? All PCI-bridges designed by Sun treat MW and MWI identically at all times. 10.14 What are the sizes and numbers of PCI write-posted buffers found in the PCI bridge device? Can a PCI initiator overflow these buffers? If so, at what PCI transfer rate will this occur? The answer depends upon the mode used - streaming or consistent.

Consistent: two write buffers Streaming : In one sense, 16 write buffers, in another sense, only one buffer (per page).

These buffers can't be overflowed in the sense of an error; PCI devices will be retried if resources aren't available to complete their transaction. Due to speed/bandwidth differences, in general this will only happen if the UPA bus is very busy. 10.15 Concerning the use of PCI MR, MRL and MRM, what is the system preference and are the implications if misused? In consistent mode, all three commands are treated identically. In streaming mode, MRL and MRM are treated identically, and the device is expected to read at least to the end of the cache line. If misused, there is a performance penalty - wasted prefetch transactions on the UPA, plus the possibly of an extra delay when the next cache line finally is reached. 10.16 Does the PCI bridge device use prefetch for PCI MRM and MRL? Is prefetch a dynamic or static assignment to address regions or pages? Does the bridge device have any priority requester capability, that is favor a PCI requester or host memory region? Consistent mode to cacheable memory prefetches for all reads in the sense that it always gets an entire cache line, regardless of the amount eventually transferred to the PCI device. (The only example of non-cacheable memory that a PCI device could perform is direct memory access (DMA) to a UPA framebuffer.) Streaming mode is somewhat similar; initial requests for data always get entire cache lines. In addition, whenever the end of a cache line is expected to be reached, the next cache line is prefetched (except at 8K page boundaries). As opposed to consistent mode, streaming mode is enabled on a per page basis, but otherwise there is no control over prefetching behavior. 10.17 How many cache lines does the bridge device prefetch for MRM or MRL? In consistent mode, one cache line is read for each MRM or MRL command (not

really a prefetch). In streaming mode, the cache line requested by the MRM or MRL is fetched and if needed, the next cache line is prefetched. 10.18 How many cache lines must be read to achieve adequate system and PCI performance, (and not negatively impact system or PCI prefetch)? All current SUN PCI bridge chips disconnects all transactions at cache line boundaries. A device that bursts 64 bytes at a time should be capable of getting the maximum performance. 10.19 Is the use of 32 bit PCI addressing a performance limitation in this system? Does using 64 bit PCI addressing increase performance? 32-bit addressing is not a limitation. 64-bit addressing does not increase performance (its only use is to bypass the IOMMU, so it should only be used by devices that provide their own IOMMU or equivalent). 10.20 How are prefetch buffers in the bridge affected by writes, interrupts or reads? The consistent mode prefetch buffers are invalidated when the read disconnects, so are no longer around to be affected by writes, interrupts, or further reads. The streaming buffers are flushed or invalidated by software, so they are in general unaffected by other PCI activity. The exception is multiple transactions in the same 8K page that use the same buffer, so a write to the same page invalidates prefetched read data, a read from the same page flushes a write buffer, etc. 10.21 How many interrupt lines are provided per PCI expansion card slot? Are the INTA# through INTD# interrupt lines shared? The number of interrupt lines is dependent on the system. In general, systems attempt to avoid sharing the interrupts. However, if there are few enough slots in the system, this is possible. Back to Top
11. PCI overview

11.1 What is PCI ? PCI is the abbreviation for 'Peripheral Component Interconnect'. It is a high performance 32/64-bit local bus that provides a host processor independent interface and an interconnect mechanism between highly integrated peripheral components, add-on devices and main memory, thus allowing high bandwidth data transfers. It

usually resides on the system board and provides the highest level of synchronization of operation with the host processor, than any other IO bus today. 11.2 What are some of the key features of the PCI bus ? Processor Independence: PCI device interface specifications are very PCI specific and are isolated from the host processor design. Support for up to 256 PCI functional devices per PCI bus: PCI bus logically supports up to 32 physical devices, each containing up to 8 separate PCI functions, thus allowing for a total 256 possible PCI functions per PCI bus. ( While this is possible theoretically, the practical story is different. A single PCI bus is limited to not more than 4 to 6 plug-in slots.) Bus Speed Revision: The PCI 2.0 specification supports PCI bus speeds up to 33 MHz. Revision 2.1 adds support for 66 MHz bus operation. 64-bit Bus Width: Apart from the 32-bit / 33 MHz support, PCI supports 64-bit devices at 66 MHz allowing a peak transfer rate of up to 528 Mbytes/sec. All PCI-bus based Sun Servers and Workstations except UltraSPARC 5 and and UltraSPARC 10, support 32/64-bit and 33/66 Mhz High Bandwidth: 132 MBytes/sec peak transfer rate for both read and write transfers at 32bit/33 MHz, 264 MBytes/sec peak transfer rate for 64-bit PCI transfers at 33 MHz, up to 528 MBytes/sec transfer rate is achievable with 64bit/66 MHz transfers. Except for low-end UltraSPARC 5 and UltraSPARC 10, most PCI based Sun systems supports both 33/66 Mhz buses and 32/64 bit. For detailed information about the number of 64/66MHz and 32/64bit 33MHz slots per platform, please refer to: http://www.sun.com/desktop/products/ http://www.sun.com/servers/

Auto-configuration: Full bit-level specification of the configuration registers necessary to support automatic peripheral detection and configuration is available. In addition to the features listed here, PCI also supports bus master, hidden bus arbitration, low-power consumption, and concurrent bus operation. 11.3 Do PCI devices require a configuration file? A PCI device is self-identifying, and hence does not require a configuration file. At boot time, the system identifies the device and its driver by reading the configuration parameters from its configuration space. However if you need to supply driver private properties, they are specified in a configuration file. See driver.conf(4) and pci(4) for more information. 11.4 Does Sun Support PCI 2.2 specification? Current Sun UltraSPARC based machines are PCI 2.1 compliant. However, many of the features which are optional in the PCI 2.1 specification and are now part of the PCI 2.2 specification, have been implemented. You can verify whether a given optional 2.1 feature is present. Sun is implementing the PCI 2.2 specification for future PCI-PCI bridge asics and next generation workstations and servers. 11.5 Does Sun Support PCI-X specification. Sun has no current plans to support PCI-X.