Author: Don Harbin
Intel Corporation

As the limits of technology have pushed the traditional parallel bus technology toward serial technologies, it makes sense that this phenomenon should occur on both sides of an I/O controller. And this indeed is what is occurring with the onset of PCI Express (PCIe). This article will discuss how I/O controllers are advancing and how PCIe will be utilized to meet the performance demands of the future.

In order to attain higher speed data buses, designs have moved from the traditional parallel bus technologies to serial-based technologies, which employ features such as differential pair transmit / receive data lines. The entry of serial technologies such as Serial ATA, Serial Attached SCSI (SAS) and PCIe, are displacing Parallel IDE, Parallel SCSI and PCI-X in turn.

SAS bandwidth starts at 3 Gbits/sec and will increase over time towards 6 and 12 Gbits/sec. Today’s PCI 64 bit / 66 MHz performance tops out at 532 Mbytes/sec while PCI-X 64 / 133 goes up to 1064 Mbytes/sec. Although PCI-X is somewhat competitive, it lacks a roadmap to keep up with SAS advances over time.

For the host side of SAS controllers to keep up with increased performance demands, PCIe-based I/O controllers are being developed. These next generation controllers will make use of this higher performance technology and provide a growth path to keep pace with the SAS performance roadmap.

PCIe is a serial technology designed to replace PCI-X. PCIe is composed of lanes, each of which is made up of two differential pairs of signals; one to transmit and one to receive. Current systems combine multiple lanes together to aggregate performance to meet high demand applications. This effectively creates a parallel bus of serial transmit / receive pairs. Implementations can consist of PCIe buses with 1, 2, 4, 8, 16, and 32 PCIe lanes. One PCIe lane at the currently supported 2.5 Gbits/sec will support about 250 Mbytes/sec in each direction simultaneously (full-duplex). For example, in order to meet PCI-X 64 / 133 performance numbers, four aggregated PCIe lanes are required.

There are however, two important points to consider. First, PCIe is full duplex, so data can be sent and received simultaneously. This will nearly double the total performance in an application that uses both transmit and receive in all four lanes simultaneously. Second, as described earlier, PCIe (currently 2.5 Gbits/sec) will provide a performance roadmap to keep pace with the increased bit rates planned for SAS. Future implementations of PCIe are being planned for 5 Gbits/sec and higher.

So how does all of this impact potential SAS controller designs? First, expect to see controller solutions that match the number of PCIe lanes with the number of SAS I/O connections. For example, if a controller has eight back-end SAS I/O channels, it makes sense to provide eight PCIe lanes on the front end, as these two technologies will likely continue to increase their performance at a similar pace. The current PCIe single lane performance of 2.5 Gbs/sec matches nicely with that of the current SAS bit rate of 3 Gbits/sec. Though the back-end I/O SAS channels will be point-to-point, it should be clarified that the eight PCIe lanes on the front end of the controller will likely be configured as a bus, aggregating the data lanes and passing the aggregate up-stream to processor(s) in the server architecture.

Chipsets are also moving to support the PCIe technologies, so controllers that are compliant with PCIe will logically become the storage south bridge of these architectures. Lastly, developers will be able to easily customize solutions to meet the demands of their storage applications due to the advantages PCIe switching technology provides. PCIe will furnish an excellent architecture transition into unique designs that include higher reliability and multi-path failover support, to name two advantages.

For more information on PCIe technologies, please see the following site: http://www.pcisig.com/specifications/pciexpress.

One can only guess what the future holds beyond SAS and PCIe. The two technologies are turning out to be excellent complements to each other in the near-term. Beyond that, performance will likely be gated by new and inventive technology in this fast-paced area of transition. For now, I/O technology is moving to serial.