Author: Sam Barnett, Product Line Manager, Serial Attached SCSI and Serial ATA Storage & Storage Networking Products, Storage Products Division,
Vitesse Semiconductor
As first generation systems based on Serial Attached SCSI (SAS) technology enter the market, leading server and storage OEMs are racing to deliver the next group of platforms based on an ever-evolving technology landscape. Vitesse opted to enhance its popular NexSAS™ product family concurrent with the release of generation 1 systems. This leap-frogging approach to the deployment of SAS technology has led to the introduction of many new products with features found only in high availability, superior performing, Fibre Channel systems today. Comprised of three new high-PHY-count expanders, two new RAID-on-CHIP offerings, extensions in intelligent mux/de-mux devices, a new enclosure management/SAS backplane controller, and high-rate, agile signal conditioners, the NexSAS™ line delivers on the promise of high-performance storage solutions for SMB (small/medium business) as well as enterprise systems.
The Vitesse NexSAS™ product family is divided into four functional areas: expander technology, RAID-on-CHIP, SAS/SATA Support devices, and Enclosure Management. This article explores the Generation 1.5 NexSAS™ expander and RAID-on-CHIP technology in detail.
NexSAS™ Expander Technology
Designed with high-performance interconnect in mind; the NexSAS™ expander family takes advantage of decades of proven switching technology. A single functional design ensures scalability from low-port offerings to the high-port-count devices needed for blade servers or storage enclosures.
All Vitesse expanders divide the core systems function into two separate components – the expander and the management functions. Included in the expander functional block are the Expander Connection Manager (ECM), Expander Connection Router (ECR), the Broadcast Primitive Processor (BPP), and the actual physical interfaces (PHYs) to the device (ranging in number from 6 to 36 depending on application need).
Rounding out the core functions is the management function block, which provides for expander management (SMP) and vendor-specific enclosure management functions (SSP/SES). Also included in this block are various peripheral interfaces needed for interaction in the overall system. These interfaces include two-wire serial, JTAG, Universal Asynchronous Receiver/Transmitter (UARTs), general purpose I/O (GPIO), and serial peripheral interfaces. The number and types of these interfaces vary with the particular expander and specific operating mode(s) employed.
Figure 1 outlines the basic functional block diagram of all expanders in the NexSAS™ family.
Figure 1 – Generation 1.5 Expander Functional Diagram
General Features
All Generation 1.5 Vitesse expanders share a common set of features:
- High Performance Switching Architecture – low latency, non-blocking switching matrix that provides up to 54 Gb/s of aggregate switching capacity (in large-port-count expanders with linear scaling for lower-port-count expanders).
- Vitesse Universal PHYs – self-configuring, rate adaptable PHYs supporting SAS initiators and SAS/SATA targets at both 1.5 and 3.0 Gb/s data rates; multiple per-phy pre-emphasis/de-emphasis capability, and multiple per-phy output levels built on over 20 years of Vitesse long-haul and ultra-long-haul transport technology. They may be combined into SAS “wide-ports” of up to n phys wide.
- Virtual Initiator and Target Capability – SMP (Serial Management Protocol), SSP (Serial SCSI Protocol), and STP (Serial ATA Tunneling Protocol) initiator and target functions enable robust value-added services including self-configuration, self-discovery, SCSI Enclosure Services (SES), rogue drive identification, etc.
- Integrated Enclosure Management Subsystem – robust EM capabilities maximize OEM investment in legacy firmware/software/custom features and enable flexibility in system designs.
- Integrated Ethernet Interfaces – two 10/100 Ethernet MACs are included in the EM subsystem to support blade server applications or other applications where Ethernet access to the expander is required.
- Multi-affiliation STP/SATA Bridge – extensions of the original STP/SATA bridge provide for two concurrent (active/active) Native Command Queuing (NCQ) affiliations between initiators and targets using Vitesse expander technology. This innovative approach eliminates the initiator starvation issues commonly associated with the single affiliation mechanisms in SAS 1.1.
- End-to-end SAS-2 Zoning – for security, control, and performance, both SAS-2 PHY-resolved and address resolved zoning are supported.
- Numerous other enhancements beyond Generation 1 devices including EPOW (Environmental and Power Warning) and parity protection on the on-chip memory.
Recent additions to the NexSAS™ family include the VSC7156 24-PHY, VSC7157 36-PHY, and VSC7158 18-PHY expanders. Like their predecessors, the VSC7153 and VSC7154 (Generation 1 expanders), these devices are also targeted at both enterprise server and enclosure applications.
The feature additions outlined above bridge the gap between the SAS 1.1 and planned SAS-2 specifications and raise the bar on both functional integration and price/performance for today’s SAS market.
Two of the key differentiating features, the multi-affiliation STP/SATA Bridge and SAS-2 zoning, are examined in more detail in the following sections.
Understanding the NexSAS™ Multi-Affiliation STP/SATA Bridge
STP/SATA Bridge Background
The SAS 1.1 specification defines the STP/SATA Bridge as the expander component that provides the “bridging function” between STP-capable initiators and SATA targets (HDDs). Without an STP/SATA bridge, Serial Attached SCSI could not support Serial ATA drives. In general, the SAS 1.1 specification defines the function of the STP/SATA Bridge as follows:
- In SAS expanders, the STP/SATA bridge allows the connection of SATA (Serial ATA) drives to the SAS domain
- The SAS protocol is used for connection setup/tear-down between the host (initiator) and the STP/SATA bridge
- The STP/SATA bridge only passes native SATA protocol to the SATA drive during connection
- Expanders typically have one STP/SATA bridge available in each PHY, but the bridge is enabled only when connected to a SATA device
Figure 2 graphically illustrates the STP/SATA bridge concept.
Figure 2 – STP/SATA Bridge
Unlike SAS drives, SATA drives have no concept of multiple host or multi-initiator concurrent access. During a “session,” one STP initiator will maintain an “affiliation” with the SATA target. During this “connected state,” any other STP initiator attempting to gain access to the same SATA device will receive an OPEN_REJECT (STP RESOURCES BUSY) and will be forced to retry the connection attempt. This simple affiliation mechanism maintains coherency by allowing commands from only one host to be active on the drive at any given time.
When all outstanding commands from a given initiator have completed, the initiator should send a special CLOSE (CLEAR AFFILIATION) primitive to the STP/SATA Bridge, resulting in release of the affiliation and allowing other hosts to access the drive/resource.
Generation 1 expander devices support only a single affiliation in compliance with the SAS 1.1.specification.
With any new technology, implementation tends to uncover certain undesired behaviors. The original STP/SATA bridging mechanism was no different. Limitations of the originally envisioned bridge involve both fairness and performance. Manifestations of these limitations are outlined briefly below:
Fairness
- There are no formal limits imposed on the longevity of a given affiliation, meaning that an affiliation could be established forever.
- Fair access to SATA devices must be enforced by the initiators and in the absence of host-to-host communications, this may not be assured.
- Software algorithms to achieve fairness have proven painful for OEMs.
- Existing real-world STP initiators don’t play fair and occasionally fail to release affiliations.
- If a valid affiliation exists between the expander STP/SATA Bridge and an initiator, other hosts cannot perform any inquiries to the drive, including diagnostic ones.
Performance
- Single threaded host access to SATA disks is slow in load-balancing systems.
Figure 3 depicts the SATA Affiliation concept in SAS.
Figure 3 – The SATA Affiliation Concept
The NexSAS™ Generation 1.5 expander products address both the fairness and performance limitations of the SAS 1.1 STP/SATA Bridge. Through the updated bridge function, two hosts can simultaneously have active commands on the same SATA drive.
Figure 4 illustrates the Multi-affiliation STP/SATA bridge operation.
Figure 4
Details of the new bridge operation are summarized below:
Intelligent Connection Management
- Flexible, non-exclusive (overlaps are allowed) disconnect policies including:
Timed disconnect (multiple modes and ranges)
Disconnect after any FIS (frame information structure) transfer
Disconnect after data FIS transfer
Disconnect after interlocked operations - Automatic callback of the proper initiator (host) based on queue tag/context
NCQ and Programmed I/O (PIO) Command Support
- Supports utilization of all 32 NCQ commands tags (or fewer by configuration) on the SATA drive
- Manages interleaving of NCQ and PIO commands from different hosts
- Tags are transparently managed and remapped to/from the drive to avoid tag number collisions between initiators (no special host software or firmware is required)
In all, the multi-affiliation STP/SATA Bridge employed by all Generation 1.5 NexSAS™ expanders enhances fairness, performance, and system design convenience. Similarly, it benefits any SAS domain using SATA disks shared by two or more hosts due to an architecture that can scale for future products requiring even more active affiliations while remaining completely compliant to the SAS 1.1 and proposed SAS-2 specifications.
Understanding the Basics of SAS-2 Zoning
As storage OEMs and integrators embarked on the introduction and deployment of SAS-based systems, many longed for the days of Fibre Channel and some of the enterprise features it offers for traffic management (performance) and security.
To address those demands, the concept of end-to-end zoning was introduced into the proposed SAS-2 specification (please refer to www.t10.org for the latest on the zoning specification) by key industry leaders.
The SAS protocol, as defined, supports up to 16,384 devices (initiators, targets, expanders, and/or virtual devices) in a given domain; however, practical implementation of such large topologies is quite problematic. In essence, end-to-end zoning provides the mechanism necessary to manage these large physical topologies in smaller logical (segmented) groups. SAS zoning provides this function as well as the mechanisms to control access among and between groups.
The SAS-2 proposed specification divides zoning into two areas: PHY-resolved zoning and address-resolved zoning. Both approaches utilize identical access control methodologies but differ in zone group allocation.
PHY-Resolved Zoning
In a PHY-resolved zoning implementation, each PHY of “zoning capable” expander is assigned to a zone group and any device attached to that PHY (or port in the case of PHY grouping for wide ports) becomes part of that zone group. A permission table in the expander contains access control permissions that outline the protocol for “sessions” between devices in differing zone groups (by default, all devices within a zone group can interact with each other).
PHY-resolved zoning is ideal for smaller topologies, blade-server, or clustering applications.
Address-Resolved Zoning
Address-resolved zoning provides a layer of abstraction not present in phy-resolved zoning. In an address-resolved scheme, self-discovering expanders interpret each device address and populate a table. This table contains routing information and zone permissions. Permissions in the table as well as zone group assignment are combined to determine what level of interaction any two devices may have with each other.
Typically, address-resolved zoning is used for larger topologies or where more granular control is needed.
NexSAS™ RAID-on-CHIP (ROC) Technology
To address the growing need for end-to-end SAS solutions, Vitesse introduced the VSC7250 and VSC7251 single-chip RAID controllers. Combined with a full-software suite, these devices provide a low-cost, high-performance alternative to traditional board-level RAID products.
Each device offers PCI-Express host connectivity linked to independent SAS PHYs, which support direct or expander connections to both SAS and SATA targets as well as other SAS 1.1-compliant devices.
The VSC7250 is ideal for high performance applications needing up to eight PCI-Express lanes and eight SAS PHYs. Similarly, the VSC7251 supports all the RAID features of the VSC7250 while providing a lower price/performance point by reducing the PCI-Express interface to four lanes communicating with four SAS PHYs.
Raw chip performance of both devices is derived from an embedded microprocessor complex for RAID calculations and an intelligent DDR-II memory controller for high-speed RAID processing functions – designed to maximize PCI-bandwidth availability while minimizing host-memory dependence. In addition to these key performance features, both the VSC7250 and VSC7251 provide a Flash/NVSRAM/SRAM interface to simplify system design and separate RAID XOR and DMA controllers have been optimized for data throughput (up to 4 GB/s).
The NexSAS™ ROC family operates with a commercially available software offering, stratified to address the needs of multiple market segments (offerings include RAID 0,1,5,6, & 10).
Figure 5 depicts a typical VSC7250 application in the ROMB (RAID-on-Motherboard) space.
Features of the VSC7250 and VSC7251 are outlined below:
- RAID-on-CHIP – minimizes bill-of-materials costs and printed circuit board real estate.
- Integrated PCI-Express – the VSC7250 supports up to eight host processor connections via x1, x2, x4, or x8 lane PCI-Express links, while the VSC7251 supports up to four PCI-Express links.
- Independent SAS/SATA PHYs – provide direct or expander-based connections to SAS or SATA HDDs or other SAS 1.1 compliant devices.
- Embedded Microprocessor Complex – RAID-optimized processor offloads host CPU from RAID calculations, volume rebuilds, and numerous interrupt requests for I/O processing.
- On-Chip RAID 5 and 6 Acceleration – RAID XOR calculations required for data generation are optimized for maximum throughput while data guarding protects against catastrophic drive failures.
- DDR-II External Memory Interface – High performance memory interface supports optimized I/O and RAID processing.
Conclusion
In all, the future of SAS storage systems based on NexSAS™ technology are dependent on the needs of the end-user. End-to-end solutions like those provided by Vitesse enable unique OEM applications while features of discrete components give way to multiple usage models.
With any new technology, be it SAS or something else, one size will never fit all. Proper solutions will differ by application, connectivity requirements, scalability, performance, and price sensitivity – all functions of innovation.
