Author: Jim Pascoe, Hitachi Corporate Communications, interviewed Gary Goodwin, Firmware Development Engineer Hitachi Global Storage Technologies
In the following interview, Gary Goodwin of Hitachi Global Storage Technologies (Hitachi GST) reviews some of the key issues related to Serial Attached SCSI (SAS) scalability and how systems integrators (SIs) and other end-users deploying SAS-based storage are expected to benefit from the transition from parallel SCSI to SAS.
Gary Goodwin is a firmware development engineer at Hitachi GST with an extensive knowledge of Ultra320 SCSI, Fibre Channel and SAS. He has had a long career working with computer storage interface technologies. SIs will benefit from Gary’s expertise presented in question and answer format.
Q: Will a cluster of SAS disk drives have functionality similar to that of a SAN?
No, not really. A SAN (Storage Area Network) is a dedicated high-performance network that interconnects different kinds of storage devices to servers, and possibly clusters of servers. SAN also implies a storage management layer that supports data mirroring, backup and restore, data archiving and data migration. A SAN may or may not have storage management, regardless of the technology used to connect the storage. SAS can be thought of as a communication technology that is used within a storage environment. It provides a dedicated connection scheme for storage that is similar to the connectivity aspect of a SAN.
Q: Will SAS expanders play a similar role to that of Fibre Channel switches in a SAN environment?
One of the goals of SAS is to be a low cost, yet high-performance method of interconnecting storage elements. SAS expanders are essentially fixed-function,
low-cost switches. To remain low cost, SAS expanders will not have advanced features and capabilities that are found in Fibre Channel director-level switches, for example. Director-level switches are high-port count (64 or more), high-availability devices that connect servers, storage and other fixed-port Fibre Channel switches to create a SAN. High availability means that when you have a port or component failure, the director can reassign and redirect traffic to different ports to avoid any disruption of the data flow.
Q: What are the different types of SAS expanders and how many drives will each support?
Conceptually, there are two basic types of SAS expanders; “edge” expanders and “fanout” expanders. Each expander can have up to 128 devices connected to it. Edge expanders, as the name implies, are connected directly to devices at the end, or the “edge,” of the SAS domain. These end devices can either be drives (or other device types) or initiators. In the case of large system configurations, an edge expander may also be connected upstream to another expander. That is where fanout expanders come into play.
Fanout expanders are typically connected to downstream edge expanders, as a means of adding additional devices to the storage domain (fanout expanders, may also be connected to end devices). If we maximize the topology, we can have a 128-port fanout expander connected to 128 edge expanders, each with 128 end devices. Thus 16,384 end devices (128X128) is a maximum configuration, where the end devices would be a logical mix of drives and initiators.
Q: What role do expanders play in enabling subsystems to include both SAS and SATA in a single enclosure?
One of the advantages of SAS expanders is that they are architected to play a larger role than just routing SAS frames between devices. SAS expanders can also provide SAS/SATA protocol transparency, enabling a native SATA device to connect to a SAS initiator. This transparency is accomplished via STP, or SATA Tunneling Protocol. By incorporating the STP functionality within the SAS expander, a system integrator now has the unprecedented capability of easily taking advantage of the complementary nature of both SAS and SATA devices.
For example, SAS disk drives can be used for mission-critical applications requiring high performance and availability, while SATA disk drives can be used as backup storage where low cost and high capacity are primary requirements. Since SAS and SATA are built on the same physical layer and have compatible connectors, all of this can be done seamlessly within the SAS domain-even within the same SAS enclosure.
Q: Does the use of expanders have any adverse impact on disk drive and/or subsystem performance?
SAS is a well-conceived architecture that takes the best features from its predecessors and improves upon them. Unlike a topology such as FCAL (Fibre Channel Arbitrated Loop), where additional delays are introduced as devices are added to the loop, a SAS device is never far from its host. Devices can be added to the SAS domain without impacting system performance. This is directly attributed to the expander topology of SAS and the point-to-point nature of the connections.
In addition, SAS has performance capabilities well beyond that of parallel SCSI. Features such as full-duplex transmission effectively double the instantaneous data rate of the link. Full duplex allows simultaneous bidirectional activities (e.g. command, data, response) to overlap on the link, thereby increasing throughput. Even though parallel Ultra320 SCSI has a peak bandwidth of 320MB/sec, compared to 300MB/sec in SAS, a SAS link can achieve higher throughput due to its full-duplex capability. And it is not just theoretical. Lab testing has proven it to be a real advantage.
Q: SAS is a dual-port interface. How does dual-porting affect reliability and performance?
Dual-port is another outstanding feature that SAS inherited from its predecessor storage interfaces, such as FCAL. Dual-port capabilities are extremely important in enterprise storage to ensure reliability and performance.
From a reliability perspective, dual porting provides the system an alternative path to the same data. This “redundant path” is important because in the event of failure, the user has an alternative path to the data. The flexibility provided in SAS can also allow for the alternate port to belong to a separate SAS domain, effectively extending the system’s redundancy capabilities.
A dual-port device can also help improve system throughput. Having alternate paths to the same data can relieve system bottlenecks and eliminate wait times for the primary path to become available. This enables the system to get to the data quicker. Intelligent SAS storage systems can implement dynamic load balancing that will help to spread I/Os evenly across host controllers and dual-port devices. In addition, these two ports can be combined into a single “wide port” for even higher throughput.
Q: What will the scalability attributes of SAS enable SIs to achieve, in terms of large-scale system design?
Parallel SCSI has a limit of 16 devices per bus. A maximum SAS domain configuration consists of 16,384 end devices. While that is the theoretical limit, in practice, systems can span from a single initiator/single device all the way to multiple initiators and thousands of devices. Massively large-scale systems are possible within a single SAS domain, all for lower cost and complexity than a comparable FCAL system. IT managers will appreciate the fact that, with SAS, they can easily extend and grow their systems to meet the storage requirements of their company. The scalability features of SAS are indeed compelling.
Q: Will SAS present any new challenges for SIs?
A new interface always presents challenges and SAS is no exception. However, SAS’ unprecedented development approach for a new storage interface has, in many ways, helped to minimize the challenges.
For example, let us consider the specification standard. It is typical for interoperability issues to arise due to different interpretations (or misinterpretations) of the standard. The SAS standards committee has done an outstanding job of documenting the standard to avoid misinterpretation. The unprecedented incorporation and extensive use of hardware state machine definitions has helped to avoid the most costly form of misinterpretation-an error in the silicon! In addition, the SCSI Trade Association (STA) did much to organize the SAS community and provide a forum for cooperation. Examples of this include the multiple SAS Plugfest events (held at the University of New Hampshire’s InterOperability Laboratory) that have occurred to date. These plugfests provide a venue for companies developing SAS products to actually test their interoperability with other companies’ products, prior to production.
This has helped to accelerate delivery of SAS products to the marketplace and will continue to smooth the industry transition from parallel SCSI to SAS.
Q: I’ve heard that SAS is the follow-on to parallel SCSI, yet some of the SAS attributes, such as dual-porting, appear to be more like Fibre Channel. Can SAS be used in multiple environments?
Absolutely. As I mentioned earlier, SAS has taken the best from predecessor storage interfaces and improved upon them. While SAS is marketed as the follow-on to parallel SCSI, it shares many positive attributes with FCAL. Features such as dual-port and full-duplex operation fall into that category. As such, it is conceivable that SAS will grow beyond the traditional parallel SCSI market and into some of the high-end environments traditionally filled by Fiber Channel. However, advanced features such as ESI (Enclosure Service Interface) and Public Loop, which are part of the Fibre world, are not included in SAS.
Q: Why use expanders, given that the SAS architecture is described as being point-to-point?
Expanders are what enable the scalability of SAS. While it is possible to build a SAS domain without an expander, it would be difficult to get beyond a single initiator/single target configuration. By inserting an expander in between the initiator and target, we can theoretically expand the target numbers up to 127. Each of the SAS links is still point-to-point (point-to-point from the initiator to the expander, point-to-point from the targets to the expander, etc). The expander functions as a router, in establishing connections between the initiator and the targets.
Q: How does the point-to-point SAS architecture compare to the loop technology of Fibre Channel?
A loop is a shared resource. When a device wants to use the loop, it needs to win arbitration first. Only then can it establish a connection with another device. While it is possible to have up to 128 devices on a single Fibre Channel loop, only one connection between any two devices can be established at any point in time. A point-to-point link (between an end device and an expander, for example) is a dedicated resource. It is always available, so no arbitration is required. With the SAS architecture, we can now have simultaneous connections that overlap with one another within the same SAS domain.
Reliability is the other important advantage of point-to-point topologies. In loop topologies, if a single link fails, it takes out all of the devices on the loop. In a point-to-point topology, if a single link fails, it only takes out the device on that link.
