Storage networking devices include processors (host servers), channel and host bus adapters (HBAs), switches (including directors), and storage devices (disk and tape, including virtual tape). Some additional storage networking devices include bridges, gateways, and routers, which were mentioned in the October/November 2003 z/Journal article, “Has Open Storage for zSeries Finally Arrived?”. Figure 1 shows a storage network with two separate fabrics (I/O paths) for redundancy (Path A and Path B) with servers attached to both. Figure 1 also shows servers that are single-attached only to Path B. Servers are attached to switching devices (director or switch) that have storage attached to them. Figure 1 also shows a switching device connected to another switch on the right using interswitch links (ISLs) in what is known as a cascade topology. This is an example of a metropolitan area network (MAN) or wide area network (WAN) for remote storage access and mirroring for data protection and access.
Switches & Directors
Switching directors have been associated with mainframe I/O; they provide:
- Static (predetermined) switching for I/O channels between processors and storage
- Storage-to-storage access for data movement, including mirroring and copies
- Processor-to-processor or channel-tochannel (CTC) communications.
A director provides any-to-any connectivity between attached processors and devices. This differs from simple point-to-point, dedicated connections where paths are predetermined and hard-wired connected. Interfaces and protocols, such as ESCON and FICON, are considered to be static with definitions via IBM mainframe host utilities, including IOCP and HCD.
Open systems interfaces and protocols, such as fibre channel and SCSI_Fibre B Y G R E G S C H U L Z addressing via discovery. Consequently, switches and directors that support FCP also support dynamic addressing and device discovery for those protocols and interfaces. Fibre channel switches and directors can support both FCP for open systems and FICON for S/390 and zSeries environments concurrently. Switches and directors that support two or more protocols, for example, FCP and FICON, are also known as multiprotocol devices. Fibre channel and FICON are often described as interfaces and protocols. Fibre channel is, in fact, an interface that supports multiple upper-level protocols (ULPs) including FCP and FICON. Thus, Fibre channel is the underlying interface that supports protocols such as FCP and FICON concurrently. This is similar to Ethernet and Gigabit Ethernet (GbE) being an interface to support network protocols such as TCP/IP (IP). Refer to the article, “Has Open Storage for zSeries Finally Arrived?” in the October/November 2003 z/Journal for more information about storage interfaces and protocols.
Switches and directors can be used as stand-alone, in pairs to create multiple, single-device SAN islands, or connected to create a fabric network. Storage networking switching devices can range from simple, four-port embedded switches to large, multiprotocol devices of 256 ports (larger devices are on the drawing board). Fibre channel switches provide scalable bandwidth between various subnets, segments, and loops as users and devices are attached.
A fibre channel director is a highly scalable, fully redundant switch that supports blades with multiple interfaces (fibre channel, Ethernet, Gb Ethernet) and protocols (FCP, FICON, IP-iSCSI, FCIP, iFCP) similar to a large network switch. Some switching devices also support application-specific blades for storage and virtualization services, including caching, storage pooling, volume management, mirroring, and backup. Directors vary in size from four to 256 ports and can replace or supplement fabrics of smaller switches. Note, however, that a standard best practice is to configure storage and I/O paths in pairs and use a pair of switches or directors for high availability. After all, availability is the sum of all components functioning together when needed. Switching devices, including directors and switches, can be networked into various topologies to create large, resilient storage networks of hundreds to thousands of ports.
ISLs are used to connect switches and directors to create a network (fabric) also referred to as SAN. On each switch or director to which the ISL is attached is an ISL expansion port (E_Port). For redundancy, ISLs should be configured in pairs. So, for every ISL needed, you’ll use at least one port per switch or more for redundancy. (For the balance of this article, unless otherwise indicated, the term “switch” will apply to both fibre channel switches and fibre channel/FICON directors.)
Figure 2 shows an example of multiple servers with any-to-any access to all storage subsystems. Here tape devices are attached to a pair of switches or ESCON directors. This example, which could be applied to different environments, shows each switch as a separate storage network fabric, which is sometimes referred to as a SAN island. The exception to this is on the far right, where two switches are networked to create a two-switch fabric. This twoswitch fabric is a cascade topology that could be used for spanning distances incorporating metropolitan and wide area devices. This example could be for FICON, FCP using fibre channel, or an ESCON environment.