The article “Performance Troubleshooting Using the FICON Director Activity Report” (March/April 2013 Enterprise Tech Journal available at http://esmpubs.com/c000r) is the latest in a series of articles on System z I/O performance. As a quick review, this series has assumed there was an application Service Level Agreement (SLA)/Service Level Objective (SLO) for transaction response time that wasn’t being met. Some basic root cause analysis was performed by checking the key RMF reports used in mainframe I/O performance management. The most recent article examined the RMF 74-7 record, the RMF FICON Director Activity Report, to determine if the FICON Storage Area Network (SAN) may have caused the performance issue.
This article focuses on the DASD array host adapters (i.e., what connects the DASD array to the FICON director and to the System z channel). The section on DASD array hardware specifics focuses on IBM DS8000 series DASD arrays, specifically the DS8800 unless otherwise noted. Space limitations preclude going into the technical details on the EMC and HDS hardware equivalents, but the same basic principles of analysis and troubleshooting apply equally well to those and other DASD arrays that support FICON attachment to IBM System z.
Figure 1 illustrates our environment and where the various I/O-related RMF reports fit.
DASD Array Host Adapter Hardware
The DS8800 supports two types of Host Adapters (HAs) installed in the array’s I/O enclosures: Fibre Channel/FICON four-port and eight-port HA cards with a nominal port speed of 8 Gbps. Fibre Channel (FC) is a technology standard that allows data to be transferred from one node to another at high speeds and great distances. This information can include commands, controls, files, graphics, video and sound. FC connections are established between FC ports that reside in I/O devices, host systems and the network that interconnects them. The network consists of elements such as switches, bridges and repeaters that are used to interconnect the FC ports. Using the DS8800 long-wave fibre port HAs, connections are possible up to a 10 km (6.2 miles). That distance isn’t user-configurable; it depends on the type of HA and switch. All ports on each HA are the same type: long wave or short wave.
The DS8800 uses FC to transmit Fibre Channel connection (FICON) traffic, which uses FC frames to carry System z I/Os. A Local Connector (LC) type cable connector is required to attach to this adapter. Each of the ports on a DS8800 HA can also independently be either FCP or FICON. You can change the port type by using the DS Storage Manager Graphical User Interface (GUI) or the DS8000 Command-Line Interface (DSCLI) commands. A port can’t be both FICON and FCP simultaneously, but can be changed as required.
The HA card is a Peripheral Component Interconnect Express (PCIe) Gen 2 bus with a dual-core 1.5 GHz PowerPC processor (Freescale MPC8572). A new high-function, high-performance, Application-Specific Integrated Circuit (ASIC) drives the card. Each FC port supports a maximum of 509 host login IDs and 1,280 paths so the user can create large FICON SANs. The front-end with the 8 Gbps ports scales up to 128 ports for a DS8800, using the eight-port HAs, which results in a theoretical aggregated host I/O bandwidth of 128 x 8 Gbps.
The DS8800 8 Gbps host adapter ports can each negotiate to 8, 4 or 2 Gbps speeds (but not 1 Gbps). The eight-port HAs offer essentially the same total maximum throughput when taking loads of all its ports together as the four-port HAs of the DS8800. So, the eight-port HAs are meant for more attachment options, but not for more performance. Random, small-block performance is usually no issue when considering HA performance because this type of port can deliver up to 100K Input/Output Operations Per Second (IOPS) for a 4K (small-block) cache-hit workload. With several HAs, potential IOPS figures from the HAs are far higher than the number of disk drives currently supported in a DS8800.
FICON host channels limit the number of devices per channel to 16,384. To fully access 65,280 devices on a storage unit, it’s necessary to connect a minimum of four FICON host channels to the storage unit. You can access the devices through a switch to a single storage unit FICON port. With this method, you can expose 64 control-unit images (16,384 devices) to each host channel.
In the System z environment, the normal practice is to provide multiple paths from each host to a disk subsystem (typically, four to eight paths are installed). The channels in each host that can access each Logical Control Unit (LCU) in the DS8800 are defined in the Hardware Configuration Definition (HCD) or I/O Configuration Data Set (IOCDS) for that host. Dynamic Path Selection (DPS) lets the channel subsystem select any available (non-busy) path to initiate an operation to the disk subsystem. Dynamic Path Reconnect (DPR) lets the DS8800 select any available path to a host to reconnect and resume a disconnected operation such as to transfer data after disconnection due to a cache miss. The channel subsystem on the host and the DS8800 manages these functions, which are part of the System z architecture.