The introduction of the FICON I/O protocol to the mainframe I/O subsystem ushered in a new era in our ability to rapidly and efficiently process data. The FICON protocol differs significantly from its predecessor, the ESCON protocol. Because of two main changes FICON made to the mainframe channel I/O infrastructure, the requirements for a new RMF record emerged. Unlike FICON, ESCON didn’t use buffer credits to account for packet delivery. In addition, FICON cascading wasn’t possible with ESCON.
Buffer-to-Buffer credits (BB credits) and their management in a FICON environment remain commonly misunderstood concepts. Moreover, the phrase “buffer-to-buffer credit management” appears to be an oxymoron. Given their impact on performance over distances in cascaded FICON environments, this is something that needs to be better addressed.
Currently, there’s no real way to manage and track BB credits being used. At initial configuration, a rule of thumb is used for allocating them. For management, we keep our fingers crossed. The typical FICON shop completely overkills the number of BB credits assigned for long-distance traffic. Just as Parallel Access Volume (PAV) overkill can lead to configuration issues due to addressing constraints, BB credit overkill can lead to director configuration issues that often require outages to resolve. Mechanisms for detecting BB credit starvation in a FICON environment are extremely limited.
This article reviews BB credits, including current schema for allocating and assigning them, and also examines a way to detect BB credit starvation on FICON directors, including a discussion on the concept of frame pacing delay. We’ll also examine a concept for a mechanism to count BB credits being used and conclude with a look at another theoretical concept: dynamic allocation of BB credits on an individual I/O basis (similar to the new HyperPAVs concept for DASD).
BB Credits: Back to Basics
Reviewing the concept of flow control is a good place to start. (For more information, see Robert Kembel’s The Fibre Channel Consultant three-volume series.) The objective of flow control is to prevent a transmitter from overrunning a receiver by letting the receiver pace the transmitter, managing each I/O as a unique instance. At extended distances, pacing signal delays can result in degraded performance.
Buffer-to-buffer credit flow control is employed to transmit frames from the transmitter to the receiver and pacing signals back from the receiver to the transmitter. The basic information carrier in the fibre channel protocol is the frame. Other than ordered sets (used for communication of low-level link conditions), all information is contained in the frames. Consider the analogy of an envelope. When you send a letter via the U.S. Postal Service, the letter is encapsulated in an envelope. When sending data via a FICON network, the data is encapsulated in a frame.
To prevent a target device (either host or storage) from being sent more frames than it has buffer memory to store (overrun), the fibre channel architecture provides a flow control mechanism based on a system of credits. Each credit represents the ability of the receiver to accept a frame. A transmitter can’t send more frames to a receiver than the receiver can store in its buffer memory. Once the transmitter exhausts the frame count of the receiver, it must wait for the receiver to credit back frames to the transmitter. Here, an analogy is a pre-paid calling card. A certain one can talk until there’s no more time on the card.
Flow control exists at the physical and logical levels. The physical level is called buffer-to-buffer flow control and manages the flow of frames between transmitters and receivers. The logical level is called end-to-end flow control; it manages the flow of a logical operation between two end nodes. A single endto- end operation may have made multiple transmitter-to-receiver pair hops (end-to-end frame transmissions) to reach its destination. However, the presence of intervening directors or Interswitch Links (ISLs) is transparent to end-to-end flow control. Since buffer- to-buffer flow control is the more crucial subject in a cascaded FICON environment, the following section provides a more in-depth discussion.
Buffer-to-Buffer Flow Control