System z I/O technology has advanced significantly over the past five years. These advances are diverse, yet synergistic, ranging from the mainframe itself to the FICON directors and the storage control units and devices. Speeds and feeds are continuously becoming faster as new technologies have emerged such as:

• Modified Indirect Data Address Words (MIDAWs)
• HyperPAVs
• z High Performance FICON (zHPF)
• 16 Gbps FICON
• FICON Dynamic Channel path Management (DCM)
• PCIe I/O drawers.

More changes are on the horizon. Changes have also occurred in IBM’s System Management Facility (SMF) and Resource Measurement Facility (RMF). Understanding these new technologies is crucial to your job, but so is understanding their performance. That will be the focus of this and subsequent articles.

Paradoxical Problem

With the technology advances seen in mainframe/mainframe storage I/O over the past five years, we have a less than optimal understanding of the root functionality of this new technology. Users understand the benefits of zHPF, but often lack an understanding of how it works and how the benefits accrue. Awareness of performance implications and the performance management of this new technology are also insufficient.

So we’ve made fantastic technical advances that, used correctly, can radically enhance the overall performance of your mainframe storage and I/O. But we don’t know all we should to take advantage of what we paid for. This yields the performance information gap paradox.

Contributing Factors

All four of the following factors have contributed to the performance information gap paradox:

• The global recession that started in 2007. Along with jobs, training and travel budgets are cut during a recession. This is a big reason that attendance at training conferences such as the annual Computer Measurement Group (CMG) International Conference and SHARE have been below historical attendance norms since 2007. Those conferences provided outstanding performance and capacity planning training, with a great deal of emphasis on mainframe topics.
• Performance management and capacity planning are getting less attention at colleges and universities. There’s little to no curricula on mainframe topics, other than the colleges and universities that participate in the IBM Academic Initiative program. So, our newly minted colleagues entering the workforce who become mainframers typically haven’t had significant coursework in the performance field. They learn on the job from colleagues with relevant experience and knowledge.
• Hardware prices continue to fall. As hardware becomes less expensive, it’s easy to deal with performance issues by “throwing more hardware at the problem.” Sadly, this can become a habit, one that eventually leads to management issues and unanticipated operational costs and complexity for the necessary power, cooling, and space the hardware requires as it multiplies on your data center floor like rabbits in spring.
• Hardware vendors haven’t recognized or responded to the paradox. Vendors haven’t found creative ways to better train their customers. Sometimes, they’ve cut their internal training and travel budgets.

Hope on the Horizon

I recently returned from a business trip to China. I was there with my friend and IBMer, Dennis Ng, to do a series of two-day training sessions focused on Mainframe I/O and FICON performance/performance management. Dennis and I started talking about the performance information gap paradox six months ago, and the training course we put together was our two companies’ effort to address the problem. We had a wonderful experience running the training for some very motivated and intelligent mainframers. One thing that stood out was that the average experience level for these mainframers was less than five years, and for many of them, their current job was their first job after graduating from university. It was inspirational to teach them! What we found really great was their enthusiasm, thirst for knowledge, and the fact that all these people had chosen the mainframe career path right out of college. Stewart Alsop, eat your heart out!

Solving the Paradox

What constitutes a performance problem? There are a wide variety of views on the subject, but most agree that a problem exists when there’s unacceptably high resource usage or unacceptably high response time. Some situations aren’t all that serious and may not be worth addressing; others will require an in-depth investigation and analysis.

The first step is acknowledging a problem exists. If a workload isn’t getting the resources needed to complete in a timely manner, or the resources are available but aren’t obtained fast enough to provide the required response time, you have a performance problem. Service-level objectives are being missed and users are complaining. It’s time to do some problem-solving.