Jan 1 ’07


by Editor in z/Journal

Imagine looking up at a steep wall. You’re a fireman, responsible for the safety and wellbeing of everyone on both sides of that wall. Suddenly, you smell something burning. Looking up at the wall, you see smoke streaming over from the other side. You rush to grab a hose, but where do you aim it? You know the fire is somewhere on the other side, but where is its source? 

This is the problem many mainframe administrators face daily when they begin to look at their system’s performance. They may know there’s a problem—something is devouring CPU capacity at an alarming rate—but isolating the problem and solving it are significantly more daunting (and usually more expensive) tasks. Just like our poor fireman, lack of visibility is the key culprit. It’s highly likely that in a mainframe environment, the undetected source of this fire will be an application. With traditional mainframe performance management, visibility is limited to the operating system and subsystems. However, automated Application Quality Management (AQM) allows for proactive monitoring and tuning on the application layer, the most likely location for performance-throttling problems. This helps you identify and correct problems before they flare up and become full-blown infernos.

Implementing automated AQM tools is a preventive measure to automatically detect discrepancies in response times of objects on the application layer, identifying them before they become a costly problem. Using smart logic, they provide detailed evaluations of computing times, resource usage trends, and potential bottlenecks quickly and accurately, allowing for proactive correction and tuning. 

The benefits of automated AQM are many and can be measured on three specific levels: 

The System Benefits  

Several market research firms have confirmed the continued importance of the mainframe and predict even a slight increase in the market over the next five or six years. These same analysts also predict a significant demand on existing machines. This combination of new, more demanding requirements on older machines has traditionally meant one thing—upgrades. Whether it’s hardware or software, this is the equivalent of “throwing money at the problem.” 

Under these real circumstances, implementing automatic AQM becomes an even more logical solution to upgrading mainframe-processing power. In contrast to the open systems world, mainframe hardware or software upgrades can be costly. In fact, the majority of money spent on upgrades has shifted from hardware to software costs to a ratio of about 30:70. 

The objective is to avoid any upgrades as long as possible. Usually, a cost/benefit analysis of the potential solutions for CPU bottlenecks favors optimization measuring and tuning at the application level. An automated AQM solution, comprised of a proactive optimization solution at the application level and automated tools for analysis and tuning, can free up 30 to 90 percent of used mainframe resources. 

Even if a minor hardware upgrade is still required, it’s a significant cost savings over a wholesale software and hardware upgrade. 

To put this into practical terms, assume your company has a transaction running on the mainframe with average CPU use of only 0.05 seconds. This single transaction may not register as a “problem,” so it doesn’t appear on your performance radar. However, if this transaction is called 150,000 times per day, half of that during peak time, the result is more than 125 minutes of CPU time daily and 31,250 minutes annually. At a rate of $12 per CPU minute, the annual cost is about $375,000. 

Obviously, less CPU usage means a more efficient mainframe, but even the most efficient mainframe will require some regular maintenance and upgrades. A second, often overlooked, benefit of an automated AQM solution is that it enables scheduling regular maintenance and upgrades, rather than ad hoc upgrades as part of putting out a fire. 

The ability to plan upgrades means you can budget financial and human resources. This future visibility will make everyone happy. 

The Human Benefits  

With demands on today’s mainframe greater than ever, IT organizations must juggle managing larger, more complex environments, with 24x7 workloads, all while delivering higher resource usage levels. Performance tuning and resource optimization, crucial to maintaining an efficient mainframe environment, have traditionally been highly manual and so labor-intensive that most IT teams defer them—at least until a critical service disruption occurs. This reactive approach often costs significantly more in the long run, as IT organizations divert more resources and time to identify and fix problems than it would take to prevent them. 

Implementing a proactive, automated AQM solution eliminates many of the manual tasks that limit mainframe performance specialists from reaching their full potential by automatically tracking an entire production batch and online system, detecting deviations from normal behavior, collecting deep application performance metrics, and highlighting areas requiring further investigation. An automated AQM solution automatically tracks applications across an enterprise-wide z/OS complex so analysts don’t have to exhaust large amounts of time scanning nightly production batch runs looking for problems, configuring stand-alone measurement tools, and reviewing reports. 

A problem that may have taken two engineers eight hours to identify and rectify is now quickly and easily identified. Your tuning and management staff is now freed up to put their energy into the fine-tuning that you hired them for. 

Another benefit of automated AQM extends to your application designers. They generally build applications in a vacuum, failing to consider the potential impact their creation will have on the entire mainframe, usually because they don’t understand the broader ramifications. By using the reporting tools and knowledge acquired from an automated AQM solution, developers and system tuners can work together to tune applications during development to ensure they perform at peak efficiency rather than having to go back and troubleshoot bugs.

This subject may need to be approached gently to avoid stifling the application designer’s creativity. The tuning professionals will require some “soft skills” to demonstrate that building applications that require less tweaking will let them spend more time building applications and less time fixing them. 

In both cases, eliminating or reducing the firefighting aspect of the job will benefit your entire IT staff by letting them put their talents to their best use. In a competitive IT job market, a happy technology professional is an important asset.

 The Company Benefits  

Besides making you the hero to the bean counters and your employees, an automated AQM solution will endear the IT department to users, too. The optimization process of an automated AQM solution starts in the data center, but isn’t restricted to it. Automated AQM can run through the entire company, from the production stages of application development to the everyday user experience. 

Proactive tuning can eradicate problems that could cause costly shutdowns of strategic applications that otherwise would have gone unnoticed. A sales support team that can’t access their Customer Relationship Management (CRM) application because it’s down for an avoidable upgrade can experience both an efficiency cost and an opportunity cost as sales may be lost or deferred. 

Better system efficiency also means faster application processing times. Users of any mainframe application will benefit from better-performing applications. An often-overlooked benefit of automating your AQM is improved response times. 

Assume your users experience an average response time of 0.9 second. Because this meets the Service-Level Agreement (SLA) of less than 1.0 second, it would again fly below your performance radar. But with concurrent requests against the database, response time during periods of peak usage is more realistically 2.0 seconds and more. If you’re running even half of your 150,000 transactions during peak time, you’re looking at just under 23 hours of lost productivity daily (75,000 x 1.1 sec), which means much less productivity.


When you’re optimizing your application management, you also optimize your business processes, and can play a major role in improving enterprisewide efficiency and effectiveness. By measuring, interpreting and subsequently implementing tuning adjustments, small changes can save large amounts of money and greatly improve productivity. 

The key benefit of automating your AQM is visibility and foresight into potential problems that it provides. This visibility is crucial to help you identify and remedy system flare-ups before they become raging infernos, burning up CPU, personnel, and fiscal resources.