Dec 1 ’08

Network and Systems Management: Bulldozing the Mountain of Complexity

by Denise P. Kalm, Rob Steiskal in z/Journal

A new age of performance management is upon us. More than ever before, IT personnel need tools that simplify management of today’s complex infrastructure, automate repetitive tasks, and ensure continuous availability of the computing systems required to run the business. Years ago, most IT infrastructures were simpler and all users were company employees; today, the user base is the entire world. Business applications span multiple operating systems and hardware platforms, and a diverse set of networking protocols facilitate communication. Companies are doing more work with fewer people. This article discusses tactical approaches to intelligently and cost-effectively manage this changing landscape.

The Way We Were

Back in the ’70s and ’80s, most data centers had one or two mainframes and a straightforward telecommunications network. Basic IBM-provided tools, packaged with hardware and systems software, were sufficient to maintain acceptable service levels, which were rarely captured as formal Service Level Agreements (SLAs). User expectations were low; few expected 100 percent availability or consistently fast service times.

Availability and performance became more critical as employees were measured on their responsiveness to business demands. Businesses expected the IT organization to support them by providing reliable systems capable of delivering consistently good service. Rapidly increasing transaction volumes precipitated the need for more comprehensive systems and network management solutions.

To address this need, many Independent Software Vendors (ISVs) delivered products that often required a fair degree of skill to install, deploy, customize, and use. The learning curve was steep. Those who believed the “mainframe is dead” myth served to exacerbate the skills issue by redeploying individuals with mainframe skills to manage new business applications on alternative computing platforms. This migration slowed the feed of new talent to the mainframe platform.

The Way We Are Now

With the dawning of the Internet, those entering the IT profession headed toward the distributed systems world of UNIX and Wintel-based servers, where they were well-compensated. These folks had no desire to work on what they viewed as Jurassic processors. However, many companies chose to maintain and expand the mainframe component of their IT infrastructure. Among the most compelling of various reasons were:

• The years, often decades, of development investment in the core business applications, designed to leverage the unique strengths of the platform

• The high cost and risk associated with redeploying these core business applications on alternative platforms

• Reliance on mainframe qualities of service (reliability, availability, scalability, etc.) that would prove difficult or impossible to match on any other computing platform

• The fact that most strategic business data resides on storage managed by the mainframe

• The existence of time-tested systems, data, and network management policies and procedures that ensure platform stability, continuous access to business data, and enable the business to continue operation—even in the face of an unexpected disaster.

The decline in mainframe skills, combined with the steady growth in mainframe-hosted workloads and an aging mainframe workforce, has created a skills void that’s nearing the critical stage. This problem is being mitigated by the emergence of ever more intelligent software-based solutions that effectively encapsulate and automate “best practices” policies and procedures developed with the benefit of years of experience.

Today, life in the virtual “glass house” is more complicated. Users are everywhere. Anyone with a computer, Personal Digital Assistant (PDA) or mobile phone with Internet connectivity is now a potential customer or consumer of business applications. These users don’t care how the applications they use are constructed or on which computing platforms the applications reside. They just want the applications to deliver the required functions when needed. This has heightened demand for continuous application availability and “always on” network connectivity. Users also typically demand these applications deliver near-instant response to their requests.

Adoption of Service-Oriented Architecture (SOA) design principles and technologies is making it possible to dynamically bring together large numbers of individual network-connected business services to create what’s essentially a virtualized application. User requirements remain unchanged, but managing the individual application components is increasingly complicated. With such virtualized applications, it has become impractical to manually create and maintain business application topology, logic, or data flow diagrams.

Managing the Mountain

The number of mainframe experts is declining while the environment becomes more complex and challenging to manage. The next generation of mainframe specialists will come from the ranks of those who are as comfortable using a PC as prior generations were using the phone. These folks rightfully expect mainframes to exhibit similar usability traits to platforms they routinely use (such as Windows). They won’t accept having to memorize countless cryptic commands or working on text-based “green screen” display devices. Network and systems management software products must evolve to deliver solutions that meet the expectations of these new mainframe technologists. The tools must be simpler to use with greater capabilities.

So what’s driving network and systems management complexity?

Today’s network administrators must manage communication systems that support a diverse set of topologies and protocols. For example, many continue to leverage their heritage investments in SNA networks; others choose to encapsulate legacy SNA traffic and physically transport it across TCP/IP-based backbones. Most companies now have well-established IP networks based on IPv4 protocols. Many of these same companies see a need to maintain dual IPv4 and IPv6 protocol stacks to facilitate cross-enterprise communication. Seemingly endless application-layer protocols (mail, file transfer, etc.) require support.

Network traffic must be managed. It arrives unabated from various sources. Message payloads are often huge in support of new technologies such as video streams. Many business workloads associated with these network payloads are mission-critical. They require stringent Quality of Service (QoS) management. Network administrators must understand these workloads to appropriately prioritize the supporting network traffic.

With multi-platform applications being the norm, the new challenge is integrating the network “picture” with distributed systems information. Mainframe network experts must work with distributed network managers, and today’s systems programmers must manage a broad spectrum of system configurations and topologies comprised of multiple machines hosting a large number of Logical Partitions (LPARs) housing a variety of operating systems. Individual system images are frequently grouped together to form one or more Sysplexes employing a “share everything” methodology. In some cases, these Sysplexes may even cross geographical boundaries.

The dynamic nature of the modern mainframe environment presents unique management challenges. The ability to dynamically change the hardware configuration, adding processing engines and altering capacity limits on the fly, introduces a new dimension to systems management. The inclusion of specialty processors—such as System z Integrated Information Processor (zIIP), System z Application Assist Processor (zAAP), and crypto engines—can, when properly managed, yield significant cost reductions. These systems can host multiple operating systems (including Linux) and standards- based software stacks (e.g., TCP/IP, Java, XML parsers, etc.) in support of modern application design and implementation methodologies (e.g., Web services and SOA). But configuring and managing these software stacks can be difficult in the absence of specific skills or tools.

Network administrators and systems programmers must contend with budget constraints; funding for new technologies is met by saving on existing run rates. Changes introduced as the result of a new project must be carefully managed to avoid disrupting required service levels. When improperly implemented, changes to applications or application components hosted on distributed platforms can result in service disruptions to mainframe-hosted application support components such as database and transaction management systems.

Both the mainframe and distributed system management teams must centrally monitor and manage the resources required to provide continuous access to critical business functions. By doing so, they’re able to quickly and efficiently identify the source of problems and resolve them. They need tools that support proactive management of the data center. Classic monitoring tools, combined with intelligent automation solutions, can deliver this capability, automatically detecting and resolving performance or service-level issues without human intervention.

Today, IT is all about delivering the services the business needs when it needs them and at the lowest possible cost. This has caused the systems management focus to shift from a platform- centric view to an end-to-end application view. Usually, the mainframe systems programmer doesn’t control application transaction service time, nor do the distributed systems or network administration folks. SLAs are achieved based on delivering endto- end computer services, not on the availability of a single infrastructure component.

Mountain Grading Tools

Network and systems managers need versatile tools that let them do more with less. The time saved by using such tools to automate routine and repetitive tasks can be redirected to addressing business and technical issues that really require their individual skills and experience.

Key properties of a good management solution are:

• The solution components must be easy to install, customize, and deploy with quick time-to-value.

• All major functions should be easily accessible. A command line interface should be available for those who prefer it and to facilitate additional automation.

• Both a Graphical User Interface (GUI) and a 3270-style interface should be included. (Each should provide access to the same core functionality so the solution can provide immediate value, regardless of the user’s experience base.)

• The solution should provide navigational shortcuts to optimize problem analysis and resolution.

Additional properties that can enhance a solution’s value include:

• A minimal resource footprint

• Built-in data/event logging functions that can be used to implement or enhance auditing capabilities • The ability to control access to functions with sufficient granularity to enable implementation of role-based security

• Support for a broad range of operating environments, standards, and technologies that enable integration with other operating system and management software components/solutions.

These features are just the basics. You need more to do these jobs well and deliver business process resilience and performance. Ideally, you’d be able to manage all infrastructure elements in your domain of control from a single viewport, where multiple products and technologies are seamlessly integrated to create a single virtualized management facility.

In a z/OS environment, network administrators must be able to monitor business application service levels and manage the IP, SNA, and tunneled traffic upon which those services depend from a single access point. This facilitates resource management from a business, rather than technology, perspective. The ability to progressively and seamlessly navigate from a business service view through its implementing subsystems and system hardware/software components provides the mainframe systems programmer with the tools required to quickly diagnose and resolve critical issues.

Both network administrators and systems programmers must be able to manage resource demand and performance from an IT and business perspective. From the IT perspective, it’s about tasks, processes, transaction processing systems, and databases. From the business perspective, it’s about business service levels, composite business transactions, and so on. But overall, it’s really about delivering service to the business: anticipating, detecting, and quickly resolving technology-based problems to maintain required business service levels.


Today, systems management needs to be proactive, with alerts designed to highlight problems before they impact users and dashboards designed to highlight issues fostering intelligent root cause analysis. Other essential capabilities include:

• Reporting tools that let you share key service level, performance, and problem management information across your consumer base to ensure all stakeholders have access to the management metrics germane to their particular role

• Comprehensive data stores that can be mined for the purpose of trending, problem analysis, establishing baselines, and that support the Information Technology Infrastructure Library (ITIL) goal of a Capacity Management Information Store (CMIS)

• Resource utilization data that can be reliably used for capacity management, including both machine and network resources, with support for resource utilization tracking (from both a technology and business perspective)— so it’s possible to forecast capacity needs based on the growth of individual services.

Are your current management tools keeping up with existing and anticipated business requirements? If your business has fewer people and less money to manage an increasingly complex data center, you need to have the right solutions in place.

  1. What if you could be more productive, spend less time on mundane, boring, or tedious tasks, and enjoy your job more? It’s quite possible you already have the right tools, but haven’t had or allocated the time to fully exploit them. The time you take to learn a new tool will be rewarded by the success you enjoy on the job. Everything is changing on an almost daily basis. Perhaps today is the day you begin to consider replacing your screwdriver with a high-speed drill. The latter does more, faster, and with less effort. The “mountain” of complexity is only getting steeper. Blast it away by ensuring the right management solutions are in place today and that those solutions position you to address tomorrow’s challenges.