Jul 1 ’07

Who’s Going to Manage Your Mainframe Apps? Cross-Platform DBAs, Linux, Mainframes & the Economy

by Editor in z/Journal

Mainframes have been reinvigorated by companies such as oracle partnering with IBM to run multiple databases on IBM System z mainframes using Linux virtual machines. But this is creating a staffing crisis in enterprise IT. Cross-platform DBAs and systems administrators who can manage distributed and mainframe database environments are becoming scarcer, and not enough new professionals are being trained to fill the gap. However, there are some ways IT organizations can plan for this challenge and minimize its impact. 

IBM unveiled its System 360 more than 40 years ago and it has since been witness to several IT evolutions: first, client/server; then n-tier environments, but the evolution is coming full circle. Increasingly reliant on Linux to assist in reducing costs and developing Software-as-a-Service (SaaS) strategies, businesses are seeking the computing power and centralization that enable them to achieve a greater return on their technology investments. for IT organizations, the ability to run multiple databases in Linux on System z address spaces provides power, reliability, and efficiency that offer another key economic benefit: It scales easily and lets businesses continue to run their database or applications of choice without requiring a major sea change.

The Skills Shortage Challenge

Dr. Walter E. Williams, a George Mason University economist, had as a core of his Ph.D. microeconomics theory course the premise that scarcity is the key to understanding both supply and demand. This premise can be applied to today’s business IT environment. Businesses require more computing power, greater speeds, ease of scalability, more storage, and staffing resources whose skills can be continually stretched. For businesses, these requirements represent what Williams calls “wants.” However, the ability to satisfy these wants is limited by both budget constraints and the extent to which science enables their achievement. There’s a finite amount of any of the above resources (supply), and the value of each often has to be rationalized in favor of another to meet customer needs (demand).

Over the past decade, many organizations have invested in massive networks based on servers and PCs, which required more staff. Adoption of Linux on the mainframe means a different mindset for these organizations and even more cost rationalization. Among these rationalizations are the productivity costs delivered by existing staff and the need to maximize the value of senior staff, many of whom have mainframe database management backgrounds.

This bodes well for multi-platform professionals, particularly cross-platform DBAs and systems administrators who can manage databases in both distributed and mainframe environments. Because mainframe DBAs possess a skillset that’s relatively scarce, the value of these skills should continue to increase. Outside the efforts of the IBM mainframe education initiative, there’s limited focus on training mainframe professionals Therefore, the ability to manage DB2 on z/OS subsystems and other databases in z/Linux address spaces is a great asset to any shop.

This evolution of IT architecture requires fundamental changes in the way skills are distributed across the IT organization. Traditionally, skills in the IT space have been vertically focused. DBAs or developers have been encouraged to “go deep” in a particular vertical technical area (e.g., IBM mainframe database skills) so they can resolve complex technical issues arising over the course of normal business operations. This approach is effective in an n-tier environment because the different components of the organization remain distinct, and these components can be managed using skills in a particular specialized vertical. As the distinction between tiers blurs, however, this practitioner specialization becomes insufficient when managing a complex, interrelated IT environment.

Cross-training across the broader IT organization isn’t necessarily a viable option. Attempting to turn DBAs into IT generalists (for example, adding network administration as a secondary focus) is unlikely to work. Likewise, cultivating database management skills in application developers isn’t realistic. The skills aren’t readily transferable, and the costs of developing those skills are significant.

For many organizations, the best alternative is to identify the job-specific (as opposed to platform-specific) skills that are transferable to other technologies in the same generalized technical vertical. Given the right toolset, a DBA who has focused his efforts and education in the Oracle space should be able to apply that experience and skillset toward the administration of other Relational Database Management System (RDBMS) platforms, whether it’s IBM DB2, Microsoft SQL Server, or MySQL. This represents a broadening of a specialized skillset. Typically, those with mainframe DBMS experience have found the transition to distributed databases less daunting than those attempting to manage the mainframe for the first time after years of distributed database management. This isn’t to suggest that there’s a toolset that serves as a panacea for managing the complete realm of issues associated with multiple DBMS systems, nor that any toolset is without its own issues. To be sure, DBAs deal with challenges associated both with DBMS management and toolset management on a daily basis; that reality won’t likely disappear tomorrow.

For DBAs, this shift can be profound because they’ve had the greatest tendency toward “going it alone” in their area of expertise, and there’s often some reluctance to step outside their comfort zone. Traditionally, they managed all aspects of their database environment and focused on maintaining a homogeneous architecture, minimizing the number of different database platforms. Increasingly, however, the databases used to power the applications are being determined by the vendor, which means DBAs are being asked to maintain multiple DBMS products, with different syntaxes, performance benchmarks, and behavior. The result is a more complex database management environment, and a much greater need for simplified administration and automation.

Furthermore, the applications used to support business activity are built differently. Problems are harder to diagnose, and DBAs, with their unique skillset, will need to partner with other disciplines to resolve issues and ensure that business systems remain available. For this reason, autonomic databases and the tools that enable lights-out diagnosis can serve essential roles in addressing—but not eliminating—this challenge.

Can Tools Bridge the Skills Gap?

In his book Basic Economics, Stanford University economist Thomas Sowell discusses the concept of productivity, framing the argument that it’s the quality of the equipment, the workers themselves, and management’s ability to recognize and provide the best tools, which influence productivity. This applies to IT organizations, too. Remove any of those three elements, and the ability of IT investments to deliver efficiency and value is hampered.

Extending Sowell’s concepts, IT organizations must partner with those who can deliver on the wants related to hardware, databases, and applications, with tools that extend the value of their technology investment and enable their staffs to migrate their existing skills to new environments at a relatively low cost. To operate effectively, new IT solutions, especially those in the database space, must have:

• The ability to enable true heterogeneous database skills migration

• Low-risk implementation

• Ease of integration

• Functionality that enables objective problem resolution.

The need for broad expertise in multiple database disciplines—coupled with the need to integrate leading-edge components to deliver comprehensive solutions— represent significant challenges. IT vendors must demonstrate these business values while positioning their solutions as integrated best-of-breed platforms, and do so without driving up the Total Cost of Ownership (TCO). As an added challenge, these integrated solutions must be capable of reconciling themselves into their discrete components that also can integrate with other vendors’ solutions. This is where z/Linux is such an asset.

The biggest barrier to the adoption of best-of-breed technology has been high TCO due to ongoing management and skills migration costs. Combining multiple databases, housed on a single mainframe machine in multiple Linux address spaces, reduces the ongoing costs of managing separate machines (each with its own Linux operating system). Of course, there are big-iron benefits— not the least of which is recovery—to running z/Linux that make it attractive for both DBAs and systems administrators. Compared with running Linux on multiple independent machines, each with its own communications, and applications nuances, recovering z/Linux is an “all at once” exercise, where all the associated processes and applications are recovered simultaneously with the operating system. Combined with platform-agnostic tools that enable IT professionals of all skill levels to proactively manage heterogeneous database environments, the total costs are reduced even further.

Retaining Expertise and Transferring Knowledge

No longer are DBAs expected to develop expertise in just a single technology. They’re increasingly considered data experts whose depth of knowledge goes beyond the database to include the applications that access the data. This has been a reality in most progressive DBA organizations for three or four years now. Many senior-level DBAs who cut their teeth in the mainframe world have found themselves much more involved with multiple layers of their production environments, including higher-level activities such as establishing data architectures.

They also have daily administration responsibilities, such as managing database objects, and often find themselves involved with tasks related to both application performance and development. This mixture, while indicative of the important part played by senior DBAs, also reveals the extent to which they’re relied upon to perform tasks that don’t always represent tangible business value, and often represent an area that causes frustration for DBAs and managers alike. Adding a heterogeneous environment to the mix is a challenge that can’t readily be accomplished without prioritization on the part of the business as a whole.

DBA managers, and those at the levels above them, wrestle with the need to derive value from their senior DBAs while ensuring that Service-Level Agreements (SLAs) are met. Because of the requirement to support both internal and external audiences, managers are faced with a difficult choice. They must meet the needs of multiple constituencies with a near-zero downtime allowance while also adding value to the business—a goal complicated by the fact that their most direct route for doing so is through the capabilities of their senior staff members.

Because DBA staffs aren’t growing at the same rate as the volume of data they’re expected to manage, DBA managers concerned with retaining top performers have to find ways to shield them from getting bogged down in daily database administration tasks. This means enabling senior staff to remain focused on tasks that deliver value to the bottom line, and providing them with the tools they need to quickly discover the root cause of an issue, diagnose and resolve it, and get back to their job. Likewise, less experienced staff need to be empowered with the tools to accomplish as many daily issue resolutions as possible without requiring senior-level intervention. While a worthy goal, the time necessary for this scenario to become a reality may be limited, making it difficult for many organizations to develop practices that clearly delineate where a less-experienced DBA’s responsibilities end and where a senior database expert’s responsibilities begin.

Looking Ahead

Adopting an entirely new platform to run business-critical databases and applications is a daunting task. While the long-term business and technical merits of a switch to Linux running on big iron are easily understood, the initial implementation hurdles are often best overcome with the help of business partners who can help navigate the landscape. DBAs stand to benefit from this partnership, too, as they’re given the opportunity to develop and hone their skills with new technology, while delivering tangible value to the business.

IBM’s zSeries IT co-op program and the IBM Academic Initiative for System z are both aimed at helping foster a new generation of mainframe professionals. Through this initiative, IBM is working with universities to provide mainframe education and practical experience worldwide to address the shortage of mainframe professionals. They also have a “Master the Mainframe” contest that gives university students the chance to show their stuff.

There’s only so much IBM can do, however. Many mainframe professionals learned through the guidance of a senior IT professional or mentor during their first job. In an era of tight budgets for IT staff, these relationships may be more difficult to foster; they’re also not always treasured among young IT professionals.

In his article, “The Coming IT Crunch,” in last year’s August/September edition of z/Journal, John William Toigo outlined the current IT skills landscape and what we can expect if organizations fail to recognize that “people are the strategic asset.” While deploying z/Linux makes sense from a fiscal standpoint, reducing technology, ongoing maintenance, and recovery costs, it’s critical that IT shops realize it’s their skilled mainframe team members who will continue to deliver the real value.