For decades, we’ve relied on the cast-iron availability and reliability of the mainframe-centric IT environment to support mission-critical applications in the banking and financial services sector. When something does go wrong, we can generally count on sophisticated processes to back out of trouble. When those supporting processes also fail, as they did recently at the Royal Bank of Scotland group in the U.K., the results can be devastating and demonstrate that true resilience depends on a lot more than the choice of hardware or software.
In that particular case, the technical root cause was apparently a failed upgrade to the bank’s workload automation software. During the recovery procedure, the scheduling queue was deleted, and what started as a manageable technical issue snowballed into something far more serious: Incoming payments weren’t registered, balances weren’t updated, and a vast amount of data had to be wiped and re-entered.
For the company, the result was unprecedented—thousands of customers of the three banks in the group lost access to their funds, direct payments were reportedly not made at all or were paid twice, credit card transactions were declined, home sales were unable to proceed at the agreed time, and in one widely publicized case, a defendant granted bail on Friday ended up spending the weekend in jail because it was unclear whether a bail payment had been made! Apart from a week or more of inconvenience for many customers, the biggest problem was the appalling PR experienced by an organization entrusted with much of the nation’s money.
It wasn’t a happy day for IT either; indeed, one of the comments that concerned me at the time was the following quote from the respected BBC political correspondent, Robert Peston: “It is striking that in the boom years before the crash of 2007-2008, none of the big banks replaced their so-called legacy systems with new modern ones—which some would see as another example of how they did too little sensible, long-term investment when the sun was shining, and are now struggling to mend their roofs in less clement weather.”
Many would question Mr. Peston’s understanding of legacy systems and how much they’re to blame for what happened at RBS. Responsibility for this particular event appears (as far as we can tell) to come down to poorly implemented change management processes, internal communication issues, and some inadvisable cost reduction and outsourcing decisions. Once the appropriate authorities have completed their investigations, I suspect they will conclude that legacy technology was a small part of the problem.
Old vs. New Mainframe Markets
It's easy to think of the worldwide mainframe business as a single marketplace, but in many ways, nothing could be further from the truth. The mature markets in America and Western Europe have always provided a steady revenue stream for IBM and the independent software vendors, but more to the point, they’ve provided a significant profit margin on both hardware and software. Most of the System z growth now is occurring in new markets in areas such as India and China, often involving MIPS on specialty processors such as the zIIP and zAAP where profit margins are considerably smaller.
For IBM, this is a worrying trend. While recent technological developments such as cloud computing and multi-platform consolidation have reawakened interest in the hybrid capabilities of the mainframe, margins will become increasingly squeezed by the shift toward new MIPS. Problems within the Eurozone have compounded this problem, as the value of the euro has declined substantially against the dollar in the last few years, meaning that hardware shipped out to European customers from the U.S. achieves a much lower dollar price than in the past.
With these large disparities between old and new markets in mind, IBM will be keen to launch the next generation of the System z as soon as possible, using new functionality to keep prices relatively high, and will make sure that as much new business as possible comes from the established markets.
Around the Vendors
CA Technologies has extended its recently launched CA Global Partner Program to include mainframe management solutions to help customers reduce costs and increase efficiency. Due to be rolled out in the fall, the program is designed to enable partners to deliver CA Technologies mainframe management tools to their customers to help reduce costs, sustain critical skills, and increase IT agility in the hybrid data center and the cloud.