Jun 26 ’08

Satisfying On-the-Go Customers: SunTrust Banks on Its Modern Mainframes

by Gabe Goldberg in Mainframe Executive

SunTrust Banks, Inc., with nearly $200 billion total assets, is one of the nation’s largest and strongest financial holding companies. 

The company and its subsidiaries provide deposit, credit, trust, and investment services to diverse retail, business, and institutional clients as well as mortgage banking, asset management, brokerage, and capital market services. 

They operate more than 1,600 retail branches and more than 2,500 ATMs in 10 states and Washington, DC. In addition, SunTrust offers several technology-based banking channels, including Internet, PC, mobile, and automated telephone banking. 

From legacy “green screen” applications, SunTrust migrated to Graphical User Interface (GUI) applications using Internet technology. Additionally, they use frontend servers for an Internet banking offering, with main transaction processing running on CICS. IBM’s WebSphere MQ serves as a transport mechanism on both AIX and the mainframe; DB2 is the database of choice on the mainframe. 

Glenn Schneck,  assistant vice president, Mainframe Services, Online Systems Services, notes that, “In the banking industry, it’s critical to provide new products before the competition; the natural evolution was by Web services enabling our backbone systems.” He feels this allowed SunTrust to introduce new products much faster and cheaper than previously without Service-Oriented Architecture (SOA). 

In fact,  the most challenging application has been Internet banking, due to the complexity of supporting servers, connections, routers, etc. used before requests reach the mainframe. ATM, mortgage loans, commercial loans, and many other critical applications also execute primarily on the mainframe. 
SunTrust,  with a decades-long history of mainframe use, relies heavily on big iron as its IT environment backbone. Its Atlanta, GA, data center runs production processing, and a slightly smaller operation in Durham, NC, handles Disaster Recovery (DR). IBM’s Geographically Dispersed Parallel Sysplex (GDPS) connects these sites for real-time data recovery. 

IT’s mandate is to get more done with fewer resources; Schneck, with his 26 years of experience in systems support roles, interprets this as the need to “provide the bank with exceptional IT solutions while reducing costs.” The bank processes approximately 800 million transactions per month, completing 98 percent in less then 300ms. Peak load is 1,000 transactions per second. 

The primary data center has two IBM z9 systems, models 2094-714/S28. Each system provides 6,215 MIPS from general purpose processors, plus five Internal Coupling Facility (ICF) engines, two Integrated Facility for Linux (IFL) processors, two IBM System z9 Integrated Information Processors (zIIPs), and 208GB of memory. Primary software components are z/OS 1.8, CICS TS 3.1/3.2, WebSphere MQ V6, and DB2 V8 with New Function Mode. The data center, designed for high-availability and peak performance, is critical to client support and the bank’s vision of quality products and customer service. 

Most applications are server-based packages; integration of the mainframe is vital to SunTrust’s success, making CICS’ SOA critical, since front-end interfaces must mesh with core solutions. Integration software lets developers quickly respond to business requirements, bringing new products to market in a fraction of the time required before Web service implementation. In addition, recent projects created granular services that can be reused in innovative and unique ways. 

In 2006, SunTrust installed CICS Transaction Gateway for transactional access from a Java front-end and began a Proof-of- Concept (POC) project to create Web services for CICS applications. The strategic goal was to make current processes within CICS available to all applications, and an SOA approach was identified as the most efficient route for current and future projects. 

The POC used realistic business functions with each of three vendors working in a one-week window. This was done to avoid what Schneck calls the “Wizard of Oz” solution, which is where a vendor visits and creates services without bank staffers knowing what was involved. For the POC, SunTrust personnel actually used the products to complete the tasks, evaluating aspects such as ease of installation and use, vendor support, reusability, training requirements, ability to create composite applications, and achieving a purely mainframe option. The POC demonstrated that the GT Ivory product best satisfied the requirements. When more projects required SOA using CICS, SunTrust purchased Ivory and began modernizing the mainframe. 

The initial POC involved two or three programmers who effectively used the product with no formal training. Since that time, two architects have trained more than 50 other developers via an internal one-day class.  GT Ivory’s visual development environment allowed SunTrust to implement a pure mainframe solution, avoiding added complexity or points of failure. It also provides flexibility to run on other platforms if desired, and supports front-end programming in languages such as Visual Basic, .NET, Java, etc. An added plus was two-phase commit database support.  

The first project to use this technology was mobile banking, which enabled customers to access accounts via cell phone and perform many functions traditionally through Voice Response Unit (VRU) or Internet banking. This was efficiently implemented by creating 35 Web services without writing any COBOL code. These services operate at a very granular level to enhance reuse, facilitating the creation of composite services for customer functions such as balance transfers and account information. 

The project started on Aug. 1, 2007, and proceeded through several stages of testing and acceptance: development, Integrated Testing/Client Acceptance (ITCA), Production Readiness/Dress Rehearsal (PRDR), and production. ITCA and PRDR require multiple iterations and PRDR includes several performance tests. On Oct. 1, 2007, 60 days later, the application entered pilot production. Since that time no problem tickets have been generated. 

Subsequently, the bank implemented—over a weekend— a sales application feature allowing U.S. citizen clients or prospects to open accounts online for consumer checking, savings, and/or money market services, with immediate access to online banking. Primary processing is done on the mainframe using CICS and GT Ivory-created Web services. This application required 48 Web services, plus two COBOL programs for DB2 access. 

These projects were justified on an ROI basis with business unit assistance. The Web services enablement tool increased ROI and enabled the removal of an antiquated vendor product. 

Not surprisingly, customer authentication and identity verification are critical for both financial protection and government regulations. Since customers must be registered for Internet banking before signing up for mobile banking, the two access methods use the same sign-on criteria. 

Challenge questions provided by Equifax confirm client identity; the bank also allows customers to use eSignature verification for agreement and/or saving electronic versions of required account disclosures. 

WebSphere MQ is used on the mainframe with the “hub” being on AIX. But Schneck feels this “increases the chances of outages, errors, and issues due to the addition of extra hops to and from AIX, as well as imposing additional costs for separate systems, support, and licenses.” He doesn’t believe AIX accomplishes anything that couldn’t be processed on z/OS and feels mainframe hubbing would reduce overall costs. 

Keeping track of SOA Web services requires expanded project management and software inventory tools. The bank uses an existing change management environment for SOA components and an inplace mechanism for publishing services. Governance is an ongoing process. 

Schneck notes that SOA is really nothing new to the mainframe, since they’ve used “services” for many years. The ability to publish Web services for use by front-end processes allows the mainframe to be used as a large server. “GT Ivory lets us efficiently and rapidly create Web service interfaces to our existing CICS business functions,” he says, “saving 75 percent of mobile banking project development time by using it vs. COBOL application coding.” 

Tom Sedlack, SunTrust’s solutions architect, adds that SOA governance is an extension of IT governance. “In addition to that,” he says, “we plan to implement WebSphere Service Registry and Repository (WSRR) and IBM Tivoli Composite Application Manager (ITCAM) for SOA to help in managing services from both the design and run-time perspectives.” 

For several years, Schneck, with his manager’s blessing, has touted benefits of Linux on System z and potential savings from server consolidation in areas such as electricity, cooling, and floor space. Linux on System z implementation is planned in the next few months, running DB2 Connect and possibly Oracle databases and WebSphere Process Server, among other applications. 

During Schneck’s time at SunTrust, there has been occasional talk about “migrating off the mainframe,” perhaps to distributed AIX or Windows; he has refuted this by demonstrating that the true cost and support of a distributed architecture doesn’t match mainframe attributes. He sees migration as infeasible, “considering the workload, availability, scalability, reliability, disaster recovery, and other requirements placed on the IT organization.” In fact, some of these application migration attempts have failed. 
He calls CICS “The world class” transaction processor and the backbone of SunTrust’s environment. “IBM Hursley does a great job of listening to their user base, and changes they make are what keep CICS number one in the transaction processing arena,” Schneck says. 

And he notes that an overall “green effect” (not to be confused with “green screen”!) lets companies save on all environmentals by consolidating workloads on a mainframe, as well as reduce software license costs. Being convincing, he feels, stems from “persistence and a belief in what is right for the company,” and the fact the mainframe is so deeply ingrained in the corporation and so successfully supports both legacy and innovative applications. 

Countering industry rumors of mainframe skills shortages, SunTrust’s technical support department finds the qualified workers it needs. And Schneck sees success in the movement within IBM and the SHARE user group organization to get younger people excited about the mainframe. 

From many SunTrust perspectives, the mainframe remains a vital platform for hosting innovative banking products and will continue to be modernized with SOA enhancing CICS. Customer service and IT-enabled innovative products such as mobile banking and the sales application center feature help make SunTrust an industry leader.