Aug 15 ’07
Mainframe Integration Using Web Services: A Survival Guide
In the IT industry, there’s a basic rule of thumb: For every problem, there’s a vendor claiming to sell a solution. The mainframe integration space is no exception; if you let word get out that you’re looking for a solution, you’re likely to be contacted by a wide variety of vendors, each with its own take on the problem and solution. So, how do you choose the right option for your enterprise?
First, understand the problem you’re trying to solve. Many people see mainframe integration using Web services as a problem of making COBOL programs consume and produce XML. That’s the approach taken with many commercial products. Actually, the problem is much more involved than this. The question you should ask is, “How can we incorporate our mainframe into an enterprise Service-Oriented Architecture (SOA)?”
An enterprise SOA is much more than converting your data communications to XML. To be successful, you must consider security, governance, registry, standards, operations, performance, reliability, change management, and many other features. These are the critical underlying elements that determine the success or failure of any enterprise endeavor. This is a difficult set of problems to solve, especially with commercial software, as most vendors see the problem as bangles and beads and have failed to invest in nuts and bolts. To truly understand the integration of the mainframe into an enterprise SOA, you need in-depth knowledge of the pitfalls and challenges of integrating (and operating) the mainframe and SOA.
You need a guidebook to help you sort out winners from also-rans. This article offers a list of 10 necessary stops on the road to an enterprise SOA solution, each with an explanation of why it’s essential to solid, holistic mainframe integration. You may not find a solution that has them all, but the more it has, the better.
1. No middle-tier gateways: This is the most important point. There should be absolutely, positively, no gateways involved in communications between mainframe applications and distributed applications. If you’re looking at a solution that requires a middle-tier gateway, look elsewhere. You need end-to-end visibility for each transaction. Each hop a transaction takes introduces a “black box” and point of failure. If you have no tools to look into the gateways (and you won’t), then you’ll literally be fumbling in the dark.
2. Run your SOAP stack on the mainframe: Relying on a distributed server to do your XML parsing, encryption, signature processing, and application of policy adds a layer of potential failure and negates the mainframe’s greatest advantage over distributed systems: reliability. Moreover, the mainframe can only expose edge interfaces to an external server that may require significant recoding to adapt mainframe applications to work within that limitation. The whole point of mainframe integration is to leverage existing assets. If you have to spend time and money modifying those assets, are you really leveraging them? There’s also the performance bottleneck a middle-tier gateway imposes, and that’s only when it’s running. Unless you enjoy stress, limit your search to only those solutions that run on the mainframe itself. The development tool should be distributed system-based, but the actual run-time that facilitates communication should run solely on the mainframe.
This isn’t easy. To run a SOAP stack entirely on the mainframe, you’ll need to focus heavily on performance.
3. Performance: If you run the SOAP stack on the mainframe, SOAP processing performance overhead must be minimized. Although it’s basically just string manipulation, XML parsing can be an expensive proposition if it isn’t designed specifically for the mainframe platform. Beware of parsers that are mainframe implementations of open source parsers.
4. Leverage human assets: The solution must leverage existing technology assets as well as human ones. Mainframe programmers know a lot about mainframes and generally little about distributed systems. The reverse applies to distributed programmers. If the solution you’re considering tasks distributed programmers with exposing mainframe applications as Web services, look elsewhere. Such a solution will require a major investment to train distributed programmers in mainframe systems and vice versa. A good solution would task mainframe programmers with publishing mainframe applications as Web services and distributed programmers with integrating those Web services into their distributed applications. The solution should require no specialized knowledge or training for either of them. No matter how shiny and spiffy the solution looks, if it doesn’t leverage human assets, it’ll be a nightmare to implement.
5. Central repository: The solution should use a central directory, such as a Universal Description, Discovery and Integration (UDDI) directory. What’s the point of publishing Web services if they aren’t available for enterprisewide reuse? A good solution will use a centralized UDDI directory and automatically document all Web services and their associated components in the directory. If the solution you’re considering stores these files on a developer’s workstation, stop considering it.
6. Thin-client: Since a good solution will leverage human assets and task mainframe programmers with publishing Web services, it shouldn’t require the installation of a fat-client application. In typical mainframe shops, these programmers make do with lower-capacity workstations incapable of effectively running a powerful, resource-intensive development tool. A good solution should be thin client; it shouldn’t require installing a resource-hogging tool on every workstation. A browser-based application is an ideal solution. Avoid fat-client solutions unless you want to spend money upgrading workstations and workstation software.
7. Flexible and adaptable: A good solution should be flexible; it should be able to adapt to just about anyone’s enterprise standards. Do you really want to spend months finding a solution only to be told by your firm’s chief architect that it’s not compatible with some bizarre requirement you didn’t even know about? This is a difficult feature to identify, but several concepts are critical, including WSDL-first, and top-down and bottom-up development.
8. Standards-based security: It was easy to rely on mainframe security back when the mainframe was connected to a closed network. There aren’t too many VTAM hackers out there, which means the mainframe security administrator usually didn’t have to worry about external threats. However, with the connection of your mainframe into the TCP/IP network, you’re vulnerable to an external threat. You’ll need a security solution that’s adapted to securing an SOA, and that means a solution that implements the WS-Security specification, with security managed by WS-Policy.
9. Bi-directional integration: A good solution must allow bi-directional communication. Having a mainframe application act as a server is great, but in most shops, that isn’t always enough. True integration means the mainframe must act as a server to a distributed client and also as a client to a distributed server. Any solution that doesn’t offer bidirectional communication is only half a solution.
10. Integrated monitoring, logging, and testing tools: Since we’re trying to avoid any and all middleware, the mainframe-based, run-time component of an ideal solution should have integrated monitoring, testing, and logging capabilities. You shouldn’t consider it a deal-breaker if a solution lacks these features, especially if it’s loaded with many of the others listed here, but a solution that does have them should score some extra points.
Hopefully, this article has helped expose the complicated, often-neglected issues involved in integrating the mainframe, and that it will aid you in selecting a viable and proven solution that will take your organization well into the future. Z on the thread titled “Mainframe Integration Using Web Services.”