Sep 1 ’03

Implementing Secure Sockets on z/OS

by Editor in z/Journal

Security is often one of the key concerns for installations using TCP/IP on z/OS systems. The migration from the apparently more secure SNA environment to TCP/IP, combined with the growth of Internet services based on z/OS, is increasing the demand for secure communications.

Firewalls and the Internet Security Protocol (IPSec) provide packet and network-level security, but these approaches are not always practical when providing services to a wider range of clients and do not necessarily protect the applications or application data. Secure sockets were introduced to protect individual applications by implementing user authentication and encryption at the transport layer.

This article provides a basic overview of secure sockets on z/OS systems and introduces some of the built-in support available from IBM. This article is meant to provide an introduction to the subject rather than be a definitive reference guide.


Netscape invented the Secure Sockets Layer (SSL) protocol to implement security at the application level. Although standard and is widely used by Internet servers and Web browsers to protect sensitive transactions such as those dealing with credit card details and other personal information. SSL became standardized in 1999 with the introduction of the Transport Layer Security (TLS) protocol (RFC 2246). Because secure sockets are an application-layer protocol, they are independent of the lower-layer security protocols and can therefore coexist with firewalls and IPSec tunnels.

Figure 1 shows the positioning of SSL/TLS when compared to firewall and IPSec technologies. Secure sockets are based on public/private key encryption, exploiting the use of digital certificates. A digital certificate is defined by the X.509 standard and is effectively a file that contains the encrypted details of the certificate owner. To be valid, a digital certificate must be requested and subsequently approved (“signed”) by a trusted Certificate Authority (CA). A certificate can be requested from a CA on behalf of an individual end user, an organization, or even a specific application. For maximum security, both the server and the client-end of a secure sockets connection should have a digital certificate signed by a trusted CA.

Secure sockets provide four key security functions:


Secure socket protocols are implemented between the Application and the TCP layer. Normal TCP communications protocols are still used between the connection’s partners. Therefore, with one exception (which I will discuss later), no changes should be required to system, router, or firewall definitions. Remember that SSL functionality is required at both ends of a connection as, not surprisingly, if one application is encrypting data, something at the other end has to decrypt it.

Immediately after establishing a connection and before any data exchange has taken place, the SSL/TLS component performs a “security handshake” with the partner SSL/TLS component. This handshake is comprised of a series of public/private key encrypted messages that are used to negotiate the level of SSL/TLS to be implemented, verify the existence and content of digital certificates, and agree on the session encryption key and algorithms to be used for the secured connection.

Once the secure handshake completes successfully and the application chooses to start data exchange with its partner, the SSL/TLS component intercepts data from the application, performs the required encryption, and sends the encrypted message to the partner application. The partner application’s SSL/TLS component decrypts the data, performs the message authentication checks, and passes the data to the receiving application. As the encryption/decryption is performed after receipt of the data by the application process, no intermediate process (including TCP or IP layer tracing) has access to an unencrypted copy of the data.


Although secure sockets are widely used by Web servers, their benefit is applicable to any other TCP application that requires the additional security features provided.

The downside of secure sockets, when compared to IPSec-based security, is that the application must replace the existing TCP socket calls with a combination of SSL/TLS and TCP socket calls, plus have its logic changed to handle SSL/TLS related settings and error detection.

Standard TCP/IP socket calls are used to listen, connect, accept, and shut down a connection, but the SSL/TLS socket calls must be used to initialize the SSL/TLS environment, initialize secure sockets, and initiate the secure handshake. SSL/TLS socket calls must also be used to send and receive data on the connection so that the data is encrypted.

In addition to the basic socket and SSL/TLS communications, an application implementing secure sockets must implement logic to identify the repository for digital certificates and the certificates to be used. It must also be able to set run-time options to define the level(s) of SSL/TLS protocols and encryption algorithms to be negotiated and supported during the SSL/TLS handshake.

Figure 2 shows a simplified example of a sockets application that implements SSL/TLS communications.


For full server and client authentication, the digital certificates must be stored in a repository accessible by the secure sockets function used by the application. The application code differs, depending on the type of certificate repository, so the type and location of the repository is generally passed as a parameter to the application. An application references a specific certificate by its “label,” which is assigned when the certificate is generated. A certificate repository can also specify one certificate to be its “default,” allowing applications to use the default certificate without having to know or specify its label.

z/OS supports three certificate repositories:


Two distinct tools have been provided to manage digital certificates on a z/OS system:

Both tools provide the range of functions required to create and manage certificates, including:

True digital certificates must be “signed” by a trusted CA (such as Thawte or Verisign). Before using a trusted CA, a certificate identifying the CA must be installed on the system where the certificates are to be used. In z/OS, certificates for some of the more common CAs are provided by default, removing the need to obtain the certificate from the CA.

To obtain a digital certificate, a certificate request must be generated and sent to the CA using one of the digital certificate management utilities. The CA processes the certificate request and a “signed” version of the certificate is returned to the requestor. This signed certificate must be installed on all systems that need the certificate for authentication using the certificate management utilities.

Although full server and client authentication can only be achieved using digital certificates signed by a trusted CA, it is not always necessary to purchase a digital certificate from a CA. In situations where secure sockets are only being used inside a private network, the organization can act as a CA for itself and its certificate can be added to all clients as another trusted CA. Certificates for other clients and servers within the private network can then be issued by the “internal CA,” giving a high level of internal security. Technically speaking, systems within the private network could use the internally issued certificates to communicate with systems outside the network. However, the outside systems would need to have knowledge (and trust) of your organization sufficiently to accept the internally signed certificates.

Finally, it is also possible to create self-signed certificates. These do not require any internal or external CA for authentication and therefore have little security value. However, self-signed certificates are very useful when implementing new applications to test for correct SSL/TLS functionality prior to requesting and receiving officially signed certificates.


Secure sockets support was introduced in OS/390 V2.6. Support for SSL protocols has been enhanced over the years with SSL V2 and V3 supported by all current versions of OS/390 and z/OS, and TLS V1 being supported by z/OS V1.2 and above.

Several IBM applications support SSL/TLS protocols, including the TN3270E Server, the FTP Server, the FTP Client, WebSphere Application Server (WAS), and the HTTP Servers.

TN3270E is perhaps the easiest place to start when implementing secure sockets for the first time. Support is enabled by the SECUREPORT and CONNTYPE SECURE keywords in the TCP/IP profile. It is common to add a new secure TN3270E port rather than replace the existing port to allow support for both secure and non-secure connections. The KEYRING keyword is used to define the repository and type of digital certificate database to be used and the CLIENTAUTH keyword determines whether a valid client certificate is mandatory. Users connecting to the secure TN3270E port must be using an SSL-capable TN3270 client (such as IBM’s Personal Communications, IBM’s Host on Demand, and Jolly Giant’s QWS3270 Secure).

The FTP Server and FTP Client support SSL/TLS to further secure and encrypt file transfers. Support is enabled using the EXTENSION AUTH_TLS keywords in the FTPDATA member. Additional keywords are supported to control the level of authentication required and to identify the digital certificate repository. FTP can allow both secure and insecure transfers on the same server, depending on the value of the SECURE_FTP keyword. The FTP protocol requires some additional consideration when implementing secure sockets. Both the control and data FTP connections are encrypted, providing a high level of security but at the same time potentially disabling some firewall functionality. The FTP protocol passes IP addresses in the data transmission and some firewalls parse this data, verifying and often converting the IP addresses for Network Address Translation (NAT). As the FTP control session is encrypted, and the firewall cannot decrypt the data, NAT and the subsequent file transfer will fail. This is a recognized limitation and extensions to the FTP protocol have been implemented to cope with this problem and will be available in a future release of z/OS.

The HTTP Server and WAS support secure socket communications for the more traditional use of SSL, Web serving. The HTTPS (HTTP + SSL) protocol typically uses port 443 (instead of the HTTP port 80) to implement secure sockets over the HTTP protocol. The WAS supports SSL for communications with other SSL-capable components and clients.


Secure sockets can, of course, be used for communications between z/OS systems and SSL-enabled clients (i.e., TN3270E and FTP) and the respective z/OS-based servers. As SSL/TLS are documented protocols, other operating system platforms and applications can implement secure sockets and communicate with z/OS-based applications.

As discussed previously, following the establishment of a connection, a security handshake takes place. Part of this handshake includes the exchange of the level of SSL/TLS and the encryption algorithms that are supported and acceptable to both partners. The application can usually define the order in which the supported options are negotiated. The handshake is successful when a specific level of SSL/TLS and encryption algorithms are supported by both applications. If no common set is found, the handshake process fails, and the applications cannot implement a secured connection.


The implementation of secure sockets does have some performance implications. After connection establishment, but prior to data exchange, the handshake protocol does require the exchange of a series of encrypted messages, leading to a small delay before the first data packet is seen. For protocols such as TN3270E, this could mean the initial sign-on screen seen by the user takes a little longer to be displayed.

Once the handshake process has completed, the same number of packets is exchanged between the connection partners, although CPU cycles are required at both ends to encrypt and decrypt the messages. Some caching storage is available with z/OS secure sockets implementation to reduce the overhead. Initial tests have shown that SSL connections do not generate a significant increase in overhead when compared with their non-SSL equivalent connections.

If the mainframe on which your application is running has cryptographic hardware features and the Integrated Cryptographic Service Facility (ICSF) has been started, the encryption/decryption will be performed on hardware rather than software, improving performance and reducing CPU cycles. The secure sockets application does not have to be changed to use the hardware if it is present.


Developing your own SSL applications is possible using the System SSL Programming Interfaces provided with z/OS. Interfaces are only provided for C/C++ and Java applications. Applications written in other languages, such as Assembler, could potentially drive the SSL/TLS functions, but would have to cope with the complications of being compatible with Language Environment (LE) and the differences in program linkage and parameter passing.

As a result, existing sockets applications written to the OE callable services must effectively be rewritten to drive the secure sockets application interfaces. Applications written using the pre-z/OS 1.2 Application Programming Interface (API) will operate on z/OS 1.2 and above, but cannot support TLS Version 1. To support TLS Version 1, applications need to be redeveloped (maybe for a second time) to the new SSL API available from z/OS Version 1.2.


Secure sockets protocols offer an additional level of security not available with IPSec or firewall technologies. This article has introduced secure socket protocols but has by no means covered all of the details required to fully understand and implement an effective security policy.

Implementing secure sockets protocols on z/OS systems is, in principle, relatively easy, but in practice is likely to be fairly complex and time-consuming. The ease of use of SSL between Web servers and browsers should not be used as a benchmark for implementing SSL in other applications. Generally, encryption is the only part of the protocol implemented by Web servers, whereas full security based on secure sockets requires requesting, issuing, and installing digital certificates for all client and server applications. Server certificates may potentially need to be manually installed on all client systems, while all certificates that identify client systems or users may need to be installed on the server system.

However, the benefits to be gained by the use of secure sockets are significant. The server can be certain of the identity of the client application and, equally important (to the client), the client can be certain of the identity of the server. No system utility or intermediate router or LAN-based sniffer can read the data on a secure connection once it has left the application process. The data on a secure connection cannot be modified without the application’s knowledge. With security now a much higher priority worldwide, the implementation of secure sockets on z/OS should allow security managers to sleep more easily. Z