Jun 2 ’14

Increasing the Strength of z/VM Secure Connectivity

by Brian Hugenbruch in Enterprise Tech Journal

As computer threats evolve, so, too, do security controls and the standards around them. In 2011, the U.S. National Institute of Standards and Technology (NIST) issued a document titled “Transitions: Recommendation for Transitioning the Use of Cryptographic Algorithms and Key Lengths,” which outlines a path of migration and upgrade for existing network security infrastructure. It specifically recommends changes in algorithm usage and an increase in the key lengths required for cryptographic operations within a particular timeline.

For example, the use of public-key cryptography for connecting two systems securely has used a default key length of 1024 bits in the past. This key length, while still reasonable for many operations, is no longer considered “secure enough” for particularly sensitive information. The standard now recommends a key length of 2048 bits to maintain a secure posture against today’s threats.

The mainframe is no exception to these new rules. The System z hardware, operating systems and hypervisors are moving not only to support the maximum values identified by these standards, but to enforce the minimum values as well. While a guest has its own security configuration and faces threats particular to it, it’s essential to note that protecting the hypervisor is an equally important part of an end-to-end security policy. Access to the hypervisor grants a special form of access to any particular guests, so it’s critical to maintain secure connectivity to and from the hypervisor.

Reintroducing the z/VM V6.3 SSL Server

z/VM has long supported a security server for TCP/IP traffic. The current server is a pool of commonly configured virtual machines associated with a particular TCP/IP stack. This server handles the Secure Socket Layer (SSL) and Transport Layer Security (TLS) protocols for z/VM. In addition to negotiating the handshake, these protocols encrypt and decrypt traffic to and from the hypervisor layer. This cryptographic barrier extends security controls to prevent unauthorized modification to the Control Program or to the guests running underneath it.

In previous versions and releases of z/VM, the SSL Server supported the SSL 3.0 and TLS 1.0 protocols. Its cryptographic library, a port of z/OS’s System SSL feature, provided security support for asymmetric key sizes up to 4096. However, z/VM’s implementation only surfaced one configuration option to augment available security—an EXEMPT statement to disable a specific cipher suite. A cipher suite, which is a combination of asymmetric key exchange, symmetric encryption standard, hashing algorithm and key length requisites may vary in applicability from protocol to protocol, and some have proven more durable than others over time. This option, while important in managing security policy, lacked granularity and didn’t account for more recent protocol versions.

z/VM Version 6.3, made Generally Available in July 2013, has upgraded the SSL Server in several ways. First, it allows for the encryption and decryption of IPv6 traffic. It supports the verification of a client’s certificate during the handshaking process. Finally, and most important, it offers new options for the configuration of security policy for the TCP/IP stack. It adds support for SSL/TLS protocol selection, and its underlying System SSL library adds support for TLS 1.2. Through a New Function APAR PM93363, z/VM V6.3 delivers the capability to place the SSL Server into an 800-131a-compliant mode through a single option.

These options not only benefit those system or network administrators beholden to obey a strict SP 800-131a requirement, but they allow for greater flexibility in managing secure connectivity to the hypervisor. 

Selecting an SSL or TLS Protocol Version

The first part of controlling secure connectivity to z/VM is control of the protocols usable by the z/VM SSL Server. By default, the SSL Server continues to provide SSL v3 and TLS 1.0. While these may be sufficient for some installations, other security policies may require a system administrator to enable other protocols or to disable the defaults entirely.

z/VM V6.3 provides an SSL Server that supports connections ranging from SSL v2 up to TLS 1.2. The protocols can be enabled or disabled through the use of a new PROTOCOL keyword. This keyword is specified in the DTCPARMS file, which defines all the other server options in a z/VM TCP/IP configuration. The DTCPARMS file is read-only during the start-up of an SSL Server—most commonly, when the TCP/IP stack is coming online. These options can’t be modified while the SSL Server is in use, so using them requires some advance planning.

In an SSL Server definition, the PROTOCOL keyword is specified as part of the raspberryarms list of options. The server defined in Figure 1 is a pool of SSL servers—multiple virtual machines that encrypt traffic for a single TCP/IP stack. They must all abide by the security rules defined in the DTCPARMS file. In this case, the parameters include the enablement of TLS 1.2, TLS 1.1 and SSL v3. TLS 1.0 and SSL v2 are disabled.

TLS 1.2 is the newest of the point-to-point encryption protocols, and it provides a couple of new features. First, it allows for hashing in the SHA2 family of ciphers (SHA-256 and above). These ciphers are exclusive to this highest level. Second, it will prevent the use of some older ciphers—DES and the “export ciphers” can’t be used through the TLS 1.2 protocol. Finally, it will automatically disable SSL v2 if it’s enabled, as the two protocols can’t coexist.

TLS 1.0 and SSL v3 are enabled by default, so they don’t have to be specified. However, as previously mentioned, a disabling statement must be specified to prevent their use.

When a protocol is enabled, this allows a client application (such as IBM Personal Communications 6.0.8) to negotiate its use during the SSL/TLS handshaking process. If a client and the server can’t agree on a common protocol, then connectivity won’t be established. As a result, you must be certain the application being used to connect to z/VM can support the security protocols being specified.

Enabling New Cryptographic Policies

z/VM V6.3 delivers key functionality in allowing for the upper boundaries of security policy to expand. However, the second half of NIST SP 800-131a is a minimum level of security. This functionality isn’t available in the base z/VM V6.3 product; it’s only available through the application of a PTF.

APAR PM93363 introduces a second new keyword called MODE, which automatically configures the SSL Server to meet a particular cryptographic policy. z/VM V6.3 supports two cryptographic modes: FIPS 140-2 and NIST SP 800-131a.

The Federal Information Processing Standard (FIPS) 140-2 has existed for some time, and z/VM V6.1 was validated as meeting its requirements when configured with a keyword FIPS. (An evaluation for z/VM V6.3 is currently in progress, per an IBM Statement of Direction released in July 2013.) FIPS 140-2 places certain restrictions on both the protocols and ciphers the z/VM SSL Server can use. Specifically, MODE FIPS-140-2 enforces only TLS protocols and restricts the use of cipher suites to ones that have been approved for FIPS-level usage.

FIPS mode can be specified through the MODE keyword on the raspberryarms statement in the DTCPARMS file (see Figure 2). Using FIPS mode also requires the use of a FIPS-mode certificate database. A FIPS-mode certificate database is digitally signed and verified during the initialization of the SSL Server to determine if tampering has occurred. This verification is accomplished through a combination of RSA Known Answer testing and HMAC-SHA256 integrity checking. This cryptographic mode requires a minimum key length of 1024 for asymmetric key exchange; either RSA or DSA signed certificates are valid in this mode.

MODE NIST-800-131A is a distinct mode of operation and is separate from FIPS mode (see Figure 3). This mode of operation carries several conditions with it. The first, as noted previously, is that only TLS 1.2 can be used with this mode. Attempting to use any other protocol with this mode will result in a warning message, and failure to enable TLS 1.2 via the PROTOCOL operand will prevent the SSL Server from commencing operation. This is because NIST SP 800-131a requires the use of the SHA2 family of hashes, and these hashes are only operable when using TLS 1.2.

The second condition is the asymmetric key exchange; the minimum key length in this mode is 2048. z/VM V6.3’s System SSL implementation is a port of similar function for z/OS V1.13 (with some added function). Because this crypto library on z/VM V6.3 supports a maximum DSA key length of 1024, no DSA-signed certificates can be used in this mode. Similarly, RSA-signed certificates of length 1024 will be rejected, even if hashed with SHA-256 or SHA-512. There are no requirements on the certificate database when using MODE NIST-800-131A. All the requirements in this case are placed on the certificates. Fortunately, the state of existing certificates (and determination of whether they meet these conditions) can be examined through the gskkyman application. This application is a port of the z/OS function of the same name; it’s executable from the GSKADMIN virtual machine or a similarly configured guest. Gskkyman allows for certificate database management as well as certificate import and export.

Can these modes of operation be combined? Yes, they can; however, because they adhere to a different set of requisites, combining them restricts the SSL Server configuration to the maximal common subset of functionality–TLS 1.2 exclusively, a minimum key exchange length of 2048, and a FIPS-mode certificate database. This will provide exemplary security strength with very little flexibility. 


z/VM V6.3 delivers some significant improvements to the strength and configurability of TCP/IP network security. The availability of TLS 1.2 allows for state-of-the-art encrypted connections to or from the hypervisor layer, and the implementation of the PROTOCOL operand allows for non-default SSL/TLS settings to be configured. APAR PM93363 further refines security capability by allowing for automatic configuration of an SSL Server (or pool thereof) to meet a particular cryptographic standard. The key determination will be what configuration provides the appropriate levels of security strength, network speed and certificate flexibility for a z/VM administrator’s environment. With the upgraded cryptographic library and configuration settings, though, an administrator has some options at his disposal.


NIST SP 800-131A: “Transitions: Recommendation for Transitioning the Use of Cryptographic Algorithms and Key Lengths,” www.nist.gov/manuscript-publication-search.cfm?pub_id=907517.