Aug 15 ’11

Mainframe Security: Securing UNIX

by Stu Henderson in z/Journal

Continuing our discussion of security over paths into the system, here we consider UNIX on the mainframe. There are three possible configurations for this, which I will describe before addressing their security. In order of increasing security and overhead, they are:

  1. Native Linux “against the metal” running Linux directly on a CPU or Logical Partition (LPAR)
  2. Linux guests under the VM operating system; instances of Linux, each running in its own virtual machine. (IBM mainframes supported virtualization [in the software with VM and in the hardware with LPARs] long before VMWare.)
  3. UNIX System Services (USS) under z/OS with additional security provided by the security software (IBM’s RACF or CA Technologies’ ACF2 or Top Secret).

How They Differ From Distributed UNIX

All three of these configurations have advantages over distributed versions of UNIX. If you have a UNIX application facing the Internet that uses files from your z/OS system, any of these three configurations will improve your response time, security, and reliability compared to a separate UNIX server.

These configurations also provide better CPU load balancing than you will find with distributed systems. Load balancing lets you specify which workload is to receive most of the CPU power, while allowing CPU power to be shared among several workloads. Load balancing is controlled, depending on the configuration, by means of LPAR management, VM priority setting, and the z/OS Workload Manager. Any of these needs less overall CPU power than if each UNIX ran in its own CPU.

How to Secure

Security includes control over access through the network, control over access to the UNIX system itself, and file access control (which we will address in a separate column, since we’re now just examining security over paths into the system).

To secure network access, you need to identify and control all the IP addresses as well as all the ports (TCP, User Datagram Protocol [UDP], and Internet Control Message Protocol [ICMP]). You do this with a combination of physical network configuration, firewalls, configuration files, and (in the case of USS) the security software.

The June/July 2010 column described how to secure TCP/IP with z/OS and USS. The same concerns and similar tools apply to the Linux configurations. Depending on the risks you need to protect against, you will use encryption (especially of userids and passwords), packet filtering, blocking of ports, Network Address Translation (NAT), and intrusion detection.

With all three configurations, much of this will be provided by firewalls. TCP/IP under z/OS provides USS additional network controls, including use of security software to control access to IP addresses and ports, control file security options, more secure support for Secure Sockets Layer (SSL) encryption (since the digital certificates can be stored in the security software database), and the Policy Agent software. The Policy Agent provides packet filtering, NAT, and intrusion detection. It may preclude the need for firewalls.

To implement these tools effectively, you need to map the IP paths into your mainframe and then identify the firewalls and other tools at each control point in the map. If you can’t explain in detail the exact paths to your mainframe UNIX, you likely have network security exposures.

To control access to the Linux systems, you will administer the /etc/passwd file, sometimes supplemented with the /etc/shadow file. These contain one record for each user with his encrypted password and other information. Many of the well-known attacks against UNIX systems are based on getting read access to these files and running password cracker programs to learn all the userids and passwords. The approach taken by USS is considered more secure (and with greater overhead), since it relies on the security software. With USS, you can also ask the security software (using the SAF resource class named APPL) to control who can come into USS.

Users in UNIX are identified by numeric UNIX User IDs (UIDs), and groups are identified by numeric Group IDs (GIDs). You will want to avoid accidently assigning the same UID to different users. Any user whose UID is zero is considered a Superuser (or “Root”) and can access any UNIX file. You will want to have effective userid and password administration (such as password length and content, and reliably identifying a user before resetting his password) to secure this path into your mainframe. Since some programs (such as FTP) permit anonymous logons, you will want to restrict such logons, or restrict what they’re able to access.

If you control the IP paths into your UNIX and control access to your UNIX as described here, you will be able to say that you’ve secured the UNIX path into your system.