May 7 ’12

Virtual Addressing With z/VSE: From 24-Bit to 64-Bit

by Ingolf Salm in z/Journal

The 64-bit virtual support introduced in z/VSE 5.1, available since November 2011, lifts a boundary and provides more options for growth and new applications. Before we consider how 64-bit virtual is implemented and what it provides, let’s examine the evolution of real and virtual addressing in VSE.

The Addressing Evolution

VSE started with a single address space and added up to three address spaces in the ’80s, followed with an address space for each of the 12 partitions. In the early ’90s, VSE added more address spaces to support dynamic partitions. Each address space had up to 16MB virtual storage. z/VSE’s name changed from VSE/SP to VSE/ESA. Shortly after, 31-bit addressing was introduced, starting with 31-bit real addressing. Now it was possible to support up to 2GB real storage. Next, 31-bit virtual addressing allowed address spaces of up to 2GB. VSE provided a 31-bit address space supporting the Enterprise Systems Architecture (ESA) hardware architecture.

Each address space has a shared area below the 16MB line, starting at zero, called shared area (24-bit) and a shared area at the end of the address space, called shared area (31-bit). All address spaces can access the shared areas. Between the shared areas is a so-called private area, unique to each address space, where the VSE partitions are allocated.

The total virtual storage size (VSIZE) VSE supports—summing up the private area sizes of all partitions and the shared areas—is 90GB. To access the area above 16 MB, a program must change its addressing mode (AMODE). In 24-bit addressing mode (AMODE 24), a program can access data up to 16MB; in 31-bit addressing mode (AMODE 31), a program can access up to 2GB.

Key VSE components and products exploited 31-bit addressing from the beginning; the CICS Transaction Server followed in 1999. In between, VSE provided another virtual storage enhancement—data spaces. A data space is a kind of address space for data only (up to 2GB). A program can use services to allocate and manage data spaces. The ESA architecture provides access registers to identify the data space. The program must switch into access register mode to access a data space. The combination of access registers and general-purpose registers locate the data in a data space. Subsystems such as VTAM and CICS exploit data spaces.

The z/Architecture, which enabled the implementation of 64-bit real and virtual addressing, debuted in 2000. The architecture extended the registers from 4 to 8 bytes, the Program Status Word (PSW) from 8 to 16 bytes, introduced new fields in the low core to hold the 64-bit interrupt information, and provided new instructions for 64-bit addresses. z/VSE continued to use the ESA/390 architecture. IBM System z processors support both the ESA/390 and z/Architecture.

In 2005, VSE changed its name to z/VSE to show its membership to the System z operating systems (z/OS and z/VM). However, z/VSE 3.1 still supported the ESA/390 architecture only.  Exploitation of 64-bit addressing had to wait for two more years. z/VSE 4.1 was the first release based on the z/Architecture and implemented 64-bit real addressing, supporting up to 8GB real storage; 64-bit registers and 64-bit addressing mode were mainly used by the page manager. The virtual address spaces were still limited to 2GB and programs could only execute in 24-bit or 31-bit addressing or access register mode. However, this was the first step to 64-bit. With the next release, z/VSE increased supported real storage to 32GB and prepared the system for 64-bit programs. New save areas were provided to hold the larger interrupt information. The interrupt handlers store the 64-bit status information into those save areas. The dispatcher restores 64-bit register before returning to the program. Programs could use 64-bit (8-byte) registers and exploit 64-bit instructions, but not yet switch into 64-bit addressing mode.

Finally, it was time for the 64-bit virtual support in z/VSE. It emerged with the new z/VSE 5.1 release. The 64-bit virtual implementation follows z/VSE’s strategy of Protect, Integrate, and Extend (PIE). Protect your existing investment (programs, data, skills, etc.) and exploit new hardware (devices, processors) transparent to the existing applications. Integrate z/VSE with non-z/VSE platforms  through connectors where z/VSE can be a back-end server connected to an applications server. For example, z/VSE as a client can connect to a database server running on Linux on System z. Extend z/VSE solutions by capabilities that Linux on System z provides. z/VSE’s 64-bit virtual support allows existing programs to run unchanged (protecting that investment).

With 64-bit virtual addressing, programs can access data above the 2GB line. The 2GB line is called the “bar.” The area above the bar can only hold data; execution of instructions isn’t supported. z/VSE provides new Application Programming Interfaces (APIs) to manage the 64-bit storage. As always, if z/VSE needs to introduce new APIs, VSE development verifies if such an API is already available for z/OS applications. If so, it’s ported to z/VSE. That’s called z/OS Affinity. z/OS Affinity eases porting of applications from z/VSE to z/OS and vice versa. On z/OS, the IARV64 services provide 64-bit services. The IARV64 macro, a High-Level Assembler (HLASM) interface, was copied to the z/VSE macro library, and z/VSE implemented a proper subset of the IARV64 services.

The virtual storage above the bar is organized as memory objects. A memory object is a contiguous area of virtual storage starting at a 1MB boundary and is a multiple of 1MB in size. Programs use IARV64 services to create private or shared memory objects. To access a memory object, the program must switch into 64-bit addressing mode (AMODE 64) via z/Architecture instructions, e.g., Set Addressing Mode 64 (SAM64).

Private memory objects are accessible only in the address space where the program that obtained the memory object resides. All VSE tasks of the program (partition) may access the memory object. Private memory objects are freed via IARV64 services or when the creating task terminates.

Shared memory objects can be shared between programs executing in different address spaces. A program must register its interest via an IARV64 service before the system provides access to the memory object. All tasks of that program have access to the shared memory object. The interest of a program can be removed with an IARV64 service or will be removed at main task termination. The shared memory object must  be freed by using an IARV64 service. It will not be freed at task termination of the creating task.

In z/VSE 5.1, any VSE task may change the PSW key to key 9 in addition to their partition key. That allows a program to change data allocated with storage protection key 9. With shared memory objects, user programs may allocate areas with storage protection key 9 and share the area with user programs in other partitions. That provides more options for applications.

The address space layout below the bar (see Figure 1) doesn’t change. A 64-bit address space layout just adds the memory objects above the bar. The partition allocation and setup don’t change. The virtual storage size (VSIZE) as defined at Initial Program Load (IPL) now includes the size of memory objects and the sizes for data spaces, and private and shared areas. The VSIZE is still limited to 90GB.

Figure 2 shows a 64-bit address space layout. Before a program can obtain a memory object, the size for all memory objects must be defined. This occurs via the MEMLIMIT parameter of the new SYSDEF MEMOBJ Attention Routine/Job Control command. The virtual size of all private and shared memory objects is included in the MEMLIMIT specification. The SHRLIMIT parameter defines the limit for shared memory objects. MEMLIMIT doesn’t reserve virtual storage; it just defines a limit. You can change the MEMLIMIT and SHRLIMIT anytime, but they become active only if all memory objects are freed. This is a flexible way to allow storage allocations without a re-IPL or change to the storage/partition layout below the bar.

There are a few considerations for z/VSE’s 64-bit virtual implementation. As mentioned, z/VSE only implemented the HLASM support. High-level languages such as COBOL, C, RPG, or PL/I don’t provide 64-bit register or 64-bit addressing mode support. z/VSE’s load services and linkage editor don’t support AMODE 64; the program may get control in 24- or 31-bit mode and can switch into AMODE 64 during program execution. All z/VSE services such as Supervisor, VSE/POWER, VSAM, or BAM services must be called in AMODE 24 or 31. Parameter lists and I/O data areas must stay below the bar. Programs in Interactive Computing and Control Facility (ICCF) pseudo partitions can’t use IARV64 services or execute in AMODE 64.

CICS programs may use 64-bit registers, but must save the high order 4 bytes of the general-purpose register before invoking a CICS service. The program must restore them after the CICS service returns. CICS services can’t be called in AMODE 64.

Benefits of 64-Bit Virtual in z/VSE



A program can easily access large amounts of data. It must only adapt to the new IARV64 services and switch into AMODE 64, where access to data above the bar is necessary. This may require additional real storage to reduce or avoid paging, depending on performance requirements. The 64-bit virtual support programs may better exploit a larger amount of real storage.

Compared to data spaces, access to the 64-bit area is less complex. The program doesn’t need to manage access register content. The area above the bar is just an extension of the address space, which also reduces program complexity. It can be used just like a program uses GETVIS (GETMAIN) storage. In AMODE 64, the program has access to data in 24-, 31-, or 64-bit virtual storage. z/VSE’s 64-bit virtual implementation has no impact on existing APIs. Programs can manage large amounts of data in virtual storage and improve the sharing of virtual areas between address spaces.

One of the first exploiters of 64-bit virtual support is the IPv6/VSE TCP/IP product. It allocates TCP/IP buffers for IPv4 or IPv6 communication in memory objects if a MEMLIMIT is defined and enough virtual storage is available.

Virtual addressing with z/VSE has progressed rapidly in a short time; IBM customers can benefit greatly from fully exploring the continuously evolving capabilities.