Jul 1 ’06
Network Job Entry for the 21st Century
Network Job Entry (NJE) is a protocol for transmitting jobs, SYSOUT data sets, operator commands and messages, and job accounting information from one computing system to another.
IBM has provided implementations of this protocol on all its mainframe systems: RSCS for z/VM, JES2 and JES3 for z/OS, and POWER for z/VSE. There’s also an implementation for iSeries OS. NJE is supported as an application over different transports, including native BSC and CTC communications, SNA networks and TCP/IP (VM, VSE, and JES2 at z/OS 1.7 only).
Although a relatively “old” protocol, there are several significant reasons to extend the facilities provided by NJE outside the mainframe and down to the distributed systems:
- Clean bi-directional integration of programmable workstations with mainframe data transfer
- Leverage Linux-based development tooling and skills in tandem with mainframe services (the right tool for the right job)
- Move data between IBM and non- IBM environments without complex automation requirements
- Eliminate SNA as a requirement to run NJE (required for pre-1.7 z/OS systems).
This article provides examples of several different scenarios for using NJE in a distributed environment, including:
- Unattended file transfer: Being able to deliver files without having to write elaborate scripts controlling File Transfer Protocol (FTP)
- Development workstation: Coding programs and jobs using the Integrated Development Environment (IDE) of choice, submitting the jobs, and receiving the job output
- Automated delivery of job output to people, programs and devices
- Remote job execution on the distributed system from jobs submitted from “classic” mainframe systems.
An Introduction to NJE
NJE is primarily an application facilitating the transfer of commands, messages, programs, and jobs among different computing systems in a network. The NJE Formats and Protocols manual (SC23-0070-03) describes NJE as follows: An NJE network is a group of two or more complexes or systems that communicate with each other. An NJE network is comprised of nodes that can transmit or receive a unit of work. The nodes in an NJE network use protocols to communicate with each other. Protocols are rules a node uses to:
- Become part of an NJE network
- Receive a unit of work
- Send a unit of work
- Indicate it was removed from the network.
NJE is an extension of Remote Job Entry (RJE), which was also developed by IBM. Before the widespread use of the Internet, NJE networks such as BITNET formed the basis of the largest collection of computers yet known. During its lifetime, NJE has accrued a heritage of tools, infrastructure, and procedures that still have merit in today’s environment.
Let’s now consider how the NJE facility can be used to establish communications, controls, and remote execution facilities between nodes running on the classic mainframe operating systems and those on the distributed platforms. Throughout this article the examples reference one of the available implementations of NJE on distributed systems.
Automated File Transfer
Automated file transfer between systems using FTP scripts is tedious and error-prone. When it works, it’s fine, but as soon as something breaks, everything crashes with no simple method for detecting the problem and recovering appropriately. Commands such as FTP were designed for interactive transfers and require things such as retry and recovery to be bolted on.
NJE was designed to handle asynchronous file transfers. It copes well with queuing and retrying. It will alert operations when anything goes wrong, so your existing automation tools can intervene.
The NJE service running on the non-classic systems also has a sendfile command that lets you send files to an NJE-capable system. NJE interactions can be scripted simply because of its single command interface, store-and-forward mechanism, extensive logging support and use of return codes to indicate success or failure.
Files can be delivered to individual users, to devices, or to a program that can act on the contents of the file. Figure 1 shows a trivial file transfer from Linux to z/VM.
Submitting Jobs From Linux
Job streams can be created on the non-classic platform and submitted using the submit command.
An NJE job contains either SYSIN or SYSOUT data and the control records used to identify the data being transmitted. A node uses these control records to transmit an NJE job:
- A job header containing information that affects scheduling, security and accounting
- A data set header if the data is SYSOUT or optionally SYSIN data
- A job trailer containing information that affects output formatting and accounting.
All NJE jobs contain both a job header and job trailer.
Under VSE, the LDEST operand of the JOB JECL statement can be used to cause the output to be returned to the submitting node (see Figure 2). NJE wraps the JCL with the required control records. The job is transmitted to VSE, where POWER will schedule the job for execution. The job is executed and output is returned via the XMIT queue. The output from the job will be placed in the submitter’s “reader queue,” a temporary spool area used to hold NJE input and output files. You can interrogate the contents of your NJE spool queues using the:
- Qrdr command to query what files are available to you
- Peek command to preview files in your reader
- Receive command to transfer a file from the NJE spool area to your current working directory
- Purge command to discard an NJE spool file without receiving it.
Redirecting Files to Programs or Devices
Besides simply routing or returning files to the “reader” of an NJE user, you can have the NJE system invoke a script or a program that acts on the file. For example, to print the output file directly on a local or networkattached printer under the control of Common Unified Printing System (CUPS), the NJE system can be configured to intercept files routed to a pseudo- user such as “LPR.” For example, under VSE, the LDEST operand can be set to “LPR” rather than a user ID. When NJE receives a file with this destination, it could invoke the lpr command to send the file to CUPS for post-processing in such ways as:
- Directing it to a real printer
- Sending it as a fax
- Converting it to a PDF and storing it for later viewing (or staging it to CD or DVD)
- Sending it as an attachment to a prepared electronic mailing list.
Executing Batch Jobs on Linux
NJE provides interfaces to an open source batch execution facility known as NQS. This allows execution of jobs on the Unix or Linux-based platforms the NJE/IP Bridge supports. Compared to other batch systems, NQS is relatively primitive but can be used as the basis of a useful job execution environment. NQS expects scripts to be sent to it and for it to return the output of stdout and stderr to the user. There’s no file staging capability (yet). Figure 3 illustrates remote task execution. This file can be created on any . NJE node. In this example, we’ve used our VM system to prepare and submit the job.
Submitting a job for execution is a matter of sending the script to the user “JOB” on the remote NJE node. When the job is complete, the stdout and stderr files from the job are returned to the originating user. The figure shows two files returned to the originating user, with the file type set to the NQS job number and using the peek command to display the contents of stdout. The platform tools can then manipulate the returned files.
Sending Commands and Messages Between Nodes
NJE provides the facility to send commands and messages between nodes. For example, you can display POWER queues on a VSE system from your Linux system. Figure 4 shows the “D A” command initiated on Linux and executed on VSE. Similarly, users residing on NJE nodes may exchange messages with each other. Unlike file transmissions, there’s no store-and-forward mechanism. Messages that can’t be delivered immediately are discarded. Tools such as NetView, NetMaster, and Tivoli’s management suite can be used to centralize management of all NJE nodes.
Bringing It All Together
NJE continues to fill an important role in the modern IT infrastructure and its benefits can be extended to distributed environments. NJE facilitates robust system management and provides a set of functions that can be exploited to help standardize the automation of operations that are typically installation-unique. Z