May 1 ’07

DFSORT: IFTHEN, Numeric Tests & Relative Dates

by Editor in z/Journal

One of the most useful features of programming languages is their ability to do conditional logic, which can be described conceptually as IF something, THEN something. Although IBM’s DFSORT isn’t a programming language, it has its own flavor of conditional logic for reformatting records. Appropriately called IFTHEN clauses, this feature can be used in a wide variety of situations and often makes it possible to replace multiple passes over the data with a single pass over the data. This article examines three examples that illustrate some things you can do with IFTHEN clauses and other new DFSORT features. Future articles will discuss additional features for manipulating data in various ways, including how to parse variable fields such as comma-separated values and delimited fields into fixed fields.

Let’s start with a simple, but useful example relating to multiple record layouts in the same input data set. Figure 1 shows an input file with a header record identified with ‘hd’ in positions one through two, detail records with no identifiers, and a trailer record with ‘trl’ in positions one through three. The three types of records (header, detail, and trailer) also have different record layouts. The header record has a date in positions 21 through 28 that we want to overlay with the date of the run. We want to sort on the bird names, but keep the header as the first record and the trailer as the last record. Just sorting on the bird names in positions 15 through 26 won’t work because the header and trailer records have different values in those positions; the output would be as shown in Figure 2. We certainly don’t want the trailer and header in the middle of the detail records.

Figure 1 shows the DFSORT control statements we need to sort the records correctly and to update the date in the header record. The new function here is the IFTHEN clause and its WHEN, OVERLAY and BUILD subparameters. We’ll discuss what each control statement does as well as the new IFTHEN function. This particular example uses the IFTHEN function while sorting, but you also can use it while copying or merging.

We use three IFTHEN clauses in the INREC statement to handle the three different record types in three different ways:

As a result, we have ‘A’ in 81 for the header record, ‘B’ in 81 for all the detail records, and ‘C’ in 81 for the trailer record.

We use the SORT statement, which is processed after the INREC statement, to sort on our new indicator in position 81, and then on the bird names in positions 15 through 26. The ‘A’ record (header) is sorted before the ‘B’ (detail) records, and the ‘C’ record (trailer) is sorted after the ‘B’ (detail) records. Since the first sort field is ‘B’ for all the detail records, they’re further sorted on the second sort field (bird name). The header record is kept as the first record and the trailer record is kept as the last record, with the detail records sorted in between by the bird name.

Finally, we use the OUTREC statement to remove the temporary indicator from position 81. Figure 1 shows the correctly sorted output records.

IFTHEN Clauses

You can use the new IFTHEN function on the INREC, OUTREC, and OUTFIL statements to conditionally reformat records. For each IFTHEN clause, you must specify a WHEN parameter to identify the records the clause applies to, and an OVERLAY or BUILD parameter to indicate the actions to be taken for those records.

WHEN=INIT clauses can be used for actions that apply to all records. WHEN=(logexp) clauses can be used for actions that apply only to records that satisfy the specified logical expression. All the logical expression capabilities available for INCLUDE also are available for WHEN, including comparisons, substring comparison tests, bit logic tests, date comparisons, numeric tests, AND and OR. WHEN=ANY clauses can be used for actions that apply only to records that satisfied a preceding WHEN=(logexp) clause. WHEN=NONE clauses can be used for actions that apply only to records that didn’t satisfy any of the WHEN=(logexp) clauses.

OVERLAY was discussed in a previous article as a parameter in its own right, but it also can be used in an IFTHEN clause to overlay specific data without affecting the rest of the record. BUILD, which also was discussed in a previous article, is a parameter in its own right, but also can be used in an IFTHEN clause to build a record itemby- item.

By default, as soon as a WHEN=(logexp) clause is satisfied for a record, IFTHEN processing for that record stops. However, you can use IFTHEN’s HIT=NEXT parameter to continue with the next IFTHEN clause as we’ll see in the next example.

This example shows how you can use IFTHEN clauses to check and correct multiple numeric fields in each input record. Figure 3 shows an input file with two ZD fields, one in positions four through eight and another in positions 10 through 14. All the negative ZD fields have a D sign (for example, ‘0521L’ = X’F0F5F2F1D3’). However, some of the positive ZD fields have a C sign (for example, ‘1234E’ = X’F1F2F3F4C5’), and others have an F sign (for example, ‘68200’ = X’F6F8F2F0F0’). We’d like to normalize the positive ZD fields to all have F signs. In addition, not all the values are valid ZD values (for example, ‘582B3’ = X’F5F8F2C2F3’ has an invalid zone of C). We’d like to change the invalid ZD values to ‘00000.’

Figure 3 shows the DFSORT control statements we need to check and change the numeric values. The new functions here are NUM and TO=ZDF. We’ll discuss what each control statement does as well as the new functions.

We use the OPTION COPY statement to tell DFSORT to copy the records without sorting or merging them.

We use four IFTHEN clauses in the OUTREC statement; two for each ZD field.

Figure 3 shows the output records with an F sign for each positive ZD value (for example, ‘12345’ = X’F1F2F3F4F5’) and ‘00000’ instead of each invalid value.

NUM

NUM is a new function you can use in a logical expression of the INCLUDE and OMIT statements, the INCLUDE and OMIT parameters of the OUTFIL statement, and the WHEN parameter of an IFTHEN clause to test whether a character, zoned-decimal, or packed-decimal value is numeric ( E Q, N UM) o r n o t n ume r i c (NE,NUM).

TO=ZDF, TO=ZDC, TO=PDF, TO=PDC

TO=ZDF and TO=ZDC are new output formats you can use in the INREC, OUTREC and OUTFIL statements wherever TO=ZD can be used (converted fields, statistics, page numbers, counts, and so on). With TO=ZDF or TO=ZD, DFSORT uses an F sign for each positive ZD value (for example, +12 = X’F1F2’). With TO=ZDC, DFSORT uses a C sign for each positive ZD value (for example, +12 = X’F1C2’). In either case, DFSORT uses a D sign for each negative ZD value (for example, -12 = X’F1D2’).

Our final example shows how you can use IFTHEN clauses to collect and display data about different date intervals. Figure 4 shows an input file with an amount field (dddd.d) in positions one through six and a date field (yyyy/ dd/mm) in positions 11 through 20. For this example, the current date (date of the run) is 2006/07/07, so the input file has both past dates and future dates relative to the date of the run, ranging from 2006/05/15 to 2006/08/01. We’d like to display the totals of the amounts for three date intervals: from seven days before the current date to three days before the current date, from two days before the current date to two days after the current date, and from three days after the current date to seven days after the current date. We’re not interested in the amounts for dates eight or more days before or after the current date.

For our current date of 2006/07/07, the actual date intervals are: 2006/06/30 through 2006/07/04, 2006/07/05 through 2006/07/09, and 2006/07/10 through 2006/07/14. Since we’re dealing with relative dates, the date intervals would change depending on the date of the run. For example, if we were to run the job on 2007/07/11, the date intervals would be: 2006/07/04 through 2006/07/08, 2006/07/09 through 2006/07/13, and 2006/07/14 through 2006/07/18. We want our report to show the actual dates for each date interval regardless of when the job runs.

Figure 4 shows the DFSORT control statements we need to display our interval report. The new functions here are past date constants (DATE1(/)- days) and future date constants (DATE1(/)+days). We’ll discuss what each control statement does as well as the new functions.

We use the OMIT statement to remove the records with dates we’re not interested in. The 11,10,CH,LT,DATE1(/)-7 test omits each record with a yyyy/mm/dd date less than the current date -7 days. The 11,10,CH,GT,DATE1(/)+7 test omits each record with a yyyy/mm/dd date greater than the current date +7 days. Since OMIT is processed before the other statements, the omitted records aren’t processed by the other statements.

We use three IFTHEN clauses in the INREC statement to handle our three date intervals:

We use the SORT statement to sort on the interval dates in positions one through 21. The records for the first interval (lowest dates) will be sorted first, followed by the records for the second interval, and then the records for the third interval (highest dates). This ensures records in the same interval are in consecutive order so we can use OUTFIL’s SECTIONS parameter to total the amounts for each interval.

Finally, we use the OUTFIL statement to display one line for each interval with the dates for that interval and the total of the amounts for that interval. HEADER2 creates header records for the report, including the date of the run. SECTIONS treats all records with the same value in positions one through 21 (the interval dates) as a group. TRAILER3 creates one trailer record for each group with the interval and total for that group. For example, with our current date of 2006/07/07, the output record for the first interval would have ‘2006/06/30-2006/07/04’ in positions one through 21 and ‘14538.8’ in positions 29 through 35 (we increased the total by one digit to avoid overflow). We want a trailer record with the interval and total for each group, but no data records, so we use NODETAIL to suppress the data records. We don’t want ANSI carriage control characters for our report, so we use REMOVECC to eliminate them.

Figure 4 shows the output report with the headings, including the run date, the actual dates for the three intervals, and the total for each interval.

Past and Future Date Constants

You can use new relative date keywords in the INCLUDE and OMIT statements, the INCLUDE and OMIT parameters of the OUTFIL statement, and the WHEN parameter of an IFTHEN clause to test date fields with appropriate formats against past and future date constants. For example, you can test:

The available date constants are as follows (of course, the same keywords without -d or +d are constants for the current date):

You can use new relative date keywords in the INREC, OUTREC, and OUTFIL statements to generate past and future date constants. The available date constants are as follows (of course, the same keywords without -d or +d generate constants for the current date):

Summary

DFSORT’s IFTHEN function can be used in a wide variety of situations that require conditional reformatting of records while sorting, merging or copying, and can often eliminate multiple passes over the data. Other new features such as numeric tests and relative dates also can make you more productive. By using the techniques discussed here, you can tap into those features whenever you need them. Future articles will discuss additional DFSORT functions you can use. Z

For More Information

Visit the DFSORT Website at www.ibm.com/storage/dfsort/for complete documentation on the features discussed in this article. In particular, my “User Guide for DFSORT PTFs UK90007 and UK90006” paper (April 2006) discusses the numeric tests, relative dates, TO=PDF and TO=PDC functions, and my “User Guide for DFSORT PTFs UQ95214 and UQ95213” paper (December 2004) discusses the IFTHEN, OVERLAY, BUILD, TO=ZDF and TO=ZDC functions. All the DFSORT documentation can be accessed online from www.ibm.com/ servers/storage/support/software/sort/mvs/srtmpub.html.