After reviewing VSAM usage for many years, one of the biggest unrecognized problems I’ve seen was (and still is) directly related to batch window processing and online performance. In fact, the issue is large enough that several vendors have created VSAM add-on enhancement products, but unfortunately, they may actually make this situation worse. What is this horrible problem, you ask? The Swami, knowing the answer even before you asked it, wrote this article to give you the answer!
BACKGROUND: BEFORE VSAM
More than 30 years ago, when we needed files indexed by a key, we fought with ISAM. ISAM had a relatively short life — from the early 1960s to the early 1970s — by which time it had been almost completely replaced by VSAM. One of the main problems with ISAM was how it handled inserted records — they were kept, unblocked, in chains in an “overflow area” at the end of the file. Adding only a few records to a file could produce very long service times for additional inserts and when processing records that had previously been inserted.
Late in ISAM’s life, “Cylinder Overflow Areas” were introduced, where you could reserve one or more tracks at the end of each cylinder, so that the chain processing might be shortened. The “Independent Overflow Area” at the end of the file was still available for use when the Cylinder Overflow Area was exhausted. Even with this enhancement, the inserted records were kept unblocked and in a chain in the overflow area(s), significantly impacting performance.
In those ISAM days, it was not uncommon to build a file with direct insert activity, and to pause after a few hundred inserts to reorganize the file in order to improve its performance. Some of the things we used to do with ISAM may have carried over into our ideas about VSAM, even though they are not generally applicable in the VSAM environment. Similarly, VSAM enhancement product features that will automatically reorganize your file for you based on VSAM insert and split activity should be monitored because in many cases, these reorganizations may not be helpful.
For the past 30 years, VSAM has provided a better way — actually several better ways. When defining a file (cluster in VSAM terminology), you can specify that some percentage of each Control Interval (CI) can be reserved as free space, and some percentage of the CIs in each Control Area (CA) can also be kept free. In the case of files with heavily clustered inserts, these techniques are not very effective, and another VSAM technique — splitting CIs and CAs to create additional free space precisely where needed — can be very useful. Often, we counter the benefits from splits by reorganizing our files more often than we should.
VSAM HANDLES INSERTS WITH CI AND CA SPLITS
CI SPLIT PROCESSING
VSAM always keeps records within DASD blocks (or groups of blocks), called CIs, and if the record size permits, many records can be contained in a CI. It doesn’t matter if the records are all the same length or are variable length. If there is more than one record contained in a CI, the records in that CI will be kept in ascending sequence by their key values.
Figure 1 shows that VSAM has plenty of free space in this CI to permit the insertion of another logical record of the length illustrated. As long as there is sufficient free space to handle the size of the logical record being inserted, it will simply be inserted into the CI in order, and the amount of free space remaining will be reduced accordingly.