Skip to content
Expertise

MF1 algorithm: Unreferenced or obsolete files

How to detect the z/OS batch processes that create files no later step uses, to cut consumption, run time and disk space.

  • Finds improvements that remove batch processes, or parts of them, that create obsolete files.

  • The goal of this detection is threefold: lower consumption, shorter duration of the affected processes, and freed disk space.

The algorithm

The aim is to detect batch processes that generate a file that is not referenced in later steps, that is, files that are not used by any later process: files that are generated but then never used.

To detect them in z/OS mainframe installations, you need access to the information in the system logs, specifically three SMF record types: SMF14, SMF15 and SMF30.

We need certain information from each of them:

  • SMF14: information about the files used as input to batch process steps.

  • SMF15: information about the files used as output of batch process steps.

  • SMF30: information about the consumption of batch processes, both per process (JCL) and the detailed consumption and duration of each step.

Execution

The algorithm works by extracting the necessary fields from each of the SMF records above and then cross-referencing the data to obtain a list of candidates.

First, we cross-reference SMF14 and SMF15 to obtain the files that are being created but that no step of any batch process in the installation is using.

This produces a result file listing every file generated in a step of a batch process that is not referenced later in any other step.

After this first cross-reference, the result is matched against SMF30 to learn the consumption and duration of each step detected as obsolete. This lets us filter candidates by a minimum CPU consumption or run time (elapsed) threshold, and sort them to see which candidates have the most impact. You can also filter by a given time range, or see which applications have the most obsolete candidates.

Bear in mind that throughout this cross-referencing of logs, you must discard the installation's system files that should be excluded from the algorithm, such as statistics or incident files, which could otherwise put false positives in the candidate list.

Results

Running this algorithm detects candidate obsolete steps in z/OS batch processes using one very specific technique: detecting files that are not referenced by other processes.

In installations where this kind of technique has never been applied, the candidate list can represent between 0.5% and 3.5% of the whole installation.

Removing these obsolete processes brings a benefit in three areas of the z/OS installation:

  • Lower CPU consumption, and therefore fewer MIPS used, which is a direct cost saving.

  • Shorter processes, and therefore more margin for processes with an SLA (those with a target end time).

  • Freed disk space, which leaves more room to store information and lowers the baseline.