Help Universe

Guide:

Data Transfer Mainframe.

 

Data Extraction and Transfer

It is highly recommended to start with a small sample file (e.g. one day’s worth of SMF Data) before transferring larger data sets. This is to ensure that the data is transferred correctly and has the correct format.

The following steps are carried out to extract and transfer the SMF data to ITBI™. Please look here for sample JCL to carry out these steps:

Extract

IFASMFDP is used to extract the SMF.

The following naming conventions should be used for the output files:

SMF 70-75, 30 and 113 should be delivered together in a file named CustomerName.DYYYYMMDD.SMF (e.g. COMPANY.D20221202.SMF).

SMF 101 should be delivered in a separate file named CustomerName.DYYYYMMDD.DB2 (e.g. COMPANY.D20221202.DB2).

SMF 110 should be delivered in a separate file named CustomerName.DYYYYMMDD.CICS (e.g. COMPANY.D20221202.CICS).

If the resulting files are very large (over 100 GB) then they should be split into smaller file:

• For very large installations the data may be split into multiple files by CEC, sysplex or by LPAR. In this case the CEC, sysplex or LPAR name should be included in the file name (e.g. COMPANY.SYSA.D2022SMF).
• If large amounts of historical data are being transferred, then the files should be split based on Date. The date qualifier can be used to indicate the last date contained in the file (e.g. COMPANY.D20171202.SMF indicates that the file contains data up to and including December 2, 2022).

IEBGENER

IEBGENER with DCB=(RECFM=U) is used to ensure the correct formatting of the data prior to transfer.

Note that the IEBGENER step is optional if standard IBM FTP is used to transfer the data files since the DCB=(RECFM=U) parameter can be specified on the DD card for the FTP step instead of running IEBGENER (see the sample JCL below).

IEBGENER copy is required for transfer methods where the DCB=(RECFM=U) parameter cannot be specified or isn’t used by non-IBM FTP tools. This is often the case if the files are manually downloaded to Windows prior to transfer. In rare cases z/OS System Storage Management override JCL specification on IEBGENER output file and a full DCB with Dsorg, Recfm and Blocksize together with parm DSNTYPE=BASIC need to be specified.

Transfer

Standard IBM FTP is the preferred method for transferring the data files from the customers mainframe to ITBI. DCB=(RECFM=U) must be specified on the DD card and the ‘binary’ parameter must be used. See the sample JCL below. The FTP destination, user-ID and password will be provided by SMT Data.

FTPS or SFTP are also options if the customer prefers an encrypted transfer.

Other transfer mechanisms are also an option in the case of a one-off transfer, for example:

• downloading the data to a removable hard disk and sending the disk to SMT Data
• downloading the data to a Windows PC or server and then uploading it through the ITBI portal.
• downloading the data to a Windows PC or server and then FTPing the data to SMT Data.

Note IEBGENER is normally required prior to downloading, and the file must be downloaded as ‘binary’ using the appropriate option for your download method. The files can be G-zipped using e.g. 7-zip or PKZIP prior to transferring them.

 

Be sure to remember everything, download the checklist here:

Download checklist

Intervals

The amount of data transferred depends on a number of factors, including:

• Which SMF types are included (SMF 101 and SMF 110 can be very large compared to the other SMF types)
• The size and complexity of the customer’s installation
• The time interval covered by the extract
• If the data is compressed or not. Compressing the files with G-zip usually result in an over 10X reduction in size

Both one-off transfer of historical SMF data and ongoing daily transfer of current data. Typically, ITBIaaS implementation starts by loading historical data, which is then updated on an ongoing basis. We prefer getting new data multiple times a day (can be down to every 5^th minute) and needs to be at least once a day just after midnight, so that are available for analysis and reporting the next day.

SMF Record Types

Minimum SMF required

The minimum requirements are SMF record types 70, 72 and 30.

SMF 70:
Hardware information in type 70 must include information for all defined LPARs. Specifically, the global performance data control option must be set to allow each z/OS system to record data for all LPARs in the configuration.
Subtype 1 – CPU, PR/SM and ICF Activity – is required as a minimum.

SMF 72:
Subtype 3 – Workload Activity – is required as a minimum

SMF 30:
The following subtypes are required as a minimum:

• Subtype 2 – Interval
• Subtype 3 – Last Interval

Recommended additional data types

In addition to the above minimum set, the following additional data types are also recommended depending on the customer’s specific configuration and needs.

SMF 101:
This SMF type is relevant for customers running DB2.
Subtype 0 – DB2 Plan accounting – is required
Subtype 1 – DB2 Package accounting – is also supported, but should be limited to selected LPARS due to data volumes

SMF 102:
IFCID 172- for DB2 deadlock data
IFCID 196- for DB2 timeout data

SMF 110:
This SMF type is relevant for customers running CICS.
Subtype 1 – CICS Monitoring Data – is required as a minimum.
A copy of the CICS dictionary record must also be included with the SMF. The CICS dictionary records can be created by turning on the CMP monitor for each region (for example using the CEMT SET MONITOR ON PERF command).

SMF 71:
This SMF type is relevant for customers who want to understand memory usage.

SMF 75:
This SMF type is relevant for customers who want to understand paging activity.

SMF 74:
This SMF type is relevant for customers who want to understand the coupling facility.
Subtype 4 – Coupling Facility Activity – is required as a minimum

SMF 113:
This SMF type is relevant for customers who want to understand the use of processor cache, RNI and CPI.
Subtypes 1 and 2

IMS log record 56FA
This record only contains performance and capacity information from the IMS log and can be created by the DFSUARC0 program. Read more here.

Other Data Types:

Other data types than the ones listed above can be included in the ITBI solution. Contact us for more information on support@smtdata.com

Sample JCL

The following JCL samples show how to extract, format and transfer the SMF data.

The sections in <brackets> must be changed based on the customer’s installation and the naming conventions specified above.

The DATE statement in SYSIN may be removed but can be used if necessary to identify a range of days to be extracted (specify start and end date in Julian format).

Note that the DD-name ‘INDD1’ may require the specification of several datasets.

//<JOBname> JOB (ACCOUNT),’DUMP SMF’

//*

//* WARNING: CHOOSE YOUR BLKSIZE WITH CARE

//* IF SMF INPUT BLKSIZE IS LESS THAN OR EQ TO 27998 CHOOSE 27998

//* IF SMF INPUT BLKSIZE IS LARGER THAN 27998 THEN CHOOSE 32760

//* IF LARGE AMOUNT OF DATA MAKE A SEPARATE DATASET

//*

//STEP1 EXEC PGM=IFASMFDP

//INDD1 DD DSN=<INPREF>.<SMFDATA>,DISP=SHR

//OUTDD1 DD DSN=<PREF>.SMF.<SYS>.<DATE>,

// DISP=(,CATLG),UNIT=(3390,9),

// SPACE=(CYL,(500,200),RLSE),

// DCB=(BLKSIZE=27998,RECFM=VBS)

//SYSPRINT DD SYSOUT=*

//SYSIN DD *

INDD(INDD1,OPTIONS(DUMP))

OUTDD(OUTDD1,TYPE(30,70,72))

DATE(2017sss,2017eee)

/*

//* PREPARE DATA FORMAT FOR TRANSFER.

//* THIS STEP IS OPTIONAL IF IBM FTP IS USED FOR THE TRANSFER

//* THE DCB OF SYSUT1 AND SYSUT2 MUST BE THE SAME

//*

//STEP2 EXEC PGM=IEBGENER

//SYSUT1 DD DSN=<PREF>.SMF.<SYS>.<DATE>,DISP=SHR,

// DCB=(RECFM=U,BLKSIZE=27998)

//SYSUT2 DD DSN=<PREF>.XFER.<SYS>.<DATE>,

// UNIT=(3390,9),DISP=(,CATLG),

// SPACE=(CYL,(500,200),RLSE),

// DCB=(RECFM=U,BLKSIZE=27998)

//SYSPRINT DD SYSOUT=*

//SYSIN DD DUMMY

//*

//* SAMPLE FTP STEP

//*

//STEP3  EXEC PGM=FTP

//SYSPRINT DD SYSOUT=*

//SYSABEND DD SYSOUT=*

//SYSOUT   DD SYSOUT=*

//DSNFTP     DD DISP=SHR, DSN=yourinputdatasetname, DCB=(RECFM=U)

//FTPOUT   DD SYSOUT=*

//SYSIN    DD *

servername

userid password

binary

put //DD:DSNFTP  ABCDCOMPANY.D20171202.SMF

quit