4. Using the Co:Z SFTP client

An enhanced sftp client (cozsftp) for z/OS is also included in the Co:Z toolkit. This client can be used to initiate transfers with a remote host and supports the same set of file transfer options as the Co:Z SFTP server. The cozsftp command is installed in the $COZ_HOME/bin directory.

4.1 Starting the Co:Z SFTP client on z/OS

$ export PATH=/opt/dovetail/coz/bin:$PATH 1
$ cozsftp user@host
Co:Z sftp version: 1.1.0 (5.0p1) 2008-10-20
Copyright (C) Dovetailed Technologies, LLC. 2008. All rights reserved.
Connecting to host...
user@host's password: ***** 
cozsftp>
1

Add the Co:Z binaries directory to your PATH. This is not necessary if symbolic links from /bin were created during installation.

4.2 Co:Z SFTP client logging

OpenSSH SFTP logging is enabled by setting the -v option on the command line. The logging level can be raised by increasing the number of v options from -v to -vvv. Additionally, the COZ_LOG environment variable can be used to set logging options for the Co:Z extension library used to add z/OS support to the cozsftp command. The default setting for COZ_LOG is I. To change this default for all users, modify /etc/ssh/cozsftp_client.rc as needed. Individual users can override this setting by exporting COZ_LOG on the command line or in the copy of cozsftp_client.rc in their individual .ssh directory.

Client logging information is sent to stderr. In order to capture logging information in a file, use the following:

$ export COZ_LOG=D; cozsftp -vvv user@hostname  2>&1  | tee ~/cozsftp-output.log

The Co:Z client loglevel can also be set using the lzopts command described in the next section. See Section B.2, “Miscellaneous options” for additional information.

For information on setting these logging options in batch, see the section called “Logging in batch”.

4.3 Setting, displaying and clearing file transfer options

The enhanced client introduces two new commands:

lzopts [-a] [option=value,...]

The lzopts command is used to set local (client) file transfer options. These options are set prior to initiating file/dataset transfers from z/OS to a remote host.

zopts [-a] [option=value,...]

The zopts command is used to set server file transfer options -- if the server is a Co:Z SFTP server. The zopts command is functionally equivalent to the ls /+<option_list> command used by existing clients to set Co:Z sftp-server file transfer options.

Multiple options can be set by separating the option=value pairs with commas. An error is returned if one or more of the options was incorrectly specified, but the remaining options are set as requested.

The active options and their settings can be displayed by issuing the commands without arguments. The -a option can be specified to list all available options, even those that are not active.

The client performs some shell-like processing of its commands. In general, this is not an issue for the setting of options, but if the supplied option value contains a hash symbol (#), the option=value pair must be quoted, either with single or double quotes. For example:

cozsftp> lzopts "dataclas=#MYCLASS"
[Note]Note

For compatibility with z/OS OpenSSH SFTP, the cozsftp command recognizes the following additional subcommands: ascii and binary. these subcommands are treated as synonyms for lzopts mode=text and lzopts mode=binary respectively.

Client session options are determined in the following priority order:

  1. The fixed: section of /etc/ssh/cozsftp_config (highest priority and non-modifiable)

  2. The first matching pattern (if any) from $HOME/.ssh/cozsftp_config

  3. The first matching pattern (if any) from /etc/ssh/cozsftp_config

  4. Previous interactive commands: lzopts (described below) in the same session

  5. The environment variable SFTP_ZOS_OPTIONS

  6. The default: section of /etc/ssh/cozsftp_config (lowest priority)

For a list of available options, see Appendix B, Co:Z SFTP options.

For a description of the cozsftp_config file format, including how to specify file name patterns, see Appendix C, Session config files.

Example: Setting and displaying local (client) transfer options

cozsftp> lzopts mode=text     1
 mode=text
cozsftp> lzopts     2
clientcp=IBM-1047    loglevel=I           mode=text     
servercp=IBM-1047
1

The local option command lzopts mode=text is used to set the transfer mode to text. mode=binary is the default.

2

The local option list command lzopts shows the options currently in effect. In this case, the codepages clientcp and servercp are set to the defaults.

Example: Setting multiple local options

cozsftp> lzopts lrecl=80,recfm=fb,space=trk.3.2     1
 lrecl=80,recfm=fb,space=trk.3.2
1

Multiple options can be specified, separated by commas. Note that the SPACE parameter uses periods for commas to avoid ambiguity.

Example: Showing all local options

cozsftp> lzopts -a      1
clientcp=IBM-1047  linerule=flexible  loglevel=I        lrecl=80            
mode=text          overflow=wrap      recfm=fb          servercp=IBM-1047   
space=trk.3.2      NOallowmount       NOblksize         NObufno             
NOcopies           NOdataclas         NOdest            NOdir               
NOdisp             NOdsorg            NOforms           NOgdgnt             
NOhold             NOlabel            NOlike            NOmaxvol            
NOmgmtclas         NOnorecall         NOoutdes          NOrelease           
NOretpd            NOsequence         NOshowall         NOspin              
NOstorclas         NOsysout           NOtrim            NOtrtch             
NOucount           NOunit             NOvol             NOwriter
1

The option command lzopts -a is used to show all of the available options, even those that are not currently active.

4.4 Coordinating Transfer Options with a Co:Z SFTP Server

The enhanced Co:Z SFTP client can connect to any sftp server, including a Co:Z SFTP server. In this case, there are two sets of transfer options in effect; the enhanced client's and the server's. Client side (local) options are controlled via the lzopts command. Server side (remote) options are controlled via the zopts command.

When transferring POSIX files between a z/OS server and z/OS client, using the default mode=binary transfer option both locally and remotely will usually yield the desired results. If codepage translations need to take place, the desired clientcp, servercp and mode=text can be set either locally (via lzopts command) or remotely (via the zopts command). The other side can be left in mode=binary.

When transferring datasets between a z/OS server and z/OS client, it is generally recommended that linerule=rdw be used for binary transfers so that record mode boundaries are preserved.

Starting with Co:Z SFTP version 5.5.0, new dsput and dsget commands may be used to transfer z/OS datasets and automatically set local and remote transfer options so that new target dataset are allocated with attributes matching the source dataset. See the section called “Transferring Datasets Between Co:Z Systems” for details on these commands.

When converting from dataset to POSIX file between a z/OS server and z/OS client, the transfer options should be set where the dataset resides.

4.5 Working with Datasets

The Co:Z implementation of sftp accepts two prefix strings to identify MVS datasets as absolute paths. The first (//) is consistent with IBM's common usage. A secondary form (/-/) is also available.

Navigating Datasets

The sftp lcd command can be used to navigate around the z/OS dataset space. Using the dataset prefix // or /-/, the dataset space can be entered. Once there, traversal up and down various dataset levels can be performed similarly to hierarchical file systems.

Partitioned datasets are treated as directories as well. Once a PDS is made the current working directory, its members can be listed and retrieved like normal files.

Just as listing the entire catalog from the root is not allowed, it is not possible to make the catalog root the current working directory. As such, the command lcd // will fail.

Example: Navigating the dataset space

cozsftp> lcd //user     1
cozsftp> lpwd     2
Local working directory: //USER
cozsftp> lcd coz.testjcl     3
cozsftp> lpwd
Local working directory: //USER.COZ.TESTJCL
cozsftp> lcd ..     4
cozsftp> lpwd
Local working directory: //USER.COZ
1

Using the dataset prefix //, the high level qualifier user is specified. For lcd commands, the dataset name is case insensitive.

2

The lpwd command will list the current working dataset level. Note that the name is properly displayed in uppercase

3

Multiple levels can be traversed at a time. Instead of using the normal separator (.), a slash can be used: lcd coz/testjcl.

4

The lcd .. command will move up a level, as expected.

Transferring Datasets

The get and put commands are used to transfer datasets and PDS members.

Any options previously set via the lzopts are in effect for any given transfer.

Example: Get a file to a text sequential dataset

$ cozsftp user@linux.com     1
Connecting to linux.com...
user@linux.com's password:
cozsftp> lzopts mode=text     2
 mode=text
cozsftp> lzopts    
clientcp=IBM-1047    loglevel=I           mode=text            
servercp=ISO8859-1
cozsftp> get /tmp/GPGDSN //USER.GPGDSN     3
Fetching /tmp/GPGDSN to //USER.GPGDSN
ZosDataset[I]: Opening dataset USER.GPGDSN for write with options: new catalog
/tmp/GPGDSN                              100% 1215     1.2KB/s   00:00    
ZosDataset[I]: Closing dataset //USER.GPGDSN - 1215 bytes received, 15 records written
ZosSmf119Record[I]: SMF Type119 recording not enabled; SMF recording disabled
1

This example shows the full connection process, using keyboard-interactive password authentication to a remote linux system.

2

The default transfer mode of binary is overridden and set to text.

3

The get command uses the dataset path prefix // (or, optionally /-/) to specify that a dataset is to be written. At the default log level of I (INFO), information is emitted about the transfer process. Note also that in this case, SMF recording is disabled because the FTP SMF records (type 119) are not currently configured for recording.

Example: Get a text file to a PDS member

cozsftp> lzopts     1
clientcp=IBM-1047     loglevel=I     mode=text     
servercp=ISO8859-1
cozsftp> lcd //user.coz.testjcl 
cozsftp> lpwd
Local working directory: //USER.COZ.TESTJCL
cozsftp> get /tmp/GPGDSN     2
Fetching /tmp/GPGDSN to //USER.COZ.TESTJCL/GPGDSN
ZosDataset[I]: Opening dataset USER.COZ.TESTJCL(GPGDSN) for write with options: old
/tmp/GPGDSN                                             100% 1215     1.2KB/s   00:00    
ZosDataset[I]: Closing dataset //USER.COZ.TESTJCL(GPGDSN) - 1215 bytes received, 15 records written
1

If this transfer is performed after the prior example, the transfer mode will still be text. Using the lzopts command quickly confirms the active options.

2

The get command uses the dataset path prefix // and pds member name in parentheses to identify the member to create.

Example: Transfer PDS members between Co:Z z/OS systems with get and put

PDS members can be transferred between Co:Z z/OS systems using the get and put command, but the target PDS must already exist. The following examples show how to transfer an individual member or all members of the source PDS. NOTE: Starting with Co:Z version 5.5.0, two new commands dsget and dsput were introduced to make it easier to transfer data sets between two z/OS systems running Co:Z. See the section called “Transferring Datasets Between Co:Z Systems”. Note that dsget and dsput can be used to transfer program objects between z/OS systems, something that is not supported with the standard get and put commands.

cozsftp> put //COZUSER.SRC.PDS(MYJCL) //COZUSER.TGT.PDS(MYJCL) 1
Uploading //COZUSER.SRC.PDS(MYJCL) to //COZUSER.TGT.PDS(MYJCL)

cozsftp> get //COZUSER.SRC.PDS(MYJCL) //COZUSER.TGT.PDS(MYJCL)
Fetching //COZUSER.SRC.PDS(MYJCL) to //COZUSER.TGT.PDS(MYJCL)

cozsftp> put //COZUSER.SRC.PDS(*) //COZUSER.TGT.PDS 2
Uploading //COZUSER.SRC.PDS(@@README) to //COZUSER.TGT.PDS/@@README
Uploading //COZUSER.SRC.PDS(MYJCL) to //COZUSER.TGT.PDS/MYJCL

cozsftp> get //COZUSER.SRC.PDS/* //COZUSER.TGT.PDS  3
Fetching //COZUSER.SRC.PDS/@@README to //COZUSER.TGT.PDS/@@README
Fetching //COZUSER.SRC.PDS/MYJCL to //COZUSER.TGT.PDS/MYJCL

cozsftp> cd //COZUSER.SRC.PDS
cozsftp> lcd //COZUSER.TGT.PDS

cozsftp> get * 4
Fetching //COZUSER.SRC.PDS/@@README to //COZUSER.TGT.PDS/@@README
Fetching //COZUSER.SRC.PDS/MYJCL to //COZUSER.TGT.PDS/MYJCL

cozsftp> put *
Uploading //COZUSER.TGT.PDS(@@README) to //COZUSER.SRC.PDS/@@README
Uploading //COZUSER.TGT.PDS(MYJCL) to //COZUSER.SRC.PDS/MYJCL
1

To transfer a specific member specify the member name explicitly in parentheses on both the source and target PDS.

2

To transfer all PDS members with a put command, use the syntax (*) after the PDS name on the source. The target should specify only the PDS name (do not include (*) here)

3

To transfer all PDS members with a get command, use the syntax //PDS.NAME/* This syntax is different because of the way that the SFTP client processes get requests.

4

Alternatively, all members of a PDS can be transferred using the * wildcard character if the source and target directories are the desired PDS names, as shown above.

Example: Get multiple files using a wild-card pattern to a GDG

In release 2.4.0, support was added to allow multiple files to be downloaded to new generations of a GDG.

cozsftp> lzopts gdgnt     1
gdgnt
cozsftp> get /tmp/*.data //USER.COZ.GDG(+1)     2
Fetching /tmp/file1.data to //USER.COZ.GDG(+1)
ZosDataset[I]: Opening dataset //USER.COZ.GDG(+1) for write with options: new catalog
/tmp/file1.data                                             100% 523     20.2KB/s   00:01    
ZosDataset[I]: Closing dataset //USER.COZ.GDG.G0001V00 - 523 bytes received, 10 records written 3
Fetching /tmp/test2.data to //USER.COZ.GDG(+1)
ZosDataset[I]: Opening dataset //USER.COZ.GDG(+1) for write with options: new catalog
/tmp/test2.data                                             100% 886     18.5KB/s   00:01    
ZosDataset[I]: Closing dataset //USER.COZ.GDG.G0002V00 - 886 bytes received, 12 records written 
1

Wild-card downloading of remote files to new GDG generations is only supported if the gdgnt option is enabled. Sites should consider adding this option to the default section of their /etc/ssh/cozsftp_server_config file.

2

This get command uses a wild-card (*) pattern to select any file in the /tmp directory that ends in ".data". Each file will be downloaded to a new generation of the target GDG: USER.COZ.GDG.

3

Each file that matches the pattern is transferred separately. The generation name that was used is printed when the data set is closed.

Example: Put PDS members

cozsftp> lpwd
Local working directory: //USER.COZ.TESTJCL
cozsftp> put ONETEST /tmp/ONETEST     1
Uploading ////USER.COZ.TESTJCL(ONETEST) to /tmp/ONETEST
ZosDataset[I]: Opening dataset USER.COZ.TESTJCL(ONETEST) for read with options: shr

ZosDataset[I]: Closing dataset //USER.COZ.TESTJCL(ONETEST) - 38 records read, 3078 bytes sent

cozsftp> put //USER.coz.testjcl(*)     2
Uploading //USER.COZ.TESTJCL(@@README) to /tmp/@@README
ZosDataset[I]: Opening dataset USER.COZ.TESTJCL(@@README) for read with options: shr

ZosDataset[I]: Closing dataset //USER.COZ.TESTJCL(@@README) - 34 records read, 2754 bytes sent
Uploading //USER.COZ.TESTJCL(ALLOCDS) to /tmp/ALLOCDS
ZosDataset[I]: Opening dataset USER.COZ.TESTJCL(ALLOCDS) for read with options: shr

ZosDataset[I]: Closing dataset //USER.COZ.TESTJCL(ALLOCDS) - 6 records read, 486 bytes sent
Uploading //USER.COZ.TESTJCL(CHKENVD) to /tmp/CHKENVD
ZosDataset[I]: Opening dataset USER.COZ.TESTJCL(CHKENVD) for read with options: shr

ZosDataset[I]: Closing dataset //USER.COZ.TESTJCL(CHKENVD) - 1 records read, 81 bytes sent
Uploading //USER.COZ.TESTJCL(CHKPOST) to /tmp/CHKPOST
ZosDataset[I]: Opening dataset USER.COZ.TESTJCL(CHKPOST) for read with options: shr

ZosDataset[I]: Closing dataset //USER.COZ.TESTJCL(CHKPOST) - 6 records read, 486 bytes sent
Uploading //USER.COZ.TESTJCL(CHKPRE) to /tmp/CHKPRE
ZosDataset[I]: Opening dataset USER.COZ.TESTJCL(CHKPRE) for read with options: shr

ZosDataset[I]: Closing dataset //USER.COZ.TESTJCL(CHKPRE) - 72 records read, 5832 bytes sent
Uploading //USER.COZ.TESTJCL(COZCFGO) to /tmp/COZCFGO
ZosDataset[I]: Opening dataset USER.COZ.TESTJCL(COZCFGO) for read with options: shr

ZosDataset[I]: Closing dataset //USER.COZ.TESTJCL(COZCFGO) - 1 records read, 81 bytes sent
Uploading //USER.COZ.TESTJCL(GPGDSN) to /tmp/GPGDSN
ZosDataset[I]: Opening dataset USER.COZ.TESTJCL(GPGDSN) for read with options: shr

ZosDataset[I]: Closing dataset //USER.COZ.TESTJCL(GPGDSN) - 15 records read, 1215 bytes sent
Uploading //USER.COZ.TESTJCL(ONETEST) to /tmp/ONETEST
ZosDataset[I]: Opening dataset USER.COZ.TESTJCL(ONETEST) for read with options: shr

ZosDataset[I]: Closing dataset //USER.COZ.TESTJCL(ONETEST) - 38 records read, 3078 bytes sent
Uploading //USER.COZ.TESTJCL(TESTPROC) to /tmp/TESTPROC
ZosDataset[I]: Opening dataset USER.COZ.TESTJCL(TESTPROC) for read with options: shr

ZosDataset[I]: Closing dataset //USER.COZ.TESTJCL(TESTPROC) - 111 records read, 8991 bytes sent
Uploading //USER.COZ.TESTJCL(USERTEST) to /tmp/USERTEST
ZosDataset[I]: Opening dataset USER.COZ.TESTJCL(USERTEST) for read with options: shr

ZosDataset[I]: Closing dataset //USER.COZ.TESTJCL(USERTEST) - 187 records read, 15147 bytes sent
1

In this case, the current local directory is the PDS. This put command will transfer a specific member from a fully qualified dataset. Alternatively, the command: put //USER.COZ.TESTJCL(ONETEST) /target

could be used without regard to the current local directory.
2

When the put command is used on a PDS with "*" specified as the member, all of the members are uploaded. Note that the ability to specify a mask, like (AB*) is not currently supported.

Example: Put all generations of a GDG

In release 2.4.0, support was added to allow all generations of a GDG to be uploaded in one put commmand.

cozsftp> ls -alf //coz.test.gdg
Volume   Referred  Ext  Tracks    Used Recfm Lrecl BlkSz Dsorg  Dsname
                                                          GDG   COZ.TEST.GDG
VPWRKA  2013/06/04   1       1       1  U        0  6144  PS    COZ.TEST.GDG.G0003V00
VPWRKA  2013/06/04   1       1       1  U        0  6144  PS    COZ.TEST.GDG.G0004V00
VPWRKC  2013/06/04   1       1       1  U        0  6144  PS    COZ.TEST.GDG.G0005V00
VPWRKB  2013/06/04   1       1       1  U        0  6144  PS    COZ.TEST.GDG.G0006V00

cozsftp> put //coz.test.gdg(*) /tmp  1
Uploading //COZ.TEST.GDG.G0003V00 to /tmp/G0003V00
ZosDataset[I]: Opening dataset COZ.TEST.GDG.G0003V00 for read with options: shr
//COZ.TEST.GDG.G0003V00                                     20%   10KB  10.0KB/s   00:03 ETA
ZosDataset[I]: Closing dataset //COZ.TEST.GDG.G0003V00 - 2 records read, 10248 bytes sent

Uploading //COZ.TEST.GDG.G0004V00 to /tmp/G0004V00
ZosDataset[I]: Opening dataset COZ.TEST.GDG.G0004V00 for read with options: shr
//COZ.TEST.GDG.G0004V00                                     20%   10KB  10.0KB/s   00:03 ETA
ZosDataset[I]: Closing dataset //COZ.TEST.GDG.G0004V00 - 2 records read, 10248 bytes sent

Uploading //COZ.TEST.GDG.G0005V00 to /tmp/G0005V00
ZosDataset[I]: Opening dataset COZ.TEST.GDG.G0005V00 for read with options: shr
//COZ.TEST.GDG.G0005V00                                      0%    5     0.0KB/s 2:43:49 ETA
ZosDataset[I]: Closing dataset //COZ.TEST.GDG.G0005V00 - 1 records read, 5 bytes sent

Uploading //COZ.TEST.GDG.G0006V00 to /tmp/G0006V00
ZosDataset[I]: Opening dataset COZ.TEST.GDG.G0006V00 for read with options: shr
//COZ.TEST.GDG.G0006V00                                      0%    5     0.0KB/s 2:43:49 ETA
ZosDataset[I]: Closing dataset //COZ.TEST.GDG.G0006V00 - 1 records read, 5 bytes sent
1

In this example, all generations of a GDG are uploaded to the /tmp directory on the target system. As with all wild-card put commands, if the target directory is not specified it defaults to the current remote working directory.

Transferring Datasets Between Co:Z Systems

Starting with 5.5.0, two z/OS systems running Co:Z can transfer data sets more easily with the new interactive commands dsget and dsput. These commands enable the transfer of datasets without having to specify allocation parameters. These commands can also be used to transfer PDSE program objects, something that is not possible with the standard get and put commands.

This feature was introduced in several phases. In order to use the following features, both the Co:Z SFTP client and server must be at the level (or newer) that introduces the feature:

Version 5.5.0 introduces support for sequential data sets only. The source data set must be a sequential data set. The target data set (if it exists) must be a sequential data set. Relative GDGs are NOT supported.

Version 6.0.0 introduces support for partitioned data sets (both PDS and PDSE). Client DD names are not supported for either the source or target (note that DD names are never supported on the server). The IBM utility IEBCOPY is used to unload the source PDS(E) and to load the target PDS(E). If IEBCOPY completes with a non-zero return code, the dsget/dsput operation will fail and the IEBCOPY error message(s) will be written to the log (either the client or server, depending on where the error occurs). Note that IEBCOPY does not preserve generations for V2 PDSE data sets. Relative GDGs are now supported for sequential data sets.

Version 6.2.0 introduces support for individual members of partitioned data sets (both PDS and PDSE). The source member is specified with the standard member syntax as a suffix to the data set name, enclosed in parentheses. The target data set must be specified without a member name. If the source data set is a PDSE, the member and its aliases are copied. If the source data set is a PDS, only the member is copied.

See cozsftp(1) for the command reference.

Example: Transfer sequential data sets between Co:Z systems

Co:Z SFTP version: 5.5.0 (6.4p1) 2019-01-10
Copyright (C) Dovetailed Technologies, LLC. 2008-2019. All rights reserved.

cozsftp> dsput //COZ.TEST.PS //COZ.TEST.PS.COPY
ZosSettings[I]: Transfer options: *linerule=l4,*mode=binary[binary]
ZosDataset[I]: Opening dataset COZ.TEST.PS for read with options: shr
Uploading //COZ.TEST.PS to //COZ.TEST.PS.COPY
//COZ.TEST.PS                                               100% 1227KB   1.2MB/s   00:00    
ZosDataset[I]: Closing dataset //COZ.TEST.PS - 100 records read, 1256400 bytes sent

cozsftp> dsget -d //COZ.TEST.PS.COPY //COZ.TEST.PS 1
Fetching //COZ.TEST.PS.COPY to //COZ.TEST.PS
ZosSettings[I]: Transfer options: *blksize=25120,*dsorg=PS,*linerule=l4,*lrecl=12560,
                                  *mode=binary[binary],*recfm=FB,*replace=realloc[reuse],
                                  *space=cyl.2.2
ZosDataset[I]: Opening dataset COZ.TEST.PS for write with options: 
                                  blksize(25120) dsorg(PS) lrecl(12560) recfm(F,B) 
                                  cyl space(2,2) new catalog
//COZ.TEST.PS                                                100% 1227KB   1.2MB/s   00:00    
ZosDataset[I]: Closing dataset //COZ.TEST.PS - 1256400 bytes received, 100 records written

1

In this dsget command, the -d switch is supplied so that the local dataset will be deleted and reallocated (replace=realloc) with the attributes of the source data set on the remote system.

Example: Transfer an entire PDSE between Co:Z systems

Co:Z SFTP version: 6.0.0 (6.4p1) 2020-02-14
Copyright (C) Dovetailed Technologies, LLC. 2008-2020. All rights reserved.

cozsftp> dsput //COZ.LLIB //COZ.LLIB.COPY
Uploading //COZ.LLIB to //COZ.LLIB.COPY
ZosSettings[I]: Transfer options: gdgnt,*linerule=l4,*mode=binary[binary]
ZosDataset[I]: Opening dataset COZ.LLIB for read
ZosDataset[I]: Closing dataset //COZ.LLIB - 296 records read, 1145668 bytes sent

Example: Transfer an entire PDS between Co:Z systems

Co:Z SFTP version: 6.0.0 (6.4p1) 2020-02-14
Copyright (C) Dovetailed Technologies, LLC. 2008-2020. All rights reserved.

cozsftp> lzopts loglevel=D 1
 loglevel=D   
cozsftp> dsget //COZ.SAMPJCL //COZ.SAMPJCL.COPY
Fetching //COZ.SAMPJCL/ to //COZ.SAMPJCL.COPY
ZosSettings[I]: Transfer options: gdgnt,*linerule=l4,*mode=binary[binary]
ZosDataset[I]: Opening dataset COZ.SAMPJCL.COPY for write
ZosDataset[I]: Closing dataset //COZ.SAMPJCL.COPY - 86804 bytes received, 25 records written
Iebcopy[D]: 1                                         IEBCOPY MESSAGES AND CONTROL STATEMENTS                              PAGE     1
Iebcopy[D]: -IEB1135I IEBCOPY  FMID HDZ2230  SERVICE LEVEL UA92265  DATED 20170618 DFSMS 
Iebcopy[D]:  IEB1035I GOETZE    *OMVSEX  10:33:26 FRI 10 JUL 2020 PARM='WORK=2M'                                                     
Iebcopy[D]: -*OMVSEX  COPY      INDD=SYS00017,OUTDD=SYS00018     GENERATED STATEMENT                                                 
Iebcopy[D]: 0IEB1013I COPYING FROM PDSU  INDD=SYS00017 VOL=       DSN=SYS20192.T103325.RA000                        
Iebcopy[D]:  IEB1014I           TO PDS  OUTDD=SYS00018 VOL=VPWRKC DSN=COZ.SAMPJCL.COPY                                        
Iebcopy[D]:  IEB167I FOLLOWING MEMBER(S) LOADED FROM INPUT DATA SET REFERENCED BY SYS00017                                           
Iebcopy[D]:  IEB154I @@README HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I COZCFGD  HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I COZPROC  HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I DTLSPAWN HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I GPGDSN   HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I GREPDSN  HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I OFFLDSMF HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I RACDCERT HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I RUNLNCH  HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I RUNLNCHK HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I RUNLNCHP HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I RUNSFTP  HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I RUNSFTPK HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I RUNSFTPS HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I RUNSHELL HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I SFTPIND  HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I SFTPPROC HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I SFTPSAMP HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I SSHIND   HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I SSHPROC  HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I SSHSAMP  HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB154I WGET2DSN HAS BEEN SUCCESSFULLY LOADED                                                                           
Iebcopy[D]:  IEB1098I 22 OF 22 MEMBERS LOADED FROM INPUT DATA SET REFERENCED BY SYS00017                                             
Iebcopy[D]:  IEB144I THERE ARE 12 UNUSED TRACKS IN OUTPUT DATA SET REFERENCED BY SYS00018                                            
Iebcopy[D]:  IEB149I THERE ARE 8 UNUSED DIRECTORY BLOCKS IN OUTPUT DIRECTORY                                                         
Iebcopy[D]:  IEB147I END OF JOB - 0 WAS HIGHEST SEVERITY CODE                                                            
1

By setting the logging level to (D)ebug, the IEBCOPY messages associated with the transfer are emitted.

Example: Transfer a program object between Co:Z systems

Co:Z SFTP version: 6.2.0 (7.6p1) 2020-08-15
Copyright (C) Dovetailed Technologies, LLC. 2008-2020. All rights reserved.
cozsftp> lzopts loglevel=D 1
 loglevel=D   
cozsftp> dsget -t //COZ.LOADLIB(COZBATCH) //COZ.NEWLIB 2
Fetching //COZ.LOADLIB(COZBATCH) to //COZ.NEWLIB
ZosSettings[I]: Transfer options: gdgnt,*linerule=l4,*mode=binary[binary]
ZosDataset[I]: Opening dataset COZ.NEWLIB for write
ZosDataset[I]: Closing dataset //COZ.NEWLIB - 284883 bytes received, 78 records written
Iebcopy[D]: 1                                         IEBCOPY MESSAGES AND CONTROL STATEMENTS
Iebcopy[D]: -IEB1135I IEBCOPY  FMID HDZ2230  SERVICE LEVEL UA92265  DATED 20170618 DFSMS 
Iebcopy[D]:  IEB1035I GOETZE    *OMVSEX  10:47:43 FRI 10 JUL 2020 PARM='WORK=2M'                                                     
Iebcopy[D]: - COPYGROUP INDD=SYS00005,OUTDD=SYS00006                                                                                      
Iebcopy[D]: 0IEB1013I COPYING FROM PDSU  INDD=SYS00005 VOL=       DSN=SYS20192.T104742.RA000                        
Iebcopy[D]:  IEB1014I           TO PDSE OUTDD=SYS00006 VOL=VPWRKE DSN=COZ.NEWLIB                                              
Iebcopy[D]:  IGW01551I MEMBER COZBATCH HAS BEEN LOADED                                                                               
Iebcopy[D]:  IGW01550I 1 OF 1  MEMBERS WERE LOADED                                                                                   
Iebcopy[D]:  IEB147I END OF JOB - 0 WAS HIGHEST SEVERITY CODE                                                                   
1

By setting the logging level to (D)ebug, the IEBCOPY messages associated with the transfer are emitted. In these messages, it can be seen that only the selected member was loaded into the new data set.

2

In this dsget command, the -t switch is supplied so that the unused space in the new data set (which was allocated with the values of the source data set) will be released.

Listing datasets and PDS directories

MVS datasets can be listed using the sftp lls command. Partitioned datasets are treated as directories with their members as entries.

When listing z/OS datasets locally with the lls command, catalog search filter keys are in effect for any wildcard requests. The catalog search wildcards *, **, and % used in the examples below are described in the IBM manual DFSMS: Managing Catalogs - SC26-7409. Note that this is different behavior from sftp clients that connect to the Co:Z sftp-server and list datasets with the ls. In that case, regular file globbing rules are in effect.

Example: Listing datasets

cozsftp> lcd //USER
cozsftp> lls -al     1
Volume  Referred  Ext  Tracks    Used Recfm Lrecl BlkSz Dsorg  Dsname
WORK84 2008/09/05   1       1       1  FB      80 27920  PS    USER.AFILE.TXT                              
WORK81 2008/09/08   1      30       ?  U        0  6144  PO-E  USER.COZ.LOADLIB                            
WORK81 2008/09/11   1      15       4  FB      80 27920  PO    USER.COZ.SAMPJCL                            
WORK84 2008/09/11   1       1       1  U        0  6144  PS    USER.COZ.TEST.SEQ                           
WORK81 2008/09/09   1      15       3  FB      80 27920  PO    USER.COZ.TESTJCL

cozsftp> lls -al //user.coz.t*     2
Volume  Referred  Ext  Tracks    Used Recfm Lrecl BlkSz Dsorg  Dsname
WORK81 2008/10/20   1      15       4  FB      80 27920  PO    USER.COZ.TESTJCL 

cozsftp> lls -al //user.c*.**     3
Volume  Referred  Ext  Tracks    Used Recfm Lrecl BlkSz Dsorg  Dsname
WORK81 2008/10/20   1      30       ?  U        0  6144  PO-E  USER.COZ.LOADLIB                            
WORK81 2008/10/20   1      15       4  FB      80 27920  PO    USER.COZ.SAMPJCL                            
WORK84 2008/09/25   1       1       1  U        0  6144  PS    USER.COZ.TEST.SEQ                           
WORK81 2008/10/20   1      15       4  FB      80 27920  PO    USER.COZ.TESTJCL
1

The long form of the list command ls -al will list detailed information from the catalog about each dataset.

2

Using the catalog search filter key syntax, a single asterisk can be used to as a wildcard for a single dataset level.

3

Using the catalog search filter key syntax, a double asterisk can be used to perform a deep listing. In this example, the single and double asterisk syntax is combined to list all of the datasets beginning with the prefix USER.C.

Example: Listing a PDS directory

...
cozsftp> lcd //user.coz.sampjcl     1
cozsftp> lls -al     2
Name           Size  Created         Changed          ID
@@README 
BPXBATCH         13 2008/04/04 2008/04/04 17:18:09  USER   
BPXBATSL         16 2008/04/03 2008/04/03 10:36:52  USER   
COZCFGD          65 2008/03/27 2008/05/12 14:28:54  USER   
COZPROC          30 2008/03/27 2008/03/27 11:54:48  USER   
DTLSPAWN         40 2008/05/05 2008/05/05 09:31:08  USER   
GPGDSN           15 2008/05/05 2008/05/05 10:40:05  USER   
GREPDSN  
GREPSED          12 2008/05/05 2008/05/05 09:30:51  USER   
OFFLDSMF 
RUNCOZ           20 2008/03/27 2008/05/12 14:08:02  USER   
RUNCOZ2          15 2008/05/05 2008/05/05 10:02:51  USER   
RUNCOZ3           8 2008/05/05 2008/05/06 08:50:37  USER   
RUNSPAWN         54 2008/05/12 2008/05/12 14:25:37  USER   
RUNSPWN2         20 2008/05/12 2008/05/12 13:19:05  USER   
TDIRK            18 2008/04/03 2008/04/03 10:19:20  USER   
WGET2DSN
1

The lcd command is used to make a PDS the current working local "directory".

2

The lls -al command (long list form) is used to display the members of the PDS, including available statistics.

4.6 Working with POSIX files

This section describes how to use the enhanced client with POSIX files (HFS, zFS) on z/OS. Standard sftp implementations (including z/OS OpenSSH) support only binary mode file transfers. The Co:Z implementation provides binary transfer mode by default, but also supports text mode transfers. Text mode transfers are controlled via the following options:

  • mode: when set to text causes file transfers to be text based.

  • clientcp and servercp: When text mode is active, these settings determine the codepage translation that will take place. The default client code page is ISO8859-1. The default server code page is the current z/OS locale.

  • linerule: When text mode is active, this setting determines how line separators are converted between the client and server.

Transferring Files

The get and put commands are used to transfer POSIX files (either on HFS or zFS filesystems).

The options (listed above) that have been previously set via the lzopts are in effect for any given transfer. All other options (used for dataset support) are ignored for POSIX file transfers.

Example: Get a text POSIX file

$ cozsftp user@linux.com     1
Connecting to linux.com...
user@linux.com's password: *****
cozsftp> lzopts mode=text,servercp=UTF-8     2
 mode=text         servercp=UTF-8  
cozsftp> lzopts     3
clientcp=IBM-1047    loglevel=I           mode=text            
servercp=UTF-8
cozsftp> pwd
Remote working directory: /tmp
cozsftp> get msgs.txt     4
Fetching /tmp/msgs.txt to msgs.txt
/tmp/msgs.txt                                        100%   19KB  19.0KB/s   00:00    
ZosPosixFile[I]: Closing file msgs.txt - 19488 bytes received, 19488 bytes written5    
cozsftp>
1

This example shows the full connection process, using keyboard-interactive password authentication to a remote linux system.

2

The default transfer mode of binary is overridden and set to text. Additionally, the server (linux) code page is explicitly set to UTF-8.

3

Displays the active options. Note that the client code page, if not explicitly set, defaults to the current z/OS locale.

4

The get command requests the transfer of the POSIX file using the options in effect.

5

Upon completion, an informational message is written that describes the number of bytes received from the server and the number of bytes written to the local file. These counts are commonly the same, but changes in line separators and codepages can result in different counts.

Example: Put a text POSIX file

cozsftp> put sftp-server.log /tmp     1
Uploading sftp-server.log to /tmp/sftp-server.log
sftp-server.log                                  100%  127     0.1KB/s   00:00    
ZosPosixFile[I]: Closing file sftp-server.log - 127 bytes read, 127 bytes sent
ZosSmf119Record[I]: SMF Type119 recording not enabled; SMF recording disabled
1

The client text file sftp-server.log is put to the remote directory /tmp. The active file transfer options are used.

4.7 Using the Co:Z SFTP client in batch

The cozsftp client command can be conveniently used in a z/OS batch job without user interaction. The COZBATCH batch utility, also installed as part of the Co:Z toolkit, makes it easy to run cozsftp (or other Unix shell scripts) directly as z/OS batch jobs.

The authentication with the remote system must be set up so as not to require any user interaction. There are three ways to do this with OpenSSH:

  • Use the SSH_ASKPASS environment variable to point to a program that will read a password.

  • Use an OpenSSH public/private keypair.

  • Use a RACF Digital Certificate.

For details on these three authentication options, see Appendix F, Client Authentication Mechanisms. Note that instructions in this appendix must be followed in order to run the examples described below.

Notes for running batch mode SFTP

When sftp is run in batch mode, it is important to know that sftp will abort if any of the supplied commands fail (i.e. complete with a non-zero return code). This behavior is different from an interactive sftp session, where a failed command will report an error, but the session will continue. In cases where a failed command is expected or acceptable (e.g. rm old_file, where old_file may not exist) it is useful to direct batch mode sftp to continue processing. To do this, prefix the command with a dash (-):

-rm old_file

Sample SFTPPROC and batch scripts

A sample SFTPPROC and batch scripts (installed in <COZ_HOME>/samples/sftp_batch) are distributed with the Co:Z toolkit to simplify maintenance and support of batch jobs using the Co:Z SFTP client. Using these samples achieves the following:

  • COZBATCH customized for running CO:Z SFTP

  • installation default options separated from individual JCL members

  • standards defined for a set of variables controlling connection, authentication, options and filenames

  • unix shell script logic separated into separate reusable script files

[Caution]NOTE:
The scripts below are designed to be "dotted in" to the COZBATCH job step STDIN DD. As such, only a single script invocation is supported per job step.

The reusable script files that are distributed with the Co:Z toolkit are the following:

  • sftp_connect.sh - Connect to a remote system using the Co:Z toolkit cozsftp command. This script connects a cozsftp client to a remote system running sshd and prepares it to accept batch subcommands. This is the script that we recommend that most customers use.

  • sftp_get.sh, sftp_put.sh - Get or put a single file to/from a remote system to/from a local (z/OS) file using the cozsftp command. cozsftp transfer options (lzopts) can be specified to customize the transfer. Please note: Due to the lack of flexibility in these two scripts, we recommend that most customers use the more flexible sftp_connect.sh script above.

  • sftp_cat.sh - Get multiple files from a remote system and concatenate them to a local (z/OS) file using the cozsftp command. This script connects a cozsftp client to a remote system running sshd and issues an ls command to get a list of files to get. Each of these files is then retrieved and written to the specified local file on z/OS. cozsftp transfer options (lzopts) can be specified to customize the transfer.

The standard set of variables to control connection, authentication, options and filenames are defined in the table below. Variables used by all scripts are required for sftp_connect.sh. sftp_cat.sh, sftp_get.sh, and sftp_put.sh invoke sftp_connect.sh to establish a connection with the remote host. Some variables are used only for specific scripts as noted in the Script column.

Table 4.1. Script Variables

VariableScriptRequiredDescription
userallrequiredSet to the remote userid
hostallrequiredSet to the remote host
portalloptionalSet to the sshd port on the remote host. Port 22 is used by default
pwdsnalloptionalSet to a fully qualified dataset name (or fully qualified dataset member) containing the user's remote system password. If so, SSH_ASKPASS authentication will be used.

Note: If neither pwdsn or cert is set, z/OS OpenSSH defaults will be used for public/private key authentication.

certalloptionalSet to the name of a SAF digital certificate using one of the following formats:

RING_NAME (no whitespace allowed; the default label will be used)

RING_NAME:LABEL_NAME (no whitespace allowed)

"RING_NAME LABEL_NAME" (whitespace between ring and label)

If the first or second format is used, the connection will be authenticated using the Co:Z Toolkit saf-ssh-agent, which is the recommended approach as hardware private keys are supported. If the third format is used, the z/OS OpenSSH identityKeyRingLabel option will be used for authentication.

Note: If neither pwdsn or cert is set, z/OS OpenSSH defaults will be used for public/private key authentication.

sftp_optsalloptionalSet to any desired SFTP options, including any ssh specific options (designated via the -o switch). Set using contatenation unless the prior variable setting is being overwritten.
cozbin_diralloptionalSet to the absolute path of the installed Co:Z Toolkit "bin" directory. If not set, this directory must be present in the current PATH environment variable.
script_diralloptionalSet to the absolute path of the directory containing the sftp_batch scripts (including this file). All scripts are assumed to be in this directory. If not set, this directory must be present in the current PATH environment variable.
lfilesftp_get.sh and sftp_put.shrequiredSet to the local file to be created or transferred.
rfilesftp_get.sh and sftp_put.shrequiredSet to the remote file to get or put.
lzoptssftp_get.sh, sftp_put.sh, and sftp_cat.shoptionalSet to cozsftp transfer options required for the transfer (e.g. mode=text,replace=no)
ldsnsftp_cat.shrequiredSet to the local dataset or DD to be written
rpatsftp_cat.shrequiredSet to the remote file pattern to get

Using the sample SFTPPROC and scripts, the following batch job conditionally creates a directory on the remote system and transfers a data set to it:

//PROCLIB JCLLIB ORDER='COZUSER.COZ.SAMPJCL'
//* 
//*********************************************************************
//* Use the sftp_connect.sh script to connect to a remote system
//* and send sftp commands.  This example uses a ssh key stored in a
//* SAF digital certificate
//*********************************************************************
//SFTPCONN EXEC PROC=SFTPPROC 
//SFTPIN DD *   1
cert="MY-RING:RSA-CERT" 
user=myuser
host=myhost 
                                                            
. $script_dir/sftp_connect.sh <<EOB 2
-mkdir outbound 3
lzopts mode=text
put //COZUSER.MYDATA outbound/mydata.txt
EOB 
//*  
1

All of the input to SFTPIN is processed by the z/OS UNIX shell, and therefore normal shell script quoting and interpolation rules apply.

2

The symbols "<< EOB" followed by an ending EOB define a HERE document which, in this example, is an inline string containing SFTP commands. Normal shell script 'here'-document substitutions will be made.

3

If an SFTP command fails in batch mode, the script will exit at the point of failure with a non-zero return code. In this case, if the target of the mkdir command already exists, the command will fail. Prefixing the command with a dash "-" will cause the return code to be ignored for this command, allowing the script to continue.

If you only need to download a remote file, you can use the somewhat simpler sftp_get.sh script:

//PROCLIB JCLLIB ORDER='COZUSER.COZ.SAMPJCL'
//* 
//*********************************************************************
//* Use the sftp_get.sh script to retrieve a remote file to a local
//* data set.  This example uses a password (via the SSH_ASKPASS 
//* protocol) to connect to the remote system
//*********************************************************************
//SFTPGET EXEC PROC=SFTPPROC 
//SFTPIN DD * 
pwdsn="COZUSER.COZ.SAMPJCL(PW)" 
user=myuser
host=myhost 
lzopts="mode=text" 
lfile=//DD:MYDD 
rfile=/etc/profile 
                                                            
. $script_dir/sftp_get.sh 

//MYDD  DD DSN=COZUSER.SFTPGET.DATA,DISP=(MOD,KEEP), 
//        DCB=(LRECL=80,RECFM=FB),SPACE=(CYL,(3,1)) 
//* 

The sections below describe the SFTPPROC, installation default options, and a few more examples.

PROC for executing the Co:Z SFTP client (cozsftp) in batch

The SFTPPROC sample JCL distributed with the Co:Z toolkit can be used as a tailorable model for customizing COZBATCH for using CO:Z SFTP.

//*********************************************************************
//* 
//* PROC for executing the Co:Z SFTP client (cozsftp) in batch 
//* 
//* Tailor the proc for your installation: 
//* 1.) Tailor LIBRARY with the PDSE that contains the 
//*     COZ load module. 
//* 2.) Tailor SFTPIND= to point to SAMPJCL member that contains
//*     site specific shell variable settings for running the
//*     Co:Z SFTP batch scripts
//* 3.) Review the Co:Z SFTP batch scripts (located in
//*     $COZHOME/samples/sftp_batch) for additional shell variables
//*     to set for individual jobs to get, put, connect, etc...
//*********************************************************************
//EXSFTP    PROC ARGS=,                      < [-L<log_opt>] 1
//   LIBRARY='COZUSER.COZ.LOADLIB',          < STEPLIB FOR COZBATCH
//   SFTPIND='COZUSER.COZ.SAMPJCL(SFTPIND)', < Installation defaults 2
//   REGSIZE='64M',                          < Execution region size
//   LEPARM=''
//RUNSFTP  EXEC PGM=COZBATCH,REGION=&REGSIZE, 3
//   PARM='&LEPARM/&ARGS'
//STEPLIB  DD DSN=&LIBRARY,DISP=SHR
//STDIN    DD DSN=&SFTPIND,DISP=SHR	4	
//         DD DDNAME=SFTPIN
//SFTPIN   DD DUMMY                          < Customized stdin to SFTP 5
//*
// PEND
  
1

COZBATCH logging may be added to ARGS for problem diagnosis.

2

Defines the member that contains the installation CO:Z SFTP defaults. These defaults can be overriden in individual jobs as necessary.

3

Defines the program to execute as COZBATCH, a utility similar to IBM's BPXBATCH. COZBATCH runs a Unix login shell in the original address space.

4

Ensures that the site specific installation defaults are included first in STDIN, before any job specific commands.

5

Defines a name for STDIN allowing jobs using this proc to include commands in STDIN.

Co:Z SFTP Batch Script Settings

The SFTPIND sample JCL member distributed with the Co:Z toolkit can be used as a tailorable model for CO:Z SFTP installation defaults.

###############################################################################
# Co:Z SFTP Batch Script Settings
# The shell variables below can be set to site specific values, but may
# be overridden in individual jobs.
###############################################################################

#
# CONFIGURATION VARIABLES:
#
# cozbin_dir - May be set to the absolute path of the installed Co:Z Toolkit
#              "bin" directory.  If not set, this directory must be present in
#              the current user's PATH environment variable.
# script_dir - May be set to the absolute path of the directory containing the
#              sftp_batch scripts (including this file).  All scripts are 
#              assumed to be in this directory.  If not set, this directory must
#              be present in the current user's PATH environment variable. 
#
cozbin_dir="/usr/local/coz/bin"   1
script_dir="/usr/local/coz/samples/sftp_batch" 2

#
# SFTP OPTIONS VARIABLE:
#
# sftp_opts  - May be set to any site specific SFTP options, including any ssh
#              options (designated via the -o switch).
#
sftp_opts=""
sftp_opts="$sftp_opts -oConnectTimeout=60" 
sftp_opts="$sftp_opts -oServerAliveInterval=60" 3
#
# Set the following option to "no" if you would like to 
# automatically accept host keys for new servers. 
sftp_opts="$sftp_opts -oStrictHostKeyChecking=yes"
  
1

Defines a variable for the location of the cozsftp executable. This variable is used by the sftp batch scripts to execute the cozsftp command.

2

Defines a variable for the location of the sample or customized version of the sftp batch scripts. This variable is used in all jobs executing the sftp batch scripts.

3

Sets global installation options as necessary. Note that the sftp_opts variable is appended as each option is added. Jobs using SFTPPROC can reset or append to these options using this variable.

Logging in batch

Co:Z SFTP Client logging is described earlier in this chapter (Section 4.2, “Co:Z SFTP client logging”). This section describes how to set these options in batch.

OpenSSH SFTP logging is enabled by passing the -v option on the sftp command. This can be done in batch by appending the desired -v option to the sftp_opts variable in the STDIN section of the job:

//SFTPIN DD *
sftp_opts="$sftp_opts -vvv"

Co:Z SFTP client logging is enabled by exporting the COZ_LOG variable in the STDIN section of the job.

//SFTPIN DD * 
export COZ_LOG=T 

COZBATCH logging is enabled by adding a parameter to the EXEC statement for the SFTPRPOC. The following example uses the -LD command switch to set the default logging level to "Debug" for COZBATCH. The t option is also used to prefix all messages with a timestamp.

//SFTPGET EXEC PROC=SFTPPROC,
        ARGS='-LD,t'
//SFTPIN DD *

Finally, it is often very helpful to see details about the Unix System Service shell script processing of the input to COZBATCH. This can be controlled by explicitly requesting a login shell (/bin/sh -L) along with the trace option (-x). The following example sets both the COZBATCH logging level to "Debug" and these shell options:

//SFTPGET EXEC PROC=SFTPPROC,
        ARGS='-LD,t /bin/sh -L -x'
//SFTPIN DD *

Batch job containing examples of running cozsftp in batch

The SFTPSAMP sample JCL distributed with the Co:Z toolkit can be used as a tailorable model for writing batch jobs using CO:Z SFTP.

//SFTPSAMP   JOB (),'DOVETAIL',MSGCLASS=H,NOTIFY=&SYSUID 
//PROCLIB JCLLIB ORDER='COZUSER.COZ.SAMPJCL'
//* 
//*********************************************************************
//* 
//* Batch job containing examples of running cozsftp in batch
//* 
//* Tailor the proc and job for your installation: 
//* 1.) Modify the Job card per your installation's requirements 
//* 2.) Modify the PROCLIB card to point to this PDS, or wherever 
//*     the SFTPPROC procedure has been installed. 
//* 
//*********************************************************************
//* 
//*********************************************************************
//* Use the sftp_connect.sh script to connect to a remote system
//* and send sftp commands.  This example uses a ssh key stored in a
//* SAF digital certificate
//*********************************************************************
//SFTPCONN EXEC PROC=SFTPPROC 
//SFTPIN DD * 
cert="MY-RING:RSA-CERT" 
user=myuser 
host=myhost 
                                                          
. $script_dir/sftp_connect.sh << EOB
lzopts mode=text
-mkdir incoming   # use the "-" prefix in case directory already exists
put mydata.txt incoming 
EOB

//* 
//*********************************************************************
//* Use the sftp_get.sh script to retrieve a remote file to a local
//* dataset.  This example uses a user ssh key stored in a SAF
//* digital certificate
//*********************************************************************
//SFTPGET EXEC PROC=SFTPPROC 1
//SFTPIN DD * 2  
cert="MY-RING:RSA-CERT" 3 
user=myuser
host=myhost 
lzopts="mode=text" 
lfile=//DD:MYDD 
rfile=/etc/profile 
                                                            
. $script_dir/sftp_get.sh 4 

//MYDD  DD DSN=COZUSER.SFTPGET.DATA,DISP=(MOD,KEEP), 
//        DCB=(LRECL=80,RECFM=FB),SPACE=(CYL,(3,1)) 
//* 
//*********************************************************************
//* Use the sftp_put.sh script to send a local file to a remote
//* file.  This example uses a password (via the SSH_ASKPASS protocol)
//* to connect to the remote system
//*********************************************************************
//SFTPPUT EXEC PROC=SFTPPROC 
//SFTPIN DD * 
pwdsn="COZUSER.COZ.SAMPJCL(PW)" 
user=myuser
host=myhost 
lzopts="mode=text" 
lfile=/etc/profile 
rfile=/home/myuser/zprofile.txt 

# Don't try to use our public key even if we have a default one
# This would not normally be a required setting
sftp_opts="$sftp_opts -oPubkeyAuthentication=no"
                                                            
. $script_dir/sftp_put.sh

//* 
//*********************************************************************
//* Use the sftp_cat.sh script to retrieve multiple files from a remote
//* system and concatenate them to a local dataset.  This example uses
//* z/OS OpenSSH defaults for public/private key authentication
//* (because neither the "cert" or "pwdsn" variables are defined)
//*********************************************************************
//SFTPCAT EXEC PROC=SFTPPROC 
//SFTPIN DD * 
user=myuser
host=myhost 
lzopts="mode=text" 
ldsn="//DD:MYDD" 
rpat=/home/myuser/doc/*.txt 
                                                          
. $script_dir/sftp_cat.sh 

/* 
//MYDD  DD DSN=COZUSER.SFTPCAT.DATA,DISP=(MOD,KEEP), 
//        DCB=(LRECL=80,RECFM=FB),SPACE=(CYL,(3,1)) 

// 
  
  
1

Each step in this sample job uses the SFTPPROC.

2

Each step uses SFTPIN defined by the SFTPPROC.

3

Sets all variables required for sftp_get.sh. Note that some required variables are in the installation global defaults. Global variables may be optionally overriden here before calling the shell script.

4

Executes sftp_get.sh to get a file from the remote system saving it in the data set defined by //MYDD.

Wild-card downloading using a DD

In release 2.4.0, support was added to allow multiple files to be downloaded using a wild-card pattern (*) to a single DD if the DD was allocated with DISP=MOD.

//SFTPGET EXEC PROC=SFTPPROC
//SFTPIN DD *
user=myuser
host=myhost
pwdsn="COZUSER.COZ.SAMPJCL(PW)"
lzopts="mode=text"
lfile=//DD:MYDD       1
rfile=/somedir/*.trn  2
. $script_dir/sftp_get.sh  3

//MYDD DD DSN=COZUSER.SFTP.MULTIGET.DATA,DISP=(MOD,CATLG,DELETE), 1
//        DCB=(LRECL=2052,RECFM=FB),SPACE=(CYL,(3,1))
1

The lfile variable references a DD in the job step that is allocated with DISP=MOD. This allows multiple files to be downloaded, in succession, to the end of the same target dataset. Each matching file will be downloaded separately, in alphabetical order.

2

The get command uses a wild-card (*) pattern to select any file in the /somedir directory that ends in ".trn".

3

The underlying get subcommand generated by the sftp_get.sh script will be:
get /somedir/*.trn //DD:MYDD

Conditional command processing

In release 7.0.0, support was added to allow basic if/then/else command processing in the Co:Z SFTP client. This section shows usage examples, but additional information can be found in the cozsftp command reference.

In the following example, a remote file remote.data will be downloaded only if it exists. The job step will get a return code 0 if either the remote file doesn't exist, or if it does and it is downloaded successfully.

//SFTPCND1 EXEC PROC=SFTPPROC
//SFTPIN DD *
user=myuser
host=myhost
. $script_dir/sftp_connect.sh <<EOD
lzopts mode=text,recfm=vb,lrecl=2048,space=cyl.1.1
%if
cd /remote/dir
ls remote.data
%then
get  remote.data //HLQ.MY.LOCAL
%fi
EOD
//

In this next example, a file is uploaded in an %if block but if that fails an %else block is used to attempt to put it to an alternate directory.

//SFTPCND2 EXEC PROC=SFTPPROC
//SFTPIN DD *
user=myuser
host=myhost
. $script_dir/sftp_connect.sh <<EOD
lzopts mode=text
%if
cd /remote/dir
put //HLQ.MY.LOCAL  remote.data
%else
cd /remote/alt/dir
put //HLQ.MY.LOCAL  remote.data
%fi
EOD
//

In the following example, several put commands are executed in an %if block. The block will terminate if any of these should fail, but the get command following the block will still be attempted. The exit code will be zero even if a command in the %if block fails.

//SFTPCND3 EXEC PROC=SFTPPROC
//SFTPIN DD *
user=myuser
host=myhost
. $script_dir/sftp_connect.sh <<EOD
lzopts mode=text
cd /remote/dir
%if
put //HLQ.MY.LOCAL1  remote1.data
put //HLQ.MY.LOCAL2  remote2.data
put //HLQ.MY.LOCAL3  remote3.data
%fi
lzopts recfm=fb,lrecl=80,space=trk.3.3
get remote4.data //HLQ.MY.LOCAL4
EOD
//

Saint Charles, Missouri
info@coztoolkit.com
+1 636.300.0901

Copyright© 2009 - 2023 Dovetailed Technologies, LLC. All rights reserved. Co:Z® is a registered trademark and Co:Z Toolkit™ is a trademark of Dovetailed Technologies, LLC.

Saint Charles, Missouri
info@coztoolkit.com
+1 636.300.0901

Copyright© 2009 - 2023 Dovetailed Technologies, LLC. All rights reserved. Co:Z® is a registered trademark and Co:Z Toolkit™ is a trademark of Dovetailed Technologies, LLC.