Overview

This section discusses some of the miscellaneous advanced features of the UPSTREAM program that have not been covered in previous chapters:

• UPSTREAM returns program return codes that can be used to determine the success or failure of UPSTREAM functions. See Section “Program Return Codes”.
  
• Remote UPSTREAM systems can run programs and batch jobs on your workstation/server and you can request that programs and batch jobs be run on the UPSTREAM Storage Server. See Section “Job Execution”.
  
• UPSTREAM functions can be specified remotely either from UPSTREAM Storage Server batch jobs or from a UPSTREAM Client. This section describes how to initiate these functions from a UPSTREAM Client. See Section “Requesting a Remote UPSTREAM Function”“Requesting a Remote UPSTREAM Function”.
  
• UPSTREAM has its own internal ASCII-to-EBCDIC translation table that it uses for communications parameters and file name specifications. Non-U.S. users can modify this table to support other single-byte character sets. See Section “User Defined Translation Tables”“User Defined Translation Tables”.
  
• Sequential disk backups require that the size of the z⁄OS data set be known in advance. Normally UPSTREAM calculates the value close enough to work. There are additional tuning parameters that allow you to ensure that the data set is allocated to the correct size. See Section “Sequential Disk Size Allocation”.
  
• If you have a number of UPSTREAM parameter files, and you wish to change one or more parameters at a time in a number of files, a program, usmodify is provided to allow you to do this easily.
  
• UPSTREAM allows you to set its parameters in other ways than through the dialog panels. Use of these parameters allows you to tailor your environment to best suit your needs. This section describes how to do this and lists these parameters. It then goes on to describe how to make it easy for untrained users to take advantage of UPSTREAM.
  

Program Return Codes

UPSTREAM returns program return codes to indicate success or failure of particular UPSTREAM functions. Program return codes can be read from batch files using the ERRORLEVEL command or as return codes from calling programs.

UPSTREAM program return codes are coded as a bit map. The values are added together to produce the total returned value.

This list shows the return code values.

• 0 - No errors.
  
• 1 - Backup failed.
  
• 2 - Restore failed.
  
• 4 - A signal (usually a program crash) occurred.
  
• 8 - Remote requested received failed.
  
• 16 - Remote requested failed
  
• 32 - Running the requested job failed.
  
• 64 - Running the requested host report failed.
  
• 128 - Internal function (for example, version deletion) failed.
  

For example, if a remote request and a backup both failed, you would get a return code of 9 (1 + 8).

A file is created in the WORKPATH or default directory, US.RET, which holds a more comprehensive return code that adds in the values above, plus values indicating which functions were performed. This list shows the added values:

• 0 - No functions performed
  
• 256 - Backup completed
  
• 512 - Restore completed
  
• 2048 - Remote request received
  
• 4096 - Remove request performed
  
• 8192 - Requested job started successfully
  
• 16384 - Requested host report run successfully
  
• 32768 - Internal function (for example version deletion) ran successfully
  

RETCODE.EXE

The program, RETCODE.EXE allows you to read the US.RET file and recover the original program return code to a fresh program return code (if it was lost due to running another program).

It also reads interprets the value of US.RET and displays the meanings on the screen. It can be quicker to use this program to determine if UPSTREAM ran successfully than checking the UPSTREAM log.

Job Execution

UPSTREAM includes a feature that allows remote computers (UPSTREAM Storage Server and other workstation/servers) to run a batch file or program on your workstation/server or for you to submit existing z⁄OS jobs. The workstation/server aspect of this facility is particularly useful, and is used, in the UPSTREAM Auto-Update facility (see the Chapter 14 “Management and Reporting” for a detailed description of that facility).

Locally Specifying

UPSTREAM includes a dialog that allows you to specify the parameters necessary to run a job. Use this dialog to build a parameter file for the workstation/server or z⁄OS system job automated execution.

In the UPSTREAM Director, press the Run Job tab to request a job.

Job Specifications

Run Command:

Enter the command line of the program or batch file that you wish to run. You must use the fully qualified path to the program or batch file to run, followed by any command line parameters that you wish to use. This field is usually required and the default is blank. This is the only required parameter for submitting z⁄OS system jobs.

If you do not want to specify the full path for the program/batch file, start the command line with the back quote character “`”. In this case all the rules of the operating system or shell running your command apply (default file extensions, PATH etc.). Otherwise, if you do not specify the leading back quote “`”, UPSTREAM verifies the existence of the program to execute and does some other processing on the spec.

(Win 32) If the program or batch file specification includes long file names with spaces you must use the following syntax:

`”fully qualified file name” [parameter1][parameter2] […]

(Back quote, double quote, fully qualified file name, double quote, space, and then parameters if needed.)

It is strongly recommended to specify the program/batch file with the appropriate extension. [Win32] if you do not specify the extension UPSTREAM adds “.exe” to your spec.

(Win 32) If a parameter has spaces within it and you are running a batch/command file (not a 32-bit program), you need to specify double-quotes around the entire list of parameters and then double quotes around each parameter. For example:

C:\winnt\system32\cmd.exe /c “”fully qualified batch file name” “first parameter” [”second one”] […]”

The run job check-boxes are:

Wait for Job To Complete

If checked this causes UPSTREAM to wait in a suspended state until the job has completed. Otherwise, UPSTREAM starts the job and continues. The default is not checked.

Wait Time Limit

Use this field to specify the number of milliseconds that UPSTREAM should wait for the job to complete processing. If the wait time limit expires before the job completes, UPSTREAM either forcefully terminates the job if the Kill Job After Wait Time Limit check-box is checked or leave the job running in an orphaned state. A value of 0 means that UPSTREAM waits indefinitely for the job to complete.

Send job output (stdout, stderr) to UPSTREAM log

If checked, the standard output (STDOUT) and standard error output (STDERR) from the job is written to the UPSTREAM.log file, but not displayed. If this check-box is checked, the Wait for Job To Complete check-box is also checked. This option is recommended as it allows you to capture the output from your jobs. Note that stderr or stdout lines should be less than 1000 bytes each.

Kill job when wait time expires - Issue message

If checked, the job is forcefully terminated after waiting the number of milliseconds specified by the Wait Time Limit value or if UPSTREAM itself is terminated. If this check-box is checked, the Wait for Job To Complete check-box is be checked.

Return Code Map

This is used only for CONV=WAIT or CONV=KEEP UPSTREAM Storage Server jobs running on a workstation/server. This parameter allows mapping the job return code to a return code consistent with the way that z⁄OS return codes work. The syntax is described below. The default is 0:0 ?:8; that maps a workstation/server 0 return code to a UPSTREAM Storage Server 0 return code and all other workstation/server return codes to a UPSTREAM Storage Server return code of 8. This field is not used for z⁄OS system jobs. You can specify ?:? to have unspecified return codes returned to the caller uninterpreted.

Job Return Codes

The UPSTREAM Storage Server CONV=WAIT batch jobs return a return code indicating whether the requested function has successfully completed. For backups or restores, this return code indicates whether the backup or restore worked successfully (return code 0), had some warnings (return code 4) and so on. These return codes are used for host reporting and are often used with a scheduling system to determine whether to perform additional functions or notify an administrator.

This process becomes more complex when you are running a workstation/server job. Is the job successful if it merely runs? What constitutes working?

This problem is exacerbated by the differences in PC and UNIX system return codes and mainframe return codes. On the mainframe, there is a convention of how return codes work (0 = success, 4 = minor problems, 8 = failure, …). If a workstation/server program returns a return code at all, it often is simply 0 for successes and non-zero for failure.

To help map workstation/server return codes to UPSTREAM Storage Server return codes, a mapping facility is provided (Return Code Map on the workstation/server dialog or the parameter JOBRETURNCODEMAP). The syntax is:

<workstation/server return code>:<UPSTREAM Storage Server return code> …Where:

<workstation/server return code>

is the ERRORLEVEL or program return code from the application or batch file. You can specify:

•  A single numeric value: 0, 1, etc.
  
•  A range of values: 1-4, 8-12, etc.
  
• A question mark (?): This indicates any value that is not directly specified as a single numeric value a value in a range.
  

<UPSTREAM Storage Server return code>

is the value that the UPSTREAM Storage Server job returns.

The combination of <workstation/server return code>:<UPSTREAM Storage Server return code> can be repeated multiple times, separated by spaces or tabs.

For example, the default of 0:0 ?:8 causes the UPSTREAM Storage Server batch job to return:

0

if the workstation/server program returns a program return code of 0.

8

if the workstation/server program returns a program return code which is non-zero.

12 or 16

depending upon the type of failure if the workstation/server does not run the job at all (due to some type of communications or other failure).

(UNIX) If there are errors in starting a remotely requested job, these errors are stored in the file usjob.out in the WORKPATH directory.

UPSTREAM Storage Server Initiation

UPSTREAM Storage Server initiation of jobs is via UPSTREAM parameters. The important ones are:

ACTION 5

Specifies that a job is to be run.

JOBOPTIONS

Usually specified as 35 to have UPSTREAM run the job, wait for it to complete, log the output, and return the return code to the UPSTREAM Storage Server. Allowing it to default to 0 causes UPSTREAM to terminate.

JOBRETURNCODEMAP

See “Job Return Codes” in Section.

FILES

The full command line for the job that you wish to execute.

Killing Jobs

If you accidentally specify a job that requires user input or your job hangs or ends up in an infinite loop, both UPSTREAM and the job has to be killed manually. This is done differently depending upon your operating system; some of the less complex ones:

• (UNIX) Use the ps command to get the process number and the kill command to kill the process.
  
• (Windows) The Task Manager allows you to kill programs, unless it was started from UPSTREAM running as a service.
  

For Windows, if UPSTREAM is running as a service, you can use USTPCFG to create a service that starts the task manager program (usually c:\winnt\system32\taskmgr.exe). This program can then terminate jobs that are running as services.

Requesting a Remote UPSTREAM Function

If you wish to route via the UPSTREAM Storage Server (to avoid issues with firewalls, because the request is initiated from the UPSTREAM Storage Server using the same method as USTBATCH) using the UPSTREAM Director, select the target to send the request to, press the Option button in the Backup or Restore window and press the Route Via Host tab:

Route via Host

The primary check-box is:

Route action through host repository to target

If you are using the UPSTREAM Director, you must check this check-box to have the request routed through the UPSTREAM Storage Server. For UPSTREAM Reservoir requests, the options below are ignored.

Through z⁄OS options:

APPLPREF

 Specifies the 5 character prefix USTBATCH uses to build its own VTAM APPLID value. It appends a character numeric value beginning at “001” to the specified APPLPREF value and use the resulting name to open a VTAM ACB with this name as the APPLID, incrementing the number until it finds a free VTAM application ID. The default is UPSTR.

USAPPL

(z/OS Storage Server ony)Specifies the VTAM APPLID of the online z⁄OS Storage Server task. This value is used to allocate an LU 6.2 session to the running online UPSTREAM main task to request the remote initiation. It must match the value that is specified for the APPLID= in the z⁄OS Storage Server configuration. The default is UPSTREAM.

TPNAME

Specifies the transaction program name recognized by SNA remote client systems. Not used otherwise. The default is UPSTREAM.

LOGMODE

The VTAM mode name to the z⁄OS Storage Server task. The default value of #INTER works in most circumstances.

The following options are used for all types of Storage Servers: z⁄OS

RESTART

Controls automatic restart of restores and restartable backups if a communication failure occurs and CONV=WAIT was also specified. In the UPSTREAM Director simply specify the number of times you wish it retried.

(UPSTREAM Local) It has two parameters, e.g., RESTART=3,2 (without parenthesis). The first is the number of times to attempt restart of an interrupted operation and it defaults to 0 so automatic restart is disabled by default. The second parameter is the number of minutes to wait between restart attempts and defaults to 10. When a communication error occurs and a wait to restart is begun, a message is issued to the z⁄OS operator in case manual intervention is required to restore communications.

Mins. to Wait Between Attempts

(Director only) The number of minutes to wait between restarted attempts.

IPADAPTER

(z/OS Storage Sever ony) Specifies the IP address of the port to which the z⁄OS Storage Server main task should find to make the connection to the client UPSTREAM. The IP address is specified in dotted decimal form.

APPLRETRY

(z/OS Storage Server only) Used if a free VTAM application ID (see APPLPREF) cannot be found. It waits 5 seconds and retries again, repeating this until an application ID is found free or the APPLRETRY limit is reached. The default is 240, which causes USTBATCH to retry every 5 seconds for about 20 minutes.

MAXRETRY

(z/OS Storage Server only) Enter the number of retries that the Storage Server attempts to establish a connection to the remote system. Note that you remain “hung” until the remote accepts the session or the number of retries has been exceeded.

TMAXRETRY

(z/OS Storage Server only) Used when attempting to connect to z⁄OS Storage Server but is informed that UPSTREAM is already at its maximum permitted task limit. The operation is retried every 10 minutes until it is accepted or the TMAXRETRY limit is exceeded. The default is 0 (no retries).

WTOCOMP

(z/OS Storage Server only) If checked for CONV=WAIT request, causes z⁄OS Storage Server to issue a UST747 message on the system console indicating the success or failure of each Client request. This may be used for visual confirmation by the operator or for use with automation products. The default is not checked.

VERIFY

No longer used.

User Defined Translation Tables

PC and most UNIX operating systems use the ASCII character set. The UPSTREAM Storage Server needs file and a number of other names in EBCDIC so that they can be displayed and selected.

UPSTREAM has its own internal ASCII-to-EBCDIC and EBCDIC-to-ASCII translation table that it uses for communications parameters and file name specifications. For users of UPSTREAM in the United States, this works correctly for most file names. However, if you have a non-U.S. codepage loaded on your PC, have a non-standard name space loaded (such as a Macintosh file system), or have overseas users, characters that use values not defined in the ASCII-7 character set (which includes characters with accents and umlauts) may find that file names displayed incorrectly on the UPSTREAM Storage Server or the UPSTREAM Client.

In particular, note that the internal UPSTREAM translation tables are not one-to-one. There are a number of characters that do not map back to their original values.

To remedy this situation, UPSTREAM supports user loadable ASCII-to-EBCDIC and EBCDIC-to-ASCII translation tables.

We have enclosed translation tables that address some of the following problems:

• There are a number of characters including non English characters (è, ö, etc.) and special locale specific characters (¢, £, etc.) which are available in the enhanced EBCDIC character set. The new translation tables allow access to these characters in file names when viewed in ISPF panels.
  
• Our default ASCII-to-EBCDIC and EBCDIC-to-ASCII tables are not anywhere near one to one. This means that a number of characters in ASCII were mapped to a single EBCDIC character. This can lead to multiple files mapping to the same name, errors attempting to view or display files in ISPF and files restored to a different file name than their original. The new tables are still not one to one (as this is impossible), but they are much closer.
  

The supplied tables map to EBCDIC (4), code page 037.

Within ASCII, there are a number of character sets. DOS, Windows 3.1, Windows 95, Novell, OS/2, AIX, and a number of other operating systems use the OEM character set that includes line drawing characters. DOS, Windows 3.1, and other operating systems require code page support loaded for support of non English characters; see the operating system manuals for locale specific details.

Windows generally uses the ANSI character set (unless you have specified a different character set for display). ANSI has no line drawing characters and supports a larger number of locale specific characters. UPSTREAM for Windows has been designed to support exact display of file names.

A set of translation tables supporting different code pages is included.

To use these tables, we recommend that you copy the appropriate tables to the usatoe.tab and usetoa.tab files in the UPSTREAM directory. If UPSTREAM sees these files UPSTREAM uses them on program start.

If you wish to change translation tables the parameter file entries ASCTOEBC (ASCII-to-EBCDIC, default USATOE.TAB) and EBCTOASC (EBCDIC-to-ASCII, default USETOA.TAB) specify user loadable translation tables. You can specify the translation table file name in the Backup or Restore More dialogs. Whenever a parameter file is opened or one of these values is received from a remote system (and a new table is specified), the tables are loaded immediately. Thus you can use different tables for backups than for file transfers.

When UPSTREAM loads a translation table it logs message #1225I with the table name in the message log.

Note that if you have existing backups, by changing your translation tables, the UPSTREAM Storage Server may see these files as new files; causing a larger than normal number of files to be transmitted in phase 3 of a merge full backup.

The recommended tables are:

• Windows: ANS2ATOE.TAB (ASCII-to-EBCDIC) and ANS2ETOA.TAB (EBCDIC-to-ASCII).
  
• All other PC operating systems: OEM2ATOE.TAB (ASCII-to-EBCDIC) and OEM2ETOA.TAB (EBCDIC-to-ASCII).
  
• UNIX: For best display on the mainframe use unixatoe.tab (ASCII-to-EBCDIC) and unixetoa.tab (EBCDIC-to-ASCII). If you need one-to-one translation tables and if you are unsure which tables to use, use ans2etoa.tab and ans2etoa.tab.
  

Translation Tables in UPSTREAM Director

Some pair of translation tables is always used when viewing Profile Versions (backups) in the restore action version content view window. It is important that the translation tables that are loaded are the ones that were used for the particular backup you are viewing. If not, the characters and file names may not be legible in the viewer.

The UPSTREAM Director loads and sets the translation tables from the target system itself. There is a new section in the “View Options” dialog for doing this. Also, when a parameter file is loaded for use, the translation tables specified in it are now also loaded from the target system.

Building Your Own Translation Tables

To build your own translation table, follow these guidelines:

• Each file should have 256 lines, one line per character to translate.
  
• Each line represents a position. Line 32 (decimal) of the ASCII-to-EBCDIC file represents the number that you wish the space character translated to.
  
• Specify the value to translate to in decimal or hexadecimal. If you are using hex, then precede each value with a 0x. For example to specify 2a hex, type 0x2a (the hex letter digits can be upper or lower case).
  
• You may enter comments on each line if you separate the value and the comment with at least one space.
  
• Remember that conversions should be symmetric. If you define a value in the ASCII-to-EBCDIC table, do not forget to define the converse value in the EBCDIC-to-ASCII table.
  
• The default ASCII-to-EBCDIC translation file is named USATOE.TAB.
  
• The default EBCDIC-to-ASCII translation file is named USETOA.TAB.
  

We recommend that you test translation by backing up and restoring to files using non-English characters specific to your language before putting these into production.

There are a number of EBCDIC values that must not be used to avoid problems in UPSTREAM; These values should not be in your ASCII-to-EBCDIC table. These include (in hex), 00, 01, 02, 03, and FF. The ASCII values that should not be in your EBCDIC to ASCII table are (in hex) 00, and FF.

Sequential Disk Size Allocation

If you are using the Sequential Disk backup type, you may occasionally have problems in that the z⁄OS Storage Server allocates the size of the z⁄OS data set either too large, which can have negative impacts on your system as a whole (though any excess space is freed when the backup is complete), or too small that keeps the backup from working correctly. For the UPSTREAM Reservoir, the size estimate determines if the backup can fit in the profile Disk Path. If it can not, it uses the Overflow Path (if specified). Otherwise it fails the backup.

UPSTREAM / SOS and the SAN Express Passthru also use the estimated size of the backup to decide how much space in the local backup volume to allocate. If this value is too small, it wraps too many times resulting in poor performance.

Also, the size estimate is used to display the percent complete on the backup or restore status screen (or command line). If the size estimate is wrong, the calculation is incorrect.

UPSTREAM goes to great lengths in calculating the size of the UPSTREAM Storage Server file where the data backed up is stored. Most users have no problems and can ignore this section. However if you are having problems, this section can help you.

The UPSTREAM Client transmits the number of bytes that it found in file data while building the backup file to the UPSTREAM Storage Server, which then converts this to the correct allocation in blocks. This value can be wrong for several reasons:

• Databases. For many database systems (SQL-BackTrack for example), there is no way to get the size estimate in advance.
  
• The UPSTREAM Client, by default, does not include the number of bytes of non-file data (e.g., NetWare Directory Services, other Novell information, extended attributes), as this can potentially add significantly to the amount of time that it takes to build the backup file. In extreme cases this can be all or almost all the data in the backup. In these cases the z⁄OS file allocation may go into extents or may even fail.
  
• The UPSTREAM Client does not include the savings of compression. If you are using high compression or your data compresses well this can cause UPSTREAM to over-allocate the size of the file.
  
• The UPSTREAM Storage Server takes the value received from the UPSTREAM Client, adds a certain amount for overhead and converts it into blocks. In most cases this is going to be close to the amount of space required. Even so, it is not exact and there may be a small variation causing a slight over or underestimation of the size.
  

If you are finding that the file created by UPSTREAM to hold the sequential backup is either over or under allocated or your local backups are wrapping too many times, you can control this from the UPSTREAM Client. We do not recommend solving this problem by modifying the DASD block size on z⁄OS Storage Server.

There are two parameter file parameters designed to help you solve this problem.

CALCDASDSIZE

The default is “N”. If “Y”, then UPSTREAM Client calculates, during the backup file build, the total number of bytes of non-file data and add this to the size of the file data in its calculation that it sends to z⁄OS Storage Server. Specify “Y” if you are performing a backup with a large percentage of non-file data compared to file data and are willing to accept a significantly longer amount of time to build the backup file. This parameter cannot be used for database backups.

This parameter is not shown on the screens but can be modified in the parameter file, from the command line, or from the environment (it is also documented later in this section).

DASDOVERRIDE

Available on the Backup and Migration More dialog. The default is 100%. Allows you to specifically tailor the calculation of the size of the backup. The calculated total referenced below either indicates the size of the file data only (if CALCDASDSIZE=N) or the size of the file data plus the non-file data. There are 4 forms of this parameter:

• +<number>: Add the given number of bytes to the total calculated.
  
• -<number>: Subtract the given number of bytes from the total calculated.
  
• <number>%: Use the given percentage against the total calculated.
  
• <number>: Use this number to override any calculated value.
  

For example, if you are performing a backup of a Novell file server with only Directory Trustee Rights checked, and high compression, you may find that UPSTREAM consistently over allocates the file twice as large as it needs to be. Therefore, you would want to specify a DASDOVERRIDE of 50% (or 55% to allow some variation in the format of the data) to have it allocated correctly.

Hard Links

Introduction

Hard links are a feature in UNIX where a file appears to be multiple places in the directory structure but the data resides on disk in only one place. Thus a file has multiple names, but there is not a single “owner” of the file; there are multiple links to the same file. In UNIX, the ln command is used to create hard links from the command line. All hard links must exist on the same disk; they cannot cross file system boundaries. Hard links are common in operating system files. UPSTREAM itself uses hard links in its terminal support; in the /termcap directory under the UPSTREAM directory in UNIX.

You can see if a file is a hard link using the directory listing command ls -l command. The second column is the link count, the number of pointers to the given data. You should ignore the link count for directories; hard links can only link files.

UNIX symbolic links are somewhat different. You create pointers in the directory to an existing file that really owns the data. Symbolic links can link directories and can cross file system or even LAN boundaries. UPSTREAM backs up the link as a pointer. The owning file or directory itself is only backed up when it is encountered in the file structure. UPSTREAM has supported symbolic links since the product’s inception.

Note that hard links in Windows are managed by the WinSS PlugIn and do not need to be considered explicitly.

Backups

There is a check-box in the Spec Detail, More dialog for UNIX systems, Hard Links . If checked, UPSTREAM performs special handling for hard links during the backup. The default is checked.The file spec parameter is HARDLINKS and should be set to the value “Y” to include them on backups or restores.

Inquires and Restores

When you perform a file inquiry in the us program (either list and restore or restore and inquiry old), the size field displays the text <H-LNK> for hard links with a shortened size for the size of the data the link points to.

Since files may move on disk, UPSTREAM only creates hard links for files selected in a single restore. Thus if you have 10 files that were originally hard linked, and only restored 7 of them back to their original locations, you would have two sets of hard links, the original 3 (that were not restored) and the 7 that were restored.

Thus if you restore a single directory that contains files with hard links to other directories, the hard links are not restored and multiple copies of the file may exist. You must specify all files linked together within a single restore to have the hard links correctly recovered.

You must check the Hard Links check-box (or set the file spec parameter HARDLINKS Y) to include hard links in a restore; otherwise, they are skipped.

When you start a restore that includes hard links, all non-hard link files are restored first. If hard links are included in the restore (even if there is only one instance of a hard link) UPSTREAM then automatically re-drives the restore process at least one more time. Thus for a single restore request, UPSTREAM performs at least two restores. Again, this process requires no user intervention.

Notes and Warnings

• Backups and restores containing hard links can be marked restartable and can be restarted in any phase that a regular backup or restore can be restarted in.
  
  Since a hard link is a pointer to a specific location on disk, UPSTREAM stores the data with a file name that is made up using the location on disk. For example, if on a UNIX system you have two hard links: \tmp\file1 and \tmp\file2, there is a link count of two for each of the files. UPSTREAM determines the device ID and file ID (the components of the location on disk) and construct a third file, internally named: <LNK:(8 character device ID):(16 character file ID)>. This file is hidden and is not included in any UPSTREAM file inquiries. However they are visible in UPSTREAM Storage Server inquiries, and UPSTREAM Storage Server and UPSTREAM Client reports. They should not be directly selected for restore.

• In hard link restores, there may be more than two passes depending on the number of hard links (data not directory entries) to be restored. Each hard link requires one file spec. By default UPSTREAM uses the maximum number of file specs in each pass (up to 65535 minus the number of original specs). You can reduce this number by specifying the environment variable USHARDLINKSET that increases the number of passes.
  
• The UPSTREAM Client preserves a file in the client work path with the name <profile name>hardlink.save. If this file is non-existent or deleted, all hard links are backed up again and the file recreated. This file is used to detect if a hard link is moved due to disk compaction.
  
• If you move your data to another disk, we strongly recommend that you do a first-time full so that the hard link references are rebuilt.
  

USMODIFY

A program called USMODIFY is included with UPSTREAM to allow command line or batch file tailoring of any overall parameter within a parameter or configuration file or group of files. It is particularly useful for regular changes to passwords for all your parameter files.

The command line syntax is:

Where:

/?

      (or /h, -?, -h or no parameters at all) displays help text.

DATSPEC=<FileList>

Designates a list of one or more UPSTREAM parameter file specifications (*.DAT) which may contain wild-cards. The prefix of DATSPEC is the default if you do not specify a prefix. In the FileList, you can specify multiple files, each separated by a semicolon (‘;’).

CFGSPEC=<FileList>

Designates a list of one or more UPSTREAM configuration file specifications (*.CFG) that may contain wild-cards. In the FileList, you can specify multiple files, each separated by a semicolon (‘;’).

@<Override File>

If you have a number of parameters that you wish to change (too many for the command line), then you can enter them in an override file, and specify it alone on the command line with the “@” prefix.

parm[=value]

One or more override parameters. If you specify just the parameter title with no equal sign and no value, you are prompted for the value. If you specify a parameter title with an equal sign but no value, a blank value is used. Override parameters specified on the command line take precedence over parameters specified in an override file.

Example 1: Changing Passwords

For example, if you wished to change the password in all your parameter files and wish to be prompted for the new password, type:

usmodify DATSPEC=*.DAT PASSWORDYou are then prompted for the new password (which is not displayed) and all the parameter files are updated. The list of files updated are found in the upstream.log file.

Example 2: Changing Several Parameters

If you wish to change your User ID, password, and backup profile name in two parameter files (DAY.DAT and WEEK.DAT), you could create an override file (OVERRIDE.MOD) as follows:

USERID NEWUSER
BACKUPPROFILE SERVER1
Then, from the command line, specify the following:

usmodify DATSPEC=DAY.DAT;WEEK.DAT @OVERRIDE.MOD PASSWORDYou are prompted for the password and then the two parameter files are updated with the results placed in upstream.log.

Notes

Some notes on this facility:

• The name of the log file is determined from the configuration file (upstream.cfg) and cannot be overridden. You should also have the upstgream.msg file available as it is used for logging.
  
• File specification level parameters (file name, archive bit, etc.) and frequency definition configuration parameters are likely to cause problems with this facility as there is no way to separate each specification or frequency. Thus, we do not recommend that you use usmodify to modify these parameters.
  

UPSTREAM Tuning

There are several ways to fine-tune UPSTREAM. These issues include:

• Setting the priority of the UPSTREAM program.
  
• UPSTREAM in Windows opens up to 3 files in advance in a backup, and reads up to 64K of data at a time. You can specify the number of files in advance and the size of this data buffer.
  
• UPSTREAM creates a temporary file during the backup. The maximum file size (including drive and path) impacts the size of the backup file. You can set this value.
  
• When idle, UPSTREAM uses a certain amount of CPU time. You can tune this amount of time.
  
• UPSTREAM supports a facility that is known as record packing where a number of logical records are combined into a large single transmission record. You can vary the transmission record size.
  

Many of these fine-tuning options are controlled through environment variables. Environment variables can be set in the operating system session that you are running in. Environment variables are set using the SET <variable>=value.

See Section “Environment Variables” for more information about the environment variables defined in UPSTREAM.

Program Priority (Windows Only)

In Windows, each process (complete program) is given a priority by the operating system. This priority is used to determine how much of the CPU’s processor time is dedicated for each task.

Extensive testing has shown that increasing the program priority level can have a significant performance improvement impact if the PC is not bottle-necked in disk or communications I/O.

On the other hand, an increased priority for UPSTREAM means that other tasks have less CPU time to execute producing unacceptable performance in other tasks. The ability to change your priority addresses both issues.

This priority is represented by two separate values: a priority class and a modifier priority level. The priority classes are Idle-time, Regular (which is the default), Time-critical and Fixed-high. The priority level is a modifier from 2-8 (in Windows, 0 or 1 are taken as 8) which indicates how your process is dispatched within the given level; the default is 5 for Windows.

UPSTREAM allows two separate ways to modify the program priority:

• In the Director, in the Options button, Priority tab.
  
• For a given request. You can save the priority settings to a parameter file that will then be used whenever you open a parameter file or specify a file from the UPSTREAM Storage Server.
  

Either method displays the priority dialog.

Set Priority

Backup Fine Tuning

When performing a backup, UPSTREAM pre-opens a certain number of files before it begins to back them up. In many environments this improves performance. UPSTREAM also reads data in large blocks, up to 256K bytes at a time. You can set these values through environment variables:

NUMBACKUPTHREADS

(Windows only) The file pre-opens are performed in a backup thread. Through this environment variable you set the number of files that the separate thread pre-opens. The default is 3. Valid range is from 1-50. If you set this value to 1, there are no file pre-opens.

BACKUPBUFFERSIZE

UPSTREAM takes 262144Linux OS on IBMZ and evenly divides it by the record size to determine the data read size. Thus if you specify a record size of 6000, UPSTREAM always reads data in 258,000 byte pieces (it still sends them to the UPSTREAM Storage Server in 6000 byte records). Valid values are the record size up to 4GB, but we recommend never using a value over 1MB.

Record Packing

UPSTREAM Workstation/Serverz⁄OS supports a facility known as record packing where multiple logical records are combined into a single, large record. Testing has shown that there is significant performance improvements and a reduction in CPU utilization in TCP/IP and to a lesser extent SNA.

You can specify the record packing value on the Backup or Restore More dialogs as Packing Size, or from the UPSTREAM Storage Server or parameter files as PACKRECSIZE. If using UPSTREAM v3.7 on both client and storage server sides use a value of up to 65400 as performance improvements and reduced CPU utilization has often been the result.

However, there are also situations where record packing reduces performance (particularly for TCP/IP). When optimizing backup performance, try disabling record racking (by setting PACKRECSIZE to 0) to see if it improves performance.

Record packing is enabled by default for both backups and restores when using a high enough version of UPSTREAM on both the UPSTREAM Storage Server and workstation/server sides.

This size can be varied using the parameter PACKRECSIZE. The default record packing size is 32700.

Starting Backups at the Same Time on Multiple Systems

There are times that you need to accomplish a task on multiple systems, synchronously. For example, if you have a multi-system database with no specific database agent (like ECM), you would like to guarantee that all the backups on all the systems run at the same time.

On modern operating systems, their clocks can be synchronized by a common time source. Thus, if an application takes an image of the disk at a common time, and does the backup of that image, you have a synchronized backup. You can use hardware providers (like TimeFinder from EMC) or software providers, like Microsoft's VSS and accomplish this same function.

A new non-repeating parameter was created named STARTTIME. The STARTTIME parameter takes a value in the form of YYMMDDHHMMSS, which is the time you want to have the backup (or any other action) started by the UPSTREAM Client. This parameter is added to your USTBATCH job parameters after the ACTION parameter.

In order for this to work properly, each of the systems, whose backups are to be time synchronized, must have their clocks synchronized by the common time source.

Because legacy (non-snapshot) backups are not point-in-time backups, the data that is being backed up may change while the backup is in progress. For this reason, the only hope of backing up synchronized data is to perform snapshot backups that use the WinSS PlugIn or a hardware provider (much as TimeFinder from EMC).

In Windows, to provide for snapshots to be taken at the same time sync point, a new WinSS PlugIn parameter was also created named STARTDELAY. The STARTDELAY PLUGINPARAMETERS sub-parameter takes a value that is the number of seconds after the STARTTIME absolute time.

Here is what a set of parameters for Windows looks like:

If you are not using Windows with WinSS, you should plan on starting a job that uses the command line interface provided by the vendor to perform the snapshot and mount the volume.

In the example above, to ensure that the creation of the snapshots is as fast as possible (and therefore mitigate any time differences across systems), the file specs should reference only the volumes that contain your ECM data (D:\ECMData\*).

Thus, to backup six ECM file server systems at the same time, the USTBATCH job contains parameters that target all ECM systems in one step.

If all works correctly, the snapshots for the ECM file server systems are created within milliseconds of one another.