Introducing UPSTREAM

UPSTREAM is a comprehensive enterprise data protection solution for file servers, workstations, and database servers. With a proven client/server design, UPSTREAM provides fast backup and restore capabilities for Servers, z⁄OS for Unix System Services (USS) services, applications, and workstations across the enterprise.

UPSTREAM features

UPSTREAM is an enterprise data protection solution for file servers, workstations, and database servers. Its many features include: 

Unattended Operation

• Set up UPSTREAM to run at any combination of times. Backups and restores can be scheduled to run daily, weekly, monthly, quarterly, and yearly in virtually unlimited numbers of combinations.
  
• Backups and restores can be started by batch jobs that can be integrated into your existing scheduling system allowing total unattended operation. You can even start backups or restores from other workstations or servers!
  
• Furthermore, you can be notified by mail or SNMP trap when an UPSTREAM operation succeeds or fails. Information can be attached to help you resolve any problems. Unattended operations have never been safer or easier to manage.
  

 Fast

• Selective levels of compression combined with our unique Merge backup assure fast transfers on even slow links.
  
• The advanced synthetic Merge backup facility allows the UPSTREAM Server to construct a complete full backup without the workstation having to read or send all the files resulting in extraordinary performance. Using a sophisticated technique, the workstation sends a “picture” of the drive(s) to be backed up to the Server software that uses prior backups plus changed files to construct the fulls. This technique is safe as it properly reflects deleted files, changed files, renamed files, and just about anything else.
  
• For Intel and z/Linux an I/O Optimizer improves the performance of UPSTREAM backups where large numbers of files are on a server by starting several threads of execution on the client system that pre-opens and caches the file data in parallel for transmission to the storage server. While it uses quite a bit of memory (over 1GB) it has the advantage of 20-35% performance improvement in some environments.
  

Efficient

UPSTREAM can store your workstation or file server data on the UPSTREAM Storage Server in compressed format. This conserves disk and tape space. UPSTREAM can also write your data directly to tape for long term storage.

Secure

z⁄OS Storage Server interfaces to the most common mainframe security systems including CAACF2, IBMRACF, and CATopSecret. UPSTREAM Reservoir adhears to the OS platforms native security model.

OS Platform Aware

A wide variety of server file systems are supported including:

NFS

All UNIX systems can be attached and backed up via NFS from any other UNIX system.

Multi-platform support

UPSTREAM operates on a variety of operating systems platforms. Each version supports the base UPSTREAM functions of merge backups, single file, directory and full volume restore and more.

Microsoft Windows

All the Windows specific features are supported including long file names, Unicode file names, registry hives, system event logs, security ACLs, alternate data streams, Windows System State and more.

UNIX

UPSTREAM/ UNIX, which supports the popular AIX (IBM), and Solaris (SUN) variants (SPARC and Intel), Intel, and Linux OS on IBMZ as well as IBM’s z⁄OS UNIX on z⁄OS platforms, is a native UNIX application allowing backups and restores of applications files and character special devices on all primary AIX hardware platforms, RISC Sun and HP platforms, Intel Solaris, and System 390 mainframes. It also contains support for symbolic and hard links, preservation of the last access date and more.

File Transfer

UPSTREAM allows you to transfer native files between your client and your z⁄OS mainframe. Its extensive support includes text and binary files, a choice of mainframe disk or tape, a variety of mainframe file formats including GDGs, PDS members, flat files, and more makes  a powerful tool for interchanging data among your different computer systems.

Restart / Recovery

UPSTREAM can restart a failed backup or restore that was terminated due to communications line failure, system unavailability, or just about anything else at the point it failed. For backups, UPSTREAM also remembers files that were unavailable due to LAN data sharing issues, and retries those files as well.

Easy to Use

Because UPSTREAM is transparent in normal operation, there is nothing for a user to maintain day-to-day. A system administrator can set up a UPSTREAM backup system in a matter of minutes, not hours. The UPSTREAM  Director screens are familiar and simple to use with point and shoot operation, thereby allowing you to select the backup or restore versions and files quickly and easily.

Intelligent

UPSTREAM always remembers where your data is. If you write your data to tape, UPSTREAM remembers exactly which tape it is stored on, thereby eliminating another user record keeping headache.

UPSTREAM can automatically restore your files back to a specific date, intelligently combining restores of full and incremental backups.

Verifiable

UPSTREAM logs backups, restores, communications failures, failed files, statistics, and just about every other significant event on BOTH sides of the operation. This ensures that both the open system and mainframe administrators can monitor, control, and verify every aspect of backups and restores.

Client-side reporting includes all files backed up and restored, files deleted, as well as version and file information requested from the UPSTREAM Storage Server.

Central Control

With UPSTREAM storing workstation and LAN server data centrally on a UPSTREAM Storage Server, all the advantages of central storage including security, data sharing, and disaster recovery are realized while continuing to enjoy the performance, ease of use and other benefits of decentralized computing.

Backups or restores can be initiated by z⁄OS batch jobs or a central workstation/server, allowing you control of your environment without user intervention.

Job Execution

UPSTREAM can run programs and batch jobs on the system using local or UPSTREAM Storage Server control. This allows you to integrate your backup and recovery plans with existing databases and other applications.

z⁄OS jobs can also be submitted under client control. This helps integrate such functions as FDRSOS into UPSTREAM processing.

What is UPSTREAM Client?

The UPSTREAM Client consists of several programs, each with their respective functions. Only the most significant programs are mentioned here. As a whole, UPSTREAM allows you to perform backups, restores, file transfer, and a variety of other useful benefits.

UPSTREAM Service (uscmd)

The UPSTREAM Service uscmd is a command line program primarily designed to be installed as a system service and process remotely initiated UPSTREAM requests from the UPSTREAM Storage Server or the UPSTREAM Director.

The UPSTREAM Service performs all UPSTREAM functions: listening for remote requests, backup and restore, logging, and more. The UPSTREAM Scheduler also runs from this program.

The uscmd program can process a single request based on command line parameters for integration into scheduling systems or as a command line API into UPSTREAM.

Local UPSTREAM (US.EXE)

Local UPSTREAM (the US.exe executable) provides the listening and processing facilities of the UPSTREAM Service, as well as some additional user interface functionality of building parameter files for unattended operations, inquiries, FDRSOS Local Backup Admin, and more. In Windows it uses a graphical interface, in UNIX it uses a character interface.

UPSTREAM Configurator (USCFG.EXE)

The UPSTREAM Configurator provides a way of specifying TCP/IP parameters for establishing communications connections and various other UPSTREAM parameters. It also allows you to specify Schedules and Personalization.

The Windows version of the Configurator also automates the task of installing, upgrading and reconfiguring UPSTREAM to run on a Windows cluster system.

Personalization allows you to restrict access to specific UPSTREAM function (backups, restores, etc.), limit access to specific backup profiles or even specific directories. Personalization profiles can be set up to be multi-user aware and follow a user across workstations on a network.

UPSTREAM Director

A full featured GUI user interface for almost all UPSTREAM platforms, the UPSTREAM Director allows you to set up parameter files, specify backups and restores, perform profile management and configuration, and more from a workstation anywhere in your network. Written in Java, the UPSTREAM Director allows you full control of UPSTREAM services anywhere in a large network.

UPSTREAM’s architecture

The UPSTREAM Storage Server maintains a repository for all backups. This repository contains all the backup information that you specified, information about the files (including file dates/times, attributes and the like), and the file data itself.

Backups are separated in several ways, hierarchically:

Backup Profiles

These are user specified ways of identifying groups of backups. You can have many backup profiles for each machine, or limit it to one.

Version Dates

Each backup is identified by a version date. This is the date and time (on the UPSTREAM Storage Server) that the backup was originally created. Combined with the backup profile, version dates allow you to identify a particular backup.

File specs

Each backup can have one or more file specs. A file spec is a file specification combined with other information. This allows you to be able to backup whole drives, partial directories, some files, or one file and have them grouped together. File specs can also be used to exclude files or specify migration options.

Backup file data can be stored in two different ways on the UPSTREAM Storage Server:

Direct to tape

Each backup is written directly to tape. This allows for the smallest amount of mainframe disk storage at the cost of a limited number of concurrent backups or restores.

Sequential disk files

Each backup is stored in a separate disk file which can be controlled by SMS or any other migration facility. This provides high backup performance and integration with existing services at the cost of temporary disk space.

Selective or complete inquiries or restore can be retrieved easily by a user from the UPSTREAM Client screens.

Backup Strategies

There are virtually an infinite number of ways to integrate UPSTREAM into your workstation or server backup strategy. The synthetic full merge backup facility is the easiest to use and the most powerful.

Merge backups allow the UPSTREAM Client to send a fraction of the total data on a disk (only slightly more than an incremental backup) and still end up with a complete full backup. Using a sophisticated technique, the client sends a “picture” of the drive(s) to be backed up to the UPSTREAM Storage Server that uses prior backups plus changed files to construct the fulls.

Advantages:

• 
  
  • A full backup is created with the client only having to send a tiny fraction of the total data.
    
  • Easier to use, understand, and manage. The beauty of a merge backup is that all the complexity is behind the scenes; it is as easy or easier to use than non-merge backups.
    
  • If you are using tapes for incremental backups, you use fewer tapes.
    

A single backup profile name is used for full and incremental backups. It is recommended that this single profile represent a single, unchanging group of file specs (a single server, a single workstation disk, etc.). The facility is flexible enough for you to be able to add or remove drives, however it is not recommended that you use a single profile for more than one entity.

The technique requires that you perform a first-time baseline full backup of the file specifications that you wish to maintain. In this backup you do transmit all the files. Once you have this full backup, you only perform incremental merge and full merge backups.

Incremental merge backups are backups where only the changed files are transmitted to the UPSTREAM Storage Server. The first incremental merge backup after a full backup begins a new tape or disk file; subsequent incremental merge backups to tape can be appended to previous incremental backups. Subsequent incremental merge backups to disk create new UPSTREAM Storage Server files.

Full merge backups are appended to the end of the incremental backup file (if the backup is on tape), or a new file is created (if the incremental backups are on disk). The client sends all the changed files as well as the directory entries for all files that it does not believe have been changed. The UPSTREAM Storage Server then examines this list of files, retrieves from old backups (the last full or any of the prior incremental backups) files that have not changed, and requests from the workstation or server files that it does not have.

Before UPSTREAM uses a file from a previous backup, it verifies a match of the complete, qualified file name, the last modified date and time, and the file size. If any one of these conditions do not match, the UPSTREAM Storage Server software requests a transmission of the file from the workstation or file server. The result is a complete full backup without the client software having to read or send the vast majority of the data, and deleted files are properly reflected.

The following scenarios should help you understand the process.

Scenario #1: Full and Incremental Backups to Tape 

Following figure shows a diagram of how a tape only system would work. This scenario’s advantages are:

• 
  
  • No intermediate disk requirements. Data goes directly to tape without having to be staged through disk. This saves on UPSTREAM Storage Server disk space.
    
  • Good for large volumes of data.
    
  • Only one tape is created per backup cycle (usually weekly). This saves on tape management.
    

Full and Incremental Backups to Tape

When the first-time full backup is run, a new tape is created that holds all the data on the workstation or server (Tape 1 in above figure). The first incremental after a full creates a new tape (Tape 2 in above figure). Subsequent incremental backups are appended to the end of the tape volume.

After the first-time full backup, subsequent full volume backups are merge full backups. In a merge full backup, the UPSTREAM Client sends up all the files changed since the last incremental as well as a directory listing. The tape holding the prior full backup is mounted (Tape 1) as well as the tape holding the incremental data (Tape 2).

Any files that have not been changed are copied from tape 1 to tape 2 unless they are in the incremental on Tape 2 already. The files that were in the prior incremental backups on Tape 2 are recorded as being part of both the full and incremental backups. The UPSTREAM Storage Server software then requests any files that could not be matched.

The result is a new full backup tape that will be used as the source for the next full backup.

Scenario #2: Full Backups on Tape and Incremental Backups on Disk

Full and Incremental Backups to Tapeshows a diagram of how a system where full backups are stored on tape and incremental backups are stored on disk would work. You may want to choose this option if you have sufficient UPSTREAM Storage Server disk space and do not wish to mount the backup tapes each day. In addition, recovery is quicker for incremental data.

Full Backups on Tape and Incremental Backups on Disk

When you run your first-time full backup, a new tape is created which holds all the data on your workstation or server (Tape 1 in above figure). The first incremental creates a file on disk (File 1). Subsequent incremental backups create new files on disk.

When you run a merge full backup, the tape holding the prior full backup is mounted (Tape 1) as well as the new tape for output (Tape 2). The UPSTREAM Client sends up all the files changed since the last incremental as well as a directory listing.

The UPSTREAM Storage Server software then copies the workstation or file servers files that were requested in the directory listing from the incremental backups or its most current backup on the last full (Tape 1) to the new full (Tape 2). The UPSTREAM Storage Server software then requests any files that could not be matched.

The result is a new full backup tape that will be used as the source for the next full backup.

Scenario #3: Full and Incremental Backups on Disk

“Full and Incremental Backups to Tape” shows a diagram of how a disk only system would work. You may want to use this scenario for small backups or where restore speed is important.

Full and Incremental Backups on Disk

When you run your first-time full backup, a new file is created on the UPSTREAM Storage Server that holds all the data on your workstation/server (File 1 in above figure). Each incremental afterwards creates a new UPSTREAM server file (File 2 and File 3).

When you run a merge full backup, the UPSTREAM Client transmits the changed files and they are written to the new full backup file (File 4). Then the UPSTREAM Storage Server software takes the directory listing transmitted and copies from the incremental backups (File 2 and File 3) and the last full (File 1) all the unchanged files.

The UPSTREAM Storage Server software then requests any files which could not be matched. The result is a new full backup file that will be used as the source for the next full backup.

Profile Sets

(UPSTREAM Reservoir Only) A Profile Set is a single definition for a number of backup profiles. Profile Sets let you define the complexities of a backup profile a single time, add Backup Profiles to that Profile Set definition and then store all the tape backups for that profile set on a single tape. This simplifies configuration and allows you to be able to take full advantage of today’s high capacity tape drives. Profile Sets are only available with the UPSTREAMReservoir Windows Storage Server.

The backups that comprise a Profile Set follow the rules described above for merge backups. The first incremental backup after a full for any profile in the profile set begins a new tape. Similarly, a full backup immediately after a full backup for any profile in the profile set starts a new tape as well. Otherwise, all backups are appended to the existing tape.

UPSTREAM Windows

UPSTREAM Windows is a powerful member of the UPSTREAM product line. Its many features include:

• Support for all standard UPSTREAM functions. These include standard backups/restores, merge backups, point-and-shoot restore of individual files, directories, and drives, UPSTREAM Storage Server status and reporting, file transfer, and more.
  
• Full operation on all current Microsoft Supported versions of Windows.
  
• Support for all features of Windows Servers including Active Directory, clusters, long file names, registry hives, system event logs, security ACLs, extended attributes, alternate data streams, mount points, compressed files, sparse files, user quotas, encrypted files and more.
  
• Support for backup, restore and display of Unicode (non-ASCII) file names.
  
• Support for automated UPSTREAM Windows Bare Metal Recovery using a boot-able ISO DVD. This allows for fast recovery of a Windows system with simple and minimal recovery steps.
  
• A full integrated scheduler. UPSTREAM includes a powerful scheduler called the UPSTREAM Dispatcher that allows administrators to schedule any combination of backups or restores. The UPSTREAM Dispatcher can even be used to schedule the execution of non-UPSTREAM programs.
  

If you are running Windows, refer to the UPSTREAM Installation and Configuration and Settings sections of UPSTREAM Client.

UPSTREAM/UNIX

UPSTREAM/UNIX is a flexible member of the UPSTREAM product line. Its many features include:

• Support for all standard UPSTREAM functions. These include standard backups/restores, merge backups, point-and-shoot restore of individual files, directories, and drives, UPSTREAM Storage Server status and reporting, file transfer and more.
  
• Support is provided for a number of UNIX specific features including long file names, case sensitivity, symbolic links, hard links, and more.
  
• NFS attached file systems.
  
• Character special logical and physical volumes.
  
• UPSTREAM/UNIX provides a full screen interface that operates from the system console, an X terminal, dumb terminals or TELNET terminal emulation (except z/OS UNIX Systems Services).
  
• Command line version of UPSTREAM allows unattended UPSTREAM Storage Server control or integration into applications.
  
• Backups and restores can be initiated locally, from the UPSTREAM Storage Server, another UNIX system, or a PC.
  
• Powerful UPSTREAM Storage Server Control. Virtually every UPSTREAM/UNIX function can be controlled from the UPSTREAM Storage Server including backups, restores, reporting, restarting failed backups, execution of local programs, and UPSTREAM Storage Server reporting.
  

If you are running UNIX refer to the Unix, UPSTREAM Installation and Configuration and Settings sections of UPSTREAM Client for further details.