IBM Tivoli Storage Manager

IBM Storage Protect (formerly IBM Spectrum Protect, Tivoli Storage Manager (TSM)) is a data protection platform that gives enterprises a single point of control and administration for backup and recovery. It is the flagship product in the IBM Spectrum Protect (Tivoli Storage Manager) family.

IBM Storage Protect

Other names	IBM Spectrum Protect, Tivoli Storage Manager
Developer(s)	IBM
Stable release	8.1.24 / September 27, 2024 (2024-09-27)
Operating system	cross-platform
Type	backup
License	Proprietary
Website	ibm.com/products/storage-protect

It enables backups and recovery for virtual, physical and cloud environments of all sizes.

This product is part of the IBM Spectrum Software Defined Storage suite of products and is unrelated to the Tivoli Management Framework.

History

TSM descended from a project done at IBM's Almaden Research Center around 1988 to back up VM/CMS systems. The first product that emerged was Workstation Data Save Facility (WDSF). WDSF's original purpose was to back up PC/DOS, OS/2, and AIX workstation data onto a VM/CMS (and later MVS) server. WDSF morphed into ADSTAR Distributed Storage Manager (ADSM) and was re-branded Tivoli Storage Manager in 1999.

Photo of an IBM Technical Award from 1991 given to the original Workstation Data Save Facility (WDSF) team at the IBM Almaden Research Center in San Jose, CA, USA.

The TSM database (through release 5.5) was a bespoke B+ tree database; although the TSM database used many of the same underlying technologies as IBM's Db2, has a SQL engine (for read-only use), and supports access through ODBC, the database had an architectural limit of approximately 530 GB, and 13 GB of log space. Starting with TSM 6.1, released in May 2009, TSM uses a Db2 instance as its database (thus eliminating the architectural limitations of the previous TSM database).

Product details

TSM maintains a relational database (limit 534GB through TSM v5.5, 4TB with TSM v6.3.3+) and recovery log (aka transaction log, limit 13 GB through TSM v5.5, 128GB with TSM v6.1+) for logging, configuration, statistical information, and object metadata. v5.5 DB pages are always 4KB, and partitions every 4MB. Single row inserts only. On average, 20GB of space is consumed for every 25 million objects. Shallow directory structures use less TSM DB space than deeper paths. This database may generally be queried via an emulated SQL-98 compliant interface, or through undocumented SHOW, CREATE or DELETE commands.

Actual user data is managed via a cascading hierarchy of storage media (Primary Storage Pools) presented as raw devices (UNIX), filesystem containers (Windows and Linux), streaming tape or optical media. Additionally, emulated tape from a Virtual Tape Library or EMC Centera WORM archival device is supported. Duplicate copies (backup sets or Copy Storage Pools) of any subset of data may be created on sequential media for redundancy or off-site management.

The 5.5 release of the TSM Server was supported on AIX, HP-UX, Linux, Solaris, Windows Server, and z/OS. The TSM Client of the same release is supported on NetWare, macOS, AIX, HP-UX, Linux, z/OS, Solaris, and Windows 32/64-bit. The 6.1 release of the TSM Server is supported on AIX, HP-UX, Linux, Solaris, and Windows Server, while the TSM Client is supported on the same operating systems as 5.5. On October 21, 2011, TSM 6.3 was released. Version 8.1.21 was released in December 2023.

Components

Tivoli Storage Manager as a system is made up of several different components. The major components of TSM include: TSM Server, TSM Client, TSM Storage Agent, TSM Data Protector, TSM Operation Center and TSM Administration Center.

Data Sources

The most Common data source for TSM is the TSM Client ("TSM Backup/Archive Client" or "B/A Client"). The B/A Client allows backup and restore of data both "selectively" and "incrementally", which is generally known as "Progressive Incremental" or "Incremental Forever", as each unique client+filespace+path+file combination is separately tracked for retention. Further, a separate method is provided by the B/A Client which is known as archive (and retrieve). This method generates groupings of objects to be retained as a single unit. This still differs from traditional full/incremental style backup products in that the files are stored separately or in smaller aggregates rather than as a monolithic image. Additionally, there is no provision for an incremental archive.

Other data injectors include policy-based hierarchical storage management (HSM) components for AIX, Linux and Windows. These allow migration of data from production disk into one or more of the TSM storage hierarchies while maintaining transparent access to that data by the use of DMAPI or NTFS reparse points.

IBM General Parallel File System (GPFS) can use TSM as a storage tier for GPFS' Information Lifecycle Management (ILM) which provides HSM for a GPFS filesystem. A GPFS filesystem can be simultaneously accessed from multiple servers running Linux, Windows, and AIX by using GPFS filesystem software installed on any of these operating system platforms. GPFS provides transparent access to data whether online on disk or migrated to tape by requesting file saves and retrieves from TSM.

Additionally, many applications provide or are provided with TSM API connections allowing the storage of databases, mail systems, system backups and even arbitrary user data within TSM's repository. Aside from TSM's UNIX HSM product, only the "Backup" and "Archive" management facilities are accessed through the client API.

Agents

The TSM architecture makes use of two special-purpose agents. The LAN-Free Storage Agent is a limited function TSM server which is configured as a library client and uses server-to-server communication to coordinate the use of storage resources which are configured to TSM but which are also presented to the storage agent. Usually this LAN-free and server-free backup agent is installed on the specific client; however, it is network accessible and could be utilized to bypass network bottlenecks. One example would be to connect via infiniband between two Bladecenter chassis, where one has SAN attachment to tape, and the other does not. This could bypass limited ethernet bandwidth without having to move the TSM server instance.

The NDMP API agent is used by NetApp devices and other network attached storage (NAS) devices to allow backup access to the appliance itself rather than having to back up the device via an attached NAS client.appliance direct access to shared tape.

Administration

Administrative functions are accessed through the TSM Administrative command line interface tool or via a web based portal application known as the TSM Operations Center.

Subproducts & Other Products

Enterprise Edition Features

• The Disaster Recovery Manager (DRM) - A set of commands which aid in the management of offsite secondary copies of data, the TSM Database backups required to access those media, and the configuration data required to recover the TSM database in case of a loss.
• IBM TSM for NDMP - A marketing name for TSM Enterprise Edition (TSM EE) features allowing both LAN and LAN-free backups of network attached storage (NAS). Specifically, NetApp filers or EMC Celerra datamovers are supported; however, any NDMP v3 or NDMP v4 client is likely functional.
• IBM TSM for Storage Archive Manager - A marketing name for TSM EE features which help maintain archive retention for regulatory purposes.

Associated Products

IBM's naming convention is to prefix every product name with "IBM Tivoli Storage Manager"; most products require an additional license. See also IBM Tivoli Storage Manager FastBack for more information on IBM's TSM product for block-level, continuous data protection.

• Client aka Backup/Archive Client for most major operating systems at supported versions
• Server for most major server operating systems
• for Advanced Copy Services (Formerly ITSM for Hardware) - Hardware-based snapshots for major database backups. Although it still exists as a product today the newer releases are re-branded as FCM (FlashCopy Manager).
• for Copy Services - Windows snapshots for Exchange and MSSQL.
• for Databases - An API for Oracle RMAN and a GUI & CLI tool for MSSQL backups.
• for Data retention - A zOS client to manage long-term archiving of data.
• for Enterprise Resource Planning - - Allows online backup of SAP R/3 stored in Oracle or Db2. Formerly backint developed by IBM Germany by the Enterprise Service Division (ESD) as a "Support Offering" to back up SAP R/3 directly into TSM. This product was acquired by Tivoli as Tivoli Data Protection for R/3.
• for Mail - External applications that tie into the API for Lotus Domino (aka Notes) and Microsoft Exchange for online backups.
• for SharePoint - A repackaged copy of DocAve, still marketed by its OEM AvePoint.
• for Space Management - (HSM) for Linux and AIX. Automatically moves inactive data to less expensive media and frees online disk space for important active data. The Windows product ("IBM TSM HSM for Windows") is OpenStore for File Servers produced by INTERCOPE GmbH.
• for Storage Area Networks (SAN) - aka "LAN Free Storage Agent" A modified version of the TSM Server itself, offering no local TSM Database. Configuration is purely for server-to-server library sharing. This allows the agent to write to tapes managed by the primary TSM server without having to pass data over the network.
• for System Backup and Recovery - A standalone product for AIX bare metal recovery. The original name was Sysback/6000, produced by Tony Johnson in the 1990s and sold by IBM as a service offering through IBM Global Services. Sysback can back up and restore files, filesystems, volume groups, and entire systems to local or remote disk, local or remote tape, NIM servers, and TSM. Current versions can also recover systems via the files backed up using the TSM B/A Client. There is a major branch of this product. When Tony Johnson left IBM in 1998, he started a company and product named Storix. Storix is feature rich and supports AIX and Linux, has a GUI management interface, and is very similar in origins to Sysback. Storix actively competes with IBM's Sysback due differing price structures and features.
• for Virtual Environments - Provides advanced data protection and flexible recovery options for VMware vSphere ESX and ESXi servers.

Interface Products

• IBM Tivoli Storage Manager Operational Reporting - A portion of the TSM Microsoft Management Console (MMC) Plug-in for Windows which can generate webpages and email out of SQL queries and simple processing of that data. Custom SQL can be added; however, the reporting tool provides no trending or graphing functionality. This is seen as one of the major faults of TSM.
• Power Administrator for TSM - Complete and easy to use GUI for administration, management, monitoring, reporting and error analysis of TSM environments. Includes a report builder to define custom reports with many different performance elements. Capable of sending SNMP traps and emails for new TSM messages, generated reports, and even trouble tickets.
• Operator for TSM (OTSM) - Debriefing Software's Windows based GUI for managing one or more TSM servers. Allows easy handling of offsite volumes through animated step-by-step instructions.
• TSMExplorer is easy and comfortable product for managing and monitoring Tivoli Storage Manager. Product allows to manage many TSM servers from single sign-on. It is written in Java using the SWT widget toolkit.
• Wizards Storage Portal - Debriefing Software's cloud based monitoring and reporting tool for TSM and SVC (SAN Volume Controller). Includes 24x7 monitoring, user defined dashboards, graphic reporting, remote TSM-management and integration with Operator for TSM.

Non-Tivoli API Clients

• ADINT - developed by IBM Germany by the Enterprise Service Division (ESD) as a "Support Offering" to back up SAP MaxDB directly into TSM.
• adsmpipe - an unsupported tool provided by IBM through its RedBook site for piping data directly into TSM. Commonly used to back up MySQL
• Archive Backup Client for OpenVMS - A product by STORServer Inc. to back up OpenVMS systems into TSM. With ABC you can back up, archive, restore, query and manage OpenVMS files stored on TSM servers as a logical extension to the on-line OpenVMS ODS-2 or ODS-5 file systems.
• Caminosoft Managed Server HSM, Tivoli Edition - Hierarchical Storage Management software for file system archiving of seldom accessed files to TSM. Supports N series, NetApp, Windows, Linux, and NetWare.
• CBMR/TBMR - A product by Cristie sold as a bare metal restore (BMR) tool for Linux, Solaris, AIX and Windows using TSM as a datastore.
• Repostor DATA Protector - a solution suite from Repostor to backup and restore database engines using IBM Spectrum Protect.
• IBM Db2 - As a major internally developed product, Db2 contains its own direct connection into the TSM API.
• SQL-Backtrack - A product by BMC Software to back up a variety of database products into TSM.
• STORServer Data Protection for Oracle RDB on OpenVMS - A product by STORServer Inc. to back up Oracle databases on OpenVMS into TSM.
• STORServer Appliance for VMware Consolidated Backup - A product by STORServer Inc. to back up VMware to TSM Servers.
• SPFS - a filesystem for Spectrum Protect - A product by spictera. to backup KVM, PostgreSQL, MongoDB, MariaDB, MySQL, OpenEdge, SAP IQ, SAP ASE, SQL Server, Ingres, FirebirdSQL, Cassandra, SAP HANA, Redis, Oracle, ElasticSearch, IBM Db2, Neo4j, SQL Server Express, SQL Anywhere, Greenplum to Spectrum Protect Servers by mounting the filespaces anywhere on your servers.
• TapeTrack Tape Management Framework - A tape tracking software product that interfaces with TSM to provide end-to-end asset management of tape volumes. Sold by GazillaByte LLC.
• Zmanda Recovery Manager - a MySQL backup product integrated with TSM provided by Zmanda

See also

• List of backup software
• Adstar
• Tape Management System