Think Time

Up to 70 percent of recovery time consists of think time – the time spent determining what to recover, how to recover, when to recover and where to recover.

By Jeffrey Peltier

Conjure an image of a person deep in thought, and you will most likely recall the universally recognized sculpture by Auguste Rodin, The Thinker. Every aspect of the figure’s pose suggests deep contemplation. Hunched forward, hand to chin, he leans into a position of quiet repose and long thought. Time stops for The Thinker.

Time, however, does not stop for the data professional in today’s non-stop, ever-changing world. Time costs.

To remain competitive, companies must meet the ever-encompassing challenges of e-commerce and globalization. Customers, accustomed to faster and faster response times, will not wait. With the competition just a click away, an unavailable database spells serious financial repercussions. Success depends upon maintaining database availability. A key component to maintaining availability is the capability to perform reliable, rapid recovery.

This is where "think time" comes in. Recovery is never simple or without consequences. It is necessary more often than you think, and it is complex.

Up to 70 percent of recovery time consists of think time – the time spent determining what to recover, how to recover, when to recover and where to recover.  IT recovery teams must find new ways to help reduce this time, thereby increasing availability. By automating recovery procedures, think time can be significantly reduced, if not eliminated.  Automation can save both time and manpower – a powerful combination that is critical when an organization’s overall goal is to maintain non-stop application availability.

The Challenge: Application Assets

Long gone are the days of the standalone application being associated with one database. The sharing of data, interrelationships of applications and merging of companies have yielded more application assets associated with each application. Keeping track of such large quantities of objects is a daunting task for even the best IT professionals. With the association of multiple databases within an application, even a simple logical error on one database can result in needing to recover multiple databases to a point of consistency.

With the advent of storage pools and storage area networks (SANs), even the exact location of databases takes time to determine. Although storage devices are extremely reliable, a storage device loss still happens from time to time. If a storage device is lost, it takes think time to account for all the data that was stored on that particular device. In restoring that single device, all related application assets on other storage devices may need to be recovered back to a point of consistency. Again, think time is needed to determine if and how many related application assets will need to be recovered.

In addition to the challenges presented by multiple applications, multiple databases and increased storage devices, the dynamically changing database problem also has appeared. To handle the large increases in data and the limited database sizes, new dynamic database object creation/deletion has been introduced. As applications detect a database object overload or imbalance situation, new database objects are created or database objects are deleted as necessary. This presents a problem for standardized recovery procedures. Since database objects are being dynamically created or deleted, the actual names of these objects cannot be preinserted into recovery procedures. The names of the database objects that need to be recovered must be inserted at the time of recovery.

Traditional Recovery Methods
After figuring out what needs to be recovered, determining the correct and most efficient recovery method will involve additional think time. For example, the simplistic methodology of restoring an entire storage device from a previous backup is not an acceptable solution for most companies. The backup could result in losing days, weeks or months of processing. With the speed at which e-business generates new and updated data, a day’s loss could be worth millions of dollars to a company. New rapid recovery methods such as online transaction recovery, with undo and redo abilities, have been added to the application and database recovery toolkit. Recovery teams can now determine the best recovery method, while simultaneously weighing in the amount of data lost with the time required to recover the information.

Recovery Testing
Companies continue to spend hundreds of thousands of dollars or more on recovery testing sites and limited testing time. But many are finding that after years of testing procedures, they still have not had a completely successful test. Perhaps one or two applications were recovered, but all the associated databases or control files required to allow end users full usage were not recovered. At other times, all databases were recovered, but not the applications. That’s fine, but in a real-world situation end users cannot do business without the applications being intact as well.

The root cause of most problems in recovery testing can be traced back to think time. Before the recovery date and time arrive, recovery teams spend time thinking about what and who will be needed at the recovery site. When the time comes for a recovery test to start, recovery teams spend time thinking about:

• What should be recovered first, second, third, etc.?
• Are all the required recovery assets available at the test site?
• Are there enough resources (tape drives, storage devices, etc.) available?
• Has everyone arrived who is needed for the test?

If there were unlimited funds and unlimited testing time available, eventually the recovery test would be successful. But, in reality, unlimited resources do not exist. As a result, new procedures must be used that provide recovery testing, as well as actual recoveries, and have the ability to reduce think time.

The Solution: Automated Recovery

One answer to the think time dilemma is to automate recovery procedures. An automated recovery procedure will:
• Automatically identify all associated database objects with the appropriate applications.
• If required, find a point of consistency.
• Generate the required procedure with all correct database asset names.
• Allow prerecovery analysis, simulation and asset validation.

The lost knowledge of an application asset location caused by movement of database objects among the multitude of storage devices can be overcome by automation. The recovery teams can have an automated approach to recovering a storage device and confidence in knowing what assets were on that device. As the database objects are moved, the automated recovery management process will record the movements.

As new database objects are dynamically added or deleted, the automated recovery management process will be able to recognize this. At the time of recovery, the recovery team will not spend time looking for database objects that do not exist or miss new database objects that have been created.

A company’s expenditure on testing sites and testing time can deliver a greater return on investment if automated recovery management procedures are used. Before the actual test date has arrived, recovery teams can perform simulation and audits on recovery assets to ensure everything will be available as well as usable.

Conclusion

Automation reduces think time and provides effective and rapid recovery.   Through automation, the time spent determining what to recover, how to recover, at what point to recover and where to recover can be reduced. The reduction in think time will allow companies to provide greater availability and satisfaction to their customers and, in turn, generate greater financial returns with an increase in customer satisfaction.

About the Author
Jeffrey Peltier is a Director for Product Management and Development in the Data Management business unit for BMC Software, Inc.  With more than 19 years of experience in the high-technology software industry, Peltier is able to utilize his varied skills and work experiences while bringing a customer perspective to BMC Software’s strategic solutions.  Peltier has a wide variety of knowledge across DB2, IMS, VSAM, OS/390 and e-business environments. BMC Software is headquartered in Houston, Texas, with offices worldwide. For more information please visit BMC Software's Web site at www.bmc.com, or call 800-841-2031 or 713-918-8800.