Document management: figure out how much storage you need

You probably know your company shouldn't save every bit of data. Given regulatory requirements and the role that electronic records play in lawsuits, some enterprises save everything just to be safe. Yet, "more companies are sensitive to the fact that we can't just keep throwing storage at the issue," says IDC analyst Rick Villars.

Storage is cheap, but it adds up. IDC, a sister company to CIO, found a 27 percent drop in the cost of disk storage between 2007 and 2008. Nevertheless, worldwide spending on storage reached US$80 billion last year. Furthermore, if your company gets sued, or even called as a witness, experts say it can cost millions to identify needed documents--$250 to $600 per hour for the work and $150 to $1,900 per gigabyte for the software.

It's better to figure out what you really need to save (there's software to help) and buy only the storage you need, says Enterprise Strategy Group analyst Brian Babineau.

What You Keep and Why

Tucson Electric Power (TEP) has procedures to carry out data retention policies set by its legal department. This helps the utility company contain its need for storage--currently 200TB and increasing, says Chris Rima, supervisor of infrastructure systems.

For starters, TEP doesn't mix backup with archiving. Analysts say this practice is critical because combining the two--saving temporary records along with files that must be kept for years--wastes capacity. TEP backs up 30 days of data for disaster recovery, while separately archiving e-mail and some other files for seven years.

Separated from backup and done with specialized tools, the archiving process runs more smoothly, Babineau says. Vendors offer applications that analyze data for backup and archiving based on corporate retention policies. TEP uses its own custom software.

Addressing e-discovery--the practice of collecting electronic evidence--is tougher. As soon as a company can reasonably anticipate it will be sued, it has to hold on to any records related to the allegations, says Wendy Curtis, special counsel for e-discovery at Orrick, Herrington & Sutcliffe. You have to be able to separate those from the rest of the data you routinely purge.

For about six months, TEP has used Symantec's Discovery Accelerator software to analyze e-mail for e-discovery and is now expanding its use to all unstructured data in its network-based storage. TEP is also incorporating e-mail from user-created Exchange folders on PCs into corporate network storage, banning the use of Exchange folders to make e-mail retention consistent, Rima says.

Having good backup, retention and e-discovery strategies helps TEP know how much storage it needs. Once an enterprise has these rules and mechanisms worked out, it can begin investigating products, says Ovum analyst Tim Stammers.

Stammers suggests a strategy involving data deduplication, which takes common elements of documents that have many copies and saves them once to cut the amount of storage capacity you need. Storing data in the cloud, through services offered by such vendors as Amazon.com and Nirvanix, may also be more economical because you won't need in-house staff to manage the systems.

This story includes information originally reported in the article "Why IT Should Start Throwing Data Away."

Stephen Lawson is senior U.S. correspondent with IDG News Service.