The Storage Networking Industry Association this month announced the release of a specification that can be used to test the performance of consumer (client) solid-state drives (SSDs) regardless of the vendor.
The specification, for the first time, creates a level playing field for determining consumer SSD performance.
The SNIA's announcement follows the release of a specification earlier this year for testing the performance of data center -class SSDs.
Like its Enterprise Performance Test Specification, the new Client Performance Test Specification defines a set of device-level tests and methodologies intended to enable comparative testing of SSD devices regardless of the manufacturer.
Previously, there has been no widely accepted test methodology for measuring SSD device performance. Each SSD manufacturer used different measurement methodologies to derive performance specs for their products, and many used unreliable methods such as mean time before failure (MTBF).
The SNIA, an industry trade group of vendors and universities that develops and promotes standards for storage systems, said its Solid State Storage Initiative (SSSI) group came up with the SSD Performance Test Specification to level the playing field in benchmark testing.
Paul Wassenberg, chairman of the SSSI Governing Board said the specification for testing consumer SSDs is "90 per cent" the same as the enterprise specification.
The main difference between the two specifications is that consumer or client-side SSDs aren't required to be fully "pre-conditioned" prior to testing, Wassenberg said. Pre-conditioning an SSD refers to the practice of writing data to an SSD and then erasing it before testing performance.
SSDs display different performance results when they're brand new as compared to when they've been used for a time. That's because once an SSD's capacity is fully utilized, firmware then begins what's known as an erase-write cycle, where old data on an SSD must first be marked for deletion prior to new data being written. The process can significantly reduce performance.
Wassenberg said SSD manufacturers argued to his committee that consumer SSDs are not typically written to as much as enterprise-class SSDs, which can be saturated with data thousands of times throughout a lifespan. Therefore, the consumer SSD specification should only require that half the capacity of an SSD be written to or "pre-conditioned" prior to testing.
To date, about 27 SSD manufacturers - less than half of the industry -- have given the specification their stamp of approval. But, like the enterprise-class spec, Wassenberg expects all manufacturers will eventually be on board.
While the specification may be approved by manufacturers, it's not something the average consumer will be able to use. The specification was created as a tool for system manufacturers, such as laptop and PC makers. The specification requires a sophisticated test bed to be set up.
Test beds using the specification can be achieved in several ways. A device manufacturer can simply go to a vendor, such as Calypso Systems, and purchase a pre-assembled hardware test platform.
"That's a system where you can plug in multiple SSDs and run a test and boom you get a report," Wassenberg said.
Test vendors such as Calypso also charge to test SSD for manufacturers and others.
A consumer could also spend thousands of dollars to set up anSSD test bed by purchasing the proper equipment and software.
"The specification is like a recipe," Wassenberg said. "It doesn't require specific hardware or software but it tells you what to do. At least a few people have written Iometer scripts. You execute them like a macro and get the results. You can use any motherboard you want as long as it's fast enough that it doesn't slow down the SSD.
"Client SSDs are different in how they're used and a little different in how they're designed," he said. "Certainly an enterprise-class SSD has more internal data checking and [error detection code]. Enterprise data is priority number one. On a PC, if you lose a few frames out of a movie, it's annoying, but not earth shattering."
Lucas Mearian covers storage, disaster recovery and business continuity, financial services infrastructure and health care IT for Computerworld. Follow Lucas on Twitter at @lucasmearian or subscribe to Lucas's RSS feed. His e-mail address is lmearian@computerworld.com.
Read more about storage in Computerworld's Storage Topic Center.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.