Synthetic Load Tests vs. Using Actual Production Clients

By Jim Bahn, Senior Director, Product Marketing

In a perfect world with limitless resources, we would do storage performance testing using the same equipment and applications as we expect to use in production. But in the real world, it’s simply not feasible, primarily due to the scale found in production.

Both synthetic models and “real” models are meant to help predict how workloads will behave in production environments, by testing.   And we know that many organizations do some level of testing on a small scale, but results at full scale are merely extrapolations.  Although that’s better than pure guesswork, real world results don’t change linearly.  To use an analogy, it’s like saying that your family sedan runs fine for many hours at 20, 40, 60, and 80 miles per hour, so, by extrapolation if you run it at 160 mph for many hours, it will work just as fine.  We know that’s unlikely to be the case with your family sedan.

So in practice, we use synthetic workloads, and load generators that can accurately simulate the I/O profile and scale as you would find in the production datacenter.  Using industry best practices, storage engineers routinely achieve test results that exceed 95% accuracy and, in many cases, show 99% accuracy.  In the figure below, the IOPS from a production monitor is overlaid with the same metric using the Load DynamiX Enterprise product (labelled LDX).  The two are nearly identical.

Even a few years ago, this level of accuracy was either unachievable, or only achievable with painfully complex and impractical scripting. And even then, as noted above, lab environments simply don’t exist at the scale of real-world environments.  But because today’s best synthetic workload generators can handle scale, temporality, metadata, burstiness, hotspot drift, compression and dedupe, you can depend them for realistic test results to make informed decisions on storage deployments.  This means eliminating unnecessary overprovisioning and never running into performance headroom limits.

Call us and ask for a demo, with YOUR application workload.