Storage Benchmarking: Repeatable & Comparable

Your goal when benchmarking should be to ensure that the results are both continuously repeatable and fairly comparable to other attempts (with or without changes to the system components or configuration). While getting the biggest possible number is a fun and favoured activity of those creating storage data sheets, for bonus points we'll also try to get something more closely approximating your real workloads. Benchmarking of any kind often presents tricky business, but storage has always presented particularly difficult challenges as even Hard Drives have interesting performance characteristics that vary greatly depending on workload. You might hope that was solved by SSDs, and that is true to an extent for real workloads, but they tend to give even more misleading results during synthetic benchmarks. Our case study is benchmarking both the individual components and over-all performance of a distributed Ceph storage cluster. Don't worry if you're not using Ceph, the majority of the content will apply equally well to any storage system, but if you are then there are a few minutes of specific traps for Ceph clusters that we'll cover. You'll walk away with an understanding of the many different factors getting in the way of great benchmarks including - Working set size - Bi-modal SSD performance due to flash block management - Thin provisioning - Bandwidth limitations of SSDs, Backplanes, PCIe buses, CPUs, Memory and Networks - Measuring and identifying the actual bottleneck you are hitting - Filesystems - Caches - Benchmarking tool configuration