A Comparison of Cache Size for Software RAID-5

Linux Software RAID Performance Comparisons (2012)

The Problem

Various web sites recommend setting /sys/block/md0/md/stripe_cache_size to 32768 for improved read performance without providing compelling evidence that changing the default from 256 has broad benefits.

The Controller

The Test System

The Test Matrix

Conclusion

As expected, the setting of stripe_cache_size did not impact read performance. For random small blocks, a smaller cache was was helpful, providing a 54% improvement over the suggested 32768. However, the absolute performance difference at this point was under 100KB/s.

For large sequential writes, however, the default setting of 256 limited throughput to about 200MB/s, while larger cache sizes allowed thoughput to scale with write size, with a 59% improvement for 1GB writes using 32768 instead of 256.

(Not shown here, but availabe in the raw data, the difference for large sequential writes was smaller with smaller chunk sizes and larger with larger chunk sizes; with similar anomalies for the 64MB point; and without significant change in peak performance.)

Recommendation: 32768

Small Block Tests Using Default Chunk Size



Large Block Tests Using Default Chunk Size



Small Block Tests Using Legacy Default Chunk Size



Large Block Tests Using Legacy Default Chunk Size