A Comparison of Payload Align Size for Cryptsetup Over Software RAID-5

Linux Software RAID Performance Comparisons (2012)

The Problem

The cryptsetup command takes an --align-payload parameter that specifies, in sectors, the alignment. The default is 1m.

Motherboard: Supermicro MBD-H8DCL-IF-O
Processors: Two 3.3GHz Opteron 4238 (Socket C32)
RAM: 64GB 1600MHz DDR3 (PC3-12800)
Slots: PCIe x8 (4000MB/s)
Drives: Seagate Barracuda 7200 3000Gbytes ST3000DM001
Drive cage: Supermicro CSE-M35T-1B 5-Bay Enclosure (fits in three 5-inch chassis bays; sells for about $100-$120 from newegg.com)
Debian Wheezy, Linux 3.2.0-3-amd64

Read Percentage: 100% (pure read), 0% (pure write)
Random Percentage: 100% (random)
Thread counts: 1
Small block sizes: 4k, 8k, 16k, 32k, 64k, 128k, 256k, 512k, 1m, 2m, 4m
Large block sizes: 4k, 8k, 16k, 32k, 64k, 128k, 256k, 512k, 1m, 2m, 4m, 8m, 16m, 32m, 64m, 128m, 256m, 512m, 1024m
Targets: All 5 driver were tested simultaneously, as well as each drive individually.
All small block I/Os are issued using the O_DIRECT flag.
All large block I/Os use a sequence of 8KB blocks, followed by an fsync, followed by a seek to the next "large block". This simulates random I/O at small block sizes and sequential I/O at large block sizes. By not using O_DIRECT and calling fsync instead, the Linux block system is tested, which simulated real-world NFS performance.
All tests last 30 seconds.

Recommendation: 2048 (default)