A Comparison of Various NFS Configurations
Linux Software RAID Performance Comparisons
The Problem
For a dedicated file server, how much difference does DRAM speed,
CPU speed, and NFS rsize/wsize settings make, given an AES-256
encrypted RAID-5.
The Controller
- Supermicro AOC-SAT2-MV8
- 8 SATA II ports
- 64-bit PCI-X133MHz (tested in 32-bit PCI slot)
- Chipset: MV88SX6081
- Linux driver: sata_mv
- Cost: $99.99 from newegg.com
- Configuration: JBOD
The Test System
- Motherboard: Asus K8N-DRE
- Processors: Two 1.8GHz Opteron 244 OR two 2.4GHz
Opteron 250
- RAM: 8GB DDR400 running at 333MHz
OR 4GB DDR400
running at 400MHz
- Slots: 1 32-bit PCI; 1 PCI Express x16
- Drives: Seagate Barracuda 7200.11 1000Gbytes ST31000333AS
- Drive cage: Supermicro CSE-M35T-1B 5-Bay Enclosure (fits in
three 5-inch chassis bays; sells for about $100-$120 from
newegg.com)
The Test Matrix
- Read Percentage: 100% (pure read), 50% (mixed), 0% (pure write)
- Random Percentage: 0% (sequential), 100% (random)
- Thread counts: 1 and 20
- Block sizes: 4k, 8k, 32k, 64k, 256k, 1m, 4m, 8m, 64m
- Targets: All 5 driver were tested simultaneously, as well as
each drive individually.
- All I/Os are issued using the O_DIRECT flag.
- All tests last 30 seconds.
- To invalidate both the server and the client cache, between
each test the remote NFS mount was unmounted on the client, the NFS
server was stopped on the server, and the local file system was
unmounted on the server. Without these steps, we would have
only been measuring cache effects.
- The server mounted the encrypted file system with
noatime,nodiratime, the server started nfs-kernel-server without
any special configuration, and the client mounted the exported
file system with noatime and with rsize and wsize set to the
specified values.
Conclusion
Data for a change in DRAM speed without a change in CPU speed is
limited. However, when data is available, this change makes little
difference. We expect that larger main store may improve long-term
performance when reads are cached, and may help to smooth a spikey
write workload that is cached before being written to disk. Hence,
using 8GB of 333MHz DRAM may be generally better than using 4GB of
400MHz DRAM.
CPU speed made little difference, except in the pure sequential read
tests, where the improvement was over 20% for larger I/O sizes. The
difference was less striking with single-threaded reads under 64KB,
suggesting the effects are purely from decryption speed.
For single threaded tests, larger [rw]size made no difference for
small I/O sizes and slighly hurt performance for larger I/O sizes
(above 32KB). For multi-threaded tests, larger [rw]sizes improved
performance for larger I/O sizes.
Sequential Single Threaded Tests
Random Single Threaded Tests
Sequential Multi-Threaded Tests
Random Multi-Threaded Tests
