|2009 Mac Pro 4-Drive RAID Benchmarks|
RocketRAID 4320 vs Onboard SATA (& Intel X25M SSD)
Posted: Oct. 12th, 2009
(from a reader mail - updated 1PM ET for info on OS X version, X25 gen info and QB cache setting)
Some stats on RAID performance in a 2009 Mac Pro
Thought you might find some of the following nuMBers interesting.
I'm using an early 2009 Mac Pro, 8-core 2.26GHz, with 24GB RAM and 4 new (i.e. fixed firmware) Seagate Barracuda 1.5TB 7200rpm drives. I also tested against a (2nd generation) Intel X-25 SSD (my boot drive), all using QuickBench (System Cache enabled) under OS X 10.5.8.
I've tested two RAID solutions: the Highpoint RocketRAID 4320, and Apple's Software RAID. I'm not interested in a $700 card that doesn't handle RAID-10.
First off, every solution is horrible at tiny reads and writes, but I imagine this is typical of most RAID setups. So rather than list out everything, I'll give nuMBers for five test case: 8K, 1M, 10M, 100M, 1G. Also, random reads and writes were universally bad in every case, but for the SSD (which is 20x faster in that area for small reads/writes, but on par for large ones).
RocketRAID 4320, RAID-10 (4 drives, 64k block size)
- 8K Read 97M/s Write 50M/s
- 1M Read 134M/s Write 703M/s (this is cache talking)
- 10M Read 280M/s Write 276M/s
- 100M Read 308M/s Write 224M/s
- 1G Read 186M/s Write 215M/s (this is an average over 10 times)
RocketRAID 4320, RAID-0 (4 drives, 64k block size)
- 8K Read 94M/s Write 61M/s
- 1M Read 482M/s Write 702M/s (this is cache talking, again)
- 10M Read 498M/s Write 742M/s (cache)
- 100M Read 466M/s Write 418M/s
- 1G Read 478M/s Write 440M/s (this is an average over 10 times)
Now, given that the drives themselves have a rated sustained throughput of 120MB/s, I expect to see 480MB/s for both read and write. The RocketRAID is underperforming, which surprised me.
Intel X-25 SSD 160GB (RocketRAID card connected)
- 8K Read 82M/s Write 64M/s
- 1M Read 245M/s Write 83M/s
- 10M Read 263M/s Write 85M/s
- 100M Read 264M/s Write 85M/s
- 1G Read 263M/s Write 84M/s (this is an average over 10 times)
One thing to be said for SSDs: When you hit their transfer limit, they're very consistent about it no matter what the data size is. Write speeds in nearly every sequential category averaged at 82MB/s, and reads at 262MB/s. (write speeds will drop over time/longer term usage w/o GC or Trim) Almost, but not quite, hitting the SATA throughput limit. This drive is mounted in the machine's second optical bay, using a 2.5->5.25 mounting bracket.
Seagate Barracuda 7200rpm, JBOD 1-disk
- 8K Read 60M/s Write 45M/s
- 1M Read 133M/s Write 131M/s
- 10M Read 130M/s Write 130M/s
- 100M Read 123M/s Write 122M/s
- 1G Read 126M/s Write 124M/s (this is an average over 10 times)
A single drive without RAID is also very consistent (and horrible at small random reads, with a throughput rate as low as 0.7MB/s!). (this is common as xbench users have seen for years with std HDs in the random 4KB tests) But here we're seeing the advertised 120MB/s, even a little bit better.
Software RAID-1, (64k block size)
- 8K Read 40M/s Write 42M/s
- 1M Read 124M/s Write 128M/s
- 10M Read 130M/s Write 130M/s
- 100M Read 124M/s Write 124M/s
- 1G Read 125M/s Write 121M/s (this is an average over 10 times)
There's a slight penalty for mirroring, but it's almost disappears as the transfer sizes go up. Plus, the CPU hit was barely noticeable. On average the kernel was taking up 12-20% (but nearer to 12) of a single core.
Software RAID-1, (32k block size)
- 8K Read 30M/s Write 39M/s
- 1M Read 120M/s Write 133M/s
- 10M Read 130M/s Write 134M/s
- 100M Read 125M/s Write 129M/s
- 1G Read 125M/s Write 122M/s (this is an average over 10 times)
I didn't expect that lowering the block size (and thus increasing the RAID complexity for the software driver) would increase the write speed slightly.
As for the RAID-0 times (all 64k block size), I won't even bother listing most of them because they aren't interesting (they scale pretty linearly up from the SATA drive times). I will show the 4 drive numbers:
Software RAID-0, (4 drives, 64k block size)
- 8K Read 47M/s Write 36M/s
- 1M Read 439M/s Write 485M/s
- 10M Read 510M/s Write 468M/s
- 100M Read 460M/s Write 486M/s
- 1G Read 485M/s Write 473M/s (this is an average over 10 times)
That 480 is pretty much what I expected to see. What's odd is that these times beat the RocketRAID at the same block size! I might have expected a match (since no real processing is going on), but not a win for Software RAID.
And finally, RAID-10 in Software, composed by striping two mirrored sets:
Software RAID-10, (32k block size)
- 8K Read 31M/s Write 34M/s
- 1M Read 105M/s Write 266M/s
- 10M Read 226M/s Write 240M/s
- 100M Read 246M/s Write 254M/s
- 1G Read 242M/s Write 238M/s (this is an average over 10 times)
Let me just note here that these numbers are only slightly higher than 2-drive RAID-0 (with 1G Read 248M/s Write 243 M/s). This is not bad at all, and right at the expected throughput of 240M/s. It seems that mirroring imposes almost no penalty at all.
Now what's really strange to me is that Software RAID-10 runs rings around the RocketRAID 4320, which is packing 256M of cache and a 1.2GHz Intel Xscale processor. I'm beginning to think something is wrong with this card, and perhaps that's why it causes my Leopard machine (10.5.8) to freeze up every other day (and hence motivated me to switch to software RAID for the time being).
I'm including the data exports for all the timing tests as attached Excel spreadsheets.