IBM Power9 9009-22A & StorWize V7000 Gen3 Benchmarks

Two storage layouts on the same all-NVMe StorWize V7000 Gen3, driven by an IBM Power9 9009-22A over 16Gb fibre channel: traditional RAID 10 and Distributed RAID 6. Below is the full XDISK run across block sizes, thread counts, and the usual read/write profiles.

The hardware

IBM Power9 9009-22A

• 512GB DDR4 memory
• 2x 8-core Power9 processor
• 2x 400GB NVMe M.2 SSD for rootvg
• 16Gb FC 2-port adapter pair
• FW910.20 (VL910_122) firmware
• AIX 7.2 ML3
• AIX_AAPCM MPIO driver
• algorithm set to shortest_queue
• reserve_policy set to no_reserve
• queue_depth set to 32

IBM StorWize V7000 Gen3

• 17x 1.92TB 2.5” NVMe flash drive
• 16Gb FC 4-port adapter pair
• 120GB M.2 boot drive pair
• Spectrum Virtualize 8.2.1.1

Test methodology

The tests were run with XDISK V8.6 for AIX 7.2 using the -ORW flags specific to AIX, to benchmark the storage array directly and bypass the VMM cache. Each layout was put through the same matrix:

• Random and sequential I/O at 8K, 16K, and 1MB block sizes
• 1 thread and 16 threads
• 100% read, 100% write, 50/50 read/write (backup profile), and 80/20 read/write (OLTP database profile)

In the tables below, Read (ms) and Write (ms) are the average latencies reported for that profile; a — means that side of the workload doesn’t apply (e.g. no writes in a 100% read run).

Traditional RAID 10

Two mdisk devices, 16 drives total plus 1 spare.

8K block size — 1 thread

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	236,587	1,848	0.003	—
Random	100% write	4,015	31.4	—	0.247
Random	50/50	7,832	61.2	0.004	0.248
Random	80/20	18,509	144.6	0.003	0.248
Sequential	100% read	345,942	2,703	0.002	—
Sequential	100% write	3,123	24.4	—	0.319
Sequential	50/50	6,710	52.4	0.003	0.293
Sequential	80/20	17,871	139.6	0.003	0.265

8K block size — 16 threads

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	1,505,683	11,763	0.003	—
Random	100% write	33,141	258.9	—	0.480
Random	50/50	62,571	488.8	0.022	0.485
Random	80/20	151,339	1,182	0.010	0.479
Sequential	100% read	3,581,072	27,977	0.004	—
Sequential	100% write	28,645	223.8	—	0.557
Sequential	50/50	58,762	459.1	0.027	0.515
Sequential	80/20	148,521	1,160	0.010	0.495

16K block size — 1 thread

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	178,765	2,793	0.004	—
Random	100% write	3,966	31.0	—	0.250
Random	50/50	7,289	113.9	0.004	0.265
Random	80/20	17,327	270.7	0.004	0.263
Sequential	100% read	238,870	3,732	0.003	—
Sequential	100% write	3,366	52.6	—	0.295
Sequential	50/50	7,028	109.8	0.005	0.277
Sequential	80/20	17,454	272.7	0.004	0.265

1MB block size — 1 thread

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	5,847	5,847	0.169	—
Random	100% write	4,161	32.5	—	0.238
Random	50/50	756	756.0	0.173	2.40
Random	80/20	1,589	1,589	0.172	2.38
Sequential	100% read	5,215	5,215	0.191	—
Sequential	100% write	410	410.0	—	2.36
Sequential	50/50	767	767.0	0.173	2.36
Sequential	80/20	1,409	1,409	0.200	2.67

1MB block size — 16 threads

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	78,682	78,682	0.201	—
Random	100% write	33,581	262.4	—	0.474
Random	50/50	2,325	2,325	0.616	13.0
Random	80/20	6,610	6,610	0.346	10.6
Sequential	100% read	76,001	76,001	0.210	—
Sequential	100% write	1,351	1,351	—	11.7
Sequential	50/50	2,659	2,659	0.612	11.3
Sequential	80/20	6,602	6,602	0.349	10.6

Distributed RAID 6 (DRAID6)

One mdisk device, 16 drives, 2 rebuild areas, 1 spare.

Pool0

• State: Online
• Capacity: 15.49 TiB / 20.23 TiB (77%)
• Volumes: 1 · MDisks: 1
• Encryption: Not encrypted
• Data reduction: Enabled
• Easy Tier: Balanced
• Extent size: 1.00 GiB
• Free physical capacity: 4.75 TiB

mdisk0

• State: Online
• Capacity: 20.26 TiB
• Pool: Pool0 · Mode: Array · Tier: Tier 0 Flash
• Write protected: No
• Encryption: Not encrypted · Deduplication: Not active
• Fast-Write state: Empty
• Thin-provisioned: No · Supports unmap: Yes
• RAID level: Distributed RAID 6 · Redundancy: 2
• Rebuild areas goal: 2 · Rebuild areas total: 2
• Strip size: 256 KiB

8K block size — 1 thread

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	228,787	1,787	0.003	—
Random	100% write	4,061	31.7	—	0.244
Random	50/50	7,766	60.7	0.004	0.250
Random	80/20	18,293	142.9	0.003	0.252
Sequential	100% read	354,277	2,768	0.002	—
Sequential	100% write	3,065	23.9	—	0.325
Sequential	50/50	6,629	51.8	0.003	0.297
Sequential	80/20	16,971	132.6	0.003	0.278

8K block size — 16 threads

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	1,501,107	11,727	0.003	—
Random	100% write	27,052	211.3	—	0.589
Random	50/50	46,707	364.9	0.027	0.653
Random	80/20	115,231	900.2	0.012	0.635
Sequential	100% read	3,475,273	27,151	0.004	—
Sequential	100% write	25,121	196.3	—	0.635
Sequential	50/50	52,680	411.6	0.027	0.578
Sequential	80/20	123,603	965.6	0.011	0.597

16K block size — 1 thread

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	181,688	2,839	0.004	—
Random	100% write	3,934	30.7	—	0.252
Random	50/50	7,076	110.6	0.004	0.274
Random	80/20	16,432	256.8	0.005	0.275
Sequential	100% read	224,488	3,508	0.004	—
Sequential	100% write	3,184	49.8	—	0.312
Sequential	50/50	6,931	108.3	0.005	0.281
Sequential	80/20	16,945	264.8	0.004	0.274

1MB block size — 1 thread

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	5,741	5,741	0.172	—
Random	100% write	3,989	31.2	—	0.248
Random	50/50	722	722.0	0.175	2.52
Random	80/20	1,532	1,532	0.172	2.49
Sequential	100% read	5,768	5,768	0.173	—
Sequential	100% write	379	379.0	—	2.57
Sequential	50/50	741	741.0	0.190	2.44
Sequential	80/20	1,555	1,555	0.172	2.45

1MB block size — 16 threads

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	80,170	80,170	0.197	—
Random	100% write	27,300	213.3	—	0.583
Random	50/50	1,622	1,622	0.779	18.8
Random	80/20	5,137	5,137	0.372	14.0
Sequential	100% read	76,222	76,222	0.209	—
Sequential	100% write	1,173	1,173	—	13.5
Sequential	50/50	2,168	2,168	0.671	14.0
Sequential	80/20	5,192	5,192	0.396	13.7

Takeaways

• As the block size and thread count go up, so does performance — up to a point.
• DRAID6 looks like a strong alternative to traditional RAID 10, even for critical workloads. Anyone moving from spinning-disk RAID 10 arrays to NVMe SSD arrays should seriously weigh the DRAID6 option.
• Failover testing of the active/active storage nodes worked as expected with redundant pathing.
• It was not possible to assign a stand-by hot spare in a DRAID6 configuration.
• Technical documentation for FC-NVMe configuration (instead of FC/SCSI) on Gen3 and/or AIX was unavailable at the time of writing.