IBM Power9 9009-22G & Flash System 7200 Benchmarks

Same exercise as the V7000 Gen3 run, this time against an IBM Flash System 7200. A Power9 9009-22G drives the all-NVMe array over fibre channel, benchmarked under both traditional RAID 10 and Distributed RAID 6.

The hardware

IBM Power9 9009-22G

• 2x AC power supply — 1400W for server (200–240 VAC)
• 2x mainstream 800GB SSD PCIe3 NVMe U.2 module for AIX/Linux
• Front IBM bezel for 4 NVMe-bay backplane
• Storage backplane with four front PCIe Gen4-capable NVMe U.2 drive slots
• 16x 64GB DDR4 memory DIMM
• PCIe3 LP 2-port 16Gb fibre channel adapter
• PCIe4 LP 32Gb 2-port optical fibre channel adapter
• 2x 8-core POWER9 processor, typical 3.4 to 3.9 GHz (max)
• 16x one-processor-core activation
• PCIe2 LP 2-port 10GbE Base-T adapter
• AIX 7.2.5 SP3
• algorithm set to shortest_queue
• reserve_policy set to no_reserve
• queue_depth set to 128
• j2_dynamicBufferPreallocation set to 64
• pv_pbuf_count set to 1024
• max_xfer_size set to 0x400000

IBM Flash System 7200 (FS7200)

• 2x 8-core Intel Xeon Silver 4208 CPU @ 2.10GHz
• 384 GiB memory
• 32Gb FC 4-port adapter card
• 240GB M.2 boot drive
• 17x 1.92TB NVMe flash drives

Test methodology

The tests were run with XDISK V8.6 for AIX 7.2 using the -ORW flags specific to AIX, to benchmark the storage array directly and bypass the VMM cache. Each layout was put through random and sequential I/O at 8K, 16K, and 1MB block sizes, at 1 and 16 threads, across four read/write profiles: 100% read, 100% write, 50/50 read/write (backup profile), and 80/20 read/write (OLTP database profile).

In the tables below, Read (ms) and Write (ms) are the average latencies for that profile; a — means that side of the workload doesn’t apply.

Traditional RAID 10

Two mdisk devices, 16 drives total plus 1 spare.

8K block size — 1 thread

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	236,587	1,848	0.003	—
Random	100% write	4,015	31.4	—	0.247
Random	50/50	7,832	61.2	0.004	0.248
Random	80/20	18,509	144.6	0.003	0.248
Sequential	100% read	345,942	2,703	0.002	—
Sequential	100% write	3,123	24.4	—	0.319
Sequential	50/50	6,710	52.4	0.003	0.293
Sequential	80/20	17,871	139.6	0.003	0.265

8K block size — 16 threads

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	1,505,683	11,763	0.003	—
Random	100% write	33,141	258.9	—	0.480
Random	50/50	62,571	488.8	0.022	0.485
Random	80/20	151,339	1,182	0.010	0.479
Sequential	100% read	3,581,072	27,977	0.004	—
Sequential	100% write	28,645	223.8	—	0.557
Sequential	50/50	58,762	459.1	0.027	0.515
Sequential	80/20	148,521	1,160	0.010	0.495

16K block size — 1 thread

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	178,765	2,793	0.004	—
Random	100% write	3,966	31.0	—	0.250
Random	50/50	7,289	113.9	0.004	0.265
Random	80/20	17,327	270.7	0.004	0.263
Sequential	100% read	238,870	3,732	0.003	—
Sequential	100% write	3,366	52.6	—	0.295
Sequential	50/50	7,028	109.8	0.005	0.277
Sequential	80/20	17,454	272.7	0.004	0.265

1MB block size — 1 thread

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	5,847	5,847	0.169	—
Random	100% write	4,161	32.5	—	0.238
Random	50/50	756	756.0	0.173	2.40
Random	80/20	1,589	1,589	0.172	2.38
Sequential	100% read	5,215	5,215	0.191	—
Sequential	100% write	410	410.0	—	2.36
Sequential	50/50	767	767.0	0.173	2.36
Sequential	80/20	1,409	1,409	0.200	2.67

1MB block size — 16 threads

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	78,682	78,682	0.201	—
Random	100% write	33,581	262.4	—	0.474
Random	50/50	2,325	2,325	0.616	13.0
Random	80/20	6,610	6,610	0.346	10.6
Sequential	100% read	76,001	76,001	0.210	—
Sequential	100% write	1,351	1,351	—	11.7
Sequential	50/50	2,659	2,659	0.612	11.3
Sequential	80/20	6,602	6,602	0.349	10.6

Distributed RAID 6 (DRAID6)

One mdisk device, 16 drives, 2 rebuild areas, 1 spare.

Pool0

• State: Online
• Capacity: 15.49 TiB / 20.23 TiB (77%)
• Volumes: 1 · MDisks: 1
• Encryption: Not encrypted
• Data reduction: Enabled
• Easy Tier: Balanced
• Extent size: 1.00 GiB
• Free physical capacity: 4.75 TiB
• Reclaimable capacity: 0 bytes

mdisk0

• State: Online
• Capacity: 20.26 TiB
• Pool: Pool0 · Mode: Array · Tier: Tier 0 Flash
• Write protected: No
• Encryption: Not encrypted · Deduplication: Not active
• Fast-Write state: Empty
• Thin-provisioned: No · Supports unmap: Yes
• RAID level: Distributed RAID 6 · Redundancy: 2
• Rebuild areas goal: 2 · Rebuild areas total: 2
• Strip size: 256 KiB

8K block size — 1 thread

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	228,787	1,787	0.003	—
Random	100% write	4,061	31.7	—	0.244
Random	50/50	7,766	60.7	0.004	0.250
Random	80/20	18,293	142.9	0.003	0.252
Sequential	100% read	354,277	2,768	0.002	—
Sequential	100% write	3,065	23.9	—	0.325
Sequential	50/50	6,629	51.8	0.003	0.297
Sequential	80/20	16,971	132.6	0.003	0.278

8K block size — 16 threads

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	1,501,107	11,727	0.003	—
Random	100% write	27,052	211.3	—	0.589
Random	50/50	46,707	364.9	0.027	0.653
Random	80/20	115,231	900.2	0.012	0.635
Sequential	100% read	3,475,273	27,151	0.004	—
Sequential	100% write	25,121	196.3	—	0.635
Sequential	50/50	52,680	411.6	0.027	0.578
Sequential	80/20	123,603	965.6	0.011	0.597

16K block size — 1 thread

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	181,688	2,839	0.004	—
Random	100% write	3,934	30.7	—	0.252
Random	50/50	7,076	110.6	0.004	0.274
Random	80/20	16,432	256.8	0.005	0.275
Sequential	100% read	224,488	3,508	0.004	—
Sequential	100% write	3,184	49.8	—	0.312
Sequential	50/50	6,931	108.3	0.005	0.281
Sequential	80/20	16,945	264.8	0.004	0.274

1MB block size — 1 thread

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	5,741	5,741	0.172	—
Random	100% write	3,989	31.2	—	0.248
Random	50/50	722	722.0	0.175	2.52
Random	80/20	1,532	1,532	0.172	2.49
Sequential	100% read	5,768	5,768	0.173	—
Sequential	100% write	379	379.0	—	2.57
Sequential	50/50	741	741.0	0.190	2.44
Sequential	80/20	1,555	1,555	0.172	2.45

1MB block size — 16 threads

Pattern	Profile	IO/s	MB/s	Read (ms)	Write (ms)
Random	100% read	80,170	80,170	0.197	—
Random	100% write	27,300	213.3	—	0.583
Random	50/50	1,622	1,622	0.779	18.8
Random	80/20	5,137	5,137	0.372	14.0
Sequential	100% read	76,222	76,222	0.209	—
Sequential	100% write	1,173	1,173	—	13.5
Sequential	50/50	2,168	2,168	0.671	14.0
Sequential	80/20	5,192	5,192	0.396	13.7

Takeaways

• As the block size and thread count go up, so does performance — up to a point.
• The hardware delivers high performance without much tweaking.
• DRAID6 is the recommended RAID type, and is suggested over traditional RAID 10 even for critical workloads. If you’re concerned about disk failures, configure it for two rebuild areas, which lets you lose two disks.
• NPIV is enabled by default.
• The latest StorWize codebase allows DRAID6 pools to be expanded.
• It was not possible to assign a stand-by hot spare in a DRAID6 configuration — you can still mark an SSD as a spare, but the software will not automatically consume it if a disk fails.
• No issues failing one of the active/active storage nodes and monitoring the events inside AIX; everything worked as expected in a redundant-path host and storage configuration.
• Technical documentation for FC-NVMe configuration (instead of FC/SCSI) on the FS7200 and/or AIX was unavailable at the time of writing.
• The array has two clustered, hot-swappable node canisters containing hot-swappable fan modules, memory DIMMs, batteries, and PCIe adapters. The enclosure also houses two redundant, hot-swappable AC power supplies, and concurrent code load keeps applications online during firmware upgrades to all components, including the flash drives.