skip to content
mirza@edge
← cd ..
ibmpower9storwizev7000aix2019-01-08 · 4 min read

IBM Power9 9009-22A & StorWize V7000 Gen3 Benchmarks

Two storage layouts on the same all-NVMe StorWize V7000 Gen3, driven by an IBM Power9 9009-22A over 16Gb fibre channel: traditional RAID 10 and Distributed RAID 6. Below is the full XDISK run across block sizes, thread counts, and the usual read/write profiles.

The hardware

IBM Power9 9009-22A

  • 512GB DDR4 memory
  • 2x 8-core Power9 processor
  • 2x 400GB NVMe M.2 SSD for rootvg
  • 16Gb FC 2-port adapter pair
  • FW910.20 (VL910_122) firmware
  • AIX 7.2 ML3
  • AIX_AAPCM MPIO driver
  • algorithm set to shortest_queue
  • reserve_policy set to no_reserve
  • queue_depth set to 32

IBM StorWize V7000 Gen3

  • 17x 1.92TB 2.5” NVMe flash drive
  • 16Gb FC 4-port adapter pair
  • 120GB M.2 boot drive pair
  • Spectrum Virtualize 8.2.1.1

Test methodology

The tests were run with XDISK V8.6 for AIX 7.2 using the -ORW flags specific to AIX, to benchmark the storage array directly and bypass the VMM cache. Each layout was put through the same matrix:

  • Random and sequential I/O at 8K, 16K, and 1MB block sizes
  • 1 thread and 16 threads
  • 100% read, 100% write, 50/50 read/write (backup profile), and 80/20 read/write (OLTP database profile)

In the tables below, Read (ms) and Write (ms) are the average latencies reported for that profile; a — means that side of the workload doesn’t apply (e.g. no writes in a 100% read run).

Traditional RAID 10

Two mdisk devices, 16 drives total plus 1 spare.

8K block size — 1 thread

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 236,587 1,848 0.003 —
Random 100% write 4,015 31.4 — 0.247
Random 50/50 7,832 61.2 0.004 0.248
Random 80/20 18,509 144.6 0.003 0.248
Sequential 100% read 345,942 2,703 0.002 —
Sequential 100% write 3,123 24.4 — 0.319
Sequential 50/50 6,710 52.4 0.003 0.293
Sequential 80/20 17,871 139.6 0.003 0.265

8K block size — 16 threads

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 1,505,683 11,763 0.003 —
Random 100% write 33,141 258.9 — 0.480
Random 50/50 62,571 488.8 0.022 0.485
Random 80/20 151,339 1,182 0.010 0.479
Sequential 100% read 3,581,072 27,977 0.004 —
Sequential 100% write 28,645 223.8 — 0.557
Sequential 50/50 58,762 459.1 0.027 0.515
Sequential 80/20 148,521 1,160 0.010 0.495

16K block size — 1 thread

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 178,765 2,793 0.004 —
Random 100% write 3,966 31.0 — 0.250
Random 50/50 7,289 113.9 0.004 0.265
Random 80/20 17,327 270.7 0.004 0.263
Sequential 100% read 238,870 3,732 0.003 —
Sequential 100% write 3,366 52.6 — 0.295
Sequential 50/50 7,028 109.8 0.005 0.277
Sequential 80/20 17,454 272.7 0.004 0.265

1MB block size — 1 thread

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 5,847 5,847 0.169 —
Random 100% write 4,161 32.5 — 0.238
Random 50/50 756 756.0 0.173 2.40
Random 80/20 1,589 1,589 0.172 2.38
Sequential 100% read 5,215 5,215 0.191 —
Sequential 100% write 410 410.0 — 2.36
Sequential 50/50 767 767.0 0.173 2.36
Sequential 80/20 1,409 1,409 0.200 2.67

1MB block size — 16 threads

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 78,682 78,682 0.201 —
Random 100% write 33,581 262.4 — 0.474
Random 50/50 2,325 2,325 0.616 13.0
Random 80/20 6,610 6,610 0.346 10.6
Sequential 100% read 76,001 76,001 0.210 —
Sequential 100% write 1,351 1,351 — 11.7
Sequential 50/50 2,659 2,659 0.612 11.3
Sequential 80/20 6,602 6,602 0.349 10.6

Distributed RAID 6 (DRAID6)

One mdisk device, 16 drives, 2 rebuild areas, 1 spare.

Pool0

  • State: Online
  • Capacity: 15.49 TiB / 20.23 TiB (77%)
  • Volumes: 1 · MDisks: 1
  • Encryption: Not encrypted
  • Data reduction: Enabled
  • Easy Tier: Balanced
  • Extent size: 1.00 GiB
  • Free physical capacity: 4.75 TiB

mdisk0

  • State: Online
  • Capacity: 20.26 TiB
  • Pool: Pool0 · Mode: Array · Tier: Tier 0 Flash
  • Write protected: No
  • Encryption: Not encrypted · Deduplication: Not active
  • Fast-Write state: Empty
  • Thin-provisioned: No · Supports unmap: Yes
  • RAID level: Distributed RAID 6 · Redundancy: 2
  • Rebuild areas goal: 2 · Rebuild areas total: 2
  • Strip size: 256 KiB

8K block size — 1 thread

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 228,787 1,787 0.003 —
Random 100% write 4,061 31.7 — 0.244
Random 50/50 7,766 60.7 0.004 0.250
Random 80/20 18,293 142.9 0.003 0.252
Sequential 100% read 354,277 2,768 0.002 —
Sequential 100% write 3,065 23.9 — 0.325
Sequential 50/50 6,629 51.8 0.003 0.297
Sequential 80/20 16,971 132.6 0.003 0.278

8K block size — 16 threads

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 1,501,107 11,727 0.003 —
Random 100% write 27,052 211.3 — 0.589
Random 50/50 46,707 364.9 0.027 0.653
Random 80/20 115,231 900.2 0.012 0.635
Sequential 100% read 3,475,273 27,151 0.004 —
Sequential 100% write 25,121 196.3 — 0.635
Sequential 50/50 52,680 411.6 0.027 0.578
Sequential 80/20 123,603 965.6 0.011 0.597

16K block size — 1 thread

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 181,688 2,839 0.004 —
Random 100% write 3,934 30.7 — 0.252
Random 50/50 7,076 110.6 0.004 0.274
Random 80/20 16,432 256.8 0.005 0.275
Sequential 100% read 224,488 3,508 0.004 —
Sequential 100% write 3,184 49.8 — 0.312
Sequential 50/50 6,931 108.3 0.005 0.281
Sequential 80/20 16,945 264.8 0.004 0.274

1MB block size — 1 thread

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 5,741 5,741 0.172 —
Random 100% write 3,989 31.2 — 0.248
Random 50/50 722 722.0 0.175 2.52
Random 80/20 1,532 1,532 0.172 2.49
Sequential 100% read 5,768 5,768 0.173 —
Sequential 100% write 379 379.0 — 2.57
Sequential 50/50 741 741.0 0.190 2.44
Sequential 80/20 1,555 1,555 0.172 2.45

1MB block size — 16 threads

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 80,170 80,170 0.197 —
Random 100% write 27,300 213.3 — 0.583
Random 50/50 1,622 1,622 0.779 18.8
Random 80/20 5,137 5,137 0.372 14.0
Sequential 100% read 76,222 76,222 0.209 —
Sequential 100% write 1,173 1,173 — 13.5
Sequential 50/50 2,168 2,168 0.671 14.0
Sequential 80/20 5,192 5,192 0.396 13.7

Takeaways

  • As the block size and thread count go up, so does performance — up to a point.
  • DRAID6 looks like a strong alternative to traditional RAID 10, even for critical workloads. Anyone moving from spinning-disk RAID 10 arrays to NVMe SSD arrays should seriously weigh the DRAID6 option.
  • Failover testing of the active/active storage nodes worked as expected with redundant pathing.
  • It was not possible to assign a stand-by hot spare in a DRAID6 configuration.
  • Technical documentation for FC-NVMe configuration (instead of FC/SCSI) on Gen3 and/or AIX was unavailable at the time of writing.

// hardware · 44 photos

// sharetweetlinkedinemail