skip to content
mirza@edge
← cd ..
ibmpower9fs7200aix2022-02-08 · 5 min read

IBM Power9 9009-22G & Flash System 7200 Benchmarks

Same exercise as the V7000 Gen3 run, this time against an IBM Flash System 7200. A Power9 9009-22G drives the all-NVMe array over fibre channel, benchmarked under both traditional RAID 10 and Distributed RAID 6.

The hardware

IBM Power9 9009-22G

  • 2x AC power supply — 1400W for server (200–240 VAC)
  • 2x mainstream 800GB SSD PCIe3 NVMe U.2 module for AIX/Linux
  • Front IBM bezel for 4 NVMe-bay backplane
  • Storage backplane with four front PCIe Gen4-capable NVMe U.2 drive slots
  • 16x 64GB DDR4 memory DIMM
  • PCIe3 LP 2-port 16Gb fibre channel adapter
  • PCIe4 LP 32Gb 2-port optical fibre channel adapter
  • 2x 8-core POWER9 processor, typical 3.4 to 3.9 GHz (max)
  • 16x one-processor-core activation
  • PCIe2 LP 2-port 10GbE Base-T adapter
  • AIX 7.2.5 SP3
  • algorithm set to shortest_queue
  • reserve_policy set to no_reserve
  • queue_depth set to 128
  • j2_dynamicBufferPreallocation set to 64
  • pv_pbuf_count set to 1024
  • max_xfer_size set to 0x400000

IBM Flash System 7200 (FS7200)

  • 2x 8-core Intel Xeon Silver 4208 CPU @ 2.10GHz
  • 384 GiB memory
  • 32Gb FC 4-port adapter card
  • 240GB M.2 boot drive
  • 17x 1.92TB NVMe flash drives

Test methodology

The tests were run with XDISK V8.6 for AIX 7.2 using the -ORW flags specific to AIX, to benchmark the storage array directly and bypass the VMM cache. Each layout was put through random and sequential I/O at 8K, 16K, and 1MB block sizes, at 1 and 16 threads, across four read/write profiles: 100% read, 100% write, 50/50 read/write (backup profile), and 80/20 read/write (OLTP database profile).

In the tables below, Read (ms) and Write (ms) are the average latencies for that profile; a — means that side of the workload doesn’t apply.

Traditional RAID 10

Two mdisk devices, 16 drives total plus 1 spare.

8K block size — 1 thread

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 236,587 1,848 0.003 —
Random 100% write 4,015 31.4 — 0.247
Random 50/50 7,832 61.2 0.004 0.248
Random 80/20 18,509 144.6 0.003 0.248
Sequential 100% read 345,942 2,703 0.002 —
Sequential 100% write 3,123 24.4 — 0.319
Sequential 50/50 6,710 52.4 0.003 0.293
Sequential 80/20 17,871 139.6 0.003 0.265

8K block size — 16 threads

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 1,505,683 11,763 0.003 —
Random 100% write 33,141 258.9 — 0.480
Random 50/50 62,571 488.8 0.022 0.485
Random 80/20 151,339 1,182 0.010 0.479
Sequential 100% read 3,581,072 27,977 0.004 —
Sequential 100% write 28,645 223.8 — 0.557
Sequential 50/50 58,762 459.1 0.027 0.515
Sequential 80/20 148,521 1,160 0.010 0.495

16K block size — 1 thread

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 178,765 2,793 0.004 —
Random 100% write 3,966 31.0 — 0.250
Random 50/50 7,289 113.9 0.004 0.265
Random 80/20 17,327 270.7 0.004 0.263
Sequential 100% read 238,870 3,732 0.003 —
Sequential 100% write 3,366 52.6 — 0.295
Sequential 50/50 7,028 109.8 0.005 0.277
Sequential 80/20 17,454 272.7 0.004 0.265

1MB block size — 1 thread

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 5,847 5,847 0.169 —
Random 100% write 4,161 32.5 — 0.238
Random 50/50 756 756.0 0.173 2.40
Random 80/20 1,589 1,589 0.172 2.38
Sequential 100% read 5,215 5,215 0.191 —
Sequential 100% write 410 410.0 — 2.36
Sequential 50/50 767 767.0 0.173 2.36
Sequential 80/20 1,409 1,409 0.200 2.67

1MB block size — 16 threads

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 78,682 78,682 0.201 —
Random 100% write 33,581 262.4 — 0.474
Random 50/50 2,325 2,325 0.616 13.0
Random 80/20 6,610 6,610 0.346 10.6
Sequential 100% read 76,001 76,001 0.210 —
Sequential 100% write 1,351 1,351 — 11.7
Sequential 50/50 2,659 2,659 0.612 11.3
Sequential 80/20 6,602 6,602 0.349 10.6

Distributed RAID 6 (DRAID6)

One mdisk device, 16 drives, 2 rebuild areas, 1 spare.

Pool0

  • State: Online
  • Capacity: 15.49 TiB / 20.23 TiB (77%)
  • Volumes: 1 · MDisks: 1
  • Encryption: Not encrypted
  • Data reduction: Enabled
  • Easy Tier: Balanced
  • Extent size: 1.00 GiB
  • Free physical capacity: 4.75 TiB
  • Reclaimable capacity: 0 bytes

mdisk0

  • State: Online
  • Capacity: 20.26 TiB
  • Pool: Pool0 · Mode: Array · Tier: Tier 0 Flash
  • Write protected: No
  • Encryption: Not encrypted · Deduplication: Not active
  • Fast-Write state: Empty
  • Thin-provisioned: No · Supports unmap: Yes
  • RAID level: Distributed RAID 6 · Redundancy: 2
  • Rebuild areas goal: 2 · Rebuild areas total: 2
  • Strip size: 256 KiB

8K block size — 1 thread

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 228,787 1,787 0.003 —
Random 100% write 4,061 31.7 — 0.244
Random 50/50 7,766 60.7 0.004 0.250
Random 80/20 18,293 142.9 0.003 0.252
Sequential 100% read 354,277 2,768 0.002 —
Sequential 100% write 3,065 23.9 — 0.325
Sequential 50/50 6,629 51.8 0.003 0.297
Sequential 80/20 16,971 132.6 0.003 0.278

8K block size — 16 threads

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 1,501,107 11,727 0.003 —
Random 100% write 27,052 211.3 — 0.589
Random 50/50 46,707 364.9 0.027 0.653
Random 80/20 115,231 900.2 0.012 0.635
Sequential 100% read 3,475,273 27,151 0.004 —
Sequential 100% write 25,121 196.3 — 0.635
Sequential 50/50 52,680 411.6 0.027 0.578
Sequential 80/20 123,603 965.6 0.011 0.597

16K block size — 1 thread

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 181,688 2,839 0.004 —
Random 100% write 3,934 30.7 — 0.252
Random 50/50 7,076 110.6 0.004 0.274
Random 80/20 16,432 256.8 0.005 0.275
Sequential 100% read 224,488 3,508 0.004 —
Sequential 100% write 3,184 49.8 — 0.312
Sequential 50/50 6,931 108.3 0.005 0.281
Sequential 80/20 16,945 264.8 0.004 0.274

1MB block size — 1 thread

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 5,741 5,741 0.172 —
Random 100% write 3,989 31.2 — 0.248
Random 50/50 722 722.0 0.175 2.52
Random 80/20 1,532 1,532 0.172 2.49
Sequential 100% read 5,768 5,768 0.173 —
Sequential 100% write 379 379.0 — 2.57
Sequential 50/50 741 741.0 0.190 2.44
Sequential 80/20 1,555 1,555 0.172 2.45

1MB block size — 16 threads

Pattern Profile IO/s MB/s Read (ms) Write (ms)
Random 100% read 80,170 80,170 0.197 —
Random 100% write 27,300 213.3 — 0.583
Random 50/50 1,622 1,622 0.779 18.8
Random 80/20 5,137 5,137 0.372 14.0
Sequential 100% read 76,222 76,222 0.209 —
Sequential 100% write 1,173 1,173 — 13.5
Sequential 50/50 2,168 2,168 0.671 14.0
Sequential 80/20 5,192 5,192 0.396 13.7

Takeaways

  • As the block size and thread count go up, so does performance — up to a point.
  • The hardware delivers high performance without much tweaking.
  • DRAID6 is the recommended RAID type, and is suggested over traditional RAID 10 even for critical workloads. If you’re concerned about disk failures, configure it for two rebuild areas, which lets you lose two disks.
  • NPIV is enabled by default.
  • The latest StorWize codebase allows DRAID6 pools to be expanded.
  • It was not possible to assign a stand-by hot spare in a DRAID6 configuration — you can still mark an SSD as a spare, but the software will not automatically consume it if a disk fails.
  • No issues failing one of the active/active storage nodes and monitoring the events inside AIX; everything worked as expected in a redundant-path host and storage configuration.
  • Technical documentation for FC-NVMe configuration (instead of FC/SCSI) on the FS7200 and/or AIX was unavailable at the time of writing.
  • The array has two clustered, hot-swappable node canisters containing hot-swappable fan modules, memory DIMMs, batteries, and PCIe adapters. The enclosure also houses two redundant, hot-swappable AC power supplies, and concurrent code load keeps applications online during firmware upgrades to all components, including the flash drives.

// hardware · 40 photos

// sharetweetlinkedinemail