-
Notifications
You must be signed in to change notification settings - Fork 58
Benchmarks
Benjamin Zaitlen edited this page Dec 18, 2019
·
8 revisions
With UCX (NVLink & ib Enabled; dgx14):
Merge benchmark
--------------------------
Chunk-size | 100000000
Frac-match | 0.3
Ignore-size | 1.05 MB
Protocol | ucx
Device(s) | 1,2
==========================
Total time | 1.51 s
Total time | 1.71 s
Total time | 1.16 s
==========================
Worker-Worker Transfer Rates
(w1,w2) | 25% 50% 75% (total nbytes)
--------------------------
(01,02) | 8.98 GB/s 11.46 GB/s 19.07 GB/s (19.20 GB)
(02,01) | 9.30 GB/s 11.52 GB/s 18.87 GB/s (19.20 GB)
With UCX (Only NVLink Enabled; dgx14):
Merge benchmark
--------------------------
Chunk-size | 100000000
Frac-match | 0.3
Ignore-size | 1.05 MB
Protocol | ucx
Device(s) | 1,2
==========================
Total time | 1.23 s
Total time | 1.22 s
Total time | 1.14 s
==========================
Worker-Worker Transfer Rates
(w1,w2) | 25% 50% 75% (total nbytes)
--------------------------
(01,02) | 20.42 GB/s 21.08 GB/s 22.13 GB/s (19.20 GB)
(02,01) | 20.04 GB/s 22.05 GB/s 22.51 GB/s (19.20 GB)
With TCP (dgx14):
Merge benchmark
--------------------------
Chunk-size | 100000000
Frac-match | 0.3
Ignore-size | 1.05 MB
Protocol | tcp
Device(s) | 1,2
==========================
Total time | 52.86 s
Total time | 64.20 s
Total time | 54.20 s
==========================
Worker-Worker Transfer Rates
(w1,w2) | 25% 50% 75% (total nbytes)
--------------------------
(01,02) | 89.30 MB/s 118.65 MB/s 141.55 MB/s (21.60 GB)
(02,01) | 81.78 MB/s 96.84 MB/s 141.15 MB/s (21.60 GB)
With UCX (NVLink & ib Enabled; dgx14):
Merge benchmark
--------------------------
Chunk-size | 100000000
Frac-match | 0.3
Ignore-size | 1.05 MB
Protocol | ucx
Device(s) | 0,1,2,3
==========================
Total time | 2.97 s
Total time | 2.89 s
Total time | 2.77 s
==========================
Worker-Worker Transfer Rates
(w1,w2) | 25% 50% 75% (total nbytes)
--------------------------
(00,01) | 1.39 GB/s 1.51 GB/s 2.94 GB/s (9.60 GB)
(00,02) | 1.34 GB/s 1.55 GB/s 2.67 GB/s (9.60 GB)
(00,03) | 1.44 GB/s 1.84 GB/s 3.42 GB/s (9.60 GB)
(01,00) | 1.38 GB/s 1.50 GB/s 2.70 GB/s (9.60 GB)
(01,02) | 1.51 GB/s 1.80 GB/s 3.51 GB/s (12.00 GB)
(01,03) | 1.59 GB/s 1.71 GB/s 1.99 GB/s (7.20 GB)
(02,00) | 1.18 GB/s 1.50 GB/s 1.71 GB/s (7.20 GB)
(02,01) | 1.32 GB/s 1.46 GB/s 3.05 GB/s (9.60 GB)
(02,03) | 1.37 GB/s 1.63 GB/s 3.72 GB/s (12.00 GB)
(03,00) | 1.39 GB/s 1.81 GB/s 3.91 GB/s (12.00 GB)
(03,01) | 1.42 GB/s 1.56 GB/s 3.11 GB/s (9.60 GB)
(03,02) | 1.33 GB/s 1.60 GB/s 2.99 GB/s (7.20 GB)
With UCX (Only NVLink Enabled; dgx14):
Merge benchmark
--------------------------
Chunk-size | 100000000
Frac-match | 0.3
Ignore-size | 1.05 MB
Protocol | ucx
Device(s) | 0,1,2,3
==========================
Total time | 1.81 s
Total time | 1.68 s
Total time | 1.67 s
==========================
Worker-Worker Transfer Rates
(w1,w2) | 25% 50% 75% (total nbytes)
--------------------------
(00,01) | 3.63 GB/s 3.88 GB/s 4.61 GB/s (12.00 GB)
(00,02) | 3.51 GB/s 3.91 GB/s 4.46 GB/s (7.20 GB)
(00,03) | 3.91 GB/s 4.13 GB/s 4.72 GB/s (9.60 GB)
(01,00) | 3.64 GB/s 4.14 GB/s 8.24 GB/s (14.40 GB)
(01,02) | 3.84 GB/s 4.18 GB/s 4.44 GB/s (7.20 GB)
(01,03) | 3.85 GB/s 3.88 GB/s 4.46 GB/s (7.20 GB)
(02,00) | 2.53 GB/s 3.80 GB/s 6.82 GB/s (7.20 GB)
(02,01) | 3.88 GB/s 4.21 GB/s 4.85 GB/s (9.60 GB)
(02,03) | 3.63 GB/s 4.24 GB/s 5.05 GB/s (12.00 GB)
(03,00) | 4.02 GB/s 4.38 GB/s 4.66 GB/s (7.20 GB)
(03,01) | 3.55 GB/s 3.86 GB/s 4.23 GB/s (7.20 GB)
(03,02) | 3.77 GB/s 3.98 GB/s 10.58 GB/s (14.40 GB)
With TCP (dgx14):
Merge benchmark
--------------------------
Chunk-size | 100000000
Frac-match | 0.3
Ignore-size | 1.05 MB
Protocol | tcp
Device(s) | 0,1,2,3
==========================
Total time | 65.14 s
Total time | 88.68 s
Total time | 90.60 s
==========================
Worker-Worker Transfer Rates
(w1,w2) | 25% 50% 75% (total nbytes)
--------------------------
(00,01) | 48.76 MB/s 56.70 MB/s 57.42 MB/s (9.60 GB)
(00,02) | 46.50 MB/s 56.02 MB/s 77.26 MB/s (9.30 GB)
(00,03) | 61.93 MB/s 65.69 MB/s 76.17 MB/s (6.90 GB)
(01,00) | 39.59 MB/s 46.15 MB/s 53.39 MB/s (7.90 GB)
(01,02) | 51.95 MB/s 56.68 MB/s 81.24 MB/s (8.60 GB)
(01,03) | 45.64 MB/s 61.88 MB/s 93.96 MB/s (12.00 GB)
(02,00) | 36.81 MB/s 43.89 MB/s 61.16 MB/s (9.60 GB)
(02,01) | 49.12 MB/s 60.38 MB/s 105.98 MB/s (11.00 GB)
(02,03) | 49.67 MB/s 55.76 MB/s 74.83 MB/s (10.30 GB)
(03,00) | 51.42 MB/s 64.27 MB/s 82.52 MB/s (11.70 GB)
(03,01) | 41.32 MB/s 53.54 MB/s 75.44 MB/s (11.00 GB)
(03,02) | 46.47 MB/s 58.26 MB/s 98.87 MB/s (10.00 GB)
With UCX (NVLink & ib Enabled; dgx14):
Merge benchmark
--------------------------
Chunk-size | 100000000
Frac-match | 0.3
Ignore-size | 1.05 MB
Protocol | ucx
Device(s) | 0,1,2,3,4,5,6,7
==========================
Total time | 12.24 s
Total time | 11.33 s
Total time | 9.17 s
Total time | 8.92 s
Total time | 5.95 s
Total time | 5.20 s
Total time | 5.24 s
==========================
Worker-Worker Transfer Rates
(w1,w2) | 25% 50% 75% (total nbytes)
--------------------------
(00,01) | 170.87 MB/s 268.29 MB/s 369.26 MB/s (8.75 GB)
(00,02) | 159.53 MB/s 263.56 MB/s 341.53 MB/s (9.10 GB)
(00,03) | 171.01 MB/s 265.22 MB/s 403.75 MB/s (11.85 GB)
(00,04) | 181.48 MB/s 254.46 MB/s 357.78 MB/s (11.50 GB)
(00,05) | 171.15 MB/s 269.89 MB/s 392.40 MB/s (11.50 GB)
(00,06) | 167.22 MB/s 289.16 MB/s 373.49 MB/s (13.55 GB)
(00,07) | 167.07 MB/s 291.28 MB/s 402.55 MB/s (10.80 GB)
(01,00) | 152.43 MB/s 242.44 MB/s 458.86 MB/s (9.10 GB)
(01,02) | 163.07 MB/s 240.60 MB/s 355.73 MB/s (9.10 GB)
(01,03) | 153.51 MB/s 263.49 MB/s 476.21 MB/s (17.35 GB)
(01,04) | 202.00 MB/s 292.19 MB/s 365.60 MB/s (10.15 GB)
(01,05) | 157.53 MB/s 241.64 MB/s 364.72 MB/s (11.85 GB)
(01,06) | 162.38 MB/s 239.02 MB/s 342.63 MB/s (11.50 GB)
(01,07) | 176.97 MB/s 322.70 MB/s 801.58 MB/s (14.25 GB)
(02,00) | 127.99 MB/s 267.93 MB/s 459.03 MB/s (13.20 GB)
(02,01) | 136.31 MB/s 244.81 MB/s 348.59 MB/s (9.10 GB)
(02,03) | 138.94 MB/s 245.86 MB/s 498.27 MB/s (7.70 GB)
(02,04) | 191.54 MB/s 248.28 MB/s 623.21 MB/s (11.15 GB)
(02,05) | 138.30 MB/s 241.75 MB/s 550.28 MB/s (10.80 GB)
(02,06) | 140.61 MB/s 246.98 MB/s 449.69 MB/s (13.90 GB)
(02,07) | 147.66 MB/s 279.35 MB/s 340.71 MB/s (11.15 GB)
(03,00) | 147.57 MB/s 269.69 MB/s 402.41 MB/s (9.80 GB)
(03,01) | 177.81 MB/s 279.29 MB/s 886.40 MB/s (17.00 GB)
(03,02) | 159.23 MB/s 244.44 MB/s 328.62 MB/s (11.85 GB)
(03,04) | 152.27 MB/s 299.56 MB/s 487.58 MB/s (11.85 GB)
(03,05) | 152.83 MB/s 287.98 MB/s 435.10 MB/s (14.25 GB)
(03,06) | 151.87 MB/s 298.05 MB/s 397.76 MB/s (9.45 GB)
(03,07) | 123.38 MB/s 240.92 MB/s 381.74 MB/s (9.45 GB)
(04,00) | 197.14 MB/s 280.78 MB/s 430.51 MB/s (12.55 GB)
(04,01) | 179.70 MB/s 292.00 MB/s 569.28 MB/s (9.80 GB)
(04,02) | 174.79 MB/s 322.72 MB/s 540.76 MB/s (11.85 GB)
(04,03) | 198.26 MB/s 322.06 MB/s 661.30 MB/s (16.30 GB)
(04,05) | 161.50 MB/s 289.76 MB/s 557.92 MB/s (11.85 GB)
(04,06) | 182.68 MB/s 236.98 MB/s 347.70 MB/s (13.90 GB)
(04,07) | 149.81 MB/s 254.47 MB/s 389.29 MB/s (9.45 GB)
(05,00) | 165.85 MB/s 263.07 MB/s 541.80 MB/s (9.45 GB)
(05,01) | 149.32 MB/s 309.41 MB/s 657.16 MB/s (14.60 GB)
(05,02) | 125.34 MB/s 297.96 MB/s 566.81 MB/s (8.75 GB)
(05,03) | 201.06 MB/s 304.08 MB/s 722.50 MB/s (11.15 GB)
(05,04) | 135.99 MB/s 282.90 MB/s 553.04 MB/s (9.80 GB)
(05,06) | 147.04 MB/s 271.71 MB/s 545.92 MB/s (11.50 GB)
(05,07) | 164.19 MB/s 314.66 MB/s 616.88 MB/s (14.60 GB)
(06,00) | 184.96 MB/s 293.26 MB/s 543.58 MB/s (9.80 GB)
(06,01) | 172.11 MB/s 268.53 MB/s 528.09 MB/s (11.85 GB)
(06,02) | 174.05 MB/s 210.60 MB/s 362.06 MB/s (9.45 GB)
(06,03) | 186.72 MB/s 333.26 MB/s 651.96 MB/s (9.45 GB)
(06,04) | 196.32 MB/s 274.06 MB/s 508.44 MB/s (14.95 GB)
(06,05) | 184.75 MB/s 277.72 MB/s 459.42 MB/s (17.00 GB)
(06,07) | 160.21 MB/s 299.32 MB/s 446.58 MB/s (12.20 GB)
(07,00) | 150.14 MB/s 264.18 MB/s 519.00 MB/s (13.90 GB)
(07,01) | 128.83 MB/s 292.49 MB/s 526.69 MB/s (9.45 GB)
(07,02) | 137.73 MB/s 300.03 MB/s 773.75 MB/s (10.80 GB)
(07,03) | 136.52 MB/s 227.20 MB/s 363.01 MB/s (9.10 GB)
(07,04) | 155.90 MB/s 255.37 MB/s 488.56 MB/s (11.15 GB)
(07,05) | 123.74 MB/s 254.76 MB/s 365.85 MB/s (8.75 GB)
(07,06) | 137.92 MB/s 268.29 MB/s 479.20 MB/s (13.55 GB)
With TCP (dgx14):
Merge benchmark
--------------------------
Chunk-size | 100000000
Frac-match | 0.3
Ignore-size | 1.05 MB
Protocol | tcp
Device(s) | 0,1,2,3,4,5,6,7
==========================
Total time | 86.69 s
Total time | 101.35 s
Total time | 108.21 s
==========================
Worker-Worker Transfer Rates
(w1,w2) | 25% 50% 75% (total nbytes)
--------------------------
(00,01) | 19.59 MB/s 26.60 MB/s 35.07 MB/s (3.95 GB)
(00,02) | 24.35 MB/s 34.26 MB/s 53.47 MB/s (8.75 GB)
(00,03) | 26.40 MB/s 31.16 MB/s 43.46 MB/s (6.35 GB)
(00,04) | 24.12 MB/s 29.29 MB/s 35.10 MB/s (3.60 GB)
(00,05) | 19.22 MB/s 22.44 MB/s 46.70 MB/s (3.60 GB)
(00,06) | 26.06 MB/s 29.12 MB/s 34.80 MB/s (3.95 GB)
(00,07) | 17.63 MB/s 25.02 MB/s 41.12 MB/s (8.75 GB)
(01,00) | 26.31 MB/s 31.75 MB/s 41.44 MB/s (3.95 GB)
(01,02) | 22.20 MB/s 28.00 MB/s 35.52 MB/s (3.95 GB)
(01,03) | 20.61 MB/s 28.88 MB/s 38.56 MB/s (4.30 GB)
(01,04) | 23.38 MB/s 30.86 MB/s 35.09 MB/s (3.95 GB)
(01,05) | 19.21 MB/s 21.89 MB/s 34.56 MB/s (6.70 GB)
(01,06) | 22.67 MB/s 24.49 MB/s 28.53 MB/s (6.35 GB)
(01,07) | 20.78 MB/s 26.22 MB/s 36.89 MB/s (6.70 GB)
(02,00) | 23.70 MB/s 28.09 MB/s 57.90 MB/s (4.65 GB)
(02,01) | 32.89 MB/s 60.63 MB/s 79.73 MB/s (9.45 GB)
(02,03) | 27.97 MB/s 29.65 MB/s 36.55 MB/s (3.95 GB)
(02,04) | 20.64 MB/s 26.73 MB/s 33.34 MB/s (4.65 GB)
(02,05) | 19.88 MB/s 35.94 MB/s 55.60 MB/s (4.30 GB)
(02,06) | 27.13 MB/s 29.68 MB/s 56.03 MB/s (4.30 GB)
(02,07) | 19.35 MB/s 29.75 MB/s 30.75 MB/s (3.95 GB)
(03,00) | 25.92 MB/s 32.77 MB/s 42.75 MB/s (3.10 GB)
(03,01) | 21.35 MB/s 31.39 MB/s 43.57 MB/s (5.65 GB)
(03,02) | 20.29 MB/s 32.34 MB/s 38.91 MB/s (4.15 GB)
(03,04) | 35.25 MB/s 59.73 MB/s 60.46 MB/s (3.45 GB)
(03,05) | 26.99 MB/s 34.27 MB/s 50.60 MB/s (3.45 GB)
(03,06) | 20.03 MB/s 37.08 MB/s 42.02 MB/s (3.80 GB)
(03,07) | 43.82 MB/s 53.66 MB/s 82.59 MB/s (9.40 GB)
(04,00) | 26.70 MB/s 37.78 MB/s 48.50 MB/s (7.05 GB)
(04,01) | 20.01 MB/s 25.75 MB/s 54.05 MB/s (3.95 GB)
(04,02) | 17.01 MB/s 30.15 MB/s 38.18 MB/s (4.30 GB)
(04,03) | 19.65 MB/s 24.22 MB/s 31.63 MB/s (4.30 GB)
(04,05) | 30.06 MB/s 45.21 MB/s 60.45 MB/s (9.45 GB)
(04,06) | 22.11 MB/s 30.65 MB/s 37.39 MB/s (3.95 GB)
(04,07) | 21.77 MB/s 27.99 MB/s 34.59 MB/s (4.30 GB)
(05,00) | 18.43 MB/s 21.65 MB/s 28.90 MB/s (3.95 GB)
(05,01) | 16.39 MB/s 24.25 MB/s 27.77 MB/s (3.95 GB)
(05,02) | 19.00 MB/s 27.80 MB/s 41.88 MB/s (6.70 GB)
(05,03) | 30.40 MB/s 33.86 MB/s 63.56 MB/s (4.30 GB)
(05,04) | 32.38 MB/s 38.12 MB/s 49.22 MB/s (4.65 GB)
(05,06) | 25.84 MB/s 44.11 MB/s 59.88 MB/s (7.05 GB)
(05,07) | 17.45 MB/s 43.34 MB/s 55.06 MB/s (6.35 GB)
(06,00) | 22.80 MB/s 29.27 MB/s 56.39 MB/s (5.70 GB)
(06,01) | 20.88 MB/s 33.09 MB/s 46.56 MB/s (5.70 GB)
(06,02) | 22.63 MB/s 33.25 MB/s 41.93 MB/s (7.40 GB)
(06,03) | 18.70 MB/s 27.05 MB/s 33.61 MB/s (3.95 GB)
(06,04) | 25.33 MB/s 35.18 MB/s 61.78 MB/s (10.50 GB)
(06,05) | 24.70 MB/s 29.76 MB/s 42.74 MB/s (6.35 GB)
(06,07) | 21.52 MB/s 26.05 MB/s 38.47 MB/s (3.95 GB)
(07,00) | 16.39 MB/s 23.22 MB/s 33.17 MB/s (3.80 GB)
(07,01) | 15.76 MB/s 23.19 MB/s 33.79 MB/s (4.65 GB)
(07,02) | 20.98 MB/s 28.23 MB/s 41.44 MB/s (4.65 GB)
(07,03) | 24.76 MB/s 52.00 MB/s 56.94 MB/s (5.15 GB)
(07,04) | 18.56 MB/s 25.99 MB/s 58.61 MB/s (4.15 GB)
(07,05) | 18.20 MB/s 43.64 MB/s 54.24 MB/s (4.15 GB)
(07,06) | 20.18 MB/s 31.06 MB/s 36.45 MB/s (4.15 GB)
With UCX (Only NVLink Enabled; dgx14):
Merge benchmark
--------------------------
Chunk-size | 100000000
Frac-match | 0.3
Ignore-size | 1.05 MB
Protocol | ucx
Device(s) | 0,1,2,3,4,5,6,7
==========================
Total time | 11.34 s
Total time | 10.43 s
Total time | 10.47 s
==========================
Worker-Worker Transfer Rates
(w1,w2) | 25% 50% 75% (total nbytes)
--------------------------
(00,01) | 254.00 MB/s 513.77 MB/s 838.71 MB/s (2.80 GB)
(00,02) | 266.62 MB/s 302.47 MB/s 557.03 MB/s (5.55 GB)
(00,03) | 334.01 MB/s 435.66 MB/s 4.36 GB/s (5.20 GB)
(00,04) | 98.57 MB/s 124.89 MB/s 479.46 MB/s (5.55 GB)
(00,05) | 82.38 MB/s 87.01 MB/s 97.09 MB/s (2.45 GB)
(00,06) | 69.51 MB/s 74.75 MB/s 199.61 MB/s (4.15 GB)
(00,07) | 83.66 MB/s 86.23 MB/s 109.52 MB/s (2.45 GB)
(01,00) | 288.21 MB/s 526.00 MB/s 1.22 GB/s (3.50 GB)
(01,02) | 192.12 MB/s 421.93 MB/s 726.91 MB/s (2.80 GB)
(01,03) | 140.04 MB/s 265.66 MB/s 357.55 MB/s (5.20 GB)
(01,04) | 46.03 MB/s 56.02 MB/s 108.98 MB/s (1.40 GB)
(01,05) | 112.66 MB/s 146.25 MB/s 161.14 MB/s (2.10 GB)
(01,06) | 56.56 MB/s 83.10 MB/s 101.28 MB/s (2.45 GB)
(01,07) | 51.98 MB/s 61.95 MB/s 140.31 MB/s (4.50 GB)
(02,00) | 147.37 MB/s 263.72 MB/s 536.50 MB/s (2.80 GB)
(02,01) | 216.53 MB/s 416.35 MB/s 635.28 MB/s (3.15 GB)
(02,03) | 247.11 MB/s 469.44 MB/s 844.11 MB/s (2.80 GB)
(02,04) | 68.20 MB/s 109.48 MB/s 168.20 MB/s (6.90 GB)
(02,05) | 73.38 MB/s 79.80 MB/s 100.35 MB/s (2.45 GB)
(02,06) | 136.05 MB/s 141.95 MB/s 146.95 MB/s (4.15 GB)
(02,07) | 69.77 MB/s 105.93 MB/s 112.89 MB/s (4.50 GB)
(03,00) | 484.21 MB/s 1.50 GB/s 5.52 GB/s (8.30 GB)
(03,01) | 283.58 MB/s 552.94 MB/s 1.31 GB/s (3.50 GB)
(03,02) | 216.72 MB/s 647.79 MB/s 961.33 MB/s (4.20 GB)
(03,04) | 59.10 MB/s 88.74 MB/s 113.32 MB/s (3.15 GB)
(03,05) | 63.16 MB/s 75.99 MB/s 86.53 MB/s (2.80 GB)
(03,06) | 49.91 MB/s 98.08 MB/s 130.07 MB/s (5.20 GB)
(03,07) | 86.16 MB/s 118.22 MB/s 144.84 MB/s (3.85 GB)
(04,00) | 77.73 MB/s 128.88 MB/s 193.89 MB/s (3.15 GB)
(04,01) | 71.80 MB/s 75.98 MB/s 78.97 MB/s (1.75 GB)
(04,02) | 47.95 MB/s 70.25 MB/s 89.96 MB/s (2.10 GB)
(04,03) | 56.97 MB/s 76.66 MB/s 89.15 MB/s (2.45 GB)
(04,05) | 186.71 MB/s 2.43 GB/s 4.69 GB/s (6.55 GB)
(04,06) | 155.27 MB/s 190.28 MB/s 524.52 MB/s (2.80 GB)
(04,07) | 163.34 MB/s 324.46 MB/s 1.35 GB/s (4.85 GB)
(05,00) | 89.56 MB/s 98.86 MB/s 112.51 MB/s (2.45 GB)
(05,01) | 97.85 MB/s 129.70 MB/s 253.26 MB/s (2.80 GB)
(05,02) | 85.63 MB/s 95.64 MB/s 141.05 MB/s (2.45 GB)
(05,03) | 84.74 MB/s 95.96 MB/s 111.97 MB/s (5.20 GB)
(05,04) | 149.25 MB/s 181.48 MB/s 271.61 MB/s (2.80 GB)
(05,06) | 109.16 MB/s 206.66 MB/s 361.80 MB/s (4.15 GB)
(05,07) | 140.37 MB/s 163.69 MB/s 257.51 MB/s (2.80 GB)
(06,00) | 60.86 MB/s 69.51 MB/s 89.35 MB/s (2.80 GB)
(06,01) | 64.78 MB/s 72.00 MB/s 107.79 MB/s (3.15 GB)
(06,02) | 92.29 MB/s 105.28 MB/s 213.23 MB/s (3.15 GB)
(06,03) | 74.23 MB/s 103.20 MB/s 130.14 MB/s (7.25 GB)
(06,04) | 174.92 MB/s 300.91 MB/s 562.39 MB/s (3.15 GB)
(06,05) | 189.45 MB/s 648.71 MB/s 4.74 GB/s (5.55 GB)
(06,07) | 115.96 MB/s 300.81 MB/s 430.57 MB/s (3.15 GB)
(07,00) | 62.87 MB/s 90.63 MB/s 94.57 MB/s (2.45 GB)
(07,01) | 68.59 MB/s 75.21 MB/s 101.45 MB/s (4.85 GB)
(07,02) | 82.95 MB/s 90.17 MB/s 120.03 MB/s (2.80 GB)
(07,03) | 109.30 MB/s 120.74 MB/s 140.04 MB/s (3.50 GB)
(07,04) | 128.11 MB/s 191.59 MB/s 384.80 MB/s (2.10 GB)
(07,05) | 142.40 MB/s 152.86 MB/s 664.38 MB/s (1.75 GB)
(07,06) | 122.96 MB/s 756.28 MB/s 1.59 GB/s (5.20 GB)