====== Virident tachIOn 400GB ======

sysbench fileio benchmarks for Virident tachIOn 400GB PCI-E Flash card

Server specification:

  * Supermicro X8DTH series motherboard
  * 2 x Xeon E5520 (2.27GHz) processors w/HT enabled (16 cores)
  * 64GB of ECC/Registered DDR3 DRAM
  * Centos 5.3 2-6.18.164 Kernel
  * Filesystem is XFS formatted with ''mkfs.xfs -s size=4096'' option ( size=4096, sector size, is very important to have aligned IO requests) and mounted with ''nobarrier'' option
  * Benchmark: sysbench fileio on 100GB file, 16KB blocksize
  * Virident tachIOn 400GB SLC card
  * FusionIO ioDrive 160GB SLC card

===== Virident card results =====

^ size^type^      thr^bytes/sec^req/sec^ 95pct resp time, ms^ max resp time, ms^
|  100G | rndrd |    1|   110.69Mb|  7084.09|  0.1|  284.6|
|  100G | rndrd |    4|   402.63Mb|  25768.56|  0.2|  2.9|
|  100G | rndrd |    8|   665.47Mb|  42589.81|  0.3|  2.2|
|  100G | rndrd |   16|   925.94Mb|  59260.30|  0.4|  1.9|
|  100G | rndrd |   32|    1.146Gb|  75105.48|  0.7|  2.4|
|  100G | rndrd |   64|   1.1618Gb|  76140.28|  1.6|  22.6|
^ size^type^      thr^bytes/sec^req/sec^ 95pct resp time, ms^ max resp time, sec^
|  100G | rndrw |    1|   146.17Mb|  9355.10|  0.2|  279.0|
|  100G | rndrw |    4|   437.82Mb|  28020.71|  0.4|  15.5|
|  100G | rndrw |    8|   571.77Mb|  36593.58|  0.6|  28.0|
|  100G | rndrw |   16|   603.62Mb|  38631.92|  1.0|  27.1|
|  100G | rndrw |   32|   745.54Mb|  47714.74|  2.0|  23.0|
|  100G | rndrw |   64|   754.29Mb|  48274.31|  3.8|  50.4|
^ size^type^      thr^bytes/sec^req/sec^ 95pct resp time, ms^ max resp time, sec^
|  100G | rndwr |    1|   237.45Mb|  15196.65|  0.0|  312.2|
|  100G | rndwr |    4|   278.67Mb|  17834.76|  0.6|  305.3|
|  100G | rndwr |    8|   380.59Mb|  24357.50|  3.1|  23.0|
|  100G | rndwr |   16|   426.34Mb|  27285.98|  3.8|  41.2|
|  100G | rndwr |   32|   448.12Mb|  28679.96|  4.6|  96.6|
|  100G | rndwr |   64|   457.03Mb|  29249.87|  5.3|  225.8|
^ size^type^      thr^bytes/sec^req/sec^ 95pct resp time, ms^ max resp time, sec^
|  100G | seqrd |    1|   143.68Mb|  9195.62|  0.1|  1.8|
|  100G | seqrd |    4|   399.96Mb|  25597.33|  0.2|  279.5|
|  100G | seqrd |    8|   581.29Mb|  37202.39|  0.3|  379.3|
|  100G | seqrd |   16|   885.51Mb|  56672.58|  0.4|  309.2|
|  100G | seqrd |   32|    1.189Gb|  77920.80|  0.6|  1.9|
|  100G | seqrd |   64|   1.2806Gb|  83926.83|  0.9|  2.6|
^ size^type^      thr^bytes/sec^req/sec^ 95pct resp time, ms^ max resp time, sec^
|  100G | seqwr |    1|   366.66Mb|  23465.97|  0.0|  30.5|
|  100G | seqwr |    4|   383.84Mb|  24565.78|  0.2|  364.9|
|  100G | seqwr |    8|   421.66Mb|  26986.42|  0.1|  254.2|
|  100G | seqwr |   16|   314.41Mb|  20122.44|  0.1|  732.1|
|  100G | seqwr |   32|   331.68Mb|  21227.47|  0.1|  1028.3|
|  100G | seqwr |   64|   321.12Mb|  20551.80|  0.1|  1303.3|

===== 370GB file  =====

(no max response time, as this data was not gathered in the script)

^ size^type^      thr^bytes/sec^req/sec^95pct resp time, ms^
|  370G | rndrd |    1|   104.12Mb|  6663.79|  0.1|
|  370G | rndrd |    4|   401.92Mb|  25722.90|  0.2|
|  370G | rndrd |    8|   655.15Mb|  41929.48|  0.3|
|  370G | rndrd |   16|   931.11Mb|  59591.15|  0.4|
|  370G | rndrd |   32|   1.1486Gb|  75273.42|  0.7|
|  370G | rndrd |   64|   1.1801Gb|  77339.67|  1.6|
^ size^type^      thr^bytes/sec^req/sec^95pct resp time, ms^
|  370G | rndrw |    1|   142.24Mb|  9103.33|  0.2|
|  370G | rndrw |    4|   444.29Mb|  28434.46|  0.4|
|  370G | rndrw |    8|   667.25Mb|  42703.82|  0.5|
|  370G | rndrw |   16|   901.11Mb|  57670.97|  0.7|
|  370G | rndrw |   32|   711.36Mb|  45527.20|  2.2|
|  370G | rndrw |   64|   505.72Mb|  32366.00|  5.8|
^ size^type^      thr^bytes/sec^req/sec^95pct resp time, ms^
|  370G | rndwr |    1|   269.12Mb|  17223.81|  0.0|
|  370G | rndwr |    4|   425.35Mb|  27222.16|  0.0|
|  370G | rndwr |    8|   416.06Mb|  26628.00|  2.5|
|  370G | rndwr |   16|   268.52Mb|  17185.30|  5.9|
|  370G | rndwr |   32|    315.2Mb|  20173.09|  6.4|
|  370G | rndwr |   64|   367.15Mb|  23497.56|  7.5|
^ size^type^      thr^bytes/sec^req/sec^95pct resp time, ms^
|  370G | seqrd |    1|   155.05Mb|  9923.38|  0.1|
|  370G | seqrd |    4|   408.28Mb|  26130.16|  0.2|
|  370G | seqrd |    8|   578.98Mb|  37054.84|  0.3|
|  370G | seqrd |   16|   943.54Mb|  60386.50|  0.4|
|  370G | seqrd |   32|   1.2023Gb|  78792.63|  0.6|
|  370G | seqrd |   64|   1.2761Gb|  83628.29|  0.9|
^ size^type^      thr^bytes/sec^req/sec^95pct resp time, ms^
|  370G | seqwr |    1|   431.46Mb|  27613.20|  0.0|
|  370G | seqwr |    4|   449.91Mb|  28794.46|  0.2|
|  370G | seqwr |    8|   449.82Mb|  28788.79|  0.1|
|  370G | seqwr |   16|   382.96Mb|  24509.26|  0.1|
|  370G | seqwr |   32|   375.42Mb|  24026.75|  0.1|
|  370G | seqwr |   64|   358.24Mb|  22927.40|  0.1|

===== FusionIO fileio =====

Card formatted with ''-s 75%'' ( 25% of space reservation)

^ size^type^      thr^bytes/sec^req/sec^ 95pct,ms^max, ms^
|  110G | rndrd |    1|    205.7Mb|  13164.64|  0.07|  314.82|
|  110G | rndrd |    4|   532.98Mb|  34111.01|  0.14|  127.13|
|  110G | rndrd |    8|    528.7Mb|  33836.50|  0.27|  7.76|
|  110G | rndrd |   16|   530.58Mb|  33956.83|  0.52|  106.16|
|  110G | rndrd |   32|   705.69Mb|  45163.95|  1.14|  26.66|
|  110G | rndrd |   64|   716.48Mb|  45854.58|  2.89|  5.22|
^ size^type^      thr^bytes/sec^req/sec^ 95pct,ms^max, ms^
|  110G | rndrw |    1|   168.43Mb|  10779.78|  0.09|  313.39|
|  110G | rndrw |    4|   298.28Mb|  19089.89|  0.42|  63.73|
|  110G | rndrw |    8|   307.09Mb|  19654.05|  0.58|  77.95|
|  110G | rndrw |   16|   323.17Mb|  20682.65|  0.87|  100.23|
|  110G | rndrw |   32|   359.66Mb|  23018.03|  1.53|  102.64|
|  110G | rndrw |   64|   382.13Mb|  24456.18|  5.63|  127.33|
^ size^type^      thr^bytes/sec^req/sec^ 95pct,ms^max, ms^
|  110G | rndwr |    1|    212.2Mb|  13580.78|  0.07|  313.32|
|  110G | rndwr |    4|    304.3Mb|  19475.17|  0.10|  64.70|
|  110G | rndwr |    8|    191.6Mb|  12262.36|  0.50|  99.68|
|  110G | rndwr |   16|   234.35Mb|  14998.21|  1.65|  97.53|
|  110G | rndwr |   32|   211.15Mb|  13513.31|  15.73|  84.11|
|  110G | rndwr |   64|   213.41Mb|  13657.94|  24.19|  184.72|
^ size^type^      thr^bytes/sec^req/sec^ 95pct,ms^max, ms^
|  110G | seqrd |    1|   300.09Mb|  19205.65|  0.07|  318.43|
|  110G | seqrd |    4|   705.74Mb|  45167.33|  0.08|  399.21|
|  110G | seqrd |    8|   744.66Mb|  47658.43|  0.17|  336.02|
|  110G | seqrd |   16|   746.44Mb|  47772.44|  0.34|  328.91|
|  110G | seqrd |   32|   747.85Mb|  47862.55|  0.68|  105.21|
|  110G | seqrd |   64|   748.12Mb|  47879.78|  1.34|  36.45|
^ size^type^      thr^bytes/sec^req/sec^ 95pct,ms^max, ms^
|  110G | seqwr |    1|   188.95Mb|  12092.61|  0.07|  57.05|
|  110G | seqwr |    4|   176.76Mb|  11312.83|  0.48|  511.16|
|  110G | seqwr |    8|   184.86Mb|  11831.30|  0.29|  525.61|
|  110G | seqwr |   16|   167.45Mb|  10716.64|  0.30|  935.41|
|  110G | seqwr |   32|   165.05Mb|  10563.40|  0.30|  1820.43|
|  110G | seqwr |   64|   162.31Mb|  10387.79|  0.30|  2502.53|


^ size^type^      thr^bytes/sec^req/sec^ 95pct^
|  145G | rndrd |    1|   205.69Mb|  13164.09|  0.07|  427.52|
|  145G | rndrd |    4|   534.93Mb|  34235.21|  0.14|  13.74|
|  145G | rndrd |    8|   531.27Mb|  34001.59|  0.27|  3.37|
|  145G | rndrd |   16|   532.59Mb|  34085.77|  0.52|  4.41|
|  145G | rndrd |   32|    706.8Mb|  45234.94|  1.14|  10.85|
|  145G | rndrd |   64|   716.35Mb|  45846.55|  2.88|  7.44|
^ size^type^      thr^bytes/sec^req/sec^ 95pct,ms^max, ms^
|  145G | rndrw |    1|   97.572Mb|  6244.58|  0.13|  463.74|
|  145G | rndrw |    4|   137.35Mb|  8790.62|  0.44|  62.34|
|  145G | rndrw |    8|   141.59Mb|  9061.99|  0.62|  115.48|
|  145G | rndrw |   16|    131.9Mb|  8441.90|  1.54|  115.23|
|  145G | rndrw |   32|   136.46Mb|  8733.73|  15.98|  119.23|
|  145G | rndrw |   64|   146.72Mb|  9390.26|  54.06|  188.92|
^ size^type^      thr^bytes/sec^req/sec^ 95pct,ms^max, ms^
|  145G | rndwr |    1|   143.14Mb|  9160.68|  0.06|  424.10|
|  145G | rndwr |    4|   44.596Mb|  2854.16|  1.38|  62.09|
|  145G | rndwr |    8|   55.806Mb|  3571.55|  1.55|  60.43|
|  145G | rndwr |   16|   68.934Mb|  4411.75|  52.25|  59.77|
|  145G | rndwr |   32|   65.692Mb|  4204.27|  50.90|  133.05|
|  145G | rndwr |   64|   60.663Mb|  3882.45|  60.95|  334.67|
^ size^type^      thr^bytes/sec^req/sec^ 95pct,ms^max, ms^
|  145G | seqrd |    1|   328.55Mb|  21027.45|  0.06|  73.36|
|  145G | seqrd |    4|    743.9Mb|  47609.84|  0.08|  409.09|
|  145G | seqrd |    8|   749.88Mb|  47992.24|  0.17|  4.06|
|  145G | seqrd |   16|    750.3Mb|  48019.15|  0.34|  4.11|
|  145G | seqrd |   32|   750.73Mb|  48046.74|  0.68|  10.47|
|  145G | seqrd |   64|      751Mb|  48063.97|  1.34|  7.21|
^ size^type^      thr^bytes/sec^req/sec^ 95pct,ms^max, ms^
|  145G | seqwr |    1|      190Mb|  12160.05|  0.08|  3.48|
|  145G | seqwr |    4|   176.61Mb|  11302.88|  0.47|  289.89|
|  145G | seqwr |    8|   183.18Mb|  11723.75|  0.29|  530.40|
|  145G | seqwr |   16|   165.01Mb|  10560.46|  0.30|  749.92|
|  145G | seqwr |   32|   158.95Mb|  10172.95|  0.31|  953.30|
|  145G | seqwr |   64|   160.97Mb|  10302.13|  0.31|  2049.12|


==== Script ====

<code>
#!/bin/sh
set -u
set -x
set -e

for size in 100G; do
   for mode in seqwr seqrd rndrd rndwr rndrw; do
   for blksize in 16384 ; do
      ./sysbench --test=fileio --file-num=64 --file-total-size=$size prepare
      for threads in 1 4 8 16 32 64; do
         echo "====== testing $blksize in $threads threads"
         echo PARAMS $size $mode $threads $blksize > sysbench-size-$size-mode-$mode-threads-$threads-blksz-$blksize
         ./sysbench --test=fileio --file-total-size=$size --file-test-mode=$mode\
            --max-time=120 --max-requests=100000000 --num-threads=$threads --init-rng=on \
            --file-num=64 --file-extra-flags=direct --file-fsync-freq=0 --file-block-size=$blksize run \
            | tee -a sysbench-size-$size-mode-$mode-threads-$threads-blksz-$blksize 2>&1
      done
      ./sysbench --test=fileio --file-total-size=$size cleanup
   done
   done
done
</code>