ceph@ceph8:fio$ fio 1job4k.fio  

rand-read: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=128

fio-2.1.3

Starting 1 process

Jobs: 1 (f=1): [r] [100.0% done] [803.4MB/0KB/0KB /s] [206K/0/0 iops] [eta 00m:00s]

rand-read: (groupid=0, jobs=1): err= 0: pid=13061: Fri Jun  3 12:19:26 2016

  read : io=8,008.3MB, bw=825,470KB/s, iops=206,367, runt=  9,934msec

    slat (usec): min=1, max=95, avg= 1.08, stdev= 0.33

    clat (usec): min=0, max=142, avg= 0.29, stdev= 0.48

     lat (usec): min=1, max=143, avg= 1.44, stdev= 0.56

    clat percentiles (usec):

     |  1.00th=[    0],  5.00th=[    0], 10.00th=[    0], 20.00th=[    0],
     | 30.00th=[    0], 40.00th=[    0], 50.00th=[    0], 60.00th=[    0],
     | 70.00th=[    0], 80.00th=[    1], 90.00th=[    1], 95.00th=[    1],
     | 99.00th=[    1], 99.50th=[    1], 99.90th=[    1], 99.95th=[    1],
     | 99.99th=[    3]

    lat (usec) : 2=99.97%, 4=0.02%, 10=0.01%, 20=0.01%, 100=0.01%, 250=0.01%


  cpu          : usr=37.40%, sys=62.52%, ctx=852, majf=0, minf=8

  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%

     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     issued    : total=r=2,050,055/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):

   READ: io=8,008.3MB,     aggrb=825,470KB/s,
         minb=825,470KB/s, maxb=825,470KB/s,
         mint=9,934msec,   maxt=9,934msec

=============================================================


>> twemproxy

  "service": "nutcracker",
  "source": "hcd6",
   ...

  "testpoolhcd2": {

   ...

    "latency_min": 0,
    "latency_max": 0,
    "latency_p50": 0,
    "latency_p90": 0,
    "latency_p95": 0,
    "latency_p99": 0,
  }


>> libaio

$ ./fio 1job4k.fio --output-format=json

jobs pid=19322 started
{
  "fio version" : "fio-2.1.3",
  "jobs" : [
    {

      "jobname" : "rand-read",
      "groupid" : 0,
      "error" : 0,

"read":{"io_bytes" : 8,342,576,  "bw" : 840,223,

        "iops" : 210,055,     "runtime" : 9,929,

        "slat" : {"min" : 0,   "max"    : 116,
                  "mean": 1.01,"stddev" : 1.06},

        "clat" : {"min" : 0,   "max"    : 114,
                  "mean": 0.29,"stddev" : 0.75,
                  "percentile" : {
                 "1.000000" : 0,
                     "5.000000" : 0,
                "10.000000" : 0,
                "20.000000" : 0,
                "30.000000" : 0,
                "40.000000" : 0,
                "50.000000" : 0,
                "60.000000" : 0,
                "70.000000" : 0,
                "80.000000" : 1,
                "90.000000" : 1,
                "95.000000" : 1,
                "99.000000" : 1,
                "99.500000" : 1,
                "99.900000" : 1,
                "99.950000" : 1,
                "99.990000" : 4,}
            },

        "lat" : {
          "min" : 1,
          "max" : 134,
          "mean" : 1.38,
          "stddev" : 1.35
        },

        "bw_min" : 0,
        "bw_max" : 0,
        "bw_agg" : 0.00,
        "bw_mean" : 0.00,
        "bw_dev" : 0.00
      },
      "usr_cpu" : 37.52,
      "sys_cpu" : 62.40,
      "ctx" : 859,
      "majf" : 0,
      "minf" : 7,
      "iodepth_level" : {
        "1" : 100.00,
        "2" : 0.00,
          ...
        ">=64" : 0.00
      },

      "latency_us" : {
        "2"   : 99.96,  4"  : 0.02,  "10" : 0.01,    "20" : 0.01, "50" : 0.01, "100" : 0.01,
        "250" : 0.01, "500" : 0.00, "750" : 0.00,  "1000" : 0.00
      },
    }
  ]
}


>> sync

$ ./fio 1job4k.fio --output-format=json

jobs pid=19234 started
{
  "fio version" : "fio-2.1.3",
  "jobs" : [
    {

      "jobname" : "rand-read",
      "groupid" : 0,
      "error" : 0,
      "read":{"io_bytes" : 9,680,076, "bw"      : 974,536,
              "iops"     : 243,634,   "runtime" : 9,933,
      "slat":{"min"  : 0,    "max" : 0,
              "mean" : 0.00, "stddev" : 0.00},
      "clat":{"min"  : 0,    "max" : 121,
              "mean" : 0.80,  "stddev" : 0.80,
           "percentile" : {
                  "1.000000" : 0,
                  "5.000000" : 0,
                 "10.000000" : 0,
                  "20.000000" : 0,
                 "30.000000" : 1,
                 "40.000000" : 1,
                 "50.000000" : 1,
                 "60.000000" : 1,
                 "70.000000" : 1,
                 "80.000000" : 1,
                 "90.000000" : 1,
                 "95.000000" : 1,
                 "99.000000" : 1,
                  "99.500000" : 1,
                 "99.900000" : 2,
                 "99.950000" : 4,
                    "99.990000" : 8,}
             },
       "lat" : {"min" : 0, "max" : 121, "mean" : 0.84, "stddev" : 0.79},
       "bw_min" : 0,
       "bw_max" : 0,
       "bw_agg" : 0.00,
       "bw_mean" : 0.00,
       "bw_dev" : 0.00
      },

      "latency_us" : {
         "2" : 99.79,
         "4" : 0.14,
         "10" : 0.07,
         "20" : 0.01,
         "50": 0.01,
         "100": 0.01,
         "250" : 0.01 },
      "usr_cpu" : 37.68,
      "sys_cpu" : 62.20,
      "ctx" : 1038,
      "majf" : 0,
      "minf" : 6,

      "iodepth_level" : {
        "1" : 100.00,
        "2" :   0.00,
        ...
        ">=64": 0.00
      },

========================================================================

numjobs=1

[rand-read]
rw=randread
stonewall

[rand-read1]
rw=randread
stonewall


ceph@ceph8:fio-2.1.3$ ./fio 1job4k.fio --output-format=json

rand-read: (groupid=0, jobs=1): err= 0: pid=26866: Thu Jun 16 14:08:10 2016

  read : io=7957.1MB, bw=820062KB/s, iops=205015, runt=  9937msec
    slat (usec): min=0, max=138, avg= 1.09, stdev= 1.32

rand-read1: (groupid=1, jobs=1): err= 0: pid=26867: Thu Jun 16 14:08:10 2016

  read : io=8175.5MB, bw=842383KB/s, iops=210595, runt=  9938msec
    slat (usec): min=0, max=112, avg= 1.05, stdev= 0.28


{
  "fio version" : "fio-2.1.3",

  "jobs" : [
    {
      "jobname" : "rand-read",
      "groupid" : 0,
...
    },

    {
      "jobname" : "rand-read1",
      "groupid" : 1,
...
    }
  ]
}



ceph@ceph8:fio-2.1.3$ ./fio 1job4k.fio

 rand-read: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=128
 rand-read: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=128
 rand-read1: (g=1): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=128
 rand-read1: (g=1): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=128

  Starting 4 processes

  Jobs: 2 (f=2): [__rr] [100.0% done] [1369MB/0KB/0KB /s] [350K/0/0 iops] [eta 00m:00s]

rand-read:  (groupid=0, jobs=2): err= 0: pid=26982: Thu Jun 16 14:21:00 2016
  read : io=13875MB, bw=1396.2MB/s, iops=357417, runt=  9938msec

rand-read1: (groupid=1, jobs=2): err= 0: pid=26984: Thu Jun 16 14:21:00 2016
  read : io=13630MB, bw=1371.3MB/s, iops=351040, runt=  9940msec


Run status group 0 (all jobs):
   READ: io=13875MB, aggrb=1396.2MB/s, minb=1396.2MB/s, maxb=1396.2MB/s, mint=9938msec, maxt=9938msec

Run status group 1 (all jobs):
   READ: io=13630MB, aggrb=1371.3MB/s, minb=1371.3MB/s, maxb=1371.3MB/s, mint=9940msec, maxt=99