ceph@ceph8:fio$ fio 1job4k.fio
rand-read: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=128
fio-2.1.3
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [803.4MB/0KB/0KB /s] [206K/0/0 iops] [eta 00m:00s]
rand-read: (groupid=0, jobs=1): err= 0: pid=13061: Fri Jun 3 12:19:26 2016
read : io=8,008.3MB, bw=825,470KB/s, iops=206,367, runt= 9,934msec
slat (usec): min=1, max=95, avg= 1.08, stdev= 0.33
clat (usec): min=0, max=142, avg= 0.29, stdev= 0.48
lat (usec): min=1, max=143, avg= 1.44, stdev= 0.56
clat percentiles (usec):
| 1.00th=[ 0], 5.00th=[ 0], 10.00th=[ 0], 20.00th=[ 0],
| 30.00th=[ 0], 40.00th=[ 0], 50.00th=[ 0], 60.00th=[ 0],
| 70.00th=[ 0], 80.00th=[ 1], 90.00th=[ 1], 95.00th=[ 1],
| 99.00th=[ 1], 99.50th=[ 1], 99.90th=[ 1], 99.95th=[ 1],
| 99.99th=[ 3]
lat (usec) : 2=99.97%, 4=0.02%, 10=0.01%, 20=0.01%, 100=0.01%, 250=0.01%
cpu : usr=37.40%, sys=62.52%, ctx=852, majf=0, minf=8
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=2,050,055/w=0/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=8,008.3MB, aggrb=825,470KB/s,
minb=825,470KB/s, maxb=825,470KB/s,
mint=9,934msec, maxt=9,934msec
=============================================================
>> twemproxy
"service": "nutcracker",
"source": "hcd6",
...
"testpoolhcd2": {
...
"latency_min": 0,
"latency_max": 0,
"latency_p50": 0,
"latency_p90": 0,
"latency_p95": 0,
"latency_p99": 0,
}
>> libaio
$ ./fio 1job4k.fio --output-format=json
jobs pid=19322 started
{
"fio version" : "fio-2.1.3",
"jobs" : [
{
"jobname" : "rand-read",
"groupid" : 0,
"error" : 0,
"read":{"io_bytes" : 8,342,576, "bw" : 840,223,
"iops" : 210,055, "runtime" : 9,929,
"slat" : {"min" : 0, "max" : 116,
"mean": 1.01,"stddev" : 1.06},
"clat" : {"min" : 0, "max" : 114,
"mean": 0.29,"stddev" : 0.75,
"percentile" : {
"1.000000" : 0,
"5.000000" : 0,
"10.000000" : 0,
"20.000000" : 0,
"30.000000" : 0,
"40.000000" : 0,
"50.000000" : 0,
"60.000000" : 0,
"70.000000" : 0,
"80.000000" : 1,
"90.000000" : 1,
"95.000000" : 1,
"99.000000" : 1,
"99.500000" : 1,
"99.900000" : 1,
"99.950000" : 1,
"99.990000" : 4,}
},
"lat" : {
"min" : 1,
"max" : 134,
"mean" : 1.38,
"stddev" : 1.35
},
"bw_min" : 0,
"bw_max" : 0,
"bw_agg" : 0.00,
"bw_mean" : 0.00,
"bw_dev" : 0.00
},
"usr_cpu" : 37.52,
"sys_cpu" : 62.40,
"ctx" : 859,
"majf" : 0,
"minf" : 7,
"iodepth_level" : {
"1" : 100.00,
"2" : 0.00,
...
">=64" : 0.00
},
"latency_us" : {
"2" : 99.96, 4" : 0.02, "10" : 0.01, "20" : 0.01, "50" : 0.01, "100" : 0.01,
"250" : 0.01, "500" : 0.00, "750" : 0.00, "1000" : 0.00
},
}
]
}
>> sync
$ ./fio 1job4k.fio --output-format=json
jobs pid=19234 started
{
"fio version" : "fio-2.1.3",
"jobs" : [
{
"jobname" : "rand-read",
"groupid" : 0,
"error" : 0,
"read":{"io_bytes" : 9,680,076, "bw" : 974,536,
"iops" : 243,634, "runtime" : 9,933,
"slat":{"min" : 0, "max" : 0,
"mean" : 0.00, "stddev" : 0.00},
"clat":{"min" : 0, "max" : 121,
"mean" : 0.80, "stddev" : 0.80,
"percentile" : {
"1.000000" : 0,
"5.000000" : 0,
"10.000000" : 0,
"20.000000" : 0,
"30.000000" : 1,
"40.000000" : 1,
"50.000000" : 1,
"60.000000" : 1,
"70.000000" : 1,
"80.000000" : 1,
"90.000000" : 1,
"95.000000" : 1,
"99.000000" : 1,
"99.500000" : 1,
"99.900000" : 2,
"99.950000" : 4,
"99.990000" : 8,}
},
"lat" : {"min" : 0, "max" : 121, "mean" : 0.84, "stddev" : 0.79},
"bw_min" : 0,
"bw_max" : 0,
"bw_agg" : 0.00,
"bw_mean" : 0.00,
"bw_dev" : 0.00
},
"latency_us" : {
"2" : 99.79,
"4" : 0.14,
"10" : 0.07,
"20" : 0.01,
"50": 0.01,
"100": 0.01,
"250" : 0.01 },
"usr_cpu" : 37.68,
"sys_cpu" : 62.20,
"ctx" : 1038,
"majf" : 0,
"minf" : 6,
"iodepth_level" : {
"1" : 100.00,
"2" : 0.00,
...
">=64": 0.00
},
========================================================================
numjobs=1
[rand-read]
rw=randread
stonewall
[rand-read1]
rw=randread
stonewall
ceph@ceph8:fio-2.1.3$ ./fio 1job4k.fio --output-format=json
rand-read: (groupid=0, jobs=1): err= 0: pid=26866: Thu Jun 16 14:08:10 2016
read : io=7957.1MB, bw=820062KB/s, iops=205015, runt= 9937msec
slat (usec): min=0, max=138, avg= 1.09, stdev= 1.32
rand-read1: (groupid=1, jobs=1): err= 0: pid=26867: Thu Jun 16 14:08:10 2016
read : io=8175.5MB, bw=842383KB/s, iops=210595, runt= 9938msec
slat (usec): min=0, max=112, avg= 1.05, stdev= 0.28
{
"fio version" : "fio-2.1.3",
"jobs" : [
{
"jobname" : "rand-read",
"groupid" : 0,
...
},
{
"jobname" : "rand-read1",
"groupid" : 1,
...
}
]
}
ceph@ceph8:fio-2.1.3$ ./fio 1job4k.fio
rand-read: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=128
rand-read: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=128
rand-read1: (g=1): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=128
rand-read1: (g=1): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=128
Starting 4 processes
Jobs: 2 (f=2): [__rr] [100.0% done] [1369MB/0KB/0KB /s] [350K/0/0 iops] [eta 00m:00s]
rand-read: (groupid=0, jobs=2): err= 0: pid=26982: Thu Jun 16 14:21:00 2016
read : io=13875MB, bw=1396.2MB/s, iops=357417, runt= 9938msec
rand-read1: (groupid=1, jobs=2): err= 0: pid=26984: Thu Jun 16 14:21:00 2016
read : io=13630MB, bw=1371.3MB/s, iops=351040, runt= 9940msec
Run status group 0 (all jobs):
READ: io=13875MB, aggrb=1396.2MB/s, minb=1396.2MB/s, maxb=1396.2MB/s, mint=9938msec, maxt=9938msec
Run status group 1 (all jobs):
READ: io=13630MB, aggrb=1371.3MB/s, minb=1371.3MB/s, maxb=1371.3MB/s, mint=9940msec, maxt=99