| Demonstrations of biolatpcts, the Linux eBPF/bcc version. |
| |
| |
| biolatpcts traces block device I/O (disk I/O), and prints the latency |
| percentiles per I/O type. Example: |
| |
| # ./biolatpcts.py /dev/nvme0n1 |
| nvme0n1 p1 p5 p10 p16 p25 p50 p75 p84 p90 p95 p99 p100 |
| read 95us 175us 305us 515us 895us 985us 995us 1.5ms 2.5ms 3.5ms 4.5ms 10ms |
| write 5us 5us 5us 15us 25us 135us 765us 855us 885us 895us 965us 1.5ms |
| discard 5us 5us 5us 5us 135us 145us 165us 205us 385us 875us 1.5ms 2.5ms |
| flush 5us 5us 5us 5us 5us 5us 5us 5us 5us 1.5ms 4.5ms 5.5ms |
| [...] |
| |
| Unless changed with the -i option, the latency percentiles are printed every 3 |
| seconds. |
| |
| |
| Any number of custom percentiles can be requested with the -p option: |
| |
| # ./biolatpcts.py /dev/nvme0n1 -p 01,90.0,99.9,99.99,100.0 |
| |
| nvme0n1 01 90.0 99.9 99.99 100.0 |
| read 5us 4.5ms 16ms 22ms 26ms |
| write 15us 255us 365us 515us 2.5ms |
| discard - - - - - |
| flush 5us 5us 5us 5us 24ms |
| [...] |
| |
| Note that the target percentile strings are preserved as-is to facilitate |
| exact key matching when the output is consumed by another program. |
| |
| |
| When the output is consumed by another program, parsing can be tedious. The -j |
| option makes biolatpcts output results in json, one line per interval. |
| |
| # ./tools/biolatpcts.py /dev/nvme0n1 -j |
| {"read": {"1": 2.5e-05, "5": 3.5e-05, "10": 4.5e-05, "16": 0.000145, "25": 0.000195, "50": 0.000355, "75": 0.000605, "84": 0.000775, "90": 0.000965, "95": 0.0015, "99": 0.0025, "100": 0.0235}, "write": {"1": 5e-06, "5": 5e-06, "10": 5e-06, "16": 5e-06, "25": 1.5e-05, "50": 2.5e-05, "75": 4.5e-05, "84": 7.5e-05, "90": 0.000195, "95": 0.000665, "99": 0.0015, "100": 0.0035}, "discard": {"1": 0.0, "5": 0.0, "10": 0.0, "16": 0.0, "25": 0.0, "50": 0.0, "75": 0.0, "84": 0.0, "90": 0.0, "95": 0.0, "99": 0.0, "100": 0.0}, "flush": {"1": 0.0, "5": 0.0, "10": 0.0, "16": 0.0, "25": 0.0, "50": 0.0, "75": 0.0, "84": 0.0, "90": 0.0, "95": 0.0, "99": 0.0, "100": 0.0}} |
| [...] |
| |
| |
| By default biolatpcts measures the duration each IO was on the device. It can |
| be changed using the -w option. |
| |
| |
| USAGE message: |
| |
| usage: biolatpcts.py [-h] [-i INTERVAL] |
| [-w {from-rq-alloc,after-rq-alloc,on-device}] |
| [-p PCT,...] [-j] [--verbose] |
| DEV |
| |
| Monitor IO latency distribution of a block device |
| |
| positional arguments: |
| DEV Target block device (/dev/DEVNAME, DEVNAME or MAJ:MIN) |
| |
| optional arguments: |
| -h, --help show this help message and exit |
| -i INTERVAL, --interval INTERVAL |
| Report interval (default: 3) |
| -w {from-rq-alloc,after-rq-alloc,on-device}, --which {from-rq-alloc,after-rq-alloc,on-device} |
| Which latency to measure (default: on-device) |
| -p PCT,..., --pcts PCT,... |
| Percentiles to calculate (default: |
| 1,5,10,16,25,50,75,84,90,95,99,100) |
| -j, --json Output in json (default: False) |
| --verbose, -v |