blob: 674be4998954d30c43669a592da73406de044983 [file] [log] [blame]
.TH cpuunclaimed 8 "2016-12-21" "USER COMMANDS"
.SH NAME
cpuunclaimed \- Sample CPU run queues and calculate unclaimed idle CPU. Uses Linux eBPF/bcc.
.SH SYNOPSIS
.B cpuunclaimed
[\-T] [\-j] [\-J] [interval [count]]
.SH DESCRIPTION
This tool samples the length of the run queues and determine when there are idle
CPUs, yet queued threads waiting their turn. It reports the amount of idle
(yet unclaimed by waiting threads) CPU as a system-wide percentage.
This situation can happen for a number of reasons:
.IP -
An application has been bound to some, but not all, CPUs, and has runnable
threads that cannot migrate to other CPUs due to this configuration.
.IP -
CPU affinity: an optimization that leaves threads on CPUs where the CPU
caches are warm, even if this means short periods of waiting while other
CPUs are idle. The wait period is tunale (see sysctl, kernel.sched*).
.IP -
Scheduler bugs.
.P
An unclaimed idle of < 1% is likely to be CPU affinity, and not usually a
cause for concern. By leaving the CPU idle, overall throughput of the system
may be improved. This tool is best for identifying larger issues, > 2%, due
to the coarseness of its 99 Hertz samples.
This is an experimental tool that currently works by use of sampling to
keep overheads low. Tool assumptions:
.IP -
CPU samples consistently fire around the same offset. There will sometimes
be a lag as a sample is delayed by higher-priority interrupts, but it is
assumed the subsequent samples will catch up to the expected offsets (as
is seen in practice). You can use -J to inspect sample offsets. Some
systems can power down CPUs when idle, and when they wake up again they
may begin firing at a skewed offset: this tool will detect the skew, print
an error, and exit.
.IP -
All CPUs are online (see ncpu).
.P
If this identifies unclaimed CPU, you can double check it by dumping raw
samples (-j), as well as using other tracing tools to instrument scheduler
events (although this latter approach has much higher overhead).
Since this uses BPF, only the root user can use this tool.
.SH REQUIREMENTS
CONFIG_BPF and bcc.
.SH EXAMPLES
.TP
Sample and calculate unclaimed idle CPUs, output every 1 second (default:
#
.B cpuunclaimed
.TP
Print 5 second summaries, 10 times:
#
.B cpuunclaimed 5 10
.TP
Print 1 second summaries with timestamps:
#
.B cpuunclaimed \-T 1
.TP
Raw dump of all samples (verbose), as comma-separated values:
#
.B cpuunclaimed \-j
.SH FIELDS
.TP
%CPU
CPU utilization as a system-wide percentage.
.TP
unclaimed idle
Percentage of CPU resources that were idle when work was queued on other CPUs,
as a system-wide percentage.
.TP
TIME
Time (HH:MM:SS)
.TP
TIMESTAMP_ns
Timestamp, nanoseconds.
.TP
CPU#
CPU ID.
.TP
OFFSET_ns_CPU#
Time offset that a sample fired within a sample group for this CPU.
.SH OVERHEAD
The overhead is expected to be low/negligible as this tool uses sampling at
99 Hertz (on all CPUs), which has a fixed and low cost, rather than sampling
every scheduler event as many other approaches use (which can involve
instrumenting millions of events per second). Sampled CPUs, run queue lengths,
and timestamps are written to ring buffers that are periodically read by
user space for reporting. Measure overhead in a test environment.
.SH SOURCE
This is from bcc.
.IP
https://github.com/iovisor/bcc
.PP
Also look in the bcc distribution for a companion _examples.txt file containing
example usage, output, and commentary for this tool.
.SH OS
Linux
.SH STABILITY
Unstable - in development.
.SH AUTHOR
Brendan Gregg
.SH SEE ALSO
runqlen(8)