| Demonstrations of opensnoop, the Linux eBPF/bcc version. |
| |
| |
| opensnoop traces the open() syscall system-wide, and prints various details. |
| Example output: |
| |
| # ./opensnoop |
| PID COMM FD ERR PATH |
| 17326 <...> 7 0 /sys/kernel/debug/tracing/trace_pipe |
| 1576 snmpd 9 0 /proc/net/dev |
| 1576 snmpd 11 0 /proc/net/if_inet6 |
| 1576 snmpd 11 0 /proc/sys/net/ipv4/neigh/eth0/retrans_time_ms |
| 1576 snmpd 11 0 /proc/sys/net/ipv6/neigh/eth0/retrans_time_ms |
| 1576 snmpd 11 0 /proc/sys/net/ipv6/conf/eth0/forwarding |
| 1576 snmpd 11 0 /proc/sys/net/ipv6/neigh/eth0/base_reachable_time_ms |
| 1576 snmpd 11 0 /proc/sys/net/ipv4/neigh/lo/retrans_time_ms |
| 1576 snmpd 11 0 /proc/sys/net/ipv6/neigh/lo/retrans_time_ms |
| 1576 snmpd 11 0 /proc/sys/net/ipv6/conf/lo/forwarding |
| 1576 snmpd 11 0 /proc/sys/net/ipv6/neigh/lo/base_reachable_time_ms |
| 1576 snmpd 9 0 /proc/diskstats |
| 1576 snmpd 9 0 /proc/stat |
| 1576 snmpd 9 0 /proc/vmstat |
| 1956 supervise 9 0 supervise/status.new |
| 1956 supervise 9 0 supervise/status.new |
| 17358 run 3 0 /etc/ld.so.cache |
| 17358 run 3 0 /lib/x86_64-linux-gnu/libtinfo.so.5 |
| 17358 run 3 0 /lib/x86_64-linux-gnu/libdl.so.2 |
| 17358 run 3 0 /lib/x86_64-linux-gnu/libc.so.6 |
| 17358 run -1 6 /dev/tty |
| 17358 run 3 0 /proc/meminfo |
| 17358 run 3 0 /etc/nsswitch.conf |
| 17358 run 3 0 /etc/ld.so.cache |
| 17358 run 3 0 /lib/x86_64-linux-gnu/libnss_compat.so.2 |
| 17358 run 3 0 /lib/x86_64-linux-gnu/libnsl.so.1 |
| 17358 run 3 0 /etc/ld.so.cache |
| 17358 run 3 0 /lib/x86_64-linux-gnu/libnss_nis.so.2 |
| 17358 run 3 0 /lib/x86_64-linux-gnu/libnss_files.so.2 |
| 17358 run 3 0 /etc/passwd |
| 17358 run 3 0 ./run |
| ^C |
| |
| While tracing, the snmpd process opened various /proc files (reading metrics), |
| and a "run" process read various libraries and config files (looks like it |
| was starting up: a new process). |
| |
| opensnoop can be useful for discovering configuration and log files, if used |
| during application startup. |
| |
| |
| The -p option can be used to filter on a PID, which is filtered in-kernel. Here |
| I've used it with -T to print timestamps: |
| |
| ./opensnoop -Tp 1956 |
| TIME(s) PID COMM FD ERR PATH |
| 0.000000000 1956 supervise 9 0 supervise/status.new |
| 0.000289999 1956 supervise 9 0 supervise/status.new |
| 1.023068000 1956 supervise 9 0 supervise/status.new |
| 1.023381997 1956 supervise 9 0 supervise/status.new |
| 2.046030000 1956 supervise 9 0 supervise/status.new |
| 2.046363000 1956 supervise 9 0 supervise/status.new |
| 3.068203997 1956 supervise 9 0 supervise/status.new |
| 3.068544999 1956 supervise 9 0 supervise/status.new |
| |
| This shows the supervise process is opening the status.new file twice every |
| second. |
| |
| |
| The -U option include UID on output: |
| |
| # ./opensnoop -U |
| UID PID COMM FD ERR PATH |
| 0 27063 vminfo 5 0 /var/run/utmp |
| 103 628 dbus-daemon -1 2 /usr/local/share/dbus-1/system-services |
| 103 628 dbus-daemon 18 0 /usr/share/dbus-1/system-services |
| 103 628 dbus-daemon -1 2 /lib/dbus-1/system-services |
| |
| |
| The -u option filtering UID: |
| |
| # ./opensnoop -Uu 1000 |
| UID PID COMM FD ERR PATH |
| 1000 30240 ls 3 0 /etc/ld.so.cache |
| 1000 30240 ls 3 0 /lib/x86_64-linux-gnu/libselinux.so.1 |
| 1000 30240 ls 3 0 /lib/x86_64-linux-gnu/libc.so.6 |
| 1000 30240 ls 3 0 /lib/x86_64-linux-gnu/libpcre.so.3 |
| 1000 30240 ls 3 0 /lib/x86_64-linux-gnu/libdl.so.2 |
| 1000 30240 ls 3 0 /lib/x86_64-linux-gnu/libpthread.so.0 |
| |
| The -x option only prints failed opens: |
| |
| # ./opensnoop -x |
| PID COMM FD ERR PATH |
| 18372 run -1 6 /dev/tty |
| 18373 run -1 6 /dev/tty |
| 18373 multilog -1 13 lock |
| 18372 multilog -1 13 lock |
| 18384 df -1 2 /usr/share/locale/en_US.UTF-8/LC_MESSAGES/coreutils.mo |
| 18384 df -1 2 /usr/share/locale/en_US.utf8/LC_MESSAGES/coreutils.mo |
| 18384 df -1 2 /usr/share/locale/en_US/LC_MESSAGES/coreutils.mo |
| 18384 df -1 2 /usr/share/locale/en.UTF-8/LC_MESSAGES/coreutils.mo |
| 18384 df -1 2 /usr/share/locale/en.utf8/LC_MESSAGES/coreutils.mo |
| 18384 df -1 2 /usr/share/locale/en/LC_MESSAGES/coreutils.mo |
| 18385 run -1 6 /dev/tty |
| 18386 run -1 6 /dev/tty |
| |
| This caught a df command failing to open a coreutils.mo file, and trying from |
| different directories. |
| |
| The ERR column is the system error number. Error number 2 is ENOENT: no such |
| file or directory. |
| |
| |
| A maximum tracing duration can be set with the -d option. For example, to trace |
| for 2 seconds: |
| |
| # ./opensnoop -d 2 |
| PID COMM FD ERR PATH |
| 2191 indicator-multi 11 0 /sys/block |
| 2191 indicator-multi 11 0 /sys/block |
| 2191 indicator-multi 11 0 /sys/block |
| 2191 indicator-multi 11 0 /sys/block |
| 2191 indicator-multi 11 0 /sys/block |
| |
| |
| The -n option can be used to filter on process name using partial matches: |
| |
| # ./opensnoop -n ed |
| |
| PID COMM FD ERR PATH |
| 2679 sed 3 0 /etc/ld.so.cache |
| 2679 sed 3 0 /lib/x86_64-linux-gnu/libselinux.so.1 |
| 2679 sed 3 0 /lib/x86_64-linux-gnu/libc.so.6 |
| 2679 sed 3 0 /lib/x86_64-linux-gnu/libpcre.so.3 |
| 2679 sed 3 0 /lib/x86_64-linux-gnu/libdl.so.2 |
| 2679 sed 3 0 /lib/x86_64-linux-gnu/libpthread.so.0 |
| 2679 sed 3 0 /proc/filesystems |
| 2679 sed 3 0 /usr/lib/locale/locale-archive |
| 2679 sed -1 2 |
| 2679 sed 3 0 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache |
| 2679 sed 3 0 /dev/null |
| 2680 sed 3 0 /etc/ld.so.cache |
| 2680 sed 3 0 /lib/x86_64-linux-gnu/libselinux.so.1 |
| 2680 sed 3 0 /lib/x86_64-linux-gnu/libc.so.6 |
| 2680 sed 3 0 /lib/x86_64-linux-gnu/libpcre.so.3 |
| 2680 sed 3 0 /lib/x86_64-linux-gnu/libdl.so.2 |
| 2680 sed 3 0 /lib/x86_64-linux-gnu/libpthread.so.0 |
| 2680 sed 3 0 /proc/filesystems |
| 2680 sed 3 0 /usr/lib/locale/locale-archive |
| 2680 sed -1 2 |
| ^C |
| |
| This caught the 'sed' command because it partially matches 'ed' that's passed |
| to the '-n' option. |
| |
| |
| The -e option prints out extra columns; for example, the following output |
| contains the flags passed to open(2), in octal: |
| |
| # ./opensnoop -e |
| PID COMM FD ERR FLAGS PATH |
| 28512 sshd 10 0 00101101 /proc/self/oom_score_adj |
| 28512 sshd 3 0 02100000 /etc/ld.so.cache |
| 28512 sshd 3 0 02100000 /lib/x86_64-linux-gnu/libwrap.so.0 |
| 28512 sshd 3 0 02100000 /lib/x86_64-linux-gnu/libaudit.so.1 |
| 28512 sshd 3 0 02100000 /lib/x86_64-linux-gnu/libpam.so.0 |
| 28512 sshd 3 0 02100000 /lib/x86_64-linux-gnu/libselinux.so.1 |
| 28512 sshd 3 0 02100000 /lib/x86_64-linux-gnu/libsystemd.so.0 |
| 28512 sshd 3 0 02100000 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.2 |
| 28512 sshd 3 0 02100000 /lib/x86_64-linux-gnu/libutil.so.1 |
| |
| |
| The -f option filters based on flags to the open(2) call, for example: |
| |
| # ./opensnoop -e -f O_WRONLY -f O_RDWR |
| PID COMM FD ERR FLAGS PATH |
| 28084 clear_console 3 0 00100002 /dev/tty |
| 28084 clear_console -1 13 00100002 /dev/tty0 |
| 28084 clear_console -1 13 00100001 /dev/tty0 |
| 28084 clear_console -1 13 00100002 /dev/console |
| 28084 clear_console -1 13 00100001 /dev/console |
| 28051 sshd 8 0 02100002 /var/run/utmp |
| 28051 sshd 7 0 00100001 /var/log/wtmp |
| |
| |
| The --cgroupmap option filters based on a cgroup set. It is meant to be used |
| with an externally created map. |
| |
| # ./opensnoop --cgroupmap /sys/fs/bpf/test01 |
| |
| For more details, see docs/special_filtering.md |
| |
| |
| USAGE message: |
| |
| # ./opensnoop -h |
| usage: opensnoop.py [-h] [-T] [-U] [-x] [-p PID] [-t TID] |
| [--cgroupmap CGROUPMAP] [--mntnsmap MNTNSMAP] [-u UID] |
| [-d DURATION] [-n NAME] [-e] [-f FLAG_FILTER] [-F] |
| [-b BUFFER_PAGES] |
| |
| Trace open() syscalls |
| |
| optional arguments: |
| -h, --help show this help message and exit |
| -T, --timestamp include timestamp on output |
| -U, --print-uid print UID column |
| -x, --failed only show failed opens |
| -p PID, --pid PID trace this PID only |
| -t TID, --tid TID trace this TID only |
| --cgroupmap CGROUPMAP |
| trace cgroups in this BPF map only |
| --mntnsmap MNTNSMAP trace mount namespaces in this BPF map only |
| -u UID, --uid UID trace this UID only |
| -d DURATION, --duration DURATION |
| total duration of trace in seconds |
| -n NAME, --name NAME only print process names containing this name |
| -e, --extended_fields |
| show extended fields |
| -f FLAG_FILTER, --flag_filter FLAG_FILTER |
| filter on flags argument (e.g., O_WRONLY) |
| -F, --full-path show full path for an open file with relative path |
| -b BUFFER_PAGES, --buffer-pages BUFFER_PAGES |
| size of the perf ring buffer (must be a power of two |
| number of pages and defaults to 64) |
| |
| examples: |
| ./opensnoop # trace all open() syscalls |
| ./opensnoop -T # include timestamps |
| ./opensnoop -U # include UID |
| ./opensnoop -x # only show failed opens |
| ./opensnoop -p 181 # only trace PID 181 |
| ./opensnoop -t 123 # only trace TID 123 |
| ./opensnoop -u 1000 # only trace UID 1000 |
| ./opensnoop -d 10 # trace for 10 seconds only |
| ./opensnoop -n main # only print process names containing "main" |
| ./opensnoop -e # show extended fields |
| ./opensnoop -f O_WRONLY -f O_RDWR # only print calls for writing |
| ./opensnoop -F # show full path for an open file with relative path |
| ./opensnoop --cgroupmap mappath # only trace cgroups in this BPF map |
| ./opensnoop --mntnsmap mappath # only trace mount namespaces in the map |
