Merge branch 'upstream-master' into 'master'

* upstream-master:
  ci: Add a step to patch kernel with temporary fixes
  sync: latest libbpf changes from kernel
  libbpf: Rework feature-probing APIs
  libbpf: Avoid reading past ELF data section end when copying license
  ci: add LIRC kernel config
  vmtest: blacklist bpf_loop and get_func_args_test for s390x
  sync: latest libbpf changes from kernel
  sync: auto-generate latest BPF helpers
  libbpf: Mark bpf_object__find_program_by_title API deprecated.
  libbpf: Auto-bump RLIMIT_MEMLOCK if kernel needs it for BPF
  libbpf: Add sane strncpy alternative and use it internally
  libbpf: Fix potential uninit memory read
  libbpf: Add doc comments for bpf_program__(un)pin()
  bpf: Add get_func_[arg|ret|arg_cnt] helpers
  libbpf: Don't validate TYPE_ID relo's original imm value
  bpf: Add bpf_strncmp helper
  libbpf: Fix gen_loader assumption on number of programs.
  Bonding: add arp_missed_max option
  sync: latest libbpf changes from kernel
  sync: auto-generate latest BPF helpers
  tools: sync uapi/linux/if_link.h header
  libbpf: Add "bool skipped" to struct bpf_map
  libbpf: Fix typo in btf__dedup@LIBBPF_0.0.2 definition
  libbpf: Deprecate bpf_object__load_xattr()
  libbpf: Add per-program log buffer setter and getter
  libbpf: Preserve kernel error code and remove kprobe prog type guessing
  libbpf: Improve logging around BPF program loading
  libbpf: Allow passing user log setting through bpf_object_open_opts
  libbpf: Allow passing preallocated log_buf when loading BTF into kernel
  libbpf: Add OPTS-based bpf_btf_load() API
  libbpf: Fix bpf_prog_load() log_buf logic for log_level 0
  libbpf: Add doc comments in libbpf.h
  libbpf: Fix trivial typo
  libbpf: Reduce bpf_core_apply_relo_insn() stack usage.
  libbpf: Deprecate bpf_prog_load_xattr() API
  libbpf: Add API to get/set log_level at per-program level
  libbpf: Use __u32 fields in bpf_map_create_opts
  libbpf: Clean gen_loader's attach kind.
  libbpf: Support init of inner maps in light skeleton.
  libbpf: Use CO-RE in the kernel in light skeleton.
  libbpf: Cleanup struct bpf_core_cand.
  bpf: Pass a set of bpf_core_relo-s to prog_load command.
  bpf: Define enum bpf_core_relo_kind as uapi.
  bpf: Prepare relo_core.c for kernel duty.
  libbpf: Replace btf__type_by_id() with btf_type_by_id().
  libbpf: Avoid reload of imm for weak, unresolved, repeating ksym
  libbpf: Avoid double stores for success/failure case of ksym relocations
  bpf: Add bpf_loop helper
  libbpf: Remove duplicate assignments
  sync: improve patch application process by using patch command
  Migrate vmtest to modular actions in libbpf/ci
  ci: upgrade s390x runner to v2.285.0
  ci: fix test on /exitstatus existence and size
  ci: whitelist legacy_printk tests on 4.9 and 5.5
  ci: add folding markers to avoid getting output out of sections
  ci: carry on after selftest failure and report test group results
  ci: change VM's /exitstatus format to prepare it for several results
  ci: create helpers for formatting errors and notices
  sync: latest libbpf changes from kernel
  libbpf: Silence uninitialized warning/error in btf_dump_dump_type_data
  libbpf: Support static initialization of BPF_MAP_TYPE_PROG_ARRAY
  bpf, mips: Fix build errors about __NR_bpf undeclared
  sync: try harder when git am -3 fails
  sync: latest libbpf changes from kernel
  libbpf: Fix using invalidated memory in bpf_linker
  libbpf: Fix glob_syms memory leak in bpf_linker
  libbpf: Don't call libc APIs with NULL pointers
  libbpf: Fix potential misaligned memory access in btf_ext__new()
  libbpf: Prevent deprecation warnings in xsk.c
  libbpf: Use bpf_map_create() consistently internally
  libbpf: Unify low-level map creation APIs w/ new bpf_map_create()
  libbpf: Load global data maps lazily on legacy kernels
  sync: latest libbpf changes from kernel
  libbpf: Change bpf_program__set_extra_flags to bpf_program__set_flags
  libbpf: Accommodate DWARF/compiler bug with duplicated structs
  libbpf: Add runtime APIs to query libbpf version
  bpf, docs: Fix ordering of bpf documentation
  ci: create summary for tests and account for bpftool checks result
  ci: pass shutdown fold description to fold command
  Makefile: enforce gnu89 standard
  Makefile: don't hide relevant parts of file path
  sync: latest libbpf changes from kernel
  sync: auto-generate latest BPF helpers
  libbpf: Perform map fd cleanup for gen_loader in case of error
  bpf: Change value of MAX_TAIL_CALL_CNT from 32 to 33
  libbpf: Fix a couple of missed btf_type_tag handling in btf.c
  ci: add s390x vmtest
  vmtest: use libguestfs for disk image manipulations
  vmtest: add s390x blacklist
  vmtest: tweak qemu invocation for s390x
  vmtest: add s390x image
  vmtest: add s390x config
  vmtest: add debootstrap-based mkrootfs script
  vmtest: do not install lld
  vmtest: use python3-docutils instead of python-docutils
  sync: latest libbpf changes from kernel
  sync: auto-generate latest BPF helpers
  libbpf: Support BTF_KIND_TYPE_TAG
  bpf: Support BTF_KIND_TYPE_TAG for btf_type_tag attributes
  libbpf: Make perf_buffer__new() use OPTS-based interface
  libbpf: Ensure btf_dump__new() and btf_dump_opts are future-proof
  libbpf: Turn btf_dedup_opts into OPTS-based struct
  libbpf: Add ability to get/set per-program load flags
  bpf: Add ingress_ifindex to bpf_sk_lookup
  bpf: Introduce helper bpf_find_vma
  libbpf: Free up resources used by inner map definition
  libbpf: Remove deprecation attribute from struct bpf_prog_prep_result
  libbpf: Stop using to-be-deprecated APIs
  libbpf: Remove internal use of deprecated bpf_prog_load() variants
  libbpf: Unify low-level BPF_PROG_LOAD APIs into bpf_prog_load()
  libbpf: Pass number of prog load attempts explicitly
  libbpf: Rename DECLARE_LIBBPF_OPTS into LIBBPF_OPTS
  sync: latest libbpf changes from kernel
  libbpf: Fix lookup_and_delete_elem_flags error reporting
  libbpf: Fix non-C89 loop variable declaration in gen_loader.c
  libbpf: Deprecate bpf_program__load() API
  libbpf: Improve ELF relo sanitization
  libbpf: Fix section counting logic
  libbpf: Validate that .BTF and .BTF.ext sections contain data
  libbpf: Improve sanity checking during BTF fix up
  libbpf: Detect corrupted ELF symbols section
  sync: latest libbpf changes from kernel
  libbpf: Deprecate bpf_program__get_prog_info_linear
  libbpf: Deprecate AF_XDP support
  bpf: Add alignment padding for "map_extra" + consolidate holes
  include: add BPF_ALU32_IMM macro implementation
  sync: latest libbpf changes from kernel
  sync: auto-generate latest BPF helpers
  riscv, libbpf: Add RISC-V (RV64) support to bpf_tracing.h
  libbpf: Use O_CLOEXEC uniformly when opening fds
  libbpf: Ensure that BPF syscall fds are never 0, 1, or 2
  libbpf: Add weak ksym support to gen_loader
  libbpf: Add typeless ksym support to gen_loader
  bpf: Add bpf_kallsyms_lookup_name helper
  libbpf: Add "map_extra" as a per-map-type extra flag
  bpf: Add bloom filter map implementation
  libbpf: Deprecate bpf_objects_list
  libbpf: Use __BYTE_ORDER__
  libbpf: Fix endianness detection in BPF_CORE_READ_BITFIELD_PROBED()
  libbpf: Deprecate ambiguously-named bpf_program__size() API
  libbpf: Deprecate multi-instance bpf_program APIs
  libbpf: Add ability to fetch bpf_program's underlying instructions
  libbpf: Fix off-by-one bug in bpf_core_apply_relo()
  sync: remove redundant test on $BPF_BRANCH
  sync: fix comment for commit_signature() (subject instead of hash)
  README: add links to BPF CO-RE reference guide
  [ci] turn on CIFuzz
  sync: latest libbpf changes from kernel
  sync: auto-generate latest BPF helpers
  libbpf: Fix BTF header parsing checks
  libbpf: Fix overflow in BTF sanity checks
  libbpf: Use func name when pinning programs with LIBBPF_STRICT_SEC_NAME
  libbpf: Add btf__type_cnt() and btf__raw_data() APIs
  libbpf: Fix memory leak in btf__dedup()
  libbpf: Fix the use of aligned attribute
  libbpf: Simplify look up by name of internal maps
  libbpf: Support multiple .rodata.* and .data.* BPF maps
  libbpf: Remove assumptions about uniqueness of .rodata/.data/.bss maps
  libbpf: Use Elf64-specific types explicitly for dealing with ELF
  libbpf: Extract ELF processing state into separate struct
  libbpf: Deprecate btf__finalize_data() and move it into libbpf.c
  bpf: Add verified_insns to bpf_prog_info and fdinfo
  libbpf: Fix ptr_is_aligned() usages
  bpf: Add bpf_skc_to_unix_sock() helper
  libbpf: Fix dumping non-aligned __int128
  libbpf: Fix dumping big-endian bitfields
  libbpf: Migrate internal use of bpf_program__get_prog_info_linear
  bpf: Rename BTF_KIND_TAG to BTF_KIND_DECL_TAG
  README: add OSS-Fuzz badge
  [coverity] skip forks
  sync: latest libbpf changes from kernel
  libbpf: Support detecting and attaching of writable tracepoint program
  sync: latest libbpf changes from kernel
  libbpf: Fix memory leak in strset
  libbpf: Fix segfault in light skeleton for objects without BTF
  libbpf: Deprecate bpf_{map,program}__{prev,next} APIs since v0.7
  libbpf: Deprecate bpf_object__unload() API since v0.6
  libbpf: Add API documentation convention guidelines
  libbpf: Add API that copies all BTF types from one BTF object to another
  libbpf: Update gen_loader to emit BTF_KIND_FUNC relocations
  libbpf: Resolve invalid weak kfunc calls with imm = 0, off = 0
  libbpf: Support kernel module function calls
  libbpf: Support uniform BTF-defined key/value specification across all BPF maps
  libbpf: Fix skel_internal.h to set errno on loader retval < 0
  libbpf: Properly ignore STT_SECTION symbols in legacy map definitions
  libbpf: Make gen_loader data aligned.
  Remove caching of pahole test result
  sync: latest libbpf changes from kernel
  sync: auto-generate latest BPF helpers
  libbpf: Fix segfault in static linker for objects without BTF
  selftests/bpf: Switch sk_lookup selftests to strict SEC("sk_lookup") use
  libbpf: Add opt-in strict BPF program section name handling logic
  libbpf: Complete SEC() table unification for BPF_APROG_SEC/BPF_EAPROG_SEC
  libbpf: Refactor ELF section handler definitions
  libbpf: Reduce reliance of attach_fns on sec_def internals
  libbpf: Refactor internal sec_def handling to enable pluggability
  libbpf: Add "tc" SEC_DEF which is a better name for "classifier"
  libbpf: Ignore STT_SECTION symbols in 'maps' section
  libbpf: Add legacy uprobe attaching support
  libbpf: Refactor and simplify legacy kprobe code
  libbpf: Fix memory leak in legacy kprobe attach logic
  libbpf: Add doc comments in libbpf.h
  bpf: Clarify data_len param in bpf_snprintf and bpf_seq_printf comments
  libbpf: Use static const fmt string in __bpf_printk
  libbpf: Modify bpf_printk to choose helper based on arg count
  bpf: Add bpf_trace_vprintk helper
  libbpf: Constify all high-level program attach APIs
  libbpf: Schedule open_opts.attach_prog_fd deprecation since v0.7
  libbpf: Allow skipping attach_func_name in bpf_program__set_attach_target()
  libbpf: Deprecated bpf_object_open_opts.relaxed_core_relocs
  libbpf: Use pre-setup sec_def in libbpf_find_attach_btf_id()
  bpf: Update bpf_get_smp_processor_id() documentation
  ci: use -a/-d with exact string match for black/white-listing
  vmtest: temporarily blacklist spammy selftests
  sync: latest libbpf changes from kernel
  sync: auto-generate latest BPF helpers
  libbpf: Add sphinx code documentation comments
  libbpf: Add support for BTF_KIND_TAG
  libbpf: Rename btf_{hash,equal}_int to btf_{hash,equal}_int_tag
  bpf: Support for new btf kind BTF_KIND_TAG
  btf: Change BTF_KIND_* macros to enums
  libbpf: Minimize explicit iterator of section definition array
  libbpf: Simplify BPF program auto-attach code
  libbpf: Ensure BPF prog types are set before relocations
  libbpf: Introduce legacy kprobe events support
  libbpf: Make libbpf_version.h non-auto-generated
  bpf: Introduce helper bpf_get_branch_snapshot
  bpf: Add hardware timestamp field to __sk_buff
  libbpf: Add LIBBPF_DEPRECATED_SINCE macro for scheduling API deprecations
  ipv6: add IFLA_INET6_RA_MTU to expose mtu value
  makefile: add libbpf_version.h to the list of installed headers
  ci: use test_progs -l option to generate the list of whitelisted tests
  Add enum to be displayed in documentation
  libbpf: Fix build with latest gcc/binutils with LTO
  libbpf: Change bpf_object_skeleton data field to const pointer
  libbpf: Don't crash on object files with no symbol tables
  ci: blacklist few new selftests for 5.5
  sync: latest libbpf changes from kernel
  sync: auto-generate latest BPF helpers
  bpf: Add bpf_task_pt_regs() helper
  bpf: Migrate cgroup_bpf to internal cgroup_bpf_attach_type enum
  ci: don't hang on kernel crashes in qemu
  sync: latest libbpf changes from kernel
  libbpf: Rename libbpf documentation index file
  Fix path to Doxygen source code input
  ci: blacklist new selftests on 5.5
  sync: latest libbpf changes from kernel
  sync: auto-generate latest BPF helpers
  libbpf: Add uprobe ref counter offset support for USDT semaphores
  libbpf: Add bpf_cookie to perf_event, kprobe, uprobe, and tp attach APIs
  libbpf: Add bpf_cookie support to bpf_link_create() API
  libbpf: Use BPF perf link when supported by kernel
  libbpf: Remove unused bpf_link's destroy operation, but add dealloc
  bpf: Add bpf_get_attach_cookie() BPF helper to access bpf_cookie value
  bpf: Allow to specify user-provided bpf_cookie for BPF perf links
  bpf: Implement minimal BPF perf link
  libbpf: Support weak typed ksyms.
  libbpf, doc: Eliminate warnings in libbpf_naming_convention
  bonding: add new option lacp_active
  sync: fix up docs sync path mapping
  docs: initial set of libbpf docs
  docs: reconfigure libbpf documentation syncing
  ci: run script to test bpftool types/options sync
  Make CI tests compatible with vanilla kernel tree
  ci: restore all temporary disabled tests
  Add a test step to produce a minimal binary using libbpf.
  makefile: fix missing object for static compilation
  ci: make CI build log less verbose
  ci: blacklist newly migrated netcnt selftest
  sync: latest libbpf changes from kernel
  libbpf: Do not close un-owned FD 0 on errors
  libbpf: Fix probe for BPF_PROG_TYPE_CGROUP_SOCKOPT
  libbpf: Add btf__load_vmlinux_btf/btf__load_module_btf
  libbpf: Add split BTF support for btf__load_from_kernel_by_id()
  libbpf: Rename btf__get_from_id() as btf__load_from_kernel_by_id()
  libbpf: Rename btf__load() as btf__load_into_kernel()
  libbpf: Return non-null error on failures in libbpf_find_prog_btf_id()
  libbpf: Fix race when pinning maps in parallel
  libbpf: Fix comment typo
  libbpf: Split CO-RE logic into relo_core.c.
  libbpf: Move CO-RE types into relo_core.h.
  libbpf: Split bpf_core_apply_relo() into bpf_program independent helper.
  libbpf: Cleanup the layering between CORE and bpf_program.
  libbpf: Add bpf_map__pin_path function
  libbpf: Export bpf_program__attach_kprobe_opts function
  libbpf: Allow decimal offset for kprobes
  libbpf: Fix func leak in attach_kprobe
  Fix text grouping issue on github actions
  ci: bump nightly Clang/LLVM version to 14
  ci: improve CI UX a little by setting names a hiding debug
  README: remove Travis CI build badge
  ci: fix log folding in Github Actions
  ci: add cancel-in-progress behavior for main test CI workflow
  ci: add Github Actions status badge
  sync: latest libbpf changes from kernel
  libbpf: Propagate errors when retrieving enum value for typed data display
  libbpf: Avoid use of __int128 in typed dump display
  README: State the source origin more prominently.
  Makefile: Default LIBSUBDIR to lib64 on 64bit architectures.
  sync: latest libbpf changes from kernel
  libbpf: Fix removal of inner map in bpf_object__create_map
  libbpf: Btf typed dump does not need to allocate dump data
  libbpf: Fix compilation errors on ppc64le for btf dump typed data
  libbpf: Clarify/fix unaligned data issues for btf typed dump
  Update README.md
  Update README.md
  Run apt-get update as a first step for GH actions
  vmtest: blacklist 5.5 selftests
  ci: temporary work-around to get green CI builds back
  ci: disable -Wstringop-truncation for GCC10 configurations as well
  vmtest: regenerate latest vmlinux.h
  sync: latest libbpf changes from kernel
  sync: auto-generate latest BPF helpers
  libbpf: BTF dumper support for typed data
  libbpf: Fix the possible memory leak on error
  libbpf: Introduce 'btf_custom_path' to 'bpf_obj_open_opts'
  libbpf: Allow specification of "kprobe/function+offset"
  libbpf: Add bpf_program__attach_kprobe_opts function
  bpf: Add bpf_get_func_ip helper for kprobe programs
  bpf: Add bpf_get_func_ip helper for tracing programs
  bpf: Introduce bpf timers.
  bpf: Fix a typo of reuseport map in bpf.h.
  bpf: Sync tools/include/uapi/linux/bpf.h
  libbpf: Fix reuse of pinned map on older kernel
  libbpf: Restore errno return for functions that were already returning it
  libbpf: Switch to void * casting in netlink helpers
  libbpf: Add request buffer type for netlink messages
  libbpf: Add extra BPF_PROG_TYPE check to bpf_object__probe_loading
  include: initial sync of pkt_cls.h and pkt_sched.h
  sync uapi headers linux/pkt_cls.h and linux/pkt_sched.h
  create ondemand vmtest workflow

Bug: 203823368
Test: m libbpf
Signed-off-by: Connor O'Brien <connoro@google.com>
Change-Id: Ia9a76c029d9f528aeee612a3587887a314cd3e22
tree: 8c0537edb9dc3872ee13281e2a09e98280f24a43
  1. .github/
  2. android/
  3. docs/
  4. include/
  5. scripts/
  6. src/
  7. travis-ci/
  8. .lgtm.yml
  9. .readthedocs.yaml
  10. .travis.yml
  11. Android.bp
  12. BPF-CHECKPOINT-COMMIT
  13. CHECKPOINT-COMMIT
  14. LICENSE.BSD-2-Clause
  15. LICENSE.LGPL-2.1
  16. METADATA
  17. MODULE_LICENSE_BSD
  18. OWNERS
  19. README.md
README.md

This is a mirror of bpf-next Linux source tree's tools/lib/bpf directory plus its supporting header files.

All the gory details of syncing can be found in scripts/sync-kernel.sh script.

Some header files in this repo (include/linux/*.h) are reduced versions of their counterpart files at bpf-next's tools/include/linux/*.h to make compilation successful.

BPF/libbpf usage and questions

Please check out libbpf-bootstrap and the companion blog post for the examples of building BPF applications with libbpf. libbpf-tools are also a good source of the real-world libbpf-based tracing tools.

See also “BPF CO-RE reference guide” for the coverage of practical aspects of building BPF CO-RE applications and “BPF CO-RE” for general introduction into BPF portability issues and BPF CO-RE origins.

All general BPF questions, including kernel functionality, libbpf APIs and their application, should be sent to bpf@vger.kernel.org mailing list. You can subscribe to it here and search its archive here. Please search the archive before asking new questions. It very well might be that this was already addressed or answered before.

bpf@vger.kernel.org is monitored by many more people and they will happily try to help you with whatever issue you have. This repository's PRs and issues should be opened only for dealing with issues pertaining to specific way this libbpf mirror repo is set up and organized.

Build Github Actions Builds & Tests Total alerts Coverity OSS-Fuzz Status

libelf is an internal dependency of libbpf and thus it is required to link against and must be installed on the system for applications to work. pkg-config is used by default to find libelf, and the program called can be overridden with PKG_CONFIG.

If using pkg-config at build time is not desired, it can be disabled by setting NO_PKG_CONFIG=1 when calling make.

To build both static libbpf.a and shared libbpf.so:

$ cd src
$ make

To build only static libbpf.a library in directory build/ and install them together with libbpf headers in a staging directory root/:

$ cd src
$ mkdir build root
$ BUILD_STATIC_ONLY=y OBJDIR=build DESTDIR=root make install

To build both static libbpf.a and shared libbpf.so against a custom libelf dependency installed in /build/root/ and install them together with libbpf headers in a build directory /build/root/:

$ cd src
$ PKG_CONFIG_PATH=/build/root/lib64/pkgconfig DESTDIR=/build/root make install

Distributions

Distributions packaging libbpf from this mirror:

Benefits of packaging from the mirror over packaging from kernel sources:

  • Consistent versioning across distributions.
  • No ties to any specific kernel, transparent handling of older kernels. Libbpf is designed to be kernel-agnostic and work across multitude of kernel versions. It has built-in mechanisms to gracefully handle older kernels, that are missing some of the features, by working around or gracefully degrading functionality. Thus libbpf is not tied to a specific kernel version and can/should be packaged and versioned independently.
  • Continuous integration testing via TravisCI.
  • Static code analysis via LGTM and Coverity.

Package dependencies of libbpf, package names may vary across distros:

  • zlib
  • libelf

BPF CO-RE (Compile Once – Run Everywhere)

Libbpf supports building BPF CO-RE-enabled applications, which, in contrast to BCC, do not require Clang/LLVM runtime being deployed to target servers and doesn't rely on kernel-devel headers being available.

It does rely on kernel to be built with BTF type information, though. Some major Linux distributions come with kernel BTF already built in:

  • Fedora 31+
  • RHEL 8.2+
  • OpenSUSE Tumbleweed (in the next release, as of 2020-06-04)
  • Arch Linux (from kernel 5.7.1.arch1-1)
  • Manjaro (from kernel 5.4 if compiled after 2021-06-18)
  • Ubuntu 20.10
  • Debian 11 (amd64/arm64)

If your kernel doesn‘t come with BTF built-in, you’ll need to build custom kernel. You'll need:

  • pahole 1.16+ tool (part of dwarves package), which performs DWARF to BTF conversion;
  • kernel built with CONFIG_DEBUG_INFO_BTF=y option;
  • you can check if your kernel has BTF built-in by looking for /sys/kernel/btf/vmlinux file:
$ ls -la /sys/kernel/btf/vmlinux
-r--r--r--. 1 root root 3541561 Jun  2 18:16 /sys/kernel/btf/vmlinux

To develop and build BPF programs, you'll need Clang/LLVM 10+. The following distributions have Clang/LLVM 10+ packaged by default:

  • Fedora 32+
  • Ubuntu 20.04+
  • Arch Linux
  • Ubuntu 20.10 (LLVM 11)
  • Debian 11 (LLVM 11)
  • Alpine 3.13+

Otherwise, please make sure to update it on your system.

The following resources are useful to understand what BPF CO-RE is and how to use it:

License

This work is dual-licensed under BSD 2-clause license and GNU LGPL v2.1 license. You can choose between one of them if you use this work.

SPDX-License-Identifier: BSD-2-Clause OR LGPL-2.1