sync: latest libbpf changes from kernel

Syncing latest libbpf commits from kernel repository.
Baseline bpf-next commit:   22541a9eeb0d968c133aaebd95fa59da3208e705
Checkpoint bpf-next commit: 155f556d64b1a48710f01305e14bb860734ed1e3
Baseline bpf commit:        6185266c5a853bb0f2a459e3ff594546f277609b
Checkpoint bpf commit:      002322402dafd846c424ffa9240a937f49b48c42

Andrii Nakryiko (11):
  libbpf: Add explicit padding to bpf_xdp_set_link_opts
  libbpf: provide NULL and KERNEL_VERSION macros in bpf_helpers.h
  libbpf: Expose btf_type_by_id() internally
  libbpf: Generalize BTF and BTF.ext type ID and strings iteration
  libbpf: Rename internal memory-management helpers
  libbpf: Extract internal set-of-strings datastructure APIs
  libbpf: Add generic BTF type shallow copy API
  libbpf: Add BPF static linker APIs
  libbpf: Add BPF static linker BTF and BTF.ext support
  libbpf: Skip BTF fixup if object file has no BTF
  libbpf: Constify few bpf_program getters

Björn Töpel (3):
  libbpf, xsk: Add libbpf_smp_store_release libbpf_smp_load_acquire
  libbpf: xsk: Remove linux/compiler.h header
  libbpf: xsk: Move barriers from libbpf_util.h to xsk.h

Jean-Philippe Brucker (2):
  libbpf: Fix arm64 build
  libbpf: Fix BTF dump of pointer-to-array-of-struct

Joe Stringer (2):
  scripts/bpf: Abstract eBPF API target parameter
  tools: Sync uapi bpf.h header with latest changes

KP Singh (1):
  libbpf: Add explicit padding to btf_dump_emit_type_decl_opts

Kumar Kartikeya Dwivedi (1):
  libbpf: Use SOCK_CLOEXEC when opening the netlink socket

Lorenz Bauer (1):
  bpf: Add PROG_TEST_RUN support for sk_lookup programs

Maciej Fijalkowski (1):
  libbpf: Clear map_info before each bpf_obj_get_info_by_fd

Namhyung Kim (1):
  libbpf: Fix error path in bpf_object__elf_init()

Pedro Tammela (1):
  libbpf: Avoid inline hint definition from 'linux/stddef.h'

Rafael David Tinoco (1):
  libbpf: Add bpf object kern_version attribute setter

Xuesen Huang (1):
  bpf: Add bpf_skb_adjust_room flag BPF_F_ADJ_ROOM_ENCAP_L2_ETH

 include/uapi/linux/bpf.h |  724 +++++++++++++-
 src/bpf_helpers.h        |   21 +-
 src/btf.c                |  714 +++++++-------
 src/btf.h                |    3 +
 src/btf_dump.c           |   10 +-
 src/libbpf.c             |   32 +-
 src/libbpf.h             |   19 +-
 src/libbpf.map           |    6 +
 src/libbpf_internal.h    |   38 +-
 src/libbpf_util.h        |   47 -
 src/linker.c             | 1944 ++++++++++++++++++++++++++++++++++++++
 src/netlink.c            |    2 +-
 src/strset.c             |  176 ++++
 src/strset.h             |   21 +
 src/xsk.c                |    5 +-
 src/xsk.h                |   87 +-
 16 files changed, 3379 insertions(+), 470 deletions(-)
 delete mode 100644 src/libbpf_util.h
 create mode 100644 src/linker.c
 create mode 100644 src/strset.c
 create mode 100644 src/strset.h

--
2.30.2
2 files changed
tree: d8eb1e6dbec26efba89e01b00632797a38f6cd9a
  1. include/
  2. scripts/
  3. src/
  4. travis-ci/
  5. .lgtm.yml
  6. .travis.yml
  7. BPF-CHECKPOINT-COMMIT
  8. CHECKPOINT-COMMIT
  9. LICENSE
  10. LICENSE.BSD-2-Clause
  11. LICENSE.LGPL-2.1
  12. README.md
README.md

BPF/libbpf usage and questions

Please check out libbpf-bootstrap and the companion blog post for the examples of building BPF applications with libbpf. libbpf-tools are also a good source of the real-world libbpf-based tracing tools.

All general BPF questions, including kernel functionality, libbpf APIs and their application, should be sent to bpf@vger.kernel.org mailing list. You can subscribe to it here and search its archive here. Please search the archive before asking new questions. It very well might be that this was already addressed or answered before.

bpf@vger.kernel.org is monitored by many more people and they will happily try to help you with whatever issue you have. This repository's PRs and issues should be opened only for dealing with issues pertaining to specific way this libbpf mirror repo is set up and organized.

Build Build Status Total alerts Coverity

libelf is an internal dependency of libbpf and thus it is required to link against and must be installed on the system for applications to work. pkg-config is used by default to find libelf, and the program called can be overridden with PKG_CONFIG.

If using pkg-config at build time is not desired, it can be disabled by setting NO_PKG_CONFIG=1 when calling make.

To build both static libbpf.a and shared libbpf.so:

$ cd src
$ make

To build only static libbpf.a library in directory build/ and install them together with libbpf headers in a staging directory root/:

$ cd src
$ mkdir build root
$ BUILD_STATIC_ONLY=y OBJDIR=build DESTDIR=root make install

To build both static libbpf.a and shared libbpf.so against a custom libelf dependency installed in /build/root/ and install them together with libbpf headers in a build directory /build/root/:

$ cd src
$ PKG_CONFIG_PATH=/build/root/lib64/pkgconfig DESTDIR=/build/root make install

Distributions

Distributions packaging libbpf from this mirror:

Benefits of packaging from the mirror over packaging from kernel sources:

  • Consistent versioning across distributions.
  • No ties to any specific kernel, transparent handling of older kernels. Libbpf is designed to be kernel-agnostic and work across multitude of kernel versions. It has built-in mechanisms to gracefully handle older kernels, that are missing some of the features, by working around or gracefully degrading functionality. Thus libbpf is not tied to a specific kernel version and can/should be packaged and versioned independently.
  • Continuous integration testing via TravisCI.
  • Static code analysis via LGTM and Coverity.

Package dependencies of libbpf, package names may vary across distros:

  • zlib
  • libelf

BPF CO-RE (Compile Once – Run Everywhere)

Libbpf supports building BPF CO-RE-enabled applications, which, in contrast to BCC, do not require Clang/LLVM runtime being deployed to target servers and doesn't rely on kernel-devel headers being available.

It does rely on kernel to be built with BTF type information, though. Some major Linux distributions come with kernel BTF already built in:

  • Fedora 31+
  • RHEL 8.2+
  • OpenSUSE Tumbleweed (in the next release, as of 2020-06-04)
  • Arch Linux (from kernel 5.7.1.arch1-1)
  • Ubuntu 20.10
  • Debian 11 (amd64/arm64)

If your kernel doesn‘t come with BTF built-in, you’ll need to build custom kernel. You'll need:

  • pahole 1.16+ tool (part of dwarves package), which performs DWARF to BTF conversion;
  • kernel built with CONFIG_DEBUG_INFO_BTF=y option;
  • you can check if your kernel has BTF built-in by looking for /sys/kernel/btf/vmlinux file:
$ ls -la /sys/kernel/btf/vmlinux
-r--r--r--. 1 root root 3541561 Jun  2 18:16 /sys/kernel/btf/vmlinux

To develop and build BPF programs, you'll need Clang/LLVM 10+. The following distributions have Clang/LLVM 10+ packaged by default:

  • Fedora 32+
  • Ubuntu 20.04+
  • Arch Linux
  • Ubuntu 20.10 (LLVM 11)
  • Debian 11 (LLVM 11)

Otherwise, please make sure to update it on your system.

The following resources are useful to understand what BPF CO-RE is and how to use it:

Details

This is a mirror of bpf-next Linux source tree's tools/lib/bpf directory plus its supporting header files.

All the gory details of syncing can be found in scripts/sync-kernel.sh script.

Some header files in this repo (include/linux/*.h) are reduced versions of their counterpart files at bpf-next's tools/include/linux/*.h to make compilation successful.

License

This work is dual-licensed under BSD 2-clause license and GNU LGPL v2.1 license. You can choose between one of them if you use this work.

SPDX-License-Identifier: BSD-2-Clause OR LGPL-2.1