btf_encoder: Teach pahole to store percpu variables in vmlinux BTF.

On SMP systems, the global percpu variables are placed in a special
'.data..percpu' section, which is stored in a segment whose initial
address is set to 0, the addresses of per-CPU variables are relative
positive addresses [1].

This patch extracts these variables from vmlinux and places them with
their type information in BTF. More specifically, when BTF is encoded,
we find the index of the '.data..percpu' section and then traverse the
symbol table to find those global objects which are in this section.
For each of these objects, we push a BTF_KIND_VAR into the types buffer,
and a BTF_VAR_SECINFO into another buffer, percpu_secinfo. When all the
CUs have finished processing, we push a BTF_KIND_DATASEC into the
btfe->types buffer, followed by the percpu_secinfo's content.

In a v5.8-rc3 linux kernel, I was able to extract 288 such variables.
The build time overhead is small and the space overhead is also small.
See testings below.

A found variable can be invalid in two ways:

 - Its name found in elf_sym__name is invalid.
 - Its size identified by elf_sym__size is 0.

In either case, the BTF containing such symbols will be rejected by the
BTF verifier. Normally we should not see such symbols. But if one is
seen during BTF encoding, the encoder will exit with error. An new flag
'-j' (or '--force') is implemented to help testing, which skips the
invalid symbols and force emit a BTF.


- vmlinux size has increased by ~12kb.
   $ readelf -SW vmlinux | grep BTF
   [25] .BTF              PROGBITS        ffffffff821a905c 13a905c 2d2bf8 00
   $ pahole -J vmlinux
   $ readelf -SW vmlinux  | grep BTF
   [25] .BTF              PROGBITS        ffffffff821a905c 13a905c 2d5bca 00

- Common global percpu VARs and DATASEC are found in BTF section.
  $ bpftool btf dump file vmlinux | grep runqueues
  [14152] VAR 'runqueues' type_id=13778, linkage=global-alloc

  $ bpftool btf dump file vmlinux | grep 'cpu_stopper'
  [17582] STRUCT 'cpu_stopper' size=72 vlen=5
  [17601] VAR 'cpu_stopper' type_id=17582, linkage=static

  $ bpftool btf dump file vmlinux | grep ' DATASEC '
  [63652] DATASEC '.data..percpu' size=179288 vlen=288

- Tested bpf selftests.

- pahole exits with error if an invalid symbol is seen during encoding,
  make -f Makefile -j 36 -s
  PAHOLE: Error: Found symbol of zero size when encoding btf (sym: 'yyy', cu: 'xxx.c').
  PAHOLE: Error: Use '-j' or '--force_emit' to ignore such symbols and force emit the btf.
  scripts/ line 137: 2475712 Segmentation fault      LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1}

- With the flag '-j' or '--force', the invalid symbols are ignored.

- Further in verbose mode and with '-j' or '--force' set, a warning is generated:
  PAHOLE: Warning: Found symbol of zero size when encoding btf, ignored (sym: 'yyy', cu: 'xxx.c').
  PAHOLE: Warning: Found symbol of invalid name when encoding btf, ignored (sym: 'zzz', cu: 'sss.c').


Signed-off-by: Hao Luo <>
Tested-by: Andrii Nakryiko <>
Tested-by: Arnaldo Carvalho de Melo <>
Acked-by: Andrii Nakryiko <>
Cc: Alexei Starovoitov <>
Cc: Andrii Nakryiko <>
Cc: Daniel Borkmann <>
Cc: Martin KaFai Lau <>
Cc: Oleg Rombakh <>
Signed-off-by: Arnaldo Carvalho de Melo <>
5 files changed