tree 0a4fde31b68330171788620e83dbcf54aba627e6
parent f8b689a00b3fdc3b75e1575c40d92c897e1d09f4
author philippe <philippe@a5019735-40e9-0310-863c-91ae7b9d1cf9> 1404513398 +0000
committer philippe <philippe@a5019735-40e9-0310-863c-91ae7b9d1cf9> 1404513398 +0000

This patch decreases significantly the memory needed to store the cfsi info.

On a big executable, the trunk needs:
dinfo: 155844608/106737664  max/curr mmap'd 155572624/102276760 max/curr

With the patch, we have:
dinfo: 134873088/70389760  max/curr mmap'd 134607808/66717512 max/curr

So, peak dinfo memory decreases by 21Mb, and final by 36Mb.

The memory decrease is obtained by:

* using a dedup pool to store the machine dependent part (cfsi_m)
  of the cfsi information as this information is highly duplicated.
  For x86 and arm64, the duplication factor of cfsi machine dependent
  part is very high (up to a factor 60).
  For arm64, it is more like a factor 3.
  A 'variable size' (1, 2 or 4 bytes) is automatically used to identify
  the cfsi_m, if there is less than or more than 255/64K different cfsi_m.

* not storing explicitely the length of a range for which a cfsi_m
  is to be used: in a large majority of the cases, ranges are
  consecutive, and so the end of a range is just one byte before
  the start of the next range.
  So, we do not store the length of the ranges.
  If there is a hole between 2 ranges, the hole is stored explicitely
  as a range in which we have no cfsi_m information.
  On x86 and amd64, we have quite some holes (something like one hole
  every 7 cfsi). On arm64, we have very few holes (less than one hole
  every 50 cfsi).
  Even with the nr of holes on x86/amd64, it is more memory efficient
  to store the holes rather than to store the length of each cfsi.

* Merging consecutive ranges that have the same cfsi_m info:
  Many cfsi are "mergeable": there is no hole between 2 cfsi, and their
  machine dependent part is identical
  (I guess the unwind info needed by valgrind is subset of the full
   unwind info, and so, the cfsi entries are not merged by the compiler,
   but can be merged for simple unwind). Depending on the platform
   (x86, amd64, arm64) and of the library/object file, we can have a
   significant nr of mergeable entries. 


The patch is not very small, but a lot is mechanical changes.

The patch has been compiled and tested on x86/amd64/ppc32/ppc64
(but ppc does not use cfsi so that just verifies it compiles).
It has been compiled on arm64, and "tested" by launching valgrind on
one executable.
It has not been compiled on s390 and mips.
With some luck, maybe it will compile on these platforms.
And if that uses the whole provision of luck for 2014, it might even work
on these platforms :).
If it does not compile, the fix should be straightforward.
Runtime problems might be more tricky (but arm64 "worked out of the box"
once x86/amd64 were ok).

This has also be tested in an outer/inner setup, to verify no memory leak/bugs.


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14129 a5019735-40e9-0310-863c-91ae7b9d1cf9