mksquashfs: optimise fragment duplicate checking
Oops, one last commit before release, which I noticed I'd forgotten
to do when writing up the release notes ...
Make the uncompressed fragment cache and the reserve cache grow
to maximum size before re-using freed items on the free list.
Up until now (unless we're appending) it has been policy to re-use
items on the free list if available, rather than grow the cache.
This was done for a very good reason, up until recently Mksquashfs
lacked the ability to tune its memory usage to the amount
of memory in the machine being run on. Lacking the ability to
determine if the maximum size of the caches was a good fit
for the machine, it has always been wiser to avoid aggressively
growing the caches, and to only allow the caches to grow as a
result of readahead.
The advantages of doing this was it actively avoided growing the
caches so they were too large for the machine being run on, if it
had low memory. The flip side of this is the caches often times
never grow to maximum size.
This is important because the size of the fragment cache
directly affects fragment lookup performance in fragment
duplicate checking. The larger the cache, the more likelihood
the fragment being looked up will still be in the cache, and
this avoids a costly re-read from the filesystem and
decompress.
So on the one hand the failure to aggressively grow the caches has
been good, on the other hand it has always impacted the performance
of fragment duplicate checking.
Now that Mksquashfs accurately tunes its memory usage to
the amount of memory in the machine being run on, we don't have to
worry about aggressively growing the caches to maximum size - we
know the computed maximum size will not be larger than the machine
can handle. So change to growing the caches in preference to
taking off the free list.
Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
1 file changed
