PR28514: debuginfod: limit groom operation times

For large databases and many stale files, it was possible to starve
rescan operations by numerous groom "nuke" (database delete ops).
Under the theory that including new data is at least as important as
aging old, we now impose a rough deadline on groom queries.

In the process, we discovered that we were commiting some
undefined-behaviour sqlite ops (deleting rows while iterating), which
may explain some previous heisenbug occurrences.  So the groom nuke
operations are split into decision & action phases, with associated
progress-tracking metrics.

Testing the timeout facility requires hand-testing beyond the
testsuite (since it requires LARGE databases to show measurable query
times).  So confirmed this part by hand.

Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
5 files changed