libutils: check vsnprintf error

For encoding errors, this function will return a negative value which
causes problems down the line. Check for an error and return. Also,
integer overflows are guarded.

Bug: 161894517
Test: fuzzer test case
Change-Id: Ia85067d4258bde4b875c832d6223db5dd26b8838
Merged-In: Ia85067d4258bde4b875c832d6223db5dd26b8838
(cherry picked from commit ee22384c54d42149491c8b9dbcda0d8c5e88eddc)
1 file changed