commit | ca587934ebc2caded1f0d432636a51e168dc6e64 | [log] [tgz] |
---|---|---|
author | Xusong Wang <xusongw@google.com> | Thu Apr 02 11:26:27 2020 -0700 |
committer | Xusong Wang <xusongw@google.com> | Tue Apr 21 15:52:24 2020 -0700 |
tree | 2edd60f38d9c7017c3cccc7e7fb74a8413bb8869 | |
parent | c00b594a0b86d3e48b6e126871f78bdbb1556016 [diff] |
Increase the fp precision in dumped spec. By default, C++ will output 6 fractional digits of floating point values. This is insufficient for some of the RGG tests that requires precision < 1e-6. This CL uses hex representation for fp values in the generated spec file, while human-readable values are also dumped as a comment. Fixes: 144788409 Test: NNT_static_fuzzing and inspected the dumped spec Change-Id: I43ed11e9f73cb387ed8894713fe79361322315ba