tag 8882fecc1a353bbf3d47dfa198f4695e7bef4cda
tagger The Android Open Source Project <initial-contribution@android.com> Fri May 31 15:51:09 2019 -0700
object 4ddf1bbdc589182b1545b4d349b4a603dcb3ea2c

Android 9.0.0 release 40

commit 4ddf1bbdc589182b1545b4d349b4a603dcb3ea2c
author Xusong Wang <xusongw@google.com> Fri Aug 03 13:50:47 2018 -0700
committer Przemyslaw Szczepaniak <pszczepaniak@google.com> Wed Sep 12 11:02:57 2018 +0000
tree a8fbc0c93f9fbf06174ca121d91d579ebabdae0f
parent 0a27020cf1e8bd4c5c07025b2131f97aa88b5be5

Fix logics for floating-point comparision.

Set the acceptable error range based on both absolute tolerance and
relative tolerance.

Currently, absolute tolerance is set to 1e-5 for FP32 and 5 epsilon
(~5e-3) for FP16 relaxed computation. The relative tolerance is set to
5ULP of the corresponding precision. Add a TODO mark for potential
future adjustment on error limit based on testing.

Bug: 111768023

Test: NeuralNetworksTest_static
Change-Id: I8052d979bc45a83dba4c69ef20b8a740ae640953
Merged-In: I8052d979bc45a83dba4c69ef20b8a740ae640953
(cherry picked from commit 3afdbda56805b6ca1d8ceec66f861c57db616778)