Add resnet50/mobilenet_v2_quantized_qat in into deterministic_algorithms exclusive list (#120384)
After PR: https://github.com/pytorch/pytorch/pull/120026, 2 `Torchbench` testcases: `resnet50_quantized_qat` and `mobilenet_v2_quantized_qat` can pass the performance testing but failed with accuracy test. The failure msg is: `mobilenet_v2_quantized_qat, RuntimeError: quantized_resize_cpu_ does not have a deterministic implementation but you set 'torch.use_deterministic_algorithms(True)'. `
- `torch.use_deterministic_algorithms(True)` only setting for accuracy test. https://github.com/pytorch/pytorch/blob/fff9d98e58cda4ad67fef6d0739f684ca7d4bbec/benchmarks/dynamo/common.py#L3480
- However, `quantized_resize_cpu_` only support `nondeterministic_algorithms` because the resized output memory may be uninitialized. https://github.com/pytorch/pytorch/blob/fff9d98e58cda4ad67fef6d0739f684ca7d4bbec/aten/src/ATen/native/quantized/cpu/TensorOperators.cpp#L85-L87
Add these 2 models into the deterministic_algorithms exclusive model list in this PR.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/120384
Approved by: https://github.com/desertfire, https://github.com/jgong5
diff --git a/benchmarks/dynamo/ci_expected_accuracy/cpu_inductor_torchbench_inference.csv b/benchmarks/dynamo/ci_expected_accuracy/cpu_inductor_torchbench_inference.csv
index 9fa5964..16a4efa 100644
--- a/benchmarks/dynamo/ci_expected_accuracy/cpu_inductor_torchbench_inference.csv
+++ b/benchmarks/dynamo/ci_expected_accuracy/cpu_inductor_torchbench_inference.csv
@@ -198,7 +198,7 @@
-mobilenet_v2_quantized_qat,eager_fail_to_run,0
+mobilenet_v2_quantized_qat,pass,2
@@ -262,7 +262,7 @@
-resnet50_quantized_qat,eager_fail_to_run,0
+resnet50_quantized_qat,pass,2
diff --git a/benchmarks/dynamo/ci_expected_accuracy/dynamic_cpu_inductor_torchbench_inference.csv b/benchmarks/dynamo/ci_expected_accuracy/dynamic_cpu_inductor_torchbench_inference.csv
index bf8386b..0dba6f5 100644
--- a/benchmarks/dynamo/ci_expected_accuracy/dynamic_cpu_inductor_torchbench_inference.csv
+++ b/benchmarks/dynamo/ci_expected_accuracy/dynamic_cpu_inductor_torchbench_inference.csv
@@ -154,7 +154,7 @@
-mobilenet_v2_quantized_qat,eager_fail_to_run,0
+mobilenet_v2_quantized_qat,pass,2
@@ -214,7 +214,7 @@
-resnet50_quantized_qat,eager_fail_to_run,0
+resnet50_quantized_qat,pass,2
diff --git a/benchmarks/dynamo/common.py b/benchmarks/dynamo/common.py
index 8a7553f..a206ae5 100644
--- a/benchmarks/dynamo/common.py
+++ b/benchmarks/dynamo/common.py
@@ -3475,6 +3475,8 @@
"Wav2Vec2ForCTC",
"Wav2Vec2ForPreTraining",
"sam",
+ "resnet50_quantized_qat",
+ "mobilenet_v2_quantized_qat",
}:
# some of the models do not support use_deterministic_algorithms
torch.use_deterministic_algorithms(True)
