Move debug_util and python_util to torch/csrc/lazy (#72607)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/72607
since python isn't available from libtorch, most of lazy tensor
code can't depend on python.
separate python_util into libtorch_python library
make debug_util and IR dump work with or without python by providing
a default function for 'maybe getting python stacktrace' that returns
an empty stacktrace
use a registration mechanism on libtorch_python library load to update
the 'maybe' function to use the real python stacktrace getter
Test Plan:
OSS build tests:
- test_ptltc by itself works
- LTC_SAVE_TENSORS_FILE=log test_ptltc works, and log contains
empty stacktrces
- python examply.py by itself works
- LTC_SAVE_TENSORS_FILE=log test_ptltc works, and log contains
real stacktraces
fbcode build: rely on CI to run test/lazy
Reviewed By: desertfire
Differential Revision: D34115046
fbshipit-source-id: 8d6222963c146da36b3c1b5ff8a638bbc3f1442e
(cherry picked from commit 3717688adee1bba1314640f93594181e8a2b3831)
diff --git a/tools/build_variables.bzl b/tools/build_variables.bzl
index 21cef97..51d08bb 100644
--- a/tools/build_variables.bzl
+++ b/tools/build_variables.bzl
@@ -387,6 +387,7 @@
"torch/csrc/lazy/backend/backend_interface.cpp",
"torch/csrc/lazy/backend/lowering_context.cpp",
"torch/csrc/lazy/core/config.cpp",
+ "torch/csrc/lazy/core/debug_util.cpp",
"torch/csrc/lazy/core/hash.cpp",
"torch/csrc/lazy/core/helpers.cpp",
"torch/csrc/lazy/core/ir.cpp",
@@ -426,6 +427,11 @@
"torch/csrc/lazy/ts_backend/ts_node.cpp",
]
+lazy_tensor_core_python_sources = [
+ "torch/csrc/lazy/python/init.cpp",
+ "torch/csrc/lazy/python/python_util.cpp",
+]
+
libtorch_core_sources = sorted(
core_sources_common +
torch_unpickler_common +
@@ -885,7 +891,7 @@
"torch/csrc/utils/tensor_numpy.cpp",
"torch/csrc/utils/tensor_types.cpp",
"torch/csrc/utils/disable_torch_function.cpp",
-]
+] + lazy_tensor_core_python_sources
libtorch_python_distributed_core_sources = [
"torch/csrc/distributed/c10d/init.cpp",
