Introduce a few new type IDs:
* TFT_FOR_EACH is required to properly describe dataset type definitions
* TFT_RAGGED for variant-encoded ragged tensors
* TFT_LEGACY_VARIANT to represent the legacy DT_VARIANT tensors, and simplify compatibility translations
PiperOrigin-RevId: 412071476
Change-Id: I4a18dcf13f611d147420f9b2af313347895601c4
diff --git a/tensorflow/core/framework/full_type.proto b/tensorflow/core/framework/full_type.proto
index 22dbd23..ddf05ec 100644
--- a/tensorflow/core/framework/full_type.proto
+++ b/tensorflow/core/framework/full_type.proto
@@ -25,7 +25,7 @@
// TFT_TENSOR[TFT_VAR["T"]], TFT_TENSOR[TFT_VAR["T"]] are two tensors of
// identical element types.
// TFT_TENSOR[TFT_VAR["P"]], TFT_TENSOR[TFT_VAR["Q"]] are two tensors of
- // potentially different element types.
+ // independent element types.
//
TFT_VAR = 1;
@@ -62,6 +62,28 @@
// "bar".
TFT_NAMED = 4;
+ // Template definition. Expands the variables by repeating a template as
+ // arguments of container.
+ //
+ // Parametrization:
+ // TFT_FOR_EACH[<container_type>, <template>, <expansions>]
+ // * <container_type> is the type of the container that the template will be
+ // expanded into
+ // * <template> is any type definition that potentially contains type
+ // variables
+ // * <expansions> is a TFT_VAR and may include more types in the future
+ //
+ // Example:
+ // TFT_FOR_EACH[
+ // TFT_PRODUCT,
+ // TFT_TENSOR[TFT_VAR["t"]],
+ // TFT_VAR["t"]
+ // ]
+ // will substitute a T = TFT_INT32 to TFT_PRODUCT[TFT_TENSOR[TFT_INT32]]
+ // and a T = (TFT_INT32, TFT_INT64) to
+ // TFT_PRODUCT[TFT_TENSOR[TFT_INT32], TFT_TENSOR[TFT_INT64]].
+ TFT_FOR_EACH = 20;
+
// Callable types describe functions and ops.
//
// Parametrization:
@@ -151,39 +173,6 @@
// TFT_LITERAL[TFT_INT32]{1} is the compile-time constant 1.
TFT_LITERAL = 1003;
- // Datasets created by tf.data ops and APIs. Datasets have generator/iterable
- // semantics, that is, one can construct an iterator from them. Like
- // Array, they are considered to return elements that can be described
- // by a single type. Unlike Array, they do not support random access or
- // mutation, and can potentially produce an infinite number of elements.
- // A datasets can produce logical structures (e.g. multiple elements). This
- // is expressed using TFT_PRODUCT.
- //
- //
- // Parametrization: TFT_ARRAY[<element type>].
- // * <element type> may be a concrete type or a type symbol. It represents
- // the data type of the elements produced by the dataset.
- //
- // Examples:
- // TFT_DATSET[TFT_TENSOR[TFT_INT32]] is a Dataset producing single int32
- // Tensors of unknown shape.
- // TFT_DATSET[TFT_PRODUCT[TFT_TENSOR[TFT_INT32], TFT_TENSOR[TFT_FLOAT32]] is
- // a Dataset producing pairs of Tensors, one integer and one float.
- // Note: The high ID number is to prepare for the eventuality that Datasets
- // will be supported by user types in the future.
- TFT_DATASET = 10102;
-
- // A mutex lock tensor, produced by tf.raw_ops.MutexLock.
- // Unlike strict execution models, where ownership of a lock is denoted by
- // "running after the lock has been acquired", in non-strict mode, lock
- // ownership is in the true sense: "the op argument representing the lock is
- // available".
- // Mutex locks are the dynamic counterpart of control dependencies.
- // TODO(mdan): Properly document this thing.
- //
- // Parametrization: TFT_MUTEX_LOCK[].
- TFT_MUTEX_LOCK = 10202;
-
// Type attributes. These always appear in the parametrization of a type,
// never alone. For example, there is no such thing as a "bool" TensorFlow
// object (for now).
@@ -211,6 +200,56 @@
TFT_COMPLEX128 = 213;
// The string element type.
TFT_STRING = 214;
+
+ // Other types that we don't know yet whether they will become part of the
+ // core type system or be consisdered third-party (and consequently moved to
+ // user-defined type mechanisms). Presently, they are effectively in the core
+ // type system, because key compilation passes like Placer account for their
+ // existence.
+
+ // Datasets created by tf.data ops and APIs. Datasets have generator/iterable
+ // semantics, that is, one can construct an iterator from them. Like
+ // Array, they are considered to return elements that can be described
+ // by a single type. Unlike Array, they do not support random access or
+ // mutation, and can potentially produce an infinite number of elements.
+ // A datasets can produce logical structures (e.g. multiple elements). This
+ // is expressed using TFT_PRODUCT.
+ //
+ //
+ // Parametrization: TFT_ARRAY[<element type>].
+ // * <element type> may be a concrete type or a type symbol. It represents
+ // the data type of the elements produced by the dataset.
+ //
+ // Examples:
+ // TFT_DATSET[TFT_TENSOR[TFT_INT32]] is a Dataset producing single int32
+ // Tensors of unknown shape.
+ // TFT_DATSET[TFT_PRODUCT[TFT_TENSOR[TFT_INT32], TFT_TENSOR[TFT_FLOAT32]] is
+ // a Dataset producing pairs of Tensors, one integer and one float.
+ // Note: The high ID number is to prepare for the eventuality that Datasets
+ // will be supported by user types in the future.
+ TFT_DATASET = 10102;
+
+ // A ragged tensor created by tf.ragged ops and APIs.
+ //
+ // Parametrization: TFT_RAGGED[<element_type>].
+ TFT_RAGGED = 10103;
+
+ // A mutex lock tensor, produced by tf.raw_ops.MutexLock.
+ // Unlike strict execution models, where ownership of a lock is denoted by
+ // "running after the lock has been acquired", in non-strict mode, lock
+ // ownership is in the true sense: "the op argument representing the lock is
+ // available".
+ // Mutex locks are the dynamic counterpart of control dependencies.
+ // TODO(mdan): Properly document this thing.
+ //
+ // Parametrization: TFT_MUTEX_LOCK[].
+ TFT_MUTEX_LOCK = 10202;
+
+ // The equivalent of a Tensor with DT_VARIANT dtype, kept here to simplify
+ // translation. This type should not normally appear after type inference.
+ // Note that LEGACY_VARIANT != ANY: TENSOR[INT32] is a subtype of ANY, but is
+ // not a subtype of LEGACY_VARIANT.
+ TFT_LEGACY_VARIANT = 10203;
}
// Highly experimental and very likely to change.
