schema/README.md

The `schema.fbs` file in this directory describes the
[Flatbuffers](https://google.github.io/flatbuffers/) schema used to serialize
ExecuTorch programs.

The `bundled_program_schema.fbs` file is for serializing bundled program. It
bundles the ExecuTorch program, several sets of inputs and referenced outputs,
and other useful info together for verifying the correctness of ExecuTorch program.

The `scalar_type.fbs` file contains schema for scalar types, used in both
`schema.fbs` and `bundled_program_schema.fbs`.

## Rules to ensure forward/backward compatibility

A serialized program never knows which version of the code will try to read it.
It's important that old code can read serialized programs generated by new code
(forward compatibility; FC), and that new code can read serialized programs
generated by old code (backward compatibility; BC).

For background on the rules below, see the Flatbuffers document [Writing a
schema](https://google.github.io/flatbuffers/md__schemas.html), especially the
"Tables" and "Schema evolution examples" sections.

To ensure binary FC/BC:

- Use `table`, not `struct`, for structured data.
  - `struct` cannot handle optional fields; changes to `struct` are typically
    not FC/BC.
- When adding new fields, do not modify the ID of existing fields.
  - The easiest way to do this is to add new fields to the bottom of a `table`.
  - If it's important to reorder the declaration of fields for readability,
    all fields in the `table` must be explicitly annotated with `(ID: #)` fields
    that preserve their original zero-based indices.
- Fields must never be removed; they can only be deprecated using the
  `(deprecated)` annotation.
- Semantics for existing fields must not change without carefully auditing older
  versions of the code to understand the implications.
- Default values for fields must be very carefully managed.
  - Ideally, the default value for a field will never change.
  - If they do change, audit old and new code to understand the implications of
    the change.
  - Note that fields with unspecified defaults will default to the zero value of
    their types.
- Ideally the types of fields must not change. If they do, ensure that they are
  compatible: e.g., a `uint` could change to `int` if we are confident that no
  program in existence stored a value in that field with the most significant
  bit set.

Note that these rules do not ensure source code FC/BC. E.g., deprecating a field
will tell Flatbuffer's `flatc` to stop generating getters/setters for it, so any
code using those functions will fail to build, and will need to be fixed.

However, this serialization format and the Flatbuffer types that are generated
from it are private to ExecuTorch, so we do not need to worry about updating
external client code when the Flatbuffer API changes. This also means that we
can more easily upgrade to new versions of the Flatbuffers tools/library.

If we are forced to make a FC/BC-breaking change, it may make sense to create a
new `.fbs` file with a different `file_identifier`, and adding higher-level
logic to check the file magic before parsing the binary file with one schema or
the other.