Tools/c-analyzer/cpython/README

#######################################
# C Globals and CPython Runtime State.

CPython's C code makes extensive use of global variables (whether static
globals or static locals).  Each such variable falls into one of several
categories:

* strictly const data
* used exclusively in main or in the REPL
* process-global state (e.g. managing process-level resources
  like signals and file descriptors)
* Python "global" runtime state
* per-interpreter runtime state

The last one can be a problem as soon as anyone creates a second
interpreter (AKA "subinterpreter") in a process.  It is definitely a
problem under subinterpreters if they are no longer sharing the GIL,
since the GIL protects us from a lot of race conditions.  Keep in mind
that ultimately *all* objects (PyObject) should be treated as
per-interpreter state.  This includes "static types", freelists,
_PyIdentifier, and singletons.  Take that in for a second.  It has
significant implications on where we use static variables!

Be aware that module-global state (stored in C statics) is a kind of
per-interpreter state.  There have been efforts across many years, and
still going, to provide extension module authors mechanisms to store
that state safely (see PEPs 3121, 489, etc.).

(Note that there has been discussion around support for running multiple
Python runtimes in the same process.  That would ends up with the same
problems, relative to static variables, that subinterpreters have.)

Historically we have been bad at keeping per-interpreter state out of
static variables, mostly because until recently subinterpreters were
not widely used nor even factored in to solutions.  However, the
feature is growing in popularity and use in the community.

Mandate: "Eliminate use of static variables for per-interpreter state."

The "c-statics.py" script in this directory, along with its accompanying
data files, are part of the effort to resolve existing problems with
our use of static variables and to prevent future problems.

#-------------------------
## statics for actually-global state (and runtime state consolidation)

In general, holding any kind of state in static variables
increases maintenance burden and increases the complexity of code (e.g.
we use TSS to identify the active thread state).  So it is a good idea
to avoid using statics for state even if for the "global" runtime or
for process-global state.

Relative to maintenance burden, one problem is where the runtime
state is spread throughout the codebase in dozens of individual
globals.  Unlike the other globals, the runtime state represents a set
of values that are constantly shifting in a complex way.  When they are
spread out it's harder to get a clear picture of what the runtime
involves.  Furthermore, when they are spread out it complicates efforts
that change the runtime.

Consequently, the globals for Python's runtime state have been
consolidated under a single top-level _PyRuntime global. No new globals
should be added for runtime state.  Instead, they should be added to
_PyRuntimeState or one of its sub-structs.  The tools in this directory
are run as part of the test suite to ensure that no new globals have
been added.  The script can be run manually as well:

  ./python Lib/test/test_c_statics/c-statics.py check

If it reports any globals then they should be resolved.  If the globals
are runtime state then they should be folded into _PyRuntimeState.
Otherwise they should be marked as ignored.