pep-NNNN.txt

PEP: NNN
Title: Type Hints
Version: $Revision$
Last-Modified: $Date$
Author: Guido van Rossum <guido@python.org>, Jukka Lehtosalo <jukka.lehtosalo@iki.fi>, Łukasz Langa <lukasz@langa.pl>
Discussions-To: Python-Dev <python-dev@python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 12-Apr-2014
Post-History: 12-Apr-2014
Replaces: 3107
Resolution:


Abstract
========

This PEP introduces a standard syntax for type hints using annotations
on function definitions.


Rationale and Goals
===================

PEP 3107 [#pep-3107]_ added support for arbitrary annotations on parts
of a function definition.  Although no meaning has been assigned to
those metadata then, there has always been an implicit goal to use them
for type hinting, which is listed as the first possible use case in the
said PEP.

This PEP aims to provide a standard syntax for type annotations, opening
up Python code to easier static analysis and refactoring, potential
runtime type checking, and performance optimizations utilizing type
information.


Type Definition Syntax
======================

The syntax leverages PEP 3107-style annotations with a number of
extensions described in sections below.  In its basic form, type hinting
is used by filling function annotations with classes::

  def greeting(name: str) -> str:
      return 'Hello, {}'.format(name)

.. FIXME: Bad example because of `.format()`, also, doesn't sell type
          hinting enough right away.

This denotes that the expected type of the ``name`` argument is ``str``.
Analogically, the expected return type is ``str``.  Subclasses of
a specified argument type are also accepted as valid types for that
argument.

Abstract base classes, types available in the ``types`` module, as well
as any user-built class, can be provided as type hints as well.  In any
case, the type name used has to be available in the local scope by means
of import or direct declaration.

As a singleton, the ``None`` object is considered equivalent to
``NoneType`` for type hinting purposes.

Type aliases are also valid type hints::

  integer = int

  def retry(func: types.FunctionType, retry_count: integer): ...

While the function annotation syntax permits any arbitrary expression to
be passed as an annotation, the use of runtime-evaluated annotations is
discouraged as it limits what static analysis can infer from such
expressions.  Constant, direct type expressions are recommended.


Generics
--------

Since type information about objects kept in containers cannot be
statically inferred in a generic way, abstract base classes have been
extended to support subscription to denote expected types for container
elements.  Example::

  def notify_by_email(employees: Set[Employee], overrides: Mapping[str, str]): ...

Internally, a ``__getitem__`` operation on a type simply returns the
said type::

  >>> print(MutableMapping)
  <class 'collections.abc.MutableMapping'>
  >>> print(MutableMapping[int])
  <class 'collections.abc.MutableMapping'>

.. FIXME: Not true, we'd actually like for MutableMapping[int] to return
          a new class object that represents a MutableMappingOfInts

Generics can be parametrized by using a new factory available in
``collections.abc`` called ``Var``.  Example::

  T = Var('T')      # Declare type variable

  def first(l: Sequence[T]) -> T:   # Generic function
      return l[0]

In this case the contract is that the returning value is consistent with
the elements held by the collection.  Using the same var-type under
multiple names is considered aliasing.  The following example is
equivalent to the one above::

  X = Var('X')
  Y = Var('X')

  def first(l: Sequence[X]) -> Y:   # Generic function
      return l[0]

``Var`` supports constraining parametric types to classes with any of
the specified bases.  Example::

  X = Var('X')
  Y = Var('Y', Iterable[X])

  def filter(rule: Callable, collection: Y) -> Y:
      ...

.. FIXME: fill in the implementation and the Callable signature
          definition; add an example with multiple bases defined

In the example above we specify that ``Y`` can be any subclass of
Iterable with elements of type ``X``, as long as the return type of
``filter()`` will be the same as the type of the ``collection``
argument.  That means during type checking both ``X`` and ``Y`` must
point to the same concrete types in the ``collection`` argument as well
as the function's return value.  Union types can be used in ``base``.

As with function arguments, generics are covariant, which is in spirit
of duck typing.


Forward references
------------------

When a type hint contains names that have not been defined yet, that
definition may be expressed as a string, to be resolved later.  For
example, instead of writing::

  def notify_by_email(employees: Set[Employee]): ...

one might write::

  def notify_by_email(employees: 'Set[Employee]'): ...


Union types
-----------

Since accepting a small, limited set of expected types for a single
argument is common, the ``collections.abc`` module has been extended
with a new special factory called ``Union``.  Example::

  def handle_employee(e: Union[Employee, Sequence[Employee]]):
      if isinstance(e, Employee):
          e = [e]
      ...

A type factored by ``Union[T1, T2, ...]`` responds ``True`` to
``issubclass`` checks for ``T1`` and any of its subclasses, ``T2`` and
any of its subclasses, and so on.

One common case of union types are *optional* types, also called
*nullable* types.  By default, ``None`` is an invalid value for any
type, unless a default value of ``None`` has been provided in the
function definition.  Examples::

  def handle_employee(e: Union[Employee, None]): ...

  def handle_employee(e: Employee = None): ...

Union types might be inferred implicitly from the function body::

  if x:
      y = None     # First definition of y
  else:
      y = 'a'      # Second definition of y

Since this case may or may not be correct, the linter is expected to
generate warnings.

A specific kind of a union type is ``Any``, a class in
``collections.abc`` that responds ``True`` to ``issubclass`` of any
class.  This lets the user explicitly state that there are no
constraints on the type of a specific argument or return value.


Compatibility with other uses of function annotations
-----------------------------------------------------

A number of existing or potential use cases for function annotations
exist, which are incompatible with type hinting.  If a single scalar
annotation is provided, it is assumed to describe the type of the
argument::

  def notify_by_email(employees: List[Employee]): ...

However, if a dictionary is provided, typing information is available
under the ``'type'`` key, with other keys open for other use cases::

  def notify_by_email(employees: {'type': List[Employee], 'min_size': 1, 'max_size': 100}):
  ...


Type Hints on Local and Global Variables
========================================

No first-class syntax support for explicitly marking variables as being
of a specific type is added by this PEP.  To help with type inference in
complex cases, a comment of the following format might be used::

  x = []   # type: List[Employee]

If type hinting proves useful in general, variable typing in syntax will
be provided in a future Python version.


Explicit raised exceptions
==========================

No support for listing explicitly raised exceptions is being defined by
this PEP.  Currently the only known use case for this feature is
documentational, in which case the recommendation is to put this
information in a docstring.


The ``typing`` module
=====================

To enable generics on builtin types, a set of classes is introduced in
a new module called ``typing``. Those classes are as follows:

* Dict, used as ``Dict[key_type, value_type]``

* List, used as ``List[element_type]``

* Set, used as ``Set[element_type]``. See remark in ``AbstractSet``
  below.

* FrozenSet, used as ``FrozenSet[element_type]``

* Tuple, used as ``Tuple[index0_type, index1_type, ...]``.
  Arbitrary-length tuples might be expressed using ellipsis, in which
  case the following arguments are considered the same type as the last
  defined type on the tuple.

Subscripting the above types returns the vanilla type at runtime, just
as in the case of abstract base classes.

To open the usage of static type checking to Python 3 versions older
than 3.5, the new and modified types found in the ``collections.abc``
module are also importable from the ``typing`` module.  The following
new members:

* Any

* Union

* Var

All available abstract base classes are importable:

* ByteString

* Callable

* Container

* Hashable

* ItemsView

* Iterable

* Iterator

* KeysView

* Mapping

* MappingView

* MutableMapping

* MutableSequence

* MutableSet

* Sequence

* Set as ``AbstractSet``. This name change was required because ``Set``
  in the ``typing`` module means ``set()`` with generics.

* Sized

* ValuesView

* Mapping

* IO

* BinaryIO

* TextIO

The following helper types are also provided by the ``typing`` module:

* AnyStr, equivalent to ``Var('AnyStr', str, bytes)``

* Match and Pattern, types of ``re.match()`` and ``re.compile()`` results

The module also provides a special source encoding that enables usage of
function annotations in Python 2.7.

The place of the ``typing`` module in the standard library
----------------------------------------------------------

.. FIXME: complete this section


Usage Patterns
==============

The main use case of type hinting is static analysis using an external
tool without executing the analyzed program.  Existing tools used for
that purpose like ``pyflakes`` [#pyflakes]_ or ``pylint`` [#pylint]_
might be extended to support type checking.  New tools, like MyPy's
``mypy -S`` mode, can be adopted specifically for this purpose.

Type checking based on type hints is understood as a best-effort
mechanism.  In other words, whenever types are not annotated and cannot
be inferred, the type checker considers such code valid.  Type errors
are only reported in case of explicit or inferred conflict.  Moreover,
as a mechanism that is not tied to execution of the code, it does not
affect runtime behaviour.  In other words, even in the case of a typing
error, the program will continue running.

The implementation of a type checker, whether linting source files or
enforcing type information during runtime, is out of scope for this PEP.


Existing Approaches in Python
=============================

Cython and Numba
----------------

Numba [#numba]_ is a *just-in-time* specializing compiler producing
optimized native code from annotated Python and NumPy code.

obiwan
------

obiwan [#obiwan]_ is a library enabling runtime type checking inspired
by TypeScript [#typescript]_ (see `Existing Approaches in Other
Languages <#typescript>`_).  The syntax leverages function annotations,
extending it to validate callback functions, elements of dictionaries
and lists.  Type checkers might be functions, in which case a type is
considered valid if the type checker returns True.

Examples::

  def divide(a: int, b: float) -> number:
      return a/b

  def robodial(person: {"name":str, "phone": {"type":str, "number":str}}):
      ...

  def on_success(callback: function(int, any, ...)):
      ...

pytypedecl
----------

pytypedecl [#pytypedecl]_ consists of a type declaration language for
Python and an optional runtime type checker.  Type declarations for
``module.py`` are kept in a separate file called ``module.pytd``.  This
solves issues with declaration ordering.

While initially inspired by the PEP 3107 syntax, pytypedecl diverged to
support the following: overloading (specifying the same function
multiple times with different argument types), union types (listing
multiple possible types for a single argument), generics for
collections, and exceptions raised (for documentation purposes).

Example::

  class Logger:
    def log(messages: list<str>, buffer: Readable or Writeable) raises IOError
    def log(messages: list<str>) -> None
    def setStatus(status: int or str)

Argument Clinic
---------------

Argument Clinic [#argumentclinic]_ is a preprocessor for CPython
C files, automating maintenance of argument parsing code for “builtins”.

Example argument declaration::

  /*[clinic input]
  os.chmod

      path: path_t(allow_fd='PATH_HAVE_FCHMOD')
          Path to be modified.  May always be specified as a str or bytes.

      mode: int
          Operating-system mode bitfield.

      *

      dir_fd : dir_fd(requires='fchmodat') = None
          If not None, it should be a file descriptor open to a dir, and
          path should be relative; path will then be relative to that
          dir.

      follow_symlinks: bool = True
          If False, and the last element of the path is a symlink, chmod
          will modify the symlink itself instead of the file the link
          points to.

  Change the access permissions of a file.
  [clinic start generated code]*/


NumPy
-----

NumPy [#numpy]_ is an extension to Python adding support for
multi-dimensional arrays, matrices and operations to operate on those.

The project requires typing information in the API documentation.  There
is an unambiguous syntax for that type of documentation.  Example
documentation with types::

  ndarray.item(*args)

  Copy an element of an array to a standard Python scalar and return it.

  Parameters
  ----------
  \\*args : Arguments (variable number and type)

    * none: in this case, the method only works for arrays
      with one element (`a.size == 1`), which element is
      copied into a standard Python scalar object and returned.

    * int_type: this argument is interpreted as a flat index into
      the array, specifying which element to copy and return.

    * tuple of int_types: functions as does a single int_type argument,
      except that the argument is interpreted as an nd-index into the
      array.

  Returns
  -------
  z : Standard Python scalar object
      A copy of the specified element of the array as a suitable
      Python scalar


Existing Approaches in Other Languages
======================================

ActionScript
------------

ActionScript [#actionscript]_ is a class-based, single inheritance,
object-oriented superset of ECMAScript.  It supports inferfaces and
strong runtime-checked static typing.  Compilation supports a “strict
dialect” where type mismatches are reported at compile-time.

Example code with types::

  package {
    import flash.events.Event;

    public class BounceEvent extends Event {
      public static const BOUNCE:String = "bounce";
      private var _side:String = "none";

      public function get side():String {
        return _side;
      }

      public function BounceEvent(type:String, side:String){
        super(type, true);
        _side = side;
      }

      public override function clone():Event {
        return new BounceEvent(type, _side);
      }
    }
  }

Dart
----

Dart [#dart]_ is a class-based, single inheritance, object-oriented
language with C-style syntax.  It supports interfaces, abstract classes,
reified generics, and optional typing.

Types are inferred when possible.  The runtime differentiates between two
modes of execution: *checked mode* aimed for development (catching type
errors at runtime) and *production mode* recommended for speed execution
(ignoring types and asserts).

Example code with types::

  class Point {
      final num x, y;

      Point(this.x, this.y);

      num distanceTo(Point other) {
          var dx = x - other.x;
          var dy = y - other.y;
          return math.sqrt(dx * dx + dy * dy);
      }
  }

Hack
----

Hack [#hack]_ is a programming language that interoperates seamlessly
with PHP.  It provides opt-in static type checking, type aliasing,
generics, nullable types, and lambdas.

Example code with types::

  <?hh
  class MyClass {
    private ?string $x = null;

    public function alpha(): int {
      return 1;
    }

    public function beta(): string {
      return 'hi test';
    }
  }

  function f(MyClass $my_inst): string {
    // Will generate a hh_client error
    return $my_inst->alpha();
  }

TypeScript
----------

TypeScript [#typescript]_ is a typed superset of JavaScript that adds
interfaces, classes, mixins and modules to the language.

Type checks are duck typed.  Multiple valid function signatures are
specified by supplying overloaded function declarations.  Functions and
classes can use generics as type parametrization.  Interfaces can have
optional fields.  Interfaces can specify array and dictionary types.
Classes can have constructors that implicitly add arguments as fields.
Classes can have static fields.  Classes can have private fields.
Classes can have getters/setters for fields (like property).  Types are
inferred.

Example code with types::

  interface Drivable {
      start(): void;
      drive(distance: number): boolean;
      getPosition(): number;
  }

  class Car implements Drivable {
      private _isRunning: boolean;
      private _distanceFromStart: number;

      constructor() {
          this._isRunning = false;
          this._distanceFromStart = 0;
      }

      public start() {
          this._isRunning = true;
      }

      public drive(distance: number): boolean {
          if (this._isRunning) {
              this._distanceFromStart += distance;
              return true;
          }
          return false;
      }

      public getPosition(): number {
          return this._distanceFromStart;
      }
  }


Is type hinting Pythonic?
=========================

Type annotations provide important documentation for how a unit of code
should be used.  Programmers should therefore provide type hints on
public APIs, namely argument and return types on functions and methods
considered public.  However, because types of local and global variables
can be often inferred, they are rarely necessary.

The kind of information that type hints hold has always been possible to
achieve by means of docstrings.  In fact, a number of formalized
mini-languages for describing accepted arguments have evolved.  Moving
this information to the function declaration makes it more visible and
easier to access both at runtime and by static analysis.  Adding to that
the notion that “explicit is better than implicit”, type hints are
indeed *Pythonic*.


Acknowledgements
================

Influences include all mentioned existing languages, libraries and
frameworks.  Many thanks to their creators, in alphabetical order:
Stefan Behnel, William Edwards, Greg Ewing, Larry Hastings, Anders
Hejlsberg, Alok Menghrajani, Travis E. Oliphant, Joe Pamer,
Raoul-Gabriel Urma, and Julien Verlaguet.

I'd also like to thank Radomir Dopieralski for suggesting warnings as
the toggle for runtime checks.


References
==========

.. [#pep-3107]
   http://www.python.org/dev/peps/pep-3107/

.. [#mypy]
   https://github.com/JukkaL/mypy

.. [#obiwan]
   http://pypi.python.org/pypi/obiwan

.. [#numba]
   http://numba.pydata.org

.. [#pytypedecl]
   https://github.com/google/pytypedecl

.. [#argumentclinic]
   https://docs.python.org/3/howto/clinic.html

.. [#numpy]
   http://www.numpy.org

.. [#typescript]
   http://www.typescriptlang.org/

.. [#hack]
   http://hacklang.org/

.. [#dart]
   https://www.dartlang.org/

.. [#actionscript]
   http://livedocs.adobe.com/specs/actionscript/3/


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End: