pep-NNNN.txt

PEP: NNN
Title: Type Hints
Version: $Revision$
Last-Modified: $Date$
Author: Guido van Rossum <guido@python.org>, Jukka Lehtosalo <jukka.lehtosalo@iki.fi>, Łukasz Langa <lukasz@langa.pl>
Discussions-To: Python-Dev <python-dev@python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 29-Sep-2014
Post-History: 
Resolution:


Abstract
========

This PEP introduces a standard syntax for type hints using annotations
on function definitions.

The proposal is strongly inspired by mypy [mypy]_.


Rationale and Goals
===================

PEP 3107 [pep-3107]_ added support for arbitrary annotations on parts
of a function definition.  Although no meaning was assigned to
annotations then, there has always been an implicit goal to use them
for type hinting, which is listed as the first possible use case in
said PEP.

This PEP aims to provide a standard syntax for type annotations, opening
up Python code to easier static analysis and refactoring, potential
runtime type checking, and performance optimizations utilizing type
information.


Type Definition Syntax
======================

The syntax leverages PEP 3107-style annotations with a number of
extensions described in sections below.  In its basic form, type hinting
is used by filling function annotations with classes::

  def greeting(name: str) -> str:
      return 'Hello ' + name

This denotes that the expected type of the ``name`` argument is ``str``.
Analogically, the expected return type is ``str``.  Subclasses of
a specified argument type are also accepted as valid types for that
argument.

Abstract base classes, types available in the ``types`` module, and
user-defined classes may be used as type hints as well.  Annotations
must be valid expressions that evaluate without raising exceptions at
the time the function is defined.  In addition, the needs of static
analysis require that annotations must be simple enough to be
interpreted by static analysis tools.  (This is an intentionally
somewhat vague requirement.)

.. FIXME: Define rigorously what is/isn't supported.

When used as an annotation, the expression ``None`` is considered
equivalent to ``NoneType`` (i.e., ``type(None)`` for type hinting
purposes.

Type aliases are also valid type hints::

  integer = int

  def retry(url: str, retry_count: integer): ...


Generics
--------

.. FIXME: Actually you probably need to import these from typing.py.

Since type information about objects kept in containers cannot be
statically inferred in a generic way, abstract base classes have been
extended to support subscription to denote expected types for container
elements.  Example::

  def notify_by_email(employees: Set[Employee], overrides: Mapping[str, str]): ...

Generics can be parametrized by using a new factory available in
``collections.abc`` called ``Var``.  Example::

  T = Var('T')      # Declare type variable

  def first(l: Sequence[T]) -> T:   # Generic function
      return l[0]

.. FIXME: Maybe rename Var to TypeVar to avoid confusion with simple
          variables and a possible future syntax of the form 
          var name: type = value

In this case the contract is that the returning value is consistent with
the elements held by the collection.

``Var`` supports constraining parametric types to classes with any of
the specified bases.  Example::

  X = Var('X')
  Y = Var('Y', Iterable[X])

  def filter(rule: Callable[[X], bool], input: Y) -> Y:
      ...

.. FIXME: Add an example with multiple bases defined.

In the example above we specify that ``Y`` can be any subclass of
Iterable with elements of type ``X``, as long as the return type of
``filter()`` will be the same as the type of the ``input``
argument.

.. FIXME: Explain more about how this works.


Forward references
------------------

When a type hint contains names that have not been defined yet, that
definition may be expressed as a string, to be resolved later.  For
example, instead of writing::

  def notify_by_email(employees: Set[Employee]): ...

one might write::

  def notify_by_email(employees: 'Set[Employee]'): ...

.. FIXME: Rigorously define this.  Defend it, or find an alternative.


Union types
-----------

Since accepting a small, limited set of expected types for a single
argument is common, there is a new special factory called ``Union``.
Example::

  def handle_employees(e: Union[Employee, Sequence[Employee]]):
      if isinstance(e, Employee):
          e = [e]
      ...

A type factored by ``Union[T1, T2, ...]`` responds ``True`` to
``issubclass`` checks for ``T1`` and any of its subclasses, ``T2`` and
any of its subclasses, and so on.

One common case of union types are *optional* types.  By default,
``None`` is an invalid value for any type, unless a default value of
``None`` has been provided in the function definition.  Examples::

  def handle_employee(e: Union[Employee, None]): ...

As a shorthand for ``Union[T1, None]`` you can write ``Optional[T1]``;
for example, the above is equivalent to::

  def handle_employee(e: Optional[Employee]): ...

An optional type is also automatically assumed when the default value
is ``None``, for example:

  def handle_employee(e: Employee = None): ...

This is equivalent to::

  def handle_employee(e: Optional[Employee] = None): ...

.. FIXME: Is this really a good idea?

A special kind of union type is ``Any``, a class that responds
``True`` to ``issubclass`` of any class.  This lets the user
explicitly state that there are no constraints on the type of a
specific argument or return value.


Compatibility with other uses of function annotations
-----------------------------------------------------

A number of existing or potential use cases for function annotations
exist, which are incompatible with type hinting.  These may confuse a
static type checker.  However, since type hinting annotations have no
run time behavior (other than evaluation of the annotation expression
and storing annotations in the ``__annotations__`` attribute of the
function object), this does not make the program incorrect -- it just
makes it issue warnings when a static analyzer is used.

.. FIXME: Define a way to shut up the static analyzer for a module, class or function.


Type Hints on Local and Global Variables
========================================

No first-class syntax support for explicitly marking variables as being
of a specific type is added by this PEP.  To help with type inference in
complex cases, a comment of the following format may be used::

  x = []   # type: List[Employee]

If type hinting proves useful in general, a variable typing in syntax may
be provided in a future Python version.


Explicit raised exceptions
==========================

No support for listing explicitly raised exceptions is being defined by
this PEP.  Currently the only known use case for this feature is
documentational, in which case the recommendation is to put this
information in a docstring.


The ``typing`` module
=====================

.. FIXME: Reconsider changing collections.abc, in favor of requiring
          the new types to be import from typing.py.

To enable generics on builtin types, a set of classes is introduced in
a new module called ``typing``. Those classes are as follows:

* Dict, used as ``Dict[key_type, value_type]``

* List, used as ``List[element_type]``

* Set, used as ``Set[element_type]``. See remark for ``AbstractSet``
  below.

* FrozenSet, used as ``FrozenSet[element_type]``

* Tuple, used as ``Tuple[index0_type, index1_type, ...]``.
  Arbitrary-length tuples might be expressed using ellipsis, in which
  case the following arguments are considered the same type as the last
  defined type on the tuple.

To open the usage of static type checking to Python 3 versions older
than 3.5, the new and modified types found in the ``collections.abc``
module are also importable from the ``typing`` module.  The following
new members are defined:

* Any

* Union

* Var

All available abstract base classes are importable:

* ByteString

* Callable

* Container

* Hashable

* ItemsView

* Iterable

* Iterator

* KeysView

* Mapping

* MappingView

* MutableMapping

* MutableSequence

* MutableSet

* Sequence

* Set as ``AbstractSet``. This name change was required because ``Set``
  in the ``typing`` module means ``set()`` with generics.

* Sized

* ValuesView

* Mapping

* IO

* BinaryIO

* TextIO

The following helper types are also provided by the ``typing`` module:

* AnyStr, equivalent to ``Var('AnyStr', str, bytes)``

* Match and Pattern, types of ``re.match()`` and ``re.compile()`` results

.. FIXME: Match, Pattern and the IO types don't really belong here.


The place of the ``typing`` module in the standard library
----------------------------------------------------------

.. FIXME: complete this section


Usage Patterns
==============

The main use case of type hinting is static analysis using an external
tool without executing the analyzed program.  Existing tools used for
that purpose like ``pyflakes`` [pyflakes]_ or ``pylint`` [pylint]_
might be extended to support type checking.  New tools, like mypy's
``mypy -S`` mode, can be adopted specifically for this purpose.

Type checking based on type hints is understood as a best-effort
mechanism.  In other words, whenever types are not annotated and cannot
be inferred, the type checker considers such code valid.  Type errors
are only reported in case of explicit or inferred conflict.  Moreover,
as a mechanism that is not tied to execution of the code, it does not
affect runtime behaviour.  In other words, even in the case of a typing
error, the program will continue running.

The implementation of a type checker, whether linting source files or
enforcing type information during runtime, is out of scope for this PEP.

.. FIXME: Describe stub modules.

.. FIXME: Describe run-time behavior of generic types.


Existing Approaches in Python
=============================

Cython and Numba
----------------

Numba [numba]_ is a *just-in-time* specializing compiler producing
optimized native code from annotated Python and NumPy code.

obiwan
------

obiwan [obiwan]_ is a library enabling runtime type checking inspired
by TypeScript [typescript]_ (see `Existing Approaches in Other
Languages <#typescript>`_).  The syntax leverages function annotations,
extending it to validate callback functions, elements of dictionaries
and lists.  Type checkers might be functions, in which case a type is
considered valid if the type checker returns True.

Examples::

  def divide(a: int, b: float) -> number:
      return a/b

  def robodial(person: {"name":str, "phone": {"type":str, "number":str}}):
      ...

  def on_success(callback: function(int, any, ...)):
      ...

pytypedecl
----------

pytypedecl [pytypedecl]_ consists of a type declaration language for
Python and an optional runtime type checker.  Type declarations for
``module.py`` are kept in a separate file called ``module.pytd``.  This
solves issues with declaration ordering.

While initially inspired by the PEP 3107 syntax, pytypedecl diverged to
support the following: overloading (specifying the same function
multiple times with different argument types), union types (listing
multiple possible types for a single argument), generics for
collections, and exceptions raised (for documentation purposes).

Example::

  class Logger:
    def log(messages: list<str>, buffer: Readable or Writeable) raises IOError
    def log(messages: list<str>) -> None
    def setStatus(status: int or str)

Argument Clinic
---------------

Argument Clinic [argumentclinic]_ is a preprocessor for CPython
C files, automating maintenance of argument parsing code for “builtins”.

Example argument declaration::

  /*[clinic input]
  os.chmod

      path: path_t(allow_fd='PATH_HAVE_FCHMOD')
          Path to be modified.  May always be specified as a str or bytes.

      mode: int
          Operating-system mode bitfield.

      *

      dir_fd : dir_fd(requires='fchmodat') = None
          If not None, it should be a file descriptor open to a dir, and
          path should be relative; path will then be relative to that
          dir.

      follow_symlinks: bool = True
          If False, and the last element of the path is a symlink, chmod
          will modify the symlink itself instead of the file the link
          points to.

  Change the access permissions of a file.
  [clinic start generated code]*/


NumPy
-----

NumPy [numpy]_ is an extension to Python adding support for
multi-dimensional arrays, matrices and operations to operate on those.

The project requires typing information in the API documentation.  There
is an unambiguous syntax for that type of documentation.  Example
documentation with types::

  ndarray.item(*args)

  Copy an element of an array to a standard Python scalar and return it.

  Parameters
  ----------
  \\*args : Arguments (variable number and type)

    * none: in this case, the method only works for arrays
      with one element (`a.size == 1`), which element is
      copied into a standard Python scalar object and returned.

    * int_type: this argument is interpreted as a flat index into
      the array, specifying which element to copy and return.

    * tuple of int_types: functions as does a single int_type argument,
      except that the argument is interpreted as an nd-index into the
      array.

  Returns
  -------
  z : Standard Python scalar object
      A copy of the specified element of the array as a suitable
      Python scalar


Existing Approaches in Other Languages
======================================

ActionScript
------------

ActionScript [actionscript]_ is a class-based, single inheritance,
object-oriented superset of ECMAScript.  It supports inferfaces and
strong runtime-checked static typing.  Compilation supports a “strict
dialect” where type mismatches are reported at compile-time.

Example code with types::

  package {
    import flash.events.Event;

    public class BounceEvent extends Event {
      public static const BOUNCE:String = "bounce";
      private var _side:String = "none";

      public function get side():String {
        return _side;
      }

      public function BounceEvent(type:String, side:String){
        super(type, true);
        _side = side;
      }

      public override function clone():Event {
        return new BounceEvent(type, _side);
      }
    }
  }

Dart
----

Dart [dart]_ is a class-based, single inheritance, object-oriented
language with C-style syntax.  It supports interfaces, abstract classes,
reified generics, and optional typing.

Types are inferred when possible.  The runtime differentiates between two
modes of execution: *checked mode* aimed for development (catching type
errors at runtime) and *production mode* recommended for speed execution
(ignoring types and asserts).

Example code with types::

  class Point {
      final num x, y;

      Point(this.x, this.y);

      num distanceTo(Point other) {
          var dx = x - other.x;
          var dy = y - other.y;
          return math.sqrt(dx * dx + dy * dy);
      }
  }

Hack
----

Hack [hack]_ is a programming language that interoperates seamlessly
with PHP.  It provides opt-in static type checking, type aliasing,
generics, nullable types, and lambdas.

Example code with types::

  <?hh
  class MyClass {
    private ?string $x = null;

    public function alpha(): int {
      return 1;
    }

    public function beta(): string {
      return 'hi test';
    }
  }

  function f(MyClass $my_inst): string {
    // Will generate a hh_client error
    return $my_inst->alpha();
  }

TypeScript
----------

TypeScript [typescript]_ is a typed superset of JavaScript that adds
interfaces, classes, mixins and modules to the language.

Type checks are duck typed.  Multiple valid function signatures are
specified by supplying overloaded function declarations.  Functions and
classes can use generics as type parametrization.  Interfaces can have
optional fields.  Interfaces can specify array and dictionary types.
Classes can have constructors that implicitly add arguments as fields.
Classes can have static fields.  Classes can have private fields.
Classes can have getters/setters for fields (like property).  Types are
inferred.

Example code with types::

  interface Drivable {
      start(): void;
      drive(distance: number): boolean;
      getPosition(): number;
  }

  class Car implements Drivable {
      private _isRunning: boolean;
      private _distanceFromStart: number;

      constructor() {
          this._isRunning = false;
          this._distanceFromStart = 0;
      }

      public start() {
          this._isRunning = true;
      }

      public drive(distance: number): boolean {
          if (this._isRunning) {
              this._distanceFromStart += distance;
              return true;
          }
          return false;
      }

      public getPosition(): number {
          return this._distanceFromStart;
      }
  }


Is type hinting Pythonic?
=========================

Type annotations provide important documentation for how a unit of code
should be used.  Programmers should therefore provide type hints on
public APIs, namely argument and return types on functions and methods
considered public.  However, because types of local and global variables
can be often inferred, they are rarely necessary.

The kind of information that type hints hold has always been possible to
achieve by means of docstrings.  In fact, a number of formalized
mini-languages for describing accepted arguments have evolved.  Moving
this information to the function declaration makes it more visible and
easier to access both at runtime and by static analysis.  Adding to that
the notion that “explicit is better than implicit”, type hints are
indeed *Pythonic*.


Acknowledgements
================

Influences include all mentioned existing languages, libraries and
frameworks.  Many thanks to their creators, in alphabetical order:
Stefan Behnel, William Edwards, Greg Ewing, Larry Hastings, Anders
Hejlsberg, Alok Menghrajani, Travis E. Oliphant, Joe Pamer,
Raoul-Gabriel Urma, and Julien Verlaguet.

I'd also like to thank Radomir Dopieralski for suggesting warnings as
the toggle for runtime checks.


References
==========

.. [pep-3107]
   http://www.python.org/dev/peps/pep-3107/

.. [mypy]
   http://mypy-lang.org

.. [obiwan]
   http://pypi.python.org/pypi/obiwan

.. [numba]
   http://numba.pydata.org

.. [pytypedecl]
   https://github.com/google/pytypedecl

.. [argumentclinic]
   https://docs.python.org/3/howto/clinic.html

.. [numpy]
   http://www.numpy.org

.. [typescript]
   http://www.typescriptlang.org

.. [hack]
   http://hacklang.org

.. [dart]
   https://www.dartlang.org

.. [actionscript]
   http://livedocs.adobe.com/specs/actionscript/3/

.. [pyflakes]
   https://github.com/pyflakes/pyflakes/

.. [pylint]
   http://www.pylint.org


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End: