Misc/NEWS.d/3.10.0a2.rst

.. bpo: 42103
.. date: 2020-10-23-19-19-30
.. nonce: cILT66
.. release date: 2020-11-03
.. section: Security

Prevented potential DoS attack via CPU and RAM exhaustion when processing
malformed Apple Property List files in binary format.

..

.. bpo: 42051
.. date: 2020-10-19-10-56-27
.. nonce: EU_B7u
.. section: Security

The :mod:`plistlib` module no longer accepts entity declarations in XML
plist files to avoid XML vulnerabilities. This should not affect users as
entity declarations are not used in regular plist files.

..

.. bpo: 42236
.. date: 2020-11-01-21-21-38
.. nonce: MPx-NK
.. section: Core and Builtins

If the ``nl_langinfo(CODESET)`` function returns an empty string, Python now
uses UTF-8 as the filesystem encoding. Patch by Victor Stinner.

..

.. bpo: 42218
.. date: 2020-10-31-17-50-23
.. nonce: Dp_Z3v
.. section: Core and Builtins

Fixed a bug in the PEG parser that was causing crashes in debug mode. Now
errors are checked in left-recursive rules to avoid cases where such errors
do not get handled in time and appear as long-distance crashes in other
places.

..

.. bpo: 42214
.. date: 2020-10-30-22-16-30
.. nonce: lXskM_
.. section: Core and Builtins

Fixed a possible crash in the PEG parser when checking for the '!=' token in
the ``barry_as_flufl`` rule. Patch by Pablo Galindo.

..

.. bpo: 42206
.. date: 2020-10-30-13-11-01
.. nonce: xxssR8
.. section: Core and Builtins

Propagate and raise the errors caused by :c:func:`PyAST_Validate` in the
parser.

..

.. bpo: 41796
.. date: 2020-10-29-12-49-08
.. nonce: tkGdHq
.. section: Core and Builtins

The :mod:`ast` module internal state is now per interpreter. Patch by Victor
Stinner.

..

.. bpo: 42143
.. date: 2020-10-27-21-34-05
.. nonce: N6KXUO
.. section: Core and Builtins

Fix handling of errors during creation of ``PyFunctionObject``, which
resulted in operations on uninitialized memory. Patch by Yonatan
Goldschmidt.

..

.. bpo: 41659
.. date: 2020-10-27-18-32-49
.. nonce: d4a-8o
.. section: Core and Builtins

Fix a bug in the parser, where a curly brace following a `primary` didn't
fail immediately. This led to invalid expressions like `a {b}` to throw a
:exc:`SyntaxError` with a wrong offset, or invalid expressions ending with a
curly brace like `a {` to not fail immediately in the REPL.

..

.. bpo: 42150
.. date: 2020-10-25-21-14-18
.. nonce: b70u_T
.. section: Core and Builtins

Fix possible buffer overflow in the new parser when checking for
continuation lines. Patch by Pablo Galindo.

..

.. bpo: 42123
.. date: 2020-10-23-02-43-24
.. nonce: 64gJWC
.. section: Core and Builtins

Run the parser two times. On the first run, disable all the rules that only
generate better error messages to gain performance. If there's a parse
failure, run the parser a second time with those enabled.

..

.. bpo: 42093
.. date: 2020-10-20-04-24-07
.. nonce: ooZZNh
.. section: Core and Builtins

The ``LOAD_ATTR`` instruction now uses new "per opcode cache" mechanism and
it is about 36% faster now. Patch by Pablo Galindo and Yury Selivanov.

..

.. bpo: 42030
.. date: 2020-10-15-21-55-32
.. nonce: PmU2CA
.. section: Core and Builtins

Support for the legacy AIX-specific shared library loading support has been
removed. All versions of AIX since 4.3 have supported and defaulted to using
the common Unix mechanism instead.

..

.. bpo: 41984
.. date: 2020-10-14-16-19-43
.. nonce: SEtKMr
.. section: Core and Builtins

The garbage collector now tracks all user-defined classes. Patch by Brandt
Bucher.

..

.. bpo: 41993
.. date: 2020-10-10-13-53-52
.. nonce: YMzixQ
.. section: Core and Builtins

Fixed potential issues with removing not completely initialized module from
``sys.modules`` when import fails.

..

.. bpo: 41979
.. date: 2020-10-09-10-55-50
.. nonce: ImXIk2
.. section: Core and Builtins

Star-unpacking is now allowed for with item's targets in the PEG parser.

..

.. bpo: 41974
.. date: 2020-10-08-09-58-19
.. nonce: 8B-q8O
.. section: Core and Builtins

Removed special methods ``__int__``, ``__float__``, ``__floordiv__``,
``__mod__``, ``__divmod__``, ``__rfloordiv__``, ``__rmod__`` and
``__rdivmod__`` of the :class:`complex` class.  They always raised a
:exc:`TypeError`.

..

.. bpo: 41902
.. date: 2020-10-02-13-32-05
.. nonce: ZKTxzW
.. section: Core and Builtins

Micro optimization when compute :c:member:`~PySequenceMethods.sq_item` and
:c:member:`~PyMappingMethods.mp_subscript` of :class:`range`. Patch by
Dong-hee Na.

..

.. bpo: 41894
.. date: 2020-10-02-11-35-33
.. nonce: ffmtOt
.. section: Core and Builtins

When loading a native module and a load failure occurs, prevent a possible
UnicodeDecodeError when not running in a UTF-8 locale by decoding the load
error message using the current locale's encoding.

..

.. bpo: 41902
.. date: 2020-10-01-22-44-23
.. nonce: iLoMVF
.. section: Core and Builtins

Micro optimization for range.index if step is 1. Patch by Dong-hee Na.

..

.. bpo: 41435
.. date: 2020-08-07-13-42-48
.. nonce: qPWjJA
.. section: Core and Builtins

Add `sys._current_exceptions()` function to retrieve a dictionary mapping
each thread's identifier to the topmost exception currently active in that
thread at the time the function is called.

..

.. bpo: 38605
.. date: 2020-05-27-16-08-16
.. nonce: rcs2uK
.. section: Core and Builtins

Enable ``from __future__ import annotations`` (:pep:`563`) by default. The
values found in :attr:`__annotations__` dicts are now strings, e.g. ``{"x":
"int"}`` instead of ``{"x": int}``.

..

.. bpo: 35455
.. date: 2020-11-02-14-10-48
.. nonce: Q1xTIo
.. section: Library

On Solaris, :func:`~time.thread_time` is now implemented with
``gethrvtime()`` because ``clock_gettime(CLOCK_THREAD_CPUTIME_ID)`` is not
always available. Patch by Jakub Kulik.

..

.. bpo: 42233
.. date: 2020-11-02-01-31-15
.. nonce: YxRj-h
.. section: Library

The :func:`repr` of :mod:`typing` types containing :ref:`Generic Alias Types
<types-genericalias>` previously did not show the parameterized types in the
``GenericAlias``.  They have now been changed to do so.

..

.. bpo: 29566
.. date: 2020-10-31-13-28-36
.. nonce: 6aDbty
.. section: Library

``binhex.binhex()`` consistently writes macOS 9 line endings.

..

.. bpo: 26789
.. date: 2020-10-31-01-16-49
.. nonce: 9BdNAt
.. section: Library

The :class:`logging.FileHandler` class now keeps a reference to the builtin
:func:`open` function to be able to open or reopen the file during Python
finalization. Fix errors like: ``NameError: name 'open' is not defined``.
Patch by Victor Stinner.

..

.. bpo: 42157
.. date: 2020-10-26-23-29-16
.. nonce: 4wuwTe
.. section: Library

Removed the ``unicodedata.ucnhash_CAPI`` attribute which was an internal
PyCapsule object. The related private ``_PyUnicode_Name_CAPI`` structure was
moved to the internal C API. Patch by Victor Stinner.

..

.. bpo: 42157
.. date: 2020-10-26-19-08-07
.. nonce: Bdpa04
.. section: Library

Convert the :mod:`unicodedata` extension module to the multiphase
initialization API (:pep:`489`) and convert the ``unicodedata.UCD`` static
type to a heap type. Patch by Mohamed Koubaa and Victor Stinner.

..

.. bpo: 42146
.. date: 2020-10-25-19-25-02
.. nonce: 6A8uvS
.. section: Library

Fix memory leak in :func:`subprocess.Popen` in case an uid (gid) specified
in `user` (`group`, `extra_groups`) overflows `uid_t` (`gid_t`).

..

.. bpo: 42103
.. date: 2020-10-23-19-20-14
.. nonce: C5obK2
.. section: Library

:exc:`~plistlib.InvalidFileException` and :exc:`RecursionError` are now the
only errors caused by loading malformed binary Plist file (previously
ValueError and TypeError could be raised in some specific cases).

..

.. bpo: 41490
.. date: 2020-10-23-08-54-47
.. nonce: -Yk6OD
.. section: Library

In ``importlib.resources``, ``.path`` method is more aggressive about
releasing handles to zipfile objects early, enabling use-cases like certifi
to leave the context open but delete the underlying zip file.

..

.. bpo: 41052
.. date: 2020-10-21-23-45-02
.. nonce: 3N7J2J
.. section: Library

Pickling heap types implemented in C with protocols 0 and 1 raises now an
error instead of producing incorrect data.

..

.. bpo: 42089
.. date: 2020-10-19-16-53-19
.. nonce: R1dthW
.. section: Library

In ``importlib.metadata.PackageNotFoundError``, make reference to the
package metadata being missing to improve the user experience.

..

.. bpo: 41491
.. date: 2020-10-19-14-02-09
.. nonce: d1BUWH
.. section: Library

plistlib: fix parsing XML plists with hexadecimal integer values

..

.. bpo: 42065
.. date: 2020-10-17-23-17-18
.. nonce: 85BsRA
.. section: Library

Fix an incorrectly formatted error from :meth:`_codecs.charmap_decode` when
called with a mapped value outside the range of valid Unicode code points.
PR by Max Bernstein.

..

.. bpo: 41966
.. date: 2020-10-17-07-52-53
.. nonce: gwEQRZ
.. section: Library

Fix pickling pure Python :class:`datetime.time` subclasses. Patch by Dean
Inwood.

..

.. bpo: 19270
.. date: 2020-10-16-22-48-01
.. nonce: jd_gkA
.. section: Library

:meth:`sched.scheduler.cancel()` will now cancel the correct event, if two
events with same priority are scheduled for the same time. Patch by Bar
Harel.

..

.. bpo: 28660
.. date: 2020-10-16-16-08-04
.. nonce: eX9pvD
.. section: Library

:func:`textwrap.wrap` now attempts to break long words after hyphens when
``break_long_words=True`` and ``break_on_hyphens=True``.

..

.. bpo: 35823
.. date: 2020-10-16-07-45-35
.. nonce: SNQo56
.. section: Library

Use ``vfork()`` instead of ``fork()`` for :func:`subprocess.Popen` on Linux
to improve performance in cases where it is deemed safe.

..

.. bpo: 42043
.. date: 2020-10-15-17-20-37
.. nonce: OS0p_v
.. section: Library

Add support for ``zipfile.Path`` inheritance. ``zipfile.Path.is_file()`` now
returns False for non-existent names. ``zipfile.Path`` objects now expose a
``.filename`` attribute and rely on that to resolve ``.name`` and
``.parent`` when the ``Path`` object is at the root of the zipfile.

..

.. bpo: 42021
.. date: 2020-10-12-21-21-24
.. nonce: 8yv_8-
.. section: Library

Fix possible ref leaks in :mod:`sqlite3` module init.

..

.. bpo: 39101
.. date: 2020-10-11-21-43-03
.. nonce: -I49Pm
.. section: Library

Fixed tests using IsolatedAsyncioTestCase from hanging on BaseExceptions.

..

.. bpo: 41976
.. date: 2020-10-08-18-22-28
.. nonce: Svm0wb
.. section: Library

Fixed a bug that was causing :func:`ctypes.util.find_library` to return
``None`` when triying to locate a library in an environment when gcc>=9 is
available and ``ldconfig`` is not. Patch by Pablo Galindo

..

.. bpo: 41943
.. date: 2020-10-07-18-36-03
.. nonce: Pt55fT
.. section: Library

Fix bug where TestCase.assertLogs doesn't correctly filter messages by
level.

..

.. bpo: 41923
.. date: 2020-10-03-23-14-50
.. nonce: Buonw9
.. section: Library

Implement :pep:`613`, introducing :data:`typing.TypeAlias` annotation.

..

.. bpo: 41905
.. date: 2020-10-01-21-11-03
.. nonce: _JpjR4
.. section: Library

A new function in abc: *update_abstractmethods* to re-calculate an abstract
class's abstract status. In addition, *dataclass* has been changed to call
this function.

..

.. bpo: 23706
.. date: 2020-09-30-11-05-11
.. nonce: dHTGjF
.. section: Library

Added *newline* parameter to ``pathlib.Path.write_text()``.

..

.. bpo: 41876
.. date: 2020-09-29-16-23-54
.. nonce: QicdDU
.. section: Library

Tkinter font class repr uses font name

..

.. bpo: 41831
.. date: 2020-09-22-11-07-50
.. nonce: k-Eop_
.. section: Library

``str()`` for the ``type`` attribute of the ``tkinter.Event`` object always
returns now the numeric code returned by Tk instead of the name of the event
type.

..

.. bpo: 39337
.. date: 2020-09-13-02-02-18
.. nonce: L3NXTt
.. section: Library

:func:`encodings.normalize_encoding` now ignores non-ASCII characters.

..

.. bpo: 41747
.. date: 2020-09-08-23-41-29
.. nonce: M6wLKv
.. section: Library

Ensure all methods that generated from :func:`dataclasses.dataclass` objects
now have the proper ``__qualname__`` attribute referring to the class they
belong to. Patch by Batuhan Taskaya.

..

.. bpo: 30681
.. date: 2020-09-04-17-33-04
.. nonce: LR4fnY
.. section: Library

Handle exceptions caused by unparsable date headers when using email
"default" policy.  Patch by Tim Bell, Georges Toth

..

.. bpo: 41586
.. date: 2020-08-19-08-32-13
.. nonce: IYjmjK
.. section: Library

Add F_SETPIPE_SZ and F_GETPIPE_SZ to fcntl module. Allow setting pipesize on
subprocess.Popen.

..

.. bpo: 41229
.. date: 2020-07-19-20-10-41
.. nonce: p8rJa2
.. section: Library

Add ``contextlib.aclosing`` for deterministic cleanup of async generators
which is analogous to ``contextlib.closing`` for non-async generators. Patch
by Joongi Kim and John Belmonte.

..

.. bpo: 16396
.. date: 2020-07-08-09-45-00
.. nonce: z8o8Pn
.. section: Library

Allow ``ctypes.wintypes`` to be imported on non-Windows systems.

..

.. bpo: 4356
.. date: 2020-05-31-10-48-47
.. nonce: P8kXqp
.. section: Library

Add a key function to the bisect module.

..

.. bpo: 40592
.. date: 2020-05-14-16-01-34
.. nonce: Cmk855
.. section: Library

:func:`shutil.which` now ignores empty entries in :envvar:`PATHEXT` instead
of treating them as a match.

..

.. bpo: 40492
.. date: 2020-05-04-12-16-00
.. nonce: ONk9Na
.. section: Library

Fix ``--outfile`` for :mod:`cProfile` / :mod:`profile` not writing the
output file in the original directory when the program being profiled
changes the working directory.  PR by Anthony Sottile.

..

.. bpo: 34204
.. date: 2020-04-21-17-18-33
.. nonce: 9wXTtY
.. section: Library

The :mod:`shelve` module now uses :data:`pickle.DEFAULT_PROTOCOL` by default
instead of :mod:`pickle` protocol ``3``.

..

.. bpo: 27321
.. date: 2020-01-19-18-40-26
.. nonce: 8e6SpM
.. section: Library

Fixed KeyError exception when flattening an email to a string attempts to
replace a non-existent Content-Transfer-Encoding header.

..

.. bpo: 38976
.. date: 2019-12-05-05-22-49
.. nonce: 5MG7Uu
.. section: Library

The :mod:`http.cookiejar` module now supports the parsing of cookies in
CURL-style cookiejar files through MozillaCookieJar on all platforms.
Previously, such cookie entries would be silently ignored when loading a
cookiejar with such entries.

Additionally, the HTTP Only attribute is persisted in the object, and will
be correctly written to file if the MozillaCookieJar object is subsequently
dumped.

..

.. bpo: 42061
.. date: 2020-10-28-21-39-45
.. nonce: _x-0sg
.. section: Documentation

Document __format__ functionality for IP addresses.

..

.. bpo: 41910
.. date: 2020-10-21-14-40-54
.. nonce: CzBMit
.. section: Documentation

Document the default implementation of `object.__eq__`.

..

.. bpo: 42010
.. date: 2020-10-21-02-21-14
.. nonce: 76vJ0u
.. section: Documentation

Clarify that subscription expressions are also valid for certain
:term:`classes <class>` and :term:`types <type>` in the standard library,
and for user-defined classes and types if the classmethod
:meth:`__class_getitem__` is provided.

..

.. bpo: 41805
.. date: 2020-10-10-01-36-37
.. nonce: l-CGv5
.. section: Documentation

Documented :ref:`generic alias type <types-genericalias>` and
:data:`types.GenericAlias`. Also added an entry in glossary for
:term:`generic types <generic type>`.

..

.. bpo: 39693
.. date: 2020-02-24-09-02-05
.. nonce: QXw0Fm
.. section: Documentation

Fix tarfile's extractfile documentation

..

.. bpo: 39416
.. date: 2020-01-22-05-14-53
.. nonce: uYjhEm
.. section: Documentation

Document some restrictions on the default string representations of numeric
classes.

..

.. bpo: 41739
.. date: 2020-10-12-00-11-47
.. nonce: wSCc4K
.. section: Tests

Fix test_logging.test_race_between_set_target_and_flush(): the test now
waits until all threads complete to avoid leaking running threads.

..

.. bpo: 41970
.. date: 2020-10-08-14-00-17
.. nonce: aZ8QFf
.. section: Tests

Avoid a test failure in ``test_lib2to3`` if the module has already imported
at the time the test executes. Patch by Pablo Galindo.

..

.. bpo: 41944
.. date: 2020-10-05-17-43-46
.. nonce: rf1dYb
.. section: Tests

Tests for CJK codecs no longer call ``eval()`` on content received via HTTP.

..

.. bpo: 41306
.. date: 2020-08-03-13-44-37
.. nonce: VDoWXI
.. section: Tests

Fixed a failure in ``test_tk.test_widgets.ScaleTest`` happening when
executing the test with Tk 8.6.10.

..

.. bpo: 38980
.. date: 2020-10-21-18-31-54
.. nonce: xz7BNd
.. section: Build

Add ``-fno-semantic-interposition`` to both the compile and link line when
building with ``--enable-optimizations``. Patch by Victor Stinner and Pablo
Galindo.

..

.. bpo: 38439
.. date: 2020-10-20-13-19-42
.. nonce: eMLi-t
.. section: Windows

Updates the icons for IDLE in the Windows Store package.

..

.. bpo: 38252
.. date: 2020-10-18-18-43-45
.. nonce: 7Nlepg
.. section: Windows

Use 8-byte step to detect ASCII sequence in 64-bit Windows build.

..

.. bpo: 39107
.. date: 2020-09-24-23-09-40
.. nonce: GbUZvD
.. section: Windows

Update Tcl and Tk to 8.6.10 in Windows installer.

..

.. bpo: 41557
.. date: 2020-08-26-09-35-06
.. nonce: vt00cQ
.. section: Windows

Update Windows installer to use SQLite 3.33.0.

..

.. bpo: 38324
.. date: 2020-05-30-02-46-43
.. nonce: 476M-5
.. section: Windows

Avoid Unicode errors when accessing certain locale data on Windows.

..

.. bpo: 41471
.. date: 2020-10-19-12-25-19
.. nonce: gwA7un
.. section: macOS

Ignore invalid prefix lengths in system proxy excludes.

..

.. bpo: 33987
.. date: 2020-10-24-21-27-37
.. nonce: fIh9JL
.. section: IDLE

Mostly finish using ttk widgets, mainly for editor, settings, and searches.
Some patches by Mark Roseman.

..

.. bpo: 40511
.. date: 2020-06-16-12-16-13
.. nonce: XkihpM
.. section: IDLE

Typing opening and closing parentheses inside the parentheses of a function
call will no longer cause unnecessary "flashing" off and on of an existing
open call-tip, e.g. when typed in a string literal.

..

.. bpo: 38439
.. date: 2020-04-22-09-37-40
.. nonce: ieXL-c
.. section: IDLE

Add a 256×256 pixel IDLE icon to the Windows .ico file. Created by Andrew
Clover. Remove the low-color gif variations from the .ico file.

..

.. bpo: 42157
.. date: 2020-10-16-10-47-17
.. nonce: e3BcPM
.. section: C API

The private ``_PyUnicode_Name_CAPI`` structure of the PyCapsule API
``unicodedata.ucnhash_CAPI`` has been moved to the internal C API. Patch by
Victor Stinner.

..

.. bpo: 42015
.. date: 2020-10-12-20-13-58
.. nonce: X4H2_V
.. section: C API

Fix potential crash in deallocating method objects when dynamically
allocated `PyMethodDef`'s lifetime is managed through the ``self`` argument
of a `PyCFunction`.

..

.. bpo: 40423
.. date: 2020-10-11-19-17-44
.. nonce: GsmgEj
.. section: C API

The :mod:`subprocess` module and ``os.closerange`` will now use the
``close_range(low, high, flags)`` syscall when it is available for more
efficient closing of ranges of descriptors.

..

.. bpo: 41845
.. date: 2020-10-11-05-05-53
.. nonce: ZFvuQM
.. section: C API

:c:func:`PyObject_GenericGetDict` is available again in the limited API when
targeting 3.10 or later.

..

.. bpo: 40422
.. date: 2020-10-10-14-05-24
.. nonce: sh8IDY
.. section: C API

Add `_Py_closerange` function to provide performant closing of a range of
file descriptors.

..

.. bpo: 41986
.. date: 2020-10-09-22-50-46
.. nonce: JUPE59
.. section: C API

:c:data:`Py_FileSystemDefaultEncodeErrors` and :c:data:`Py_UTF8Mode` are
available again in limited API.

..

.. bpo: 41756
.. date: 2020-09-28-14-31-07
.. nonce: ZZ5wJG
.. section: C API

Add `PyIter_Send` function to allow sending value into
generator/coroutine/iterator without raising StopIteration exception to
signal return.

..

.. bpo: 41784
.. date: 2020-09-14-10-17-00
.. nonce: Yl4gI2
.. section: C API

Added ``PyUnicode_AsUTF8AndSize`` to the limited C API.