diff -Nru python3.10-3.10.12/debian/changelog python3.10-3.10.13/debian/changelog --- python3.10-3.10.12/debian/changelog 2023-06-11 05:26:28.000000000 +0000 +++ python3.10-3.10.13/debian/changelog 2023-09-05 06:03:44.000000000 +0000 @@ -1,3 +1,16 @@ +python3.10 (3.10.13-1~22.04) jammy-proposed; urgency=medium + + * SRU: LP: #2034098: Backport the 3.10.13 release to 22.04 LTS. + + -- Matthias Klose Tue, 05 Sep 2023 08:03:44 +0200 + +python3.10 (3.10.13-1) unstable; urgency=medium + + * Python 3.10.13 release. + - Addresses CVE-2023-40217. + + -- Matthias Klose Fri, 25 Aug 2023 13:35:07 +0200 + python3.10 (3.10.12-1~22.04.2) jammy-proposed; urgency=medium * SRU: LP: #1995504: Backport the 3.10.12 release to 22.04 LTS. @@ -9,6 +22,12 @@ python3.10 (3.10.12-1) unstable; urgency=medium + * Python 3.10.12 release. + + -- Matthias Klose Wed, 07 Jun 2023 12:09:46 +0200 + +python3.10 (3.10.12-1) unstable; urgency=medium + * Python 3.10.12 release. -- Matthias Klose Wed, 07 Jun 2023 12:09:46 +0200 diff -Nru python3.10-3.10.12/Doc/c-api/allocation.rst python3.10-3.10.13/Doc/c-api/allocation.rst --- python3.10-3.10.12/Doc/c-api/allocation.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Doc/c-api/allocation.rst 2023-08-24 12:46:25.000000000 +0000 @@ -31,9 +31,11 @@ Allocate a new Python object using the C structure type *TYPE* and the Python type object *type*. Fields not defined by the Python object header - are not initialized; the object's reference count will be one. The size of - the memory allocation is determined from the :c:member:`~PyTypeObject.tp_basicsize` field of - the type object. + are not initialized. + The caller will own the only reference to the object + (i.e. its reference count will be one). + The size of the memory allocation is determined from the + :c:member:`~PyTypeObject.tp_basicsize` field of the type object. .. c:function:: TYPE* PyObject_NewVar(TYPE, PyTypeObject *type, Py_ssize_t size) diff -Nru python3.10-3.10.12/Doc/c-api/arg.rst python3.10-3.10.13/Doc/c-api/arg.rst --- python3.10-3.10.12/Doc/c-api/arg.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Doc/c-api/arg.rst 2023-08-24 12:46:25.000000000 +0000 @@ -330,8 +330,10 @@ ``O`` (object) [PyObject \*] Store a Python object (without any conversion) in a C object pointer. The C - program thus receives the actual object that was passed. The object's reference - count is not increased. The pointer stored is not ``NULL``. + program thus receives the actual object that was passed. A new + :term:`strong reference` to the object is not created + (i.e. its reference count is not increased). + The pointer stored is not ``NULL``. ``O!`` (object) [*typeobject*, PyObject \*] Store a Python object in a C object pointer. This is similar to ``O``, but @@ -415,7 +417,8 @@ mutually exclude each other. Note that any Python object references which are provided to the caller are -*borrowed* references; do not decrement their reference count! +*borrowed* references; do not release them +(i.e. do not decrement their reference count)! Additional arguments passed to these functions must be addresses of variables whose type is determined by the format string; these are used to store values @@ -650,8 +653,10 @@ Convert a C :c:type:`Py_complex` structure to a Python complex number. ``O`` (object) [PyObject \*] - Pass a Python object untouched (except for its reference count, which is - incremented by one). If the object passed in is a ``NULL`` pointer, it is assumed + Pass a Python object untouched but create a new + :term:`strong reference` to it + (i.e. its reference count is incremented by one). + If the object passed in is a ``NULL`` pointer, it is assumed that this was caused because the call producing the argument found an error and set an exception. Therefore, :c:func:`Py_BuildValue` will return ``NULL`` but won't raise an exception. If no exception has been raised yet, :exc:`SystemError` is @@ -661,7 +666,7 @@ Same as ``O``. ``N`` (object) [PyObject \*] - Same as ``O``, except it doesn't increment the reference count on the object. + Same as ``O``, except it doesn't create a new :term:`strong reference`. Useful when the object is created by a call to an object constructor in the argument list. diff -Nru python3.10-3.10.12/Doc/c-api/buffer.rst python3.10-3.10.13/Doc/c-api/buffer.rst --- python3.10-3.10.12/Doc/c-api/buffer.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Doc/c-api/buffer.rst 2023-08-24 12:46:25.000000000 +0000 @@ -102,7 +102,9 @@ .. c:member:: PyObject *obj A new reference to the exporting object. The reference is owned by - the consumer and automatically decremented and set to ``NULL`` by + the consumer and automatically released + (i.e. reference count decremented) + and set to ``NULL`` by :c:func:`PyBuffer_Release`. The field is the equivalent of the return value of any standard C-API function. @@ -454,7 +456,8 @@ .. c:function:: void PyBuffer_Release(Py_buffer *view) - Release the buffer *view* and decrement the reference count for + Release the buffer *view* and release the :term:`strong reference` + (i.e. decrement the reference count) to the view's supporting object, ``view->obj``. This function MUST be called when the buffer is no longer being used, otherwise reference leaks may occur. diff -Nru python3.10-3.10.12/Doc/c-api/bytes.rst python3.10-3.10.13/Doc/c-api/bytes.rst --- python3.10-3.10.12/Doc/c-api/bytes.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Doc/c-api/bytes.rst 2023-08-24 12:46:25.000000000 +0000 @@ -187,8 +187,8 @@ .. c:function:: void PyBytes_ConcatAndDel(PyObject **bytes, PyObject *newpart) Create a new bytes object in *\*bytes* containing the contents of *newpart* - appended to *bytes*. This version decrements the reference count of - *newpart*. + appended to *bytes*. This version releases the :term:`strong reference` + to *newpart* (i.e. decrements its reference count). .. c:function:: int _PyBytes_Resize(PyObject **bytes, Py_ssize_t newsize) diff -Nru python3.10-3.10.12/Doc/c-api/exceptions.rst python3.10-3.10.13/Doc/c-api/exceptions.rst --- python3.10-3.10.12/Doc/c-api/exceptions.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Doc/c-api/exceptions.rst 2023-08-24 12:46:25.000000000 +0000 @@ -99,7 +99,8 @@ This is the most common way to set the error indicator. The first argument specifies the exception type; it is normally one of the standard exceptions, - e.g. :c:data:`PyExc_RuntimeError`. You need not increment its reference count. + e.g. :c:data:`PyExc_RuntimeError`. You need not create a new + :term:`strong reference` to it (e.g. with :c:func:`Py_INCREF`). The second argument is an error message; it is decoded from ``'utf-8'``. diff -Nru python3.10-3.10.12/Doc/c-api/intro.rst python3.10-3.10.13/Doc/c-api/intro.rst --- python3.10-3.10.12/Doc/c-api/intro.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Doc/c-api/intro.rst 2023-08-24 12:46:25.000000000 +0000 @@ -252,52 +252,58 @@ Reference Counts ---------------- -The reference count is important because today's computers have a finite (and -often severely limited) memory size; it counts how many different places there -are that have a reference to an object. Such a place could be another object, -or a global (or static) C variable, or a local variable in some C function. -When an object's reference count becomes zero, the object is deallocated. If -it contains references to other objects, their reference count is decremented. -Those other objects may be deallocated in turn, if this decrement makes their -reference count become zero, and so on. (There's an obvious problem with -objects that reference each other here; for now, the solution is "don't do -that.") +The reference count is important because today's computers have a finite +(and often severely limited) memory size; it counts how many different +places there are that have a :term:`strong reference` to an object. +Such a place could be another object, or a global (or static) C variable, +or a local variable in some C function. +When the last :term:`strong reference` to an object is released +(i.e. its reference count becomes zero), the object is deallocated. +If it contains references to other objects, those references are released. +Those other objects may be deallocated in turn, if there are no more +references to them, and so on. (There's an obvious problem with +objects that reference each other here; for now, the solution +is "don't do that.") .. index:: single: Py_INCREF() single: Py_DECREF() -Reference counts are always manipulated explicitly. The normal way is to use -the macro :c:func:`Py_INCREF` to increment an object's reference count by one, -and :c:func:`Py_DECREF` to decrement it by one. The :c:func:`Py_DECREF` macro +Reference counts are always manipulated explicitly. The normal way is +to use the macro :c:func:`Py_INCREF` to take a new reference to an +object (i.e. increment its reference count by one), +and :c:func:`Py_DECREF` to release that reference (i.e. decrement the +reference count by one). The :c:func:`Py_DECREF` macro is considerably more complex than the incref one, since it must check whether the reference count becomes zero and then cause the object's deallocator to be -called. The deallocator is a function pointer contained in the object's type -structure. The type-specific deallocator takes care of decrementing the -reference counts for other objects contained in the object if this is a compound +called. The deallocator is a function pointer contained in the object's type +structure. The type-specific deallocator takes care of releasing references +for other objects contained in the object if this is a compound object type, such as a list, as well as performing any additional finalization that's needed. There's no chance that the reference count can overflow; at least as many bits are used to hold the reference count as there are distinct memory locations in virtual memory (assuming ``sizeof(Py_ssize_t) >= sizeof(void*)``). Thus, the reference count increment is a simple operation. -It is not necessary to increment an object's reference count for every local -variable that contains a pointer to an object. In theory, the object's +It is not necessary to hold a :term:`strong reference` (i.e. increment +the reference count) for every local variable that contains a pointer +to an object. In theory, the object's reference count goes up by one when the variable is made to point to it and it goes down by one when the variable goes out of scope. However, these two cancel each other out, so at the end the reference count hasn't changed. The only real reason to use the reference count is to prevent the object from being deallocated as long as our variable is pointing to it. If we know that there is at least one other reference to the object that lives at least as long as -our variable, there is no need to increment the reference count temporarily. +our variable, there is no need to take a new :term:`strong reference` +(i.e. increment the reference count) temporarily. An important situation where this arises is in objects that are passed as arguments to C functions in an extension module that are called from Python; the call mechanism guarantees to hold a reference to every argument for the duration of the call. However, a common pitfall is to extract an object from a list and hold on to it -for a while without incrementing its reference count. Some other operation might -conceivably remove the object from the list, decrementing its reference count +for a while without taking a new reference. Some other operation might +conceivably remove the object from the list, releasing that reference, and possibly deallocating it. The real danger is that innocent-looking operations may invoke arbitrary Python code which could do this; there is a code path which allows control to flow back to the user from a :c:func:`Py_DECREF`, so @@ -305,7 +311,8 @@ A safe approach is to always use the generic operations (functions whose name begins with ``PyObject_``, ``PyNumber_``, ``PySequence_`` or ``PyMapping_``). -These operations always increment the reference count of the object they return. +These operations always create a new :term:`strong reference` +(i.e. increment the reference count) of the object they return. This leaves the caller with the responsibility to call :c:func:`Py_DECREF` when they are done with the result; this soon becomes second nature. @@ -321,7 +328,7 @@ reference" means being responsible for calling Py_DECREF on it when the reference is no longer needed. Ownership can also be transferred, meaning that the code that receives ownership of the reference then becomes responsible for -eventually decref'ing it by calling :c:func:`Py_DECREF` or :c:func:`Py_XDECREF` +eventually releasing it by calling :c:func:`Py_DECREF` or :c:func:`Py_XDECREF` when it's no longer needed---or passing on this responsibility (usually to its caller). When a function passes ownership of a reference on to its caller, the caller is said to receive a *new* reference. When no ownership is transferred, @@ -379,9 +386,9 @@ It is much more common to use :c:func:`PyObject_SetItem` and friends with items whose references you are only borrowing, like arguments that were passed in to -the function you are writing. In that case, their behaviour regarding reference -counts is much saner, since you don't have to increment a reference count so you -can give a reference away ("have it be stolen"). For example, this function +the function you are writing. In that case, their behaviour regarding references +is much saner, since you don't have to take a new reference just so you +can give that reference away ("have it be stolen"). For example, this function sets all items of a list (actually, any mutable sequence) to a given item:: int diff -Nru python3.10-3.10.12/Doc/c-api/module.rst python3.10-3.10.13/Doc/c-api/module.rst --- python3.10-3.10.12/Doc/c-api/module.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Doc/c-api/module.rst 2023-08-24 12:46:25.000000000 +0000 @@ -498,7 +498,7 @@ .. note:: Unlike other functions that steal references, ``PyModule_AddObject()`` - only decrements the reference count of *value* **on success**. + only releases the reference to *value* **on success**. This means that its return value must be checked, and calling code must :c:func:`Py_DECREF` *value* manually on error. diff -Nru python3.10-3.10.12/Doc/c-api/object.rst python3.10-3.10.13/Doc/c-api/object.rst --- python3.10-3.10.12/Doc/c-api/object.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Doc/c-api/object.rst 2023-08-24 12:46:25.000000000 +0000 @@ -15,8 +15,8 @@ .. c:macro:: Py_RETURN_NOTIMPLEMENTED Properly handle returning :c:data:`Py_NotImplemented` from within a C - function (that is, increment the reference count of NotImplemented and - return it). + function (that is, create a new :term:`strong reference` + to NotImplemented and return it). .. c:function:: int PyObject_Print(PyObject *o, FILE *fp, int flags) @@ -298,11 +298,12 @@ When *o* is non-``NULL``, returns a type object corresponding to the object type of object *o*. On failure, raises :exc:`SystemError` and returns ``NULL``. This - is equivalent to the Python expression ``type(o)``. This function increments the - reference count of the return value. There's really no reason to use this + is equivalent to the Python expression ``type(o)``. + This function creates a new :term:`strong reference` to the return value. + There's really no reason to use this function instead of the :c:func:`Py_TYPE()` function, which returns a - pointer of type :c:expr:`PyTypeObject*`, except when the incremented reference - count is needed. + pointer of type :c:expr:`PyTypeObject*`, except when a new + :term:`strong reference` is needed. .. c:function:: int PyObject_TypeCheck(PyObject *o, PyTypeObject *type) diff -Nru python3.10-3.10.12/Doc/c-api/refcounting.rst python3.10-3.10.13/Doc/c-api/refcounting.rst --- python3.10-3.10.12/Doc/c-api/refcounting.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Doc/c-api/refcounting.rst 2023-08-24 12:46:25.000000000 +0000 @@ -13,31 +13,36 @@ .. c:function:: void Py_INCREF(PyObject *o) - Increment the reference count for object *o*. + Indicate taking a new :term:`strong reference` to object *o*, + indicating it is in use and should not be destroyed. This function is usually used to convert a :term:`borrowed reference` to a :term:`strong reference` in-place. The :c:func:`Py_NewRef` function can be used to create a new :term:`strong reference`. + When done using the object, release it by calling :c:func:`Py_DECREF`. + The object must not be ``NULL``; if you aren't sure that it isn't ``NULL``, use :c:func:`Py_XINCREF`. + Do not expect this function to actually modify *o* in any way. + .. c:function:: void Py_XINCREF(PyObject *o) - Increment the reference count for object *o*. The object may be ``NULL``, in - which case the macro has no effect. + Similar to :c:func:`Py_INCREF`, but the object *o* can be ``NULL``, + in which case this has no effect. See also :c:func:`Py_XNewRef`. .. c:function:: PyObject* Py_NewRef(PyObject *o) - Create a new :term:`strong reference` to an object: increment the reference - count of the object *o* and return the object *o*. + Create a new :term:`strong reference` to an object: + call :c:func:`Py_INCREF` on *o* and return the object *o*. When the :term:`strong reference` is no longer needed, :c:func:`Py_DECREF` - should be called on it to decrement the object reference count. + should be called on it to release the reference. The object *o* must not be ``NULL``; use :c:func:`Py_XNewRef` if *o* can be ``NULL``. @@ -67,9 +72,12 @@ .. c:function:: void Py_DECREF(PyObject *o) - Decrement the reference count for object *o*. + Release a :term:`strong reference` to object *o*, indicating the + reference is no longer used. - If the reference count reaches zero, the object's type's deallocation + Once the last :term:`strong reference` is released + (i.e. the object's reference count reaches 0), + the object's type's deallocation function (which must not be ``NULL``) is invoked. This function is usually used to delete a :term:`strong reference` before @@ -78,6 +86,8 @@ The object must not be ``NULL``; if you aren't sure that it isn't ``NULL``, use :c:func:`Py_XDECREF`. + Do not expect this function to actually modify *o* in any way. + .. warning:: The deallocation function can cause arbitrary Python code to be invoked (e.g. @@ -92,32 +102,35 @@ .. c:function:: void Py_XDECREF(PyObject *o) - Decrement the reference count for object *o*. The object may be ``NULL``, in - which case the macro has no effect; otherwise the effect is the same as for - :c:func:`Py_DECREF`, and the same warning applies. + Similar to :c:func:`Py_DECREF`, but the object *o* can be ``NULL``, + in which case this has no effect. + The same warning from :c:func:`Py_DECREF` applies here as well. .. c:function:: void Py_CLEAR(PyObject *o) - Decrement the reference count for object *o*. The object may be ``NULL``, in + Release a :term:`strong reference` for object *o*. + The object may be ``NULL``, in which case the macro has no effect; otherwise the effect is the same as for :c:func:`Py_DECREF`, except that the argument is also set to ``NULL``. The warning for :c:func:`Py_DECREF` does not apply with respect to the object passed because the macro carefully uses a temporary variable and sets the argument to ``NULL`` - before decrementing its reference count. + before releasing the reference. - It is a good idea to use this macro whenever decrementing the reference - count of an object that might be traversed during garbage collection. + It is a good idea to use this macro whenever releasing a reference + to an object that might be traversed during garbage collection. .. c:function:: void Py_IncRef(PyObject *o) - Increment the reference count for object *o*. A function version of :c:func:`Py_XINCREF`. + Indicate taking a new :term:`strong reference` to object *o*. + A function version of :c:func:`Py_XINCREF`. It can be used for runtime dynamic embedding of Python. .. c:function:: void Py_DecRef(PyObject *o) - Decrement the reference count for object *o*. A function version of :c:func:`Py_XDECREF`. + Release a :term:`strong reference` to object *o*. + A function version of :c:func:`Py_XDECREF`. It can be used for runtime dynamic embedding of Python. diff -Nru python3.10-3.10.12/Doc/c-api/sys.rst python3.10-3.10.13/Doc/c-api/sys.rst --- python3.10-3.10.12/Doc/c-api/sys.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Doc/c-api/sys.rst 2023-08-24 12:46:25.000000000 +0000 @@ -8,8 +8,9 @@ .. c:function:: PyObject* PyOS_FSPath(PyObject *path) Return the file system representation for *path*. If the object is a - :class:`str` or :class:`bytes` object, then its reference count is - incremented. If the object implements the :class:`os.PathLike` interface, + :class:`str` or :class:`bytes` object, then a new + :term:`strong reference` is returned. + If the object implements the :class:`os.PathLike` interface, then :meth:`~os.PathLike.__fspath__` is returned as long as it is a :class:`str` or :class:`bytes` object. Otherwise :exc:`TypeError` is raised and ``NULL`` is returned. diff -Nru python3.10-3.10.12/Doc/c-api/typeobj.rst python3.10-3.10.13/Doc/c-api/typeobj.rst --- python3.10-3.10.12/Doc/c-api/typeobj.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Doc/c-api/typeobj.rst 2023-08-24 12:46:25.000000000 +0000 @@ -688,7 +688,8 @@ } Finally, if the type is heap allocated (:const:`Py_TPFLAGS_HEAPTYPE`), the - deallocator should decrement the reference count for its type object after + deallocator should release the owned reference to its type object + (via :c:func:`Py_DECREF`) after calling the type deallocator. In order to avoid dangling pointers, the recommended way to achieve this is: @@ -1394,9 +1395,10 @@ } The :c:func:`Py_CLEAR` macro should be used, because clearing references is - delicate: the reference to the contained object must not be decremented until + delicate: the reference to the contained object must not be released + (via :c:func:`Py_DECREF`) until after the pointer to the contained object is set to ``NULL``. This is because - decrementing the reference count may cause the contained object to become trash, + releasing the reference may cause the contained object to become trash, triggering a chain of reclamation activity that may include invoking arbitrary Python code (due to finalizers, or weakref callbacks, associated with the contained object). If it's possible for such code to reference *self* again, @@ -1472,7 +1474,7 @@ they may be C ints or floats). The third argument specifies the requested operation, as for :c:func:`PyObject_RichCompare`. - The return value's reference count is properly incremented. + The returned value is a new :term:`strong reference`. On error, sets an exception and returns ``NULL`` from the function. diff -Nru python3.10-3.10.12/Doc/c-api/unicode.rst python3.10-3.10.13/Doc/c-api/unicode.rst --- python3.10-3.10.12/Doc/c-api/unicode.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Doc/c-api/unicode.rst 2023-08-24 12:46:25.000000000 +0000 @@ -1715,11 +1715,11 @@ Intern the argument *\*string* in place. The argument must be the address of a pointer variable pointing to a Python Unicode string object. If there is an existing interned string that is the same as *\*string*, it sets *\*string* to - it (decrementing the reference count of the old string object and incrementing - the reference count of the interned string object), otherwise it leaves - *\*string* alone and interns it (incrementing its reference count). - (Clarification: even though there is a lot of talk about reference counts, think - of this function as reference-count-neutral; you own the object after the call + it (releasing the reference to the old string object and creating a new + :term:`strong reference` to the interned string object), otherwise it leaves + *\*string* alone and interns it (creating a new :term:`strong reference`). + (Clarification: even though there is a lot of talk about references, think + of this function as reference-neutral; you own the object after the call if and only if you owned it before the call.) diff -Nru python3.10-3.10.12/Doc/glossary.rst python3.10-3.10.13/Doc/glossary.rst --- python3.10-3.10.12/Doc/glossary.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Doc/glossary.rst 2023-08-24 12:46:25.000000000 +0000 @@ -168,8 +168,9 @@ :class:`str` objects. borrowed reference - In Python's C API, a borrowed reference is a reference to an object. - It does not modify the object reference count. It becomes a dangling + In Python's C API, a borrowed reference is a reference to an object, + where the code using the object does not own the reference. + It becomes a dangling pointer if the object is destroyed. For example, a garbage collection can remove the last :term:`strong reference` to the object and so destroy it. @@ -1142,8 +1143,10 @@ strong reference In Python's C API, a strong reference is a reference to an object - which increments the object's reference count when it is created and - decrements the object's reference count when it is deleted. + which is owned by the code holding the reference. The strong + reference is taken by calling :c:func:`Py_INCREF` when the + reference is created and released with :c:func:`Py_DECREF` + when the reference is deleted. The :c:func:`Py_NewRef` function can be used to create a strong reference to an object. Usually, the :c:func:`Py_DECREF` function must be called on diff -Nru python3.10-3.10.12/Doc/library/asyncio-eventloop.rst python3.10-3.10.13/Doc/library/asyncio-eventloop.rst --- python3.10-3.10.12/Doc/library/asyncio-eventloop.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Doc/library/asyncio-eventloop.rst 2023-08-24 12:46:25.000000000 +0000 @@ -830,6 +830,9 @@ object only because the coder caches *protocol*-side data and sporadically exchanges extra TLS session packets with *transport*. + In some situations (e.g. when the passed transport is already closing) this + may return ``None``. + Parameters: * *transport* and *protocol* instances that methods like diff -Nru python3.10-3.10.12/Doc/library/ssl.rst python3.10-3.10.13/Doc/library/ssl.rst --- python3.10-3.10.12/Doc/library/ssl.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Doc/library/ssl.rst 2023-08-24 12:46:25.000000000 +0000 @@ -2690,7 +2690,7 @@ >>> client_context.maximum_version = ssl.TLSVersion.TLSv1_3 -The SSL context created above will only allow TLSv1.2 and later (if +The SSL context created above will only allow TLSv1.3 and later (if supported by your system) connections to a server. :const:`PROTOCOL_TLS_CLIENT` implies certificate validation and hostname checks by default. You have to load certificates into the context. diff -Nru python3.10-3.10.12/Doc/library/sys.rst python3.10-3.10.13/Doc/library/sys.rst --- python3.10-3.10.12/Doc/library/sys.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Doc/library/sys.rst 2023-08-24 12:46:25.000000000 +0000 @@ -709,6 +709,9 @@ higher than you might expect, because it includes the (temporary) reference as an argument to :func:`getrefcount`. + Note that the returned value may not actually reflect how many + references to the object are actually held. Consequently, do not rely + on the returned value to be accurate, other than a value of 0 or 1. .. function:: getrecursionlimit() diff -Nru python3.10-3.10.12/Doc/library/tarfile.rst python3.10-3.10.13/Doc/library/tarfile.rst --- python3.10-3.10.12/Doc/library/tarfile.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Doc/library/tarfile.rst 2023-08-24 12:46:25.000000000 +0000 @@ -732,6 +732,11 @@ Name of the target file name, which is only present in :class:`TarInfo` objects of type :const:`LNKTYPE` and :const:`SYMTYPE`. + For symbolic links (``SYMTYPE``), the *linkname* is relative to the directory + that contains the link. + For hard links (``LNKTYPE``), the *linkname* is relative to the root of + the archive. + .. attribute:: TarInfo.uid :type: int diff -Nru python3.10-3.10.12/Doc/library/tokenize.rst python3.10-3.10.13/Doc/library/tokenize.rst --- python3.10-3.10.12/Doc/library/tokenize.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Doc/library/tokenize.rst 2023-08-24 12:46:25.000000000 +0000 @@ -22,6 +22,15 @@ type can be determined by checking the ``exact_type`` property on the :term:`named tuple` returned from :func:`tokenize.tokenize`. + +.. warning:: + + Note that the functions in this module are only designed to parse + syntactically valid Python code (code that does not raise when parsed + using :func:`ast.parse`). The behavior of the functions in this module is + **undefined** when providing invalid Python code and it can change at any + point. + Tokenizing Input ---------------- diff -Nru python3.10-3.10.12/Doc/reference/expressions.rst python3.10-3.10.13/Doc/reference/expressions.rst --- python3.10-3.10.12/Doc/reference/expressions.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Doc/reference/expressions.rst 2023-08-24 12:46:25.000000000 +0000 @@ -735,7 +735,8 @@ because there is no yield expression that could receive the value. -.. coroutinemethod:: agen.athrow(type[, value[, traceback]]) +.. coroutinemethod:: agen.athrow(value) + agen.athrow(type[, value[, traceback]]) Returns an awaitable that raises an exception of type ``type`` at the point where the asynchronous generator was paused, and returns the next value diff -Nru python3.10-3.10.12/Include/patchlevel.h python3.10-3.10.13/Include/patchlevel.h --- python3.10-3.10.12/Include/patchlevel.h 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Include/patchlevel.h 2023-08-24 12:46:25.000000000 +0000 @@ -18,12 +18,12 @@ /*--start constants--*/ #define PY_MAJOR_VERSION 3 #define PY_MINOR_VERSION 10 -#define PY_MICRO_VERSION 12 +#define PY_MICRO_VERSION 13 #define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_FINAL #define PY_RELEASE_SERIAL 0 /* Version as a string */ -#define PY_VERSION "3.10.12" +#define PY_VERSION "3.10.13" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff -Nru python3.10-3.10.12/Lib/pydoc_data/topics.py python3.10-3.10.13/Lib/pydoc_data/topics.py --- python3.10-3.10.12/Lib/pydoc_data/topics.py 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Lib/pydoc_data/topics.py 2023-08-24 12:46:25.000000000 +0000 @@ -1,5 +1,5 @@ # -*- coding: utf-8 -*- -# Autogenerated by Sphinx on Tue Jun 6 23:30:19 2023 +# Autogenerated by Sphinx on Thu Aug 24 13:45:52 2023 topics = {'assert': 'The "assert" statement\n' '**********************\n' '\n' diff -Nru python3.10-3.10.12/Lib/ssl.py python3.10-3.10.13/Lib/ssl.py --- python3.10-3.10.12/Lib/ssl.py 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Lib/ssl.py 2023-08-24 12:46:25.000000000 +0000 @@ -1033,7 +1033,7 @@ ) self = cls.__new__(cls, **kwargs) super(SSLSocket, self).__init__(**kwargs) - self.settimeout(sock.gettimeout()) + sock_timeout = sock.gettimeout() sock.detach() self._context = context @@ -1052,9 +1052,42 @@ if e.errno != errno.ENOTCONN: raise connected = False + blocking = self.getblocking() + self.setblocking(False) + try: + # We are not connected so this is not supposed to block, but + # testing revealed otherwise on macOS and Windows so we do + # the non-blocking dance regardless. Our raise when any data + # is found means consuming the data is harmless. + notconn_pre_handshake_data = self.recv(1) + except OSError as e: + # EINVAL occurs for recv(1) on non-connected on unix sockets. + if e.errno not in (errno.ENOTCONN, errno.EINVAL): + raise + notconn_pre_handshake_data = b'' + self.setblocking(blocking) + if notconn_pre_handshake_data: + # This prevents pending data sent to the socket before it was + # closed from escaping to the caller who could otherwise + # presume it came through a successful TLS connection. + reason = "Closed before TLS handshake with data in recv buffer." + notconn_pre_handshake_data_error = SSLError(e.errno, reason) + # Add the SSLError attributes that _ssl.c always adds. + notconn_pre_handshake_data_error.reason = reason + notconn_pre_handshake_data_error.library = None + try: + self.close() + except OSError: + pass + try: + raise notconn_pre_handshake_data_error + finally: + # Explicitly break the reference cycle. + notconn_pre_handshake_data_error = None else: connected = True + self.settimeout(sock_timeout) # Must come after setblocking() calls. self._connected = connected if connected: # create the SSL object diff -Nru python3.10-3.10.12/Lib/tarfile.py python3.10-3.10.13/Lib/tarfile.py --- python3.10-3.10.12/Lib/tarfile.py 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Lib/tarfile.py 2023-08-24 12:46:25.000000000 +0000 @@ -741,7 +741,7 @@ class AbsoluteLinkError(FilterError): def __init__(self, tarinfo): self.tarinfo = tarinfo - super().__init__(f'{tarinfo.name!r} is a symlink to an absolute path') + super().__init__(f'{tarinfo.name!r} is a link to an absolute path') class LinkOutsideDestinationError(FilterError): def __init__(self, tarinfo, path): @@ -801,7 +801,14 @@ if member.islnk() or member.issym(): if os.path.isabs(member.linkname): raise AbsoluteLinkError(member) - target_path = os.path.realpath(os.path.join(dest_path, member.linkname)) + if member.issym(): + target_path = os.path.join(dest_path, + os.path.dirname(name), + member.linkname) + else: + target_path = os.path.join(dest_path, + member.linkname) + target_path = os.path.realpath(target_path) if os.path.commonpath([target_path, dest_path]) != dest_path: raise LinkOutsideDestinationError(member, target_path) return new_attrs diff -Nru python3.10-3.10.12/Lib/test/test_capi/test_codecs.py python3.10-3.10.13/Lib/test/test_capi/test_codecs.py --- python3.10-3.10.12/Lib/test/test_capi/test_codecs.py 1970-01-01 00:00:00.000000000 +0000 +++ python3.10-3.10.13/Lib/test/test_capi/test_codecs.py 2023-08-24 12:46:25.000000000 +0000 @@ -0,0 +1,54 @@ +import unittest +from test.support import import_helper + +_testcapi = import_helper.import_module('_testcapi') + + +class CAPITest(unittest.TestCase): + + def test_decodeutf8(self): + """Test PyUnicode_DecodeUTF8()""" + decodeutf8 = _testcapi.unicode_decodeutf8 + + for s in ['abc', '\xa1\xa2', '\u4f60\u597d', 'a\U0001f600']: + b = s.encode('utf-8') + self.assertEqual(decodeutf8(b), s) + self.assertEqual(decodeutf8(b, 'strict'), s) + + self.assertRaises(UnicodeDecodeError, decodeutf8, b'\x80') + self.assertRaises(UnicodeDecodeError, decodeutf8, b'\xc0') + self.assertRaises(UnicodeDecodeError, decodeutf8, b'\xff') + self.assertRaises(UnicodeDecodeError, decodeutf8, b'a\xf0\x9f') + self.assertEqual(decodeutf8(b'a\xf0\x9f', 'replace'), 'a\ufffd') + self.assertEqual(decodeutf8(b'a\xf0\x9fb', 'replace'), 'a\ufffdb') + + self.assertRaises(LookupError, decodeutf8, b'a\x80', 'foo') + # TODO: Test PyUnicode_DecodeUTF8() with NULL as data and + # negative size. + + def test_decodeutf8stateful(self): + """Test PyUnicode_DecodeUTF8Stateful()""" + decodeutf8stateful = _testcapi.unicode_decodeutf8stateful + + for s in ['abc', '\xa1\xa2', '\u4f60\u597d', 'a\U0001f600']: + b = s.encode('utf-8') + self.assertEqual(decodeutf8stateful(b), (s, len(b))) + self.assertEqual(decodeutf8stateful(b, 'strict'), (s, len(b))) + + self.assertRaises(UnicodeDecodeError, decodeutf8stateful, b'\x80') + self.assertRaises(UnicodeDecodeError, decodeutf8stateful, b'\xc0') + self.assertRaises(UnicodeDecodeError, decodeutf8stateful, b'\xff') + self.assertEqual(decodeutf8stateful(b'a\xf0\x9f'), ('a', 1)) + self.assertEqual(decodeutf8stateful(b'a\xf0\x9f', 'replace'), ('a', 1)) + self.assertRaises(UnicodeDecodeError, decodeutf8stateful, b'a\xf0\x9fb') + self.assertEqual(decodeutf8stateful(b'a\xf0\x9fb', 'replace'), ('a\ufffdb', 4)) + + self.assertRaises(LookupError, decodeutf8stateful, b'a\x80', 'foo') + # TODO: Test PyUnicode_DecodeUTF8Stateful() with NULL as data and + # negative size. + # TODO: Test PyUnicode_DecodeUTF8Stateful() with NULL as the address of + # "consumed". + + +if __name__ == "__main__": + unittest.main() diff -Nru python3.10-3.10.12/Lib/test/test_ssl.py python3.10-3.10.13/Lib/test/test_ssl.py --- python3.10-3.10.12/Lib/test/test_ssl.py 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Lib/test/test_ssl.py 2023-08-24 12:46:25.000000000 +0000 @@ -9,11 +9,14 @@ from test.support import socket_helper from test.support import threading_helper from test.support import warnings_helper +import re import socket import select +import struct import time import datetime import gc +import http.client import os import errno import pprint @@ -4904,6 +4907,259 @@ s.connect((HOST, server.port)) +def set_socket_so_linger_on_with_zero_timeout(sock): + sock.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER, struct.pack('ii', 1, 0)) + + +class TestPreHandshakeClose(unittest.TestCase): + """Verify behavior of close sockets with received data before to the handshake. + """ + + class SingleConnectionTestServerThread(threading.Thread): + + def __init__(self, *, name, call_after_accept, timeout=None): + self.call_after_accept = call_after_accept + self.received_data = b'' # set by .run() + self.wrap_error = None # set by .run() + self.listener = None # set by .start() + self.port = None # set by .start() + if timeout is None: + self.timeout = support.SHORT_TIMEOUT + else: + self.timeout = timeout + super().__init__(name=name) + + def __enter__(self): + self.start() + return self + + def __exit__(self, *args): + try: + if self.listener: + self.listener.close() + except OSError: + pass + self.join() + self.wrap_error = None # avoid dangling references + + def start(self): + self.ssl_ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH) + self.ssl_ctx.verify_mode = ssl.CERT_REQUIRED + self.ssl_ctx.load_verify_locations(cafile=ONLYCERT) + self.ssl_ctx.load_cert_chain(certfile=ONLYCERT, keyfile=ONLYKEY) + self.listener = socket.socket() + self.port = socket_helper.bind_port(self.listener) + self.listener.settimeout(self.timeout) + self.listener.listen(1) + super().start() + + def run(self): + try: + conn, address = self.listener.accept() + except TimeoutError: + # on timeout, just close the listener + return + finally: + self.listener.close() + + with conn: + if self.call_after_accept(conn): + return + try: + tls_socket = self.ssl_ctx.wrap_socket(conn, server_side=True) + except OSError as err: # ssl.SSLError inherits from OSError + self.wrap_error = err + else: + try: + self.received_data = tls_socket.recv(400) + except OSError: + pass # closed, protocol error, etc. + + def non_linux_skip_if_other_okay_error(self, err): + if sys.platform == "linux": + return # Expect the full test setup to always work on Linux. + if (isinstance(err, ConnectionResetError) or + (isinstance(err, OSError) and err.errno == errno.EINVAL) or + re.search('wrong.version.number', getattr(err, "reason", ""), re.I)): + # On Windows the TCP RST leads to a ConnectionResetError + # (ECONNRESET) which Linux doesn't appear to surface to userspace. + # If wrap_socket() winds up on the "if connected:" path and doing + # the actual wrapping... we get an SSLError from OpenSSL. Typically + # WRONG_VERSION_NUMBER. While appropriate, neither is the scenario + # we're specifically trying to test. The way this test is written + # is known to work on Linux. We'll skip it anywhere else that it + # does not present as doing so. + try: + self.skipTest(f"Could not recreate conditions on {sys.platform}:" + f" {err=}") + finally: + # gh-108342: Explicitly break the reference cycle + err = None + + # If maintaining this conditional winds up being a problem. + # just turn this into an unconditional skip anything but Linux. + # The important thing is that our CI has the logic covered. + + def test_preauth_data_to_tls_server(self): + server_accept_called = threading.Event() + ready_for_server_wrap_socket = threading.Event() + + def call_after_accept(unused): + server_accept_called.set() + if not ready_for_server_wrap_socket.wait(support.SHORT_TIMEOUT): + raise RuntimeError("wrap_socket event never set, test may fail.") + return False # Tell the server thread to continue. + + server = self.SingleConnectionTestServerThread( + call_after_accept=call_after_accept, + name="preauth_data_to_tls_server") + server.__enter__() # starts it + self.addCleanup(server.__exit__) # ... & unittest.TestCase stops it. + + with socket.socket() as client: + client.connect(server.listener.getsockname()) + # This forces an immediate connection close via RST on .close(). + set_socket_so_linger_on_with_zero_timeout(client) + client.setblocking(False) + + server_accept_called.wait() + client.send(b"DELETE /data HTTP/1.0\r\n\r\n") + client.close() # RST + + ready_for_server_wrap_socket.set() + server.join() + + wrap_error = server.wrap_error + server.wrap_error = None + try: + self.assertEqual(b"", server.received_data) + self.assertIsInstance(wrap_error, OSError) # All platforms. + self.non_linux_skip_if_other_okay_error(wrap_error) + self.assertIsInstance(wrap_error, ssl.SSLError) + self.assertIn("before TLS handshake with data", wrap_error.args[1]) + self.assertIn("before TLS handshake with data", wrap_error.reason) + self.assertNotEqual(0, wrap_error.args[0]) + self.assertIsNone(wrap_error.library, msg="attr must exist") + finally: + # gh-108342: Explicitly break the reference cycle + wrap_error = None + server = None + + def test_preauth_data_to_tls_client(self): + server_can_continue_with_wrap_socket = threading.Event() + client_can_continue_with_wrap_socket = threading.Event() + + def call_after_accept(conn_to_client): + if not server_can_continue_with_wrap_socket.wait(support.SHORT_TIMEOUT): + print("ERROR: test client took too long") + + # This forces an immediate connection close via RST on .close(). + set_socket_so_linger_on_with_zero_timeout(conn_to_client) + conn_to_client.send( + b"HTTP/1.0 307 Temporary Redirect\r\n" + b"Location: https://example.com/someone-elses-server\r\n" + b"\r\n") + conn_to_client.close() # RST + client_can_continue_with_wrap_socket.set() + return True # Tell the server to stop. + + server = self.SingleConnectionTestServerThread( + call_after_accept=call_after_accept, + name="preauth_data_to_tls_client") + server.__enter__() # starts it + self.addCleanup(server.__exit__) # ... & unittest.TestCase stops it. + + # Redundant; call_after_accept sets SO_LINGER on the accepted conn. + set_socket_so_linger_on_with_zero_timeout(server.listener) + + with socket.socket() as client: + client.connect(server.listener.getsockname()) + server_can_continue_with_wrap_socket.set() + + if not client_can_continue_with_wrap_socket.wait(support.SHORT_TIMEOUT): + self.fail("test server took too long") + ssl_ctx = ssl.create_default_context() + try: + tls_client = ssl_ctx.wrap_socket( + client, server_hostname="localhost") + except OSError as err: # SSLError inherits from OSError + wrap_error = err + received_data = b"" + else: + wrap_error = None + received_data = tls_client.recv(400) + tls_client.close() + + server.join() + try: + self.assertEqual(b"", received_data) + self.assertIsInstance(wrap_error, OSError) # All platforms. + self.non_linux_skip_if_other_okay_error(wrap_error) + self.assertIsInstance(wrap_error, ssl.SSLError) + self.assertIn("before TLS handshake with data", wrap_error.args[1]) + self.assertIn("before TLS handshake with data", wrap_error.reason) + self.assertNotEqual(0, wrap_error.args[0]) + self.assertIsNone(wrap_error.library, msg="attr must exist") + finally: + # gh-108342: Explicitly break the reference cycle + wrap_error = None + server = None + + def test_https_client_non_tls_response_ignored(self): + server_responding = threading.Event() + + class SynchronizedHTTPSConnection(http.client.HTTPSConnection): + def connect(self): + # Call clear text HTTP connect(), not the encrypted HTTPS (TLS) + # connect(): wrap_socket() is called manually below. + http.client.HTTPConnection.connect(self) + + # Wait for our fault injection server to have done its thing. + if not server_responding.wait(support.SHORT_TIMEOUT) and support.verbose: + sys.stdout.write("server_responding event never set.") + self.sock = self._context.wrap_socket( + self.sock, server_hostname=self.host) + + def call_after_accept(conn_to_client): + # This forces an immediate connection close via RST on .close(). + set_socket_so_linger_on_with_zero_timeout(conn_to_client) + conn_to_client.send( + b"HTTP/1.0 402 Payment Required\r\n" + b"\r\n") + conn_to_client.close() # RST + server_responding.set() + return True # Tell the server to stop. + + timeout = 2.0 + server = self.SingleConnectionTestServerThread( + call_after_accept=call_after_accept, + name="non_tls_http_RST_responder", + timeout=timeout) + server.__enter__() # starts it + self.addCleanup(server.__exit__) # ... & unittest.TestCase stops it. + + # Redundant; call_after_accept sets SO_LINGER on the accepted conn. + set_socket_so_linger_on_with_zero_timeout(server.listener) + + connection = SynchronizedHTTPSConnection( + server.listener.getsockname()[0], + port=server.port, + context=ssl.create_default_context(), + timeout=timeout, + ) + + # There are lots of reasons this raises as desired, long before this + # test was added. Sending the request requires a successful TLS wrapped + # socket; that fails if the connection is broken. It may seem pointless + # to test this. It serves as an illustration of something that we never + # want to happen... properly not happening. + with self.assertRaises(OSError): + connection.request("HEAD", "/test", headers={"Host": "localhost"}) + response = connection.getresponse() + + server.join() + + def setUpModule(): if support.verbose: plats = { diff -Nru python3.10-3.10.12/Lib/test/test_tarfile.py python3.10-3.10.13/Lib/test/test_tarfile.py --- python3.10-3.10.12/Lib/test/test_tarfile.py 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Lib/test/test_tarfile.py 2023-08-24 12:46:25.000000000 +0000 @@ -3209,10 +3209,12 @@ self.bio = None def add(self, name, *, type=None, symlink_to=None, hardlink_to=None, - mode=None, **kwargs): + mode=None, size=None, **kwargs): """Add a member to the test archive. Call within `with`.""" name = str(name) tarinfo = tarfile.TarInfo(name).replace(**kwargs) + if size is not None: + tarinfo.size = size if mode: tarinfo.mode = _filemode_to_int(mode) if symlink_to is not None: @@ -3276,7 +3278,8 @@ raise self.raised_exception self.assertEqual(self.expected_paths, set()) - def expect_file(self, name, type=None, symlink_to=None, mode=None): + def expect_file(self, name, type=None, symlink_to=None, mode=None, + size=None): """Check a single file. See check_context.""" if self.raised_exception: raise self.raised_exception @@ -3310,6 +3313,8 @@ self.assertTrue(path.is_fifo()) else: raise NotImplementedError(type) + if size is not None: + self.assertEqual(path.stat().st_size, size) for parent in path.parents: self.expected_paths.discard(parent) @@ -3355,8 +3360,15 @@ # Test interplaying symlinks # Inspired by 'dirsymlink2a' in jwilk/traversal-archives with ArchiveMaker() as arc: + + # `current` links to `.` which is both: + # - the destination directory + # - `current` itself arc.add('current', symlink_to='.') + + # effectively points to ./../ arc.add('parent', symlink_to='current/..') + arc.add('parent/evil') if os_helper.can_symlink(): @@ -3397,9 +3409,46 @@ def test_parent_symlink2(self): # Test interplaying symlinks # Inspired by 'dirsymlink2b' in jwilk/traversal-archives + + # Posix and Windows have different pathname resolution: + # either symlink or a '..' component resolve first. + # Let's see which we are on. + if os_helper.can_symlink(): + testpath = os.path.join(TEMPDIR, 'resolution_test') + os.mkdir(testpath) + + # testpath/current links to `.` which is all of: + # - `testpath` + # - `testpath/current` + # - `testpath/current/current` + # - etc. + os.symlink('.', os.path.join(testpath, 'current')) + + # we'll test where `testpath/current/../file` ends up + with open(os.path.join(testpath, 'current', '..', 'file'), 'w'): + pass + + if os.path.exists(os.path.join(testpath, 'file')): + # Windows collapses 'current\..' to '.' first, leaving + # 'testpath\file' + dotdot_resolves_early = True + elif os.path.exists(os.path.join(testpath, '..', 'file')): + # Posix resolves 'current' to '.' first, leaving + # 'testpath/../file' + dotdot_resolves_early = False + else: + raise AssertionError('Could not determine link resolution') + with ArchiveMaker() as arc: + + # `current` links to `.` which is both the destination directory + # and `current` itself arc.add('current', symlink_to='.') + + # `current/parent` is also available as `./parent`, + # and effectively points to `./../` arc.add('current/parent', symlink_to='..') + arc.add('parent/evil') with self.check_context(arc.open(), 'fully_trusted'): @@ -3413,6 +3462,7 @@ with self.check_context(arc.open(), 'tar'): if os_helper.can_symlink(): + # Fail when extracting a file outside destination self.expect_exception( tarfile.OutsideDestinationError, "'parent/evil' would be extracted to " @@ -3423,10 +3473,24 @@ self.expect_file('parent/evil') with self.check_context(arc.open(), 'data'): - self.expect_exception( - tarfile.LinkOutsideDestinationError, - """'current/parent' would link to ['"].*['"], """ - + "which is outside the destination") + if os_helper.can_symlink(): + if dotdot_resolves_early: + # Fail when extracting a file outside destination + self.expect_exception( + tarfile.OutsideDestinationError, + "'parent/evil' would be extracted to " + + """['"].*evil['"], which is outside """ + + "the destination") + else: + # Fail as soon as we have a symlink outside the destination + self.expect_exception( + tarfile.LinkOutsideDestinationError, + "'current/parent' would link to " + + """['"].*outerdir['"], which is outside """ + + "the destination") + else: + self.expect_file('current/') + self.expect_file('parent/evil') def test_absolute_symlink(self): # Test symlink to an absolute path @@ -3455,11 +3519,29 @@ with self.check_context(arc.open(), 'data'): self.expect_exception( tarfile.AbsoluteLinkError, - "'parent' is a symlink to an absolute path") + "'parent' is a link to an absolute path") + + def test_absolute_hardlink(self): + # Test hardlink to an absolute path + # Inspired by 'dirsymlink' in https://github.com/jwilk/traversal-archives + with ArchiveMaker() as arc: + arc.add('parent', hardlink_to=self.outerdir / 'foo') + + with self.check_context(arc.open(), 'fully_trusted'): + self.expect_exception(KeyError, ".*foo. not found") + + with self.check_context(arc.open(), 'tar'): + self.expect_exception(KeyError, ".*foo. not found") + + with self.check_context(arc.open(), 'data'): + self.expect_exception( + tarfile.AbsoluteLinkError, + "'parent' is a link to an absolute path") def test_sly_relative0(self): # Inspired by 'relative0' in jwilk/traversal-archives with ArchiveMaker() as arc: + # points to `../../tmp/moo` arc.add('../moo', symlink_to='..//tmp/moo') try: @@ -3509,6 +3591,54 @@ + """['"].*moo['"], which is outside the """ + "destination") + def test_deep_symlink(self): + # Test that symlinks and hardlinks inside a directory + # point to the correct file (`target` of size 3). + # If links aren't supported we get a copy of the file. + with ArchiveMaker() as arc: + arc.add('targetdir/target', size=3) + # a hardlink's linkname is relative to the archive + arc.add('linkdir/hardlink', hardlink_to=os.path.join( + 'targetdir', 'target')) + # a symlink's linkname is relative to the link's directory + arc.add('linkdir/symlink', symlink_to=os.path.join( + '..', 'targetdir', 'target')) + + for filter in 'tar', 'data', 'fully_trusted': + with self.check_context(arc.open(), filter): + self.expect_file('targetdir/target', size=3) + self.expect_file('linkdir/hardlink', size=3) + if os_helper.can_symlink(): + self.expect_file('linkdir/symlink', size=3, + symlink_to='../targetdir/target') + else: + self.expect_file('linkdir/symlink', size=3) + + def test_chains(self): + # Test chaining of symlinks/hardlinks. + # Symlinks are created before the files they point to. + with ArchiveMaker() as arc: + arc.add('linkdir/symlink', symlink_to='hardlink') + arc.add('symlink2', symlink_to=os.path.join( + 'linkdir', 'hardlink2')) + arc.add('targetdir/target', size=3) + arc.add('linkdir/hardlink', hardlink_to='targetdir/target') + arc.add('linkdir/hardlink2', hardlink_to='linkdir/symlink') + + for filter in 'tar', 'data', 'fully_trusted': + with self.check_context(arc.open(), filter): + self.expect_file('targetdir/target', size=3) + self.expect_file('linkdir/hardlink', size=3) + self.expect_file('linkdir/hardlink2', size=3) + if os_helper.can_symlink(): + self.expect_file('linkdir/symlink', size=3, + symlink_to='hardlink') + self.expect_file('symlink2', size=3, + symlink_to='linkdir/hardlink2') + else: + self.expect_file('linkdir/symlink', size=3) + self.expect_file('symlink2', size=3) + def test_modes(self): # Test how file modes are extracted # (Note that the modes are ignored on platforms without working chmod) diff -Nru python3.10-3.10.12/Misc/NEWS python3.10-3.10.13/Misc/NEWS --- python3.10-3.10.12/Misc/NEWS 2023-06-06 22:43:10.000000000 +0000 +++ python3.10-3.10.13/Misc/NEWS 2023-08-24 12:59:26.000000000 +0000 @@ -2,6 +2,42 @@ Python News +++++++++++ +What's New in Python 3.10.13 final? +=================================== + +*Release date: 2023-08-24* + +Security +-------- + +- gh-issue-108310: Fixed an issue where instances of :class:`ssl.SSLSocket` + were vulnerable to a bypass of the TLS handshake and included protections + (like certificate verification) and treating sent unencrypted data as if + it were post-handshake TLS encrypted data. Security issue reported as + `CVE-2023-40217 + `_ by Aapo + Oksman. Patch by Gregory P. Smith. + +Library +------- + +- gh-issue-107845: :func:`tarfile.data_filter` now takes the location of + symlinks into account when determining their target, so it will no longer + reject some valid tarballs with ``LinkOutsideDestinationError``. + +Tools/Demos +----------- + +- gh-issue-107565: Update multissltests and GitHub CI workflows to use + OpenSSL 1.1.1v, 3.0.10, and 3.1.2. + +C API +----- + +- gh-issue-99612: Fix :c:func:`PyUnicode_DecodeUTF8Stateful` for ASCII-only + data: ``*consumed`` was not set. + + What's New in Python 3.10.12 final? =================================== diff -Nru python3.10-3.10.12/Modules/_testcapimodule.c python3.10-3.10.13/Modules/_testcapimodule.c --- python3.10-3.10.12/Modules/_testcapimodule.c 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Modules/_testcapimodule.c 2023-08-24 12:46:25.000000000 +0000 @@ -2112,6 +2112,40 @@ return Py_BuildValue("(Nn)", result, utf8_len); } +/* Test PyUnicode_DecodeUTF8() */ +static PyObject * +unicode_decodeutf8(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + return PyUnicode_DecodeUTF8(data, size, errors); +} + +/* Test PyUnicode_DecodeUTF8Stateful() */ +static PyObject * +unicode_decodeutf8stateful(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + Py_ssize_t consumed = 123456789; + PyObject *result; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + result = PyUnicode_DecodeUTF8Stateful(data, size, errors, &consumed); + if (!result) { + return NULL; + } + return Py_BuildValue("(Nn)", result, consumed); +} + static PyObject * unicode_findchar(PyObject *self, PyObject *args) { @@ -5846,7 +5880,8 @@ {"unicode_asucs4", unicode_asucs4, METH_VARARGS}, {"unicode_asutf8", unicode_asutf8, METH_VARARGS}, {"unicode_asutf8andsize", unicode_asutf8andsize, METH_VARARGS}, - {"unicode_findchar", unicode_findchar, METH_VARARGS}, + {"unicode_decodeutf8", unicode_decodeutf8, METH_VARARGS}, + {"unicode_decodeutf8stateful",unicode_decodeutf8stateful, METH_VARARGS}, {"unicode_findchar", unicode_findchar, METH_VARARGS}, {"unicode_copycharacters", unicode_copycharacters, METH_VARARGS}, #if USE_UNICODE_WCHAR_CACHE {"unicode_encodedecimal", unicode_encodedecimal, METH_VARARGS}, diff -Nru python3.10-3.10.12/Objects/unicodeobject.c python3.10-3.10.13/Objects/unicodeobject.c --- python3.10-3.10.12/Objects/unicodeobject.c 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Objects/unicodeobject.c 2023-08-24 12:46:25.000000000 +0000 @@ -5206,6 +5206,9 @@ } s += ascii_decode(s, end, PyUnicode_1BYTE_DATA(u)); if (s == end) { + if (consumed) { + *consumed = size; + } return u; } diff -Nru python3.10-3.10.12/README.rst python3.10-3.10.13/README.rst --- python3.10-3.10.12/README.rst 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/README.rst 2023-08-24 12:46:25.000000000 +0000 @@ -1,4 +1,4 @@ -This is Python version 3.10.12 +This is Python version 3.10.13 ============================== .. image:: https://travis-ci.com/python/cpython.svg?branch=master diff -Nru python3.10-3.10.12/Tools/ssl/multissltests.py python3.10-3.10.13/Tools/ssl/multissltests.py --- python3.10-3.10.12/Tools/ssl/multissltests.py 2023-06-06 22:30:33.000000000 +0000 +++ python3.10-3.10.13/Tools/ssl/multissltests.py 2023-08-24 12:46:25.000000000 +0000 @@ -47,9 +47,9 @@ ] OPENSSL_RECENT_VERSIONS = [ - "1.1.1u", - "3.0.9", - "3.1.1", + "1.1.1v", + "3.0.10", + "3.1.2", ] LIBRESSL_OLD_VERSIONS = [