Merge branch 'main' into monitoring-capi

.pre-commit-config.yaml

@@ -1,6 +1,6 @@
 repos:
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.2.0
+    rev: v0.3.4
     hooks:
       - id: ruff
         name: Run Ruff on Lib/test/
@@ -14,6 +14,8 @@ repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks
     rev: v4.5.0
     hooks:
+      - id: check-case-conflict
+      - id: check-merge-conflict
       - id: check-toml
         exclude: ^Lib/test/test_tomllib/
       - id: check-yaml

Doc/library/os.path.rst

@@ -4,7 +4,7 @@
 .. module:: os.path
    :synopsis: Operations on pathnames.
 
-**Source code:** :source:`Lib/posixpath.py` (for POSIX) and
+**Source code:** :source:`Lib/genericpath.py`, :source:`Lib/posixpath.py` (for POSIX) and
 :source:`Lib/ntpath.py` (for Windows).
 
 .. index:: single: path; operations
@@ -85,8 +85,6 @@ the :mod:`glob` module.)
    if *paths* is empty.  Unlike :func:`commonprefix`, this returns a
    valid path.
 
-   .. availability:: Unix, Windows.
-
    .. versionadded:: 3.5
 
    .. versionchanged:: 3.6
@@ -324,10 +322,11 @@ the :mod:`glob` module.)
    Dev Drives. See `the Windows documentation <https://learn.microsoft.com/windows/dev-drive/>`_
    for information on enabling and creating Dev Drives.
 
-   .. availability:: Windows.
-
    .. versionadded:: 3.12
 
+   .. versionchanged:: 3.13
+      The function is now available on all platforms, and will always return ``False`` on those that have no support for Dev Drives
+
 
 .. function:: isreserved(path)
 
@@ -442,8 +441,6 @@ the :mod:`glob` module.)
 
    *start* defaults to :data:`os.curdir`.
 
-   .. availability:: Unix, Windows.
-
    .. versionchanged:: 3.6
       Accepts a :term:`path-like object`.
 
@@ -454,8 +451,6 @@ the :mod:`glob` module.)
    This is determined by the device number and i-node number and raises an
    exception if an :func:`os.stat` call on either pathname fails.
 
-   .. availability:: Unix, Windows.
-
    .. versionchanged:: 3.2
       Added Windows support.
 
@@ -470,8 +465,6 @@ the :mod:`glob` module.)
 
    Return ``True`` if the file descriptors *fp1* and *fp2* refer to the same file.
 
-   .. availability:: Unix, Windows.
-
    .. versionchanged:: 3.2
       Added Windows support.
 
@@ -486,8 +479,6 @@ the :mod:`glob` module.)
    :func:`os.lstat`, or :func:`os.stat`.  This function implements the
    underlying comparison used by :func:`samefile` and :func:`sameopenfile`.
 
-   .. availability:: Unix, Windows.
-
    .. versionchanged:: 3.4
       Added Windows support.
 

Doc/library/statistics.rst

@@ -80,7 +80,7 @@ or sample.
 :func:`median`           Median (middle value) of data.
 :func:`median_low`       Low median of data.
 :func:`median_high`      High median of data.
-:func:`median_grouped`   Median, or 50th percentile, of grouped data.
+:func:`median_grouped`   Median (50th percentile) of grouped data.
 :func:`mode`             Single mode (most common value) of discrete or nominal data.
 :func:`multimode`        List of modes (most common values) of discrete or nominal data.
 :func:`quantiles`        Divide data into intervals with equal probability.
@@ -381,55 +381,56 @@ However, for reading convenience, most of the examples show sorted sequences.
    be an actual data point rather than interpolated.
 
 
-.. function:: median_grouped(data, interval=1)
+.. function:: median_grouped(data, interval=1.0)
 
-   Return the median of grouped continuous data, calculated as the 50th
-   percentile, using interpolation.  If *data* is empty, :exc:`StatisticsError`
-   is raised.  *data* can be a sequence or iterable.
+   Estimates the median for numeric data that has been `grouped or binned
+   <https://en.wikipedia.org/wiki/Data_binning>`_ around the midpoints
+   of consecutive, fixed-width intervals.
 
-   .. doctest::
+   The *data* can be any iterable of numeric data with each value being
+   exactly the midpoint of a bin.  At least one value must be present.
 
-      >>> median_grouped([52, 52, 53, 54])
-      52.5
+   The *interval* is the width of each bin.
 
-   In the following example, the data are rounded, so that each value represents
-   the midpoint of data classes, e.g. 1 is the midpoint of the class 0.5--1.5, 2
-   is the midpoint of 1.5--2.5, 3 is the midpoint of 2.5--3.5, etc.  With the data
-   given, the middle value falls somewhere in the class 3.5--4.5, and
-   interpolation is used to estimate it:
+   For example, demographic information may have been summarized into
+   consecutive ten-year age groups with each group being represented
+   by the 5-year midpoints of the intervals:
 
    .. doctest::
 
-      >>> median_grouped([1, 2, 2, 3, 4, 4, 4, 4, 4, 5])
-      3.7
-
-   Optional argument *interval* represents the class interval, and defaults
-   to 1.  Changing the class interval naturally will change the interpolation:
+      >>> from collections import Counter
+      >>> demographics = Counter({
+      ...    25: 172,   # 20 to 30 years old
+      ...    35: 484,   # 30 to 40 years old
+      ...    45: 387,   # 40 to 50 years old
+      ...    55:  22,   # 50 to 60 years old
+      ...    65:   6,   # 60 to 70 years old
+      ... })
+      ...
+
+   The 50th percentile (median) is the 536th person out of the 1071
+   member cohort.  That person is in the 30 to 40 year old age group.
+
+   The regular :func:`median` function would assume that everyone in the
+   tricenarian age group was exactly 35 years old.  A more tenable
+   assumption is that the 484 members of that age group are evenly
+   distributed between 30 and 40.  For that, we use
+   :func:`median_grouped`:
 
    .. doctest::
 
-      >>> median_grouped([1, 3, 3, 5, 7], interval=1)
-      3.25
-      >>> median_grouped([1, 3, 3, 5, 7], interval=2)
-      3.5
-
-   This function does not check whether the data points are at least
-   *interval* apart.
-
-   .. impl-detail::
-
-      Under some circumstances, :func:`median_grouped` may coerce data points to
-      floats.  This behaviour is likely to change in the future.
-
-   .. seealso::
+       >>> data = list(demographics.elements())
+       >>> median(data)
+       35
+       >>> round(median_grouped(data, interval=10), 1)
+       37.5
 
-      * "Statistics for the Behavioral Sciences", Frederick J Gravetter and
-        Larry B Wallnau (8th Edition).
+   The caller is responsible for making sure the data points are separated
+   by exact multiples of *interval*.  This is essential for getting a
+   correct result.  The function does not check this precondition.
 
-      * The `SSMEDIAN
-        <https://help.gnome.org/users/gnumeric/stable/gnumeric.html#gnumeric-function-SSMEDIAN>`_
-        function in the Gnome Gnumeric spreadsheet, including `this discussion
-        <https://mail.gnome.org/archives/gnumeric-list/2011-April/msg00018.html>`_.
+   Inputs may be any numeric type that can be coerced to a float during
+   the interpolation step.
 
 
 .. function:: mode(data)

Doc/reference/datamodel.rst

@@ -299,14 +299,17 @@ Sequences
 These represent finite ordered sets indexed by non-negative numbers. The
 built-in function :func:`len` returns the number of items of a sequence. When
 the length of a sequence is *n*, the index set contains the numbers 0, 1,
-..., *n*-1.  Item *i* of sequence *a* is selected by ``a[i]``.
+..., *n*-1.  Item *i* of sequence *a* is selected by ``a[i]``. Some sequences,
+including built-in sequences, interpret negative subscripts by adding the
+sequence length. For example, ``a[-2]`` equals ``a[n-2]``, the second to last
+item of sequence a with length ``n``.
 
 .. index:: single: slicing
 
 Sequences also support slicing: ``a[i:j]`` selects all items with index *k* such
 that *i* ``<=`` *k* ``<`` *j*.  When used as an expression, a slice is a
-sequence of the same type.  This implies that the index set is renumbered so
-that it starts at 0.
+sequence of the same type. The comment above about negative indexes also applies
+to negative slice positions.
 
 Some sequences also support "extended slicing" with a third "step" parameter:
 ``a[i:j:k]`` selects all items of *a* with index *x* where ``x = i + n*k``, *n*

Doc/whatsnew/3.13.rst

@@ -1742,6 +1742,9 @@ New Features
   :term:`strong reference` to the constant zero.
   (Contributed by Victor Stinner in :gh:`115754`.)
 
+* Add :c:func:`PyType_GetModuleByDef` to the limited C API
+  (Contributed by Victor Stinner in :gh:`116936`.)
+
 
 Porting to Python 3.13
 ----------------------

Grammar/python.gram

@@ -778,6 +778,7 @@ bitwise_and[expr_ty]:
 shift_expr[expr_ty]:
     | a=shift_expr '<<' b=sum { _PyAST_BinOp(a, LShift, b, EXTRA) }
     | a=shift_expr '>>' b=sum { _PyAST_BinOp(a, RShift, b, EXTRA) }
+    | invalid_arithmetic
     | sum
 
 # Arithmetic operators
@@ -794,6 +795,7 @@ term[expr_ty]:
     | a=term '//' b=factor { _PyAST_BinOp(a, FloorDiv, b, EXTRA) }
     | a=term '%' b=factor { _PyAST_BinOp(a, Mod, b, EXTRA) }
     | a=term '@' b=factor { CHECK_VERSION(expr_ty, 5, "The '@' operator is", _PyAST_BinOp(a, MatMult, b, EXTRA)) }
+    | invalid_factor
     | factor
 
 factor[expr_ty] (memo):
@@ -1415,3 +1417,8 @@ invalid_replacement_field:
 invalid_conversion_character:
     | '!' &(':' | '}') { RAISE_SYNTAX_ERROR_ON_NEXT_TOKEN("f-string: missing conversion character") }
     | '!' !NAME { RAISE_SYNTAX_ERROR_ON_NEXT_TOKEN("f-string: invalid conversion character") }
+
+invalid_arithmetic:
+    | sum ('+'|'-'|'*'|'/'|'%'|'//'|'@') a='not' b=inversion { RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "'not' after an operator must be parenthesized") }
+invalid_factor:
+    | ('+' | '-' | '~') a='not' b=factor { RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "'not' after an operator must be parenthesized") }

Include/cpython/object.h

@@ -275,7 +275,6 @@ typedef struct _heaptypeobject {
 
 PyAPI_FUNC(const char *) _PyType_Name(PyTypeObject *);
 PyAPI_FUNC(PyObject *) _PyType_Lookup(PyTypeObject *, PyObject *);
-PyAPI_FUNC(PyObject *) PyType_GetModuleByDef(PyTypeObject *, PyModuleDef *);
 PyAPI_FUNC(PyObject *) PyType_GetDict(PyTypeObject *);
 
 PyAPI_FUNC(int) PyObject_Print(PyObject *, FILE *, int);

Include/cpython/optimizer.h

@@ -30,16 +30,63 @@ typedef struct {
     PyCodeObject *code;  // Weak (NULL if no corresponding ENTER_EXECUTOR).
 } _PyVMData;
 
+#define UOP_FORMAT_TARGET 0
+#define UOP_FORMAT_EXIT 1
+#define UOP_FORMAT_JUMP 2
+#define UOP_FORMAT_UNUSED 3
+
+/* Depending on the format,
+ * the 32 bits between the oparg and operand are:
+ * UOP_FORMAT_TARGET:
+ *    uint32_t target;
+ * UOP_FORMAT_EXIT
+ *    uint16_t exit_index;
+ *    uint16_t error_target;
+ * UOP_FORMAT_JUMP
+ *    uint16_t jump_target;
+ *    uint16_t error_target;
+ */
 typedef struct {
-    uint16_t opcode;
+    uint16_t opcode:14;
+    uint16_t format:2;
     uint16_t oparg;
     union {
         uint32_t target;
-        uint32_t exit_index;
+        struct {
+            union {
+                uint16_t exit_index;
+                uint16_t jump_target;
+            };
+            uint16_t error_target;
+        };
     };
     uint64_t operand;  // A cache entry
 } _PyUOpInstruction;
 
+static inline uint32_t uop_get_target(const _PyUOpInstruction *inst)
+{
+    assert(inst->format == UOP_FORMAT_TARGET);
+    return inst->target;
+}
+
+static inline uint16_t uop_get_exit_index(const _PyUOpInstruction *inst)
+{
+    assert(inst->format == UOP_FORMAT_EXIT);
+    return inst->exit_index;
+}
+
+static inline uint16_t uop_get_jump_target(const _PyUOpInstruction *inst)
+{
+    assert(inst->format == UOP_FORMAT_JUMP);
+    return inst->jump_target;
+}
+
+static inline uint16_t uop_get_error_target(const _PyUOpInstruction *inst)
+{
+    assert(inst->format != UOP_FORMAT_TARGET);
+    return inst->error_target;
+}
+
 typedef struct _exit_data {
     uint32_t target;
     int16_t temperature;

Include/cpython/pystats.h

@@ -118,7 +118,7 @@ typedef struct _optimization_stats {
     uint64_t recursive_call;
     uint64_t low_confidence;
     uint64_t executors_invalidated;
-    UOpStats opcode[MAX_UOP_ID];
+    UOpStats opcode[MAX_UOP_ID+1];
     uint64_t unsupported_opcode[256];
     uint64_t trace_length_hist[_Py_UOP_HIST_SIZE];
     uint64_t trace_run_length_hist[_Py_UOP_HIST_SIZE];
@@ -128,7 +128,7 @@ typedef struct _optimization_stats {
     uint64_t optimizer_failure_reason_no_memory;
     uint64_t remove_globals_builtins_changed;
     uint64_t remove_globals_incorrect_keys;
-    uint64_t error_in_opcode[MAX_UOP_ID];
+    uint64_t error_in_opcode[MAX_UOP_ID+1];
 } OptimizationStats;
 
 typedef struct _rare_event_stats {

Include/internal/pycore_gc.h

@@ -113,7 +113,19 @@ static inline void _PyObject_GC_SET_SHARED_INLINE(PyObject *op) {
 /* Bit 1 is set when the object is in generation which is GCed currently. */
 #define _PyGC_PREV_MASK_COLLECTING 2
 
-/* Bit 0 is set if the object belongs to old space 1 */
+/* Bit 0 in _gc_next is the old space bit.
+ * It is set as follows:
+ * Young: gcstate->visited_space
+ * old[0]: 0
+ * old[1]: 1
+ * permanent: 0
+ *
+ * During a collection all objects handled should have the bit set to
+ * gcstate->visited_space, as objects are moved from the young gen
+ * and the increment into old[gcstate->visited_space].
+ * When object are moved from the pending space, old[gcstate->visited_space^1]
+ * into the increment, the old space bit is flipped.
+*/
 #define _PyGC_NEXT_MASK_OLD_SPACE_1    1
 
 #define _PyGC_PREV_SHIFT           2

Include/internal/pycore_global_strings.h

@@ -213,6 +213,7 @@ struct _Py_global_strings {
         STRUCT_FOR_ID(__slotnames__)
         STRUCT_FOR_ID(__slots__)
         STRUCT_FOR_ID(__spec__)
+        STRUCT_FOR_ID(__static_attributes__)
         STRUCT_FOR_ID(__str__)
         STRUCT_FOR_ID(__sub__)
         STRUCT_FOR_ID(__subclasscheck__)

Include/internal/pycore_object.h

@@ -318,8 +318,8 @@ static inline void _PyObject_GC_TRACK(
     PyGC_Head *last = (PyGC_Head*)(generation0->_gc_prev);
     _PyGCHead_SET_NEXT(last, gc);
     _PyGCHead_SET_PREV(gc, last);
-    _PyGCHead_SET_NEXT(gc, generation0);
-    assert((gc->_gc_next & _PyGC_NEXT_MASK_OLD_SPACE_1) == 0);
+    /* Young objects will be moved into the visited space during GC, so set the bit here */
+    gc->_gc_next = ((uintptr_t)generation0) | interp->gc.visited_space;
     generation0->_gc_prev = (uintptr_t)gc;
 #endif
 }