Docs: add link roles with Sphinx extlinks (python#117850)

Co-authored-by: Alex Waygood <[email protected]>

Doc/conf.py

@@ -12,6 +12,8 @@
 sys.path.append(os.path.abspath('tools/extensions'))
 sys.path.append(os.path.abspath('includes'))
 
+from pyspecific import SOURCE_URI
+
 # General configuration
 # ---------------------
 
@@ -24,6 +26,7 @@
     'pyspecific',
     'sphinx.ext.coverage',
     'sphinx.ext.doctest',
+    'sphinx.ext.extlinks',
 ]
 
 # Skip if downstream redistributors haven't installed them
@@ -513,6 +516,19 @@
     r'https://unix.org/version2/whatsnew/lp64_wp.html',
 ]
 
+# Options for sphinx.ext.extlinks
+# -------------------------------
+
+# This config is a dictionary of external sites,
+# mapping unique short aliases to a base URL and a prefix.
+# https://www.sphinx-doc.org/en/master/usage/extensions/extlinks.html
+extlinks = {
+    "cve": ("https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-%s", "CVE-%s"),
+    "cwe": ("https://cwe.mitre.org/data/definitions/%s.html", "CWE-%s"),
+    "pypi": ("https://pypi.org/project/%s/", "%s"),
+    "source": (SOURCE_URI, "%s"),
+}
+extlinks_detect_hardcoded_links = True
 
 # Options for extensions
 # ----------------------

Doc/faq/library.rst

@@ -616,16 +616,15 @@ use ``p.read(n)``.
    ("ptys") instead of pipes. Or you can use a Python interface to Don Libes'
    "expect" library.  A Python extension that interfaces to expect is called
    "expy" and available from https://expectpy.sourceforge.net.  A pure Python
-   solution that works like expect is `pexpect
-   <https://pypi.org/project/pexpect/>`_.
+   solution that works like expect is :pypi:`pexpect`.
 
 
 How do I access the serial (RS232) port?
 ----------------------------------------
 
 For Win32, OSX, Linux, BSD, Jython, IronPython:
 
-   https://pypi.org/project/pyserial/
+   :pypi:`pyserial`
 
 For Unix, see a Usenet post by Mitch Chapman:
 

Doc/howto/curses.rst

@@ -43,7 +43,7 @@ appearance---and the curses library will figure out what control codes
 need to be sent to the terminal to produce the right output.  curses
 doesn't provide many user-interface concepts such as buttons, checkboxes,
 or dialogs; if you need such features, consider a user interface library such as
-`Urwid <https://pypi.org/project/urwid/>`_.
+:pypi:`Urwid`.
 
 The curses library was originally written for BSD Unix; the later System V
 versions of Unix from AT&T added many enhancements and new functions. BSD curses
@@ -56,8 +56,7 @@ versions of curses carried by some proprietary Unixes may not support
 everything, though.
 
 The Windows version of Python doesn't include the :mod:`curses`
-module.  A ported version called `UniCurses
-<https://pypi.org/project/UniCurses>`_ is available.
+module.  A ported version called :pypi:`UniCurses` is available.
 
 
 The Python curses module
@@ -429,8 +428,7 @@ User Input
 
 The C curses library offers only very simple input mechanisms. Python's
 :mod:`curses` module adds a basic text-input widget.  (Other libraries
-such as `Urwid <https://pypi.org/project/urwid/>`_ have more extensive
-collections of widgets.)
+such as :pypi:`Urwid` have more extensive collections of widgets.)
 
 There are two methods for getting input from a window:
 

Doc/howto/logging-cookbook.rst

@@ -1912,7 +1912,7 @@ Subclassing QueueHandler and QueueListener- a ``pynng`` example
 ---------------------------------------------------------------
 
 In a similar way to the above section, we can implement a listener and handler
-using `pynng <https://pypi.org/project/pynng/>`_, which is a Python binding to
+using :pypi:`pynng`, which is a Python binding to
 `NNG <https://nng.nanomsg.org/>`_, billed as a spiritual successor to ZeroMQ.
 The following snippets illustrate -- you can test them in an environment which has
 ``pynng`` installed. Just for variety, we present the listener first.
@@ -3575,9 +3575,8 @@ A Qt GUI for logging
 
 A question that comes up from time to time is about how to log to a GUI
 application. The `Qt <https://www.qt.io/>`_ framework is a popular
-cross-platform UI framework with Python bindings using `PySide2
-<https://pypi.org/project/PySide2/>`_ or `PyQt5
-<https://pypi.org/project/PyQt5/>`_ libraries.
+cross-platform UI framework with Python bindings using :pypi:`PySide2`
+or :pypi:`PyQt5` libraries.
 
 The following example shows how to log to a Qt GUI. This introduces a simple
 ``QtHandler`` class which takes a callable, which should be a slot in the main

Doc/library/codecs.rst

@@ -1478,7 +1478,7 @@ Internationalized Domain Names (IDN)). It builds upon the ``punycode`` encoding
 and :mod:`stringprep`.
 
 If you need the IDNA 2008 standard from :rfc:`5891` and :rfc:`5895`, use the
-third-party `idna module <https://pypi.org/project/idna/>`_.
+third-party :pypi:`idna` module.
 
 These RFCs together define a protocol to support non-ASCII characters in domain
 names. A domain name containing non-ASCII characters (such as

Doc/library/datetime.rst

@@ -37,7 +37,7 @@ on efficient attribute extraction for output formatting and manipulation.
    Package `dateutil <https://dateutil.readthedocs.io/en/stable/>`_
       Third-party library with expanded time zone and parsing support.
 
-   Package `DateType <https://pypi.org/project/datetype/>`_
+   Package :pypi:`DateType`
       Third-party library that introduces distinct static types to e.g. allow
       :term:`static type checkers <static type checker>`
       to differentiate between naive and aware datetimes.

Doc/library/importlib.metadata.rst

@@ -26,7 +26,7 @@ this package can eliminate the need to use the older and less efficient
 
 ``importlib.metadata`` operates on third-party *distribution packages*
 installed into Python's ``site-packages`` directory via tools such as
-`pip <https://pypi.org/project/pip/>`_.
+:pypi:`pip`.
 Specifically, it works with distributions with discoverable
 ``dist-info`` or ``egg-info`` directories,
 and metadata defined by the `Core metadata specifications <https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata>`_.
@@ -177,7 +177,7 @@ for more information on entry points, their definition, and usage.
    no parameters and always returned a dictionary of entry points, keyed
    by group. With ``importlib_metadata`` 5.0 and Python 3.12,
    ``entry_points`` always returns an ``EntryPoints`` object. See
-   `backports.entry_points_selectable <https://pypi.org/project/backports.entry-points-selectable>`_
+   :pypi:`backports.entry_points_selectable`
    for compatibility options.
 
 .. versionchanged:: 3.13

Doc/library/itertools.rst

@@ -791,7 +791,7 @@ recipes.  Currently, the ``sliding_window()``, ``iter_index()``, and ``sieve()``
 recipes are being tested to see whether they prove their worth.
 
 Substantially all of these recipes and many, many others can be installed from
-the `more-itertools project <https://pypi.org/project/more-itertools/>`_ found
+the :pypi:`more-itertools` project found
 on the Python Package Index::
 
     python -m pip install more-itertools

Doc/library/re.rst

@@ -48,7 +48,7 @@ fine-tuning parameters.
 
 .. seealso::
 
-   The third-party `regex <https://pypi.org/project/regex/>`_ module,
+   The third-party :pypi:`regex` module,
    which has an API compatible with the standard library :mod:`re` module,
    but offers additional functionality and a more thorough Unicode support.
 

Doc/library/secrets.rst

@@ -155,7 +155,7 @@ Generate an eight-character alphanumeric password:
 .. note::
 
    Applications should not
-   `store passwords in a recoverable format <https://cwe.mitre.org/data/definitions/257.html>`_,
+   :cwe:`store passwords in a recoverable format <257>`,
    whether plain text or encrypted.  They should be salted and hashed
    using a cryptographically strong one-way (irreversible) hash function.
 

Doc/library/stdtypes.rst

@@ -5559,8 +5559,7 @@ a string to a binary integer or a binary integer to a string in linear time,
 have sub-quadratic complexity. Converting a large value such as ``int('1' *
 500_000)`` can take over a second on a fast CPU.
 
-Limiting conversion size offers a practical way to avoid `CVE-2020-10735
-<https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10735>`_.
+Limiting conversion size offers a practical way to avoid :cve:`2020-10735`.
 
 The limit is applied to the number of digit characters in the input or output
 string when a non-linear conversion algorithm would be involved.  Underscores

Doc/library/tomllib.rst

@@ -19,14 +19,14 @@ support writing TOML.
 
 .. seealso::
 
-    The `Tomli-W package <https://pypi.org/project/tomli-w/>`__
+    The :pypi:`Tomli-W package <tomli-w>`
     is a TOML writer that can be used in conjunction with this module,
     providing a write API familiar to users of the standard library
     :mod:`marshal` and :mod:`pickle` modules.
 
 .. seealso::
 
-    The `TOML Kit package <https://pypi.org/project/tomlkit/>`__
+    The :pypi:`TOML Kit package <tomlkit>`
     is a style-preserving TOML library with both read and write capability.
     It is a recommended replacement for this module for editing already
     existing TOML files.

Doc/library/typing.rst

@@ -39,7 +39,7 @@ they can also be more complex. The :mod:`typing` module provides a vocabulary of
 more advanced type hints.
 
 New features are frequently added to the ``typing`` module.
-The `typing_extensions <https://pypi.org/project/typing-extensions/>`_ package
+The :pypi:`typing_extensions` package
 provides backports of these new features to older versions of Python.
 
 .. seealso::

Doc/library/unittest.mock.rst

@@ -35,7 +35,7 @@ is based on the 'action -> assertion' pattern instead of 'record -> replay'
 used by many mocking frameworks.
 
 There is a backport of :mod:`unittest.mock` for earlier versions of Python,
-available as `mock on PyPI <https://pypi.org/project/mock>`_.
+available as :pypi:`mock` on PyPI.
 
 
 Quick Guide

Doc/library/venv.rst

@@ -27,7 +27,7 @@ optionally be isolated from the packages in the base environment,
 so only those explicitly installed in the virtual environment are available.
 
 When used from within a virtual environment, common installation tools such as
-`pip`_ will install Python packages into a virtual environment
+:pypi:`pip` will install Python packages into a virtual environment
 without needing to be told to do so explicitly.
 
 A virtual environment is (amongst other things):
@@ -614,7 +614,3 @@ subclass which installs setuptools and pip into a created virtual environment::
 
 This script is also available for download `online
 <https://gist.github.com/vsajip/4673395>`_.
-
-
-.. _setuptools: https://pypi.org/project/setuptools/
-.. _pip: https://pypi.org/project/pip/

Doc/library/xml.rst

@@ -124,25 +124,23 @@ large tokens
   Expat needs to re-parse unfinished tokens; without the protection
   introduced in Expat 2.6.0, this can lead to quadratic runtime that can
   be used to cause denial of service in the application parsing XML.
-  The issue is known as
-  `CVE-2023-52425 <https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52425>`_.
+  The issue is known as :cve:`2023-52425`.
 
-The documentation for `defusedxml`_ on PyPI has further information about
+The documentation for :pypi:`defusedxml` on PyPI has further information about
 all known attack vectors with examples and references.
 
 .. _defusedxml-package:
 
 The :mod:`!defusedxml` Package
 ------------------------------
 
-`defusedxml`_ is a pure Python package with modified subclasses of all stdlib
+:pypi:`defusedxml` is a pure Python package with modified subclasses of all stdlib
 XML parsers that prevent any potentially malicious operation. Use of this
 package is recommended for any server code that parses untrusted XML data. The
 package also ships with example exploits and extended documentation on more
 XML exploits such as XPath injection.
 
 
-.. _defusedxml: https://pypi.org/project/defusedxml/
 .. _Billion Laughs: https://en.wikipedia.org/wiki/Billion_laughs
 .. _ZIP bomb: https://en.wikipedia.org/wiki/Zip_bomb
 .. _DTD: https://en.wikipedia.org/wiki/Document_type_definition

Doc/library/zipfile.rst

@@ -632,7 +632,7 @@ Path objects are traversable using the ``/`` operator or ``joinpath``.
       Prior to 3.10, ``joinpath`` was undocumented and accepted
       exactly one parameter.
 
-The `zipp <https://pypi.org/project/zipp>`_ project provides backports
+The :pypi:`zipp` project provides backports
 of the latest path object functionality to older Pythons. Use
 ``zipp.Path`` in place of ``zipfile.Path`` for early access to
 changes.

Doc/library/zoneinfo.rst

@@ -17,15 +17,15 @@ The :mod:`zoneinfo` module provides a concrete time zone implementation to
 support the IANA time zone database as originally specified in :pep:`615`. By
 default, :mod:`zoneinfo` uses the system's time zone data if available; if no
 system time zone data is available, the library will fall back to using the
-first-party `tzdata`_ package available on PyPI.
+first-party :pypi:`tzdata` package available on PyPI.
 
 .. seealso::
 
     Module: :mod:`datetime`
         Provides the :class:`~datetime.time` and :class:`~datetime.datetime`
         types with which the :class:`ZoneInfo` class is designed to be used.
 
-    Package `tzdata`_
+    Package :pypi:`tzdata`
         First-party package maintained by the CPython core developers to supply
         time zone data via PyPI.
 
@@ -93,7 +93,7 @@ Data sources
 
 The ``zoneinfo`` module does not directly provide time zone data, and instead
 pulls time zone information from the system time zone database or the
-first-party PyPI package `tzdata`_, if available. Some systems, including
+first-party PyPI package :pypi:`tzdata`, if available. Some systems, including
 notably Windows systems, do not have an IANA database available, and so for
 projects targeting cross-platform compatibility that require time zone data, it
 is recommended to declare a dependency on tzdata. If neither system data nor
@@ -413,5 +413,3 @@ Exceptions and warnings
     be filtered out, such as a relative path.
 
 .. Links and references:
-
-.. _tzdata: https://pypi.org/project/tzdata/

Doc/tools/extensions/pyspecific.py

@@ -26,7 +26,6 @@
 from sphinx.locale import _ as sphinx_gettext
 from sphinx.util import logging
 from sphinx.util.docutils import SphinxDirective
-from sphinx.util.nodes import split_explicit_title
 from sphinx.writers.text import TextWriter, TextTranslator
 
 try:
@@ -39,6 +38,7 @@
 
 ISSUE_URI = 'https://bugs.python.org/issue?@action=redirect&bpo=%s'
 GH_ISSUE_URI = 'https://github.com/python/cpython/issues/%s'
+# Used in conf.py and updated here by python/release-tools/run_release.py
 SOURCE_URI = 'https://github.com/python/cpython/tree/main/%s'
 
 # monkey-patch reST parser to disable alphabetic and roman enumerated lists
@@ -54,6 +54,7 @@
 
 std.token_re = re.compile(r'`((~?[\w-]*:)?\w+)`')
 
+
 # Support for marking up and linking to bugs.python.org issues
 
 def issue_role(typ, rawtext, text, lineno, inliner, options={}, content=[]):
@@ -85,16 +86,6 @@ def gh_issue_role(typ, rawtext, text, lineno, inliner, options={}, content=[]):
     return [refnode], []
 
 
-# Support for linking to Python source files easily
-
-def source_role(typ, rawtext, text, lineno, inliner, options={}, content=[]):
-    has_t, title, target = split_explicit_title(text)
-    title = utils.unescape(title)
-    target = utils.unescape(target)
-    refnode = nodes.reference(title, title, refuri=SOURCE_URI % target)
-    return [refnode], []
-
-
 # Support for marking up implementation details
 
 class ImplementationDetail(Directive):
@@ -194,7 +185,6 @@ def parse_platforms(self):
         return platforms
 
 
-
 # Support for documenting audit event
 
 def audit_events_purge(app, env, docname):
@@ -710,7 +700,6 @@ def patch_pairindextypes(app, _env) -> None:
 def setup(app):
     app.add_role('issue', issue_role)
     app.add_role('gh', gh_issue_role)
-    app.add_role('source', source_role)
     app.add_directive('impl-detail', ImplementationDetail)
     app.add_directive('availability', Availability)
     app.add_directive('audit-event', AuditEvent)

Doc/using/mac.rst

@@ -145,7 +145,7 @@ There are several options for building GUI applications on the Mac with Python.
 
 *PyObjC* is a Python binding to Apple's Objective-C/Cocoa framework, which is
 the foundation of most modern Mac development. Information on PyObjC is
-available from https://pypi.org/project/pyobjc/.
+available from :pypi:`pyobjc`.
 
 The standard Python GUI toolkit is :mod:`tkinter`, based on the cross-platform
 Tk toolkit (https://www.tcl.tk). An Aqua-native version of Tk is bundled with
@@ -177,7 +177,7 @@ Distributing Python Applications
 A range of tools exist for converting your Python code into a standalone
 distributable application:
 
-* `py2app <https://pypi.org/project/py2app/>`__: Supports creating macOS ``.app``
+* :pypi:`py2app`: Supports creating macOS ``.app``
   bundles from a Python project.
 
 * `Briefcase <https://briefcase.readthedocs.io>`__: Part of the `BeeWare Project

Doc/using/windows.rst

@@ -1285,7 +1285,7 @@ The Windows-specific standard modules are documented in
 PyWin32
 -------
 
-The `PyWin32 <https://pypi.org/project/pywin32>`_ module by Mark Hammond
+The :pypi:`PyWin32` module by Mark Hammond
 is a collection of modules for advanced Windows-specific support.  This includes
 utilities for:
 

Doc/whatsnew/2.6.rst

@@ -3015,8 +3015,7 @@ Changes to Python's build process and to the C API include:
   ``PyRun_SimpleString("sys.path.pop(0)\n")`` afterwards to discard
   the first ``sys.path`` component.
 
-  Security issue reported as `CVE-2008-5983
-  <http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-5983>`_;
+  Security issue reported as :cve:`2008-5983`;
   discussed in :gh:`50003`, and fixed by Antoine Pitrou.
 
 * The BerkeleyDB module now has a C API object, available as