autodoc/docutils woes in Python 3.13: autodoc-before-process-signature (update_annotations_using_type_comments), Block quote ends without a blank line #13232
Comments
To start with, ensure you are using The strategy I tend to adopt is crudely bisecting the documentation by iteratively deleting large parts of it but ensuring that the warnings being investigated remain (ignoring issues caused by deleting parts of the documentation). I then try and minimise The sphinx/sphinx/ext/autodoc/type_comment.py Lines 117 to 135 in a1510de A |
|
Hello and thanks for your response. I linked #13178 downstream for Anyways, I can try to dig more next week. Thanks, all! |
|
@pllim that's really it: the whole thing is unclear! I looked at the apparently affected modules for a type comment and there wasn't any, I looked at your I'd follow AA-Turner's advice of using P.S. And the |
|
@pllim a minimal reproducer: import inspect, astropy.modeling
obj = astropy.modeling.Fittable1DModel.__call__
print(obj)
source = inspect.getsource(obj)
print(source)In Python 3.11 and 3.12, this reports def __call__(self, *inputs, **kwargs):
"""Evaluate this model on the supplied inputs."""
return super(cls, self).__call__(*inputs, **kwargs)In Python 3.13, In Sphinx, we could just gracefully fail here rather than emmitting a warning, though this would hide a real problem. What do you think? A |
As a downstream user, warning is better than silently failing, so I would rather Sphinx keep the warning until upstream fix can be done. But I do wonder if the warning can be clearer or that is hard to do in this situation. 🤔 Thanks for tracking that down! That is very weird indeed. FWIW this is what it is supposed to look like when successful: https://docs.astropy.org/en/v7.0.0/api/astropy.modeling.Fittable1DModel.html#astropy.modeling.Fittable1DModel.__call__
|
|
I think the problem is the metaclass. Perhaps related to the new # lasagna/__init__.py
import abc
import inspect
import keyword
import os
def make_function_with_signature(func, args=(), kwargs={}, varargs=None, varkwargs=None, name=None):
pos_args = []
key_args = []
if isinstance(kwargs, dict):
iter_kwargs = kwargs.items()
else:
iter_kwargs = iter(kwargs)
# Check that all the argument names are valid
for item in (*args, *iter_kwargs):
if isinstance(item, tuple):
argname = item[0]
key_args.append(item)
else:
argname = item
pos_args.append(item)
if keyword.iskeyword(argname) or not argname.isascii() or not argname.isidentifier():
raise SyntaxError(f"invalid argument name: {argname}")
for item in (varargs, varkwargs):
if item is not None:
if keyword.iskeyword(item) or not argname.isascii() or not argname.isidentifier():
raise SyntaxError(f"invalid argument name: {item}")
def_signature = [", ".join(pos_args)]
if varargs:
def_signature.append(f", *{varargs}")
call_signature = def_signature[:]
if name is None:
name = func.__name__
global_vars = {f"__{name}__func": func}
local_vars = {}
# Make local variables to handle setting the default args
for idx, item in enumerate(key_args):
key, value = item
default_var = f"_kwargs{idx}"
local_vars[default_var] = value
def_signature.append(f", {key}={default_var}")
call_signature.append(f", {key}={key}")
if varkwargs:
def_signature.append(f", **{varkwargs}")
call_signature.append(f", **{varkwargs}")
def_signature = "".join(def_signature).lstrip(", ")
call_signature = "".join(call_signature).lstrip(", ")
frm = inspect.currentframe()
depth = 2
for i in range(depth):
frm = frm.f_back
if frm is None:
return None
mod = inspect.getmodule(frm)
assert mod is not None, frm
frm = inspect.currentframe().f_back
if mod:
filename = mod.__file__
modname = mod.__name__
if filename.endswith(".pyc"):
filename = os.path.splitext(filename)[0] + ".py"
else:
filename = "<string>"
modname = "__main__"
# Subtract 2 from the line number since the length of the template itself
# is two lines. Therefore we have to subtract those off in order for the
# pointer in tracebacks from __{name}__func to point to the right spot.
lineno = frm.f_lineno - 2
# The lstrip is in case there were *no* positional arguments (a rare case)
# in any context this will actually be used...
template = """{0}\
def {name}({sig1}):
return __{name}__func({sig2})
""".format("\n" * lineno, name=name, sig1=def_signature, sig2=call_signature)
code = compile(template, filename, "single")
eval(code, global_vars, local_vars)
new_func = local_vars[name]
new_func.__module__ = modname
new_func.__doc__ = func.__doc__
return new_func
class _ModelMeta(abc.ABCMeta):
def __init__(cls, name, bases, members, **kwds):
super().__init__(name, bases, members, **kwds)
# Handle init creation from inputs
def update_wrapper(wrapper, cls):
# Set up the new __call__'s metadata attributes as though it were
# manually defined in the class definition
# A bit like functools.update_wrapper but uses the class instead of
# the wrapped function
wrapper.__module__ = cls.__module__
wrapper.__doc__ = getattr(cls, wrapper.__name__).__doc__
if hasattr(cls, "__qualname__"):
wrapper.__qualname__ = f"{cls.__qualname__}.{wrapper.__name__}"
# THIS BIT!!!!
if (
"__call__" not in members
and "n_inputs" in members
and isinstance(members["n_inputs"], int)
and members["n_inputs"] > 0
):
# Don't create a custom __call__ for classes that already have one
# explicitly defined (this includes the Model base class, and any
# other classes that manually override __call__
def __call__(self, *inputs, **kwargs):
"""Evaluate this model on the supplied inputs."""
return super(cls, self).__call__(*inputs, **kwargs)
# When called, models can take two optional keyword arguments:
#
# * model_set_axis, which indicates (for multi-dimensional input)
# which axis is used to indicate different models
#
# * equivalencies, a dictionary of equivalencies to be applied to
# the input values, where each key should correspond to one of
# the inputs.
#
# The following code creates the __call__ function with these
# two keyword arguments.
args = ("self",)
kwargs = {
"model_set_axis": None,
"with_bounding_box": False,
"equivalencies": None,
"inputs_map": None,
}
new_call = make_function_with_signature(
__call__, args, kwargs, varargs="inputs", varkwargs="new_inputs"
)
# The following makes it look like __call__
# was defined in the class
update_wrapper(new_call, cls)
cls.__call__ = new_call
# THIS BIT!!!!
if (
"__init__" not in members
and not inspect.isabstract(cls)
#and cls._parameters_
):
# Build list of all parameters including inherited ones
# If *all* the parameters have default values we can make them
# keyword arguments; otherwise they must all be positional
# arguments
args = ("self",)
kwargs = []
def __init__(self, *params, **kwargs):
return super(cls, self).__init__(*params, **kwargs)
new_init = make_function_with_signature(
__init__, args, kwargs, varkwargs="kwargs"
)
update_wrapper(new_init, cls)
cls.__init__ = new_init
class Model(metaclass=_ModelMeta):
def __init_subclass__(cls, **kwargs):
super().__init_subclass__()
def __call__(self, *args, **kwargs):
"""
Evaluate this model using the given input(s) and the parameter values
that were specified when the model was instantiated.
"""
class FittableModel(Model):
"""
Base class for models that can be fitted using the built-in fitting
algorithms.
"""
linear = False
fit_deriv = None
col_fit_deriv = True
fittable = True
class Fittable1DModel(FittableModel):
"""
Base class for one-dimensional fittable models.
This class provides an easier interface to defining new models.
Examples can be found in `astropy.modeling.functional_models`.
"""
n_inputs = 1
n_outputs = 1
_separable = True# docs/bug.py
import sys
sys.path.append('.')
import inspect, lasagna
obj = lasagna.Fittable1DModel.__call__
print(obj)
source = inspect.getsource(obj)
print(source)
print(source == '\n')Then run either A |
|
@pllim yep, that's what happens someone strays into too much metaclass hackery. It's fine and dandy at first, but eventually things break and just the diagnosis alone is mind boggling. |
|
All I can say in its defense is... I didn't write it. 😆 |
|
More seriously, let me ping |
|
This is likely going to be a pain to fix, thanks for the insight into what we may have to do! (Maybe this will push us to finally clean up all the metaclass stuff in modeling, its something I've wanted to do since I started working on it) |
|
Workaround in the meantime 👍 from typing import TYPE_CHECKING
from sphinx.domains.python import PythonDomain
if TYPE_CHECKING:
from docutils.nodes import Element
from sphinx.application import Sphinx
from sphinx.builders import Builder
from sphinx.environment import BuildEnvironment
def setup(app: Sphinx) -> None: # noqa: D103
class PatchedPythonDomain(PythonDomain):
def resolve_xref( # noqa: PLR0913,PLR0917
self,
env: BuildEnvironment,
fromdocname: str,
builder: Builder,
type: str, # noqa: A002
target: str,
node: resolve_xref,
contnode: Element,
) -> Element:
# fixup some wrongly resolved mappings
mapping = {
"pathlib._local.Path": "pathlib.Path",
}
if target in mapping:
target = node["reftarget"] = mapping[target]
return super().resolve_xref(env, fromdocname, builder, type, target, node, contnode)
app.add_domain(PatchedPythonDomain, override=True) |
|
Thanks, @gaborbernat ! I tried your workaround in astropy/astropy#17621 but alas, most of the warnings persisted. Is your workaround only for the |
|
Yes, you can add others in the |
|
A better workaround for the import sys
import sphinx
if sys.version_info[:2] >= (3, 13) and sphinx.version_info[:2] < (8, 2):
import pathlib
from sphinx.util.typing import _INVALID_BUILTIN_CLASSES
_INVALID_BUILTIN_CLASSES[pathlib.Path] = 'pathlib.Path' # type: ignoreA |
|
I wouldn't say better because with that you end up essentially ignoring
those links, rather than fixing it to point where it should I think.
…On Wed, Jan 22, 2025, 18:21 Adam Turner ***@***.***> wrote:
A better workaround for the pathlib._local.Path problem (though still
hacky) is:
import sysimport sphinx
if sys.version_info[:2] >= (3, 13) and sphinx.version_info[:2] < (8, 2):
import pathlib
from sphinx.util.typing import _INVALID_BUILTIN_CLASSES
_INVALID_BUILTIN_CLASSES[pathlib.Path] = 'pathlib.Path' # type: ignore
A
—
Reply to this email directly, view it on GitHub
<#13232 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAFIQPRIWVTJA77A2A5U7632MBGY5AVCNFSM6AAAAABU7QUGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMBYG4YDQMJXGU>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
@gaborbernat please could you elaborate? The method I showed means that the links to A |
|
Ah, I'm mistaken; they're likely equaly good. Note sure how forward compatible your is, but I believe should work (small downside needing a type ignore). |
Committed in #13261, so the patch reflects behaviour that will be in Sphinx 8.2. A |
|
Thank you for the suggestions. However the Path thing was the least of my problem and also a separate issue already. Back to the As for the 2 docstring weirdness from |
|
@pllim see astropy/astropy#17659 which I think fixes the codegen issue. A |
|
Closing, nothing to resolve in Sphinx. A |
|
Thanks, @AA-Turner ! |
Describe the bug
Hello.
astropyhas run into two problems I am not sure how to debug when trying to build doc using Sphinx in Python 3.13. These problems do not exist in Python 3.11 and 3.12.'autodoc-before-process-signature' threw an exception (exception: list index out of range) [autodoc]originally reported in Building docs on Python 3.13.1 gives 124 warnings astropy/astropy#17614WARNING: Block quote ends without a blank line; unexpected unindent. [docutils]How to Reproduce
See log in astropy/astropy#17621
Environment Information
Sphinx extensions
Additional context
Ideas on how to debug downstream would be very nice. The warnings/exceptions are a bit too cryptic for me to understand.
For instance, the autodoc one points to
update_annotations_using_type_commentsbut I am not sure what that is supposed to do or how to disable it.Thank you!
The text was updated successfully, but these errors were encountered: