Hooking into the debugger

[Splines]

I have a special usage question / feature request for a complex debug setup as extension author. Would love to hear your input.

Context

3Blue1Brown is a YouTuber who publishes amazing math videos featuring animations. To create those animations he has written his own animation engine in Python, which is called Manim. With Manim Notebook, we created a VSCode extension to bring Manim to VSCode.

Users of Manim can write code like this:

from manimlib import *


class MySimpleManimScene(Scene):
    def construct(self):
        circle = Circle()
        circle.set_stroke(BLUE_E, width=4)
        self.play(ShowCreation(circle))

        square = Square()
        self.play(ReplacementTransform(circle, square))

        self.play(
            square.animate.set_fill(RED_D, opacity=0.5),
            self.frame.animate.set_width(25),
        )

Then, they pass this file to the manimgl binary provided by Manim to preview their animation defined inside the construct() method:

manimgl ./path/to/your/file.py <other options like line number where preview should start>

exciting-first-start.mp4

Our Manim Notebook extension is (in a nutshell) a wrapper around this such that users can more interactively preview their animations without having to pass exact line numbers in a terminal command to indicate where they want to start previewing.

Idea & Question

Now the new idea: it'd be amazing if we could provide something like a Manim Animation Debugger (MAD), see this issue. In the best case, this would feature:

• Using the in-built VSCode debug UI elements, such that users can set breakpoints and "debug" their animation by previewing it line by line (or preview until the next breakpoint etc.)
• Leveraging your already existing great Python debugger such that we can also provide the usual debug experience along the way, e.g. view the current variable states etc.

I have just started to read a bit about how debugger extensions work in VSCode. But when I already hear about Debug Adapter Protocol and Debug Adapter, I can imagine that writing a debugger from scratch is an immense task. That's why my question is if you provide some kind of hooking mechanism into your debugger, e.g. an API for other extension authors, exposed by returning it from the extension's activate() function, such that we can perform something in the background whenever the user steps to the next line (just one example).

Even if this existed, I'm not quite sure if this specific debugging scenario is even supported by your debugger since it's another binary (manimgl) that calls this file and to make things even more complicated, it executes the code inside an Interactive Python kernel (IPython).

[Splines]

Should this not be supported, maybe you have an idea to get me into the right direction for how in theory something like this could be realized? Even just keywords could be really helpful.