Thoughts on upcoming change to improve Mathics3 debugging and tracing

[rocky]

Here are my thoughts on some upcoming changes to Mathics-core that I have in mind to improve tracing and debugging...

First, here is one that is probably the least controversial in terms of changing code: Add a builtin function, Stacktrace[] to produce a Mathics3 Function stack trace. This works similar to Breakpoint[] where Mathics3 code is changed to insert this wherever you want it.

For example:

Strings::usage = "Strings[l,n] constructs all possible strings of length n from the elements of list l."
Strings[l_List,0]  := {{}, Stacktrace[]}[[0]]  (* This  assignment adds a stacktrace call before returning the empty list. *)
Strings[l_List,k_Integer?Positive] :=
	Module[{oneless = Strings[l,k-1],i,n=Length[l]},
		Apply[Join, Table[ Map[(Prepend[#,l[[i]]])&, oneless], {i,n}] ]
	]

The above would show the stack of recursive call in progress when evaluating say Strings[{1,2,3}, 2].

The implementation would borrow from the debugger where we don't show all Python frames, just the ones that come from evaluations.

Building on this we might have something in expression.messages() that calls the Python implementation for showing a stack trace. In effect, we can show a stacktrace whenever an error message is given. Of course, giving this information would only be done on request. For example, setting a Setting variable like $ShowStacktraceOnError to True. And other values other than Null and False, might allow for going into a debugger.

The next set of changes though do add more bulk into our existing definitions and increase memory, so this may be more controversial, but that aspect can be negotiated again by putting this in as an option.

In showing backtrace information we may start to show location information as well. We have locations when something is a builtin primitive, or when something defined via Get[]. If something is defined interactively, we can report it was defined interactively. But even here, we can still give information like the position this appears inside the definition string.

Let me know your thoughts.

[rocky]

Something I've been meaning to mention... All the routines that issue Python print() should be redone to return ListExpression literals. This is the spirit of how WMA Trace* functions work. There is a good reason behind this.

The basic idea is that all of this good information should be available for a Mathics3 user to use. So instead of Python print(), there would be a Mathics3 Print[]. With this, another function would print the information and it would be presented in the user's session, so that in Django or pyodide, the information appears (as opposed to on the console).

I will make feature requests for this.

There is a lot of low hanging fruit still, and a lot of easy stuff to fix. It is just a matter devoting some time and effort here.