Auto-completion for packages

[mahrud]

When running M2 from the terminal, if you load a package, all exported package symbols can be tab-auto-completed. In Emacs, however, this doesn't happen.

The reason is clear: Emacs auto-completion is tied to the syntax highlighter which is fixed to the pre-loaded packages. I suspect auto-completion can be made to work. Is it possible for emacs to communicate with readline?

A possible solution might be to also generate per-package syntax-highlighting and auto-completion lists, and somehow making Emacs know to load those additional lists whenever a package is loaded. Is this possible?

[DanGrayson]

We could simply add symbols for all packages to the default list.

[mahrud]

It would be misleading if a word gets highlighted or autocompleted by M2, but it's just a symbol in M2 because the corresponding package hasn't been loaded yet, or maybe the package is something irrelevant to the user.

Moreover, your suggestion is still incongruent with the behavior of M2 in a terminal.

[d-torrance]

I haven't tested it yet, but this Emacs package might be helpful for this: https://github.com/monsanto/readline-complete.el

[d-torrance]

I've been thinking about a possible solution for this problem, so that TAB in Emacs would give all the same symbols as in the terminal using readline.

• We could add the ability to call completions from expr.d at top-level (this is what readline uses), possibly as complete(String) since complete already exists as a method.
• The Emacs completion function could then call this and instead of printing the output as usual, parse it and use it to form the list of possible completions.

The Python major mode does something similar.

The big difference in Macaulay2, though, is that we have output line numbers and variables like oo and ooo that we wouldn't want to muddy up. One possible (but pretty hacky) idea is to introduce a new type of list that this function could return with its own AfterEval method that doesn't update the output variables or line number.

[mahrud]

This sounds promising, but before putting in much work I'd rather make sure this is the right direction for getting auto completion working on vscode and other modern editors as well.

@etdavila10 pointed me to LSP in this direction.

[d-torrance]

I have a very early draft of an M2 LSP server, so I'm definitely interested in moving in that direction at some point!

[mahrud]

Btw, apropos "^bla" might already do what you wanted, no?

Do you know if using LSP would require an instance of M2 (or perhaps a dedicated thread communicating via a side channel)?

[d-torrance]

Btw, apropos "^bla" might already do what you wanted, no?

Yes, I think it might!

Do you know if using LSP would require an instance of M2 (or perhaps a dedicated thread communicating via a side channel)?

That's currently how my draft is implemented, yes. Emacs or VS Code or whatever would be communicating with a second M2 instance running the server.

[mahrud]

Yes, I think it might!

To be clear, I think perhaps apropos should call completions from expr.d instead, assuming you can also search based on a piece in the middle of an expression.

[etdavila10]

I have a very early draft of an M2 LSP server, so I'm definitely interested in moving in that direction at some point!

This is very exciting! I would be more than happy to contribute to an M2 LSP as I think it would be extremely beneficial to future M2 development.

[d-torrance]

@etdavila10 -- Awesome!

This is the branch that I've been working on: https://github.com/d-torrance/M2/tree/lsp

Everything is in M2/Macaulay2/packages/LanguageServer.m2 or the corresponding subdirectory.

So far, it can successfully send and receive JSON-RPC messages with the editor, but the initialization step is failing for some reason I never quite figured out. I haven't touched it since last summer, so my memory is a little hazy. :)