Jou Langserver

[Moosems]

No description provided.

[Akuli]

Ideally, the compiler internals would be used as is (you can import the self-hosted stuff easily into Jou code). But as it is, it wouldn't be very good for langserver, because it exits the whole process whenever it finds an error. A misspelled variable in the beginning of the file shouldn't prevent you from working on other parts.

That said, I would rather have "first error blocks the rest" langserver than no langserver at all... :)

Langservers also need sockets and JSON. Sockets will be a bunch of platform-specific declare and structs (very messy, but doable). Before #77, the easiest way to do JSON is probably to take cJSON (or other json library) and declare what we need, similarly to how the Jou compiler uses LLVM.

[Akuli]

(This comment assumes that Jou files only contain functions and imports. I will think more about classes etc later.)

I've thought about this more. The error handling of the Jou compiler is bad in at least the following two scenarios:

1. Error in a function near the beginning of the file: langserver should keep going and analyze other functions
2. Error in import: langserver should ignore the unimportable file, or even better, analyze it as much as it can despite its errors

This is difficult, because I don't want to change how error handling works in the compiler. The "first error stops compiling" error handling style makes the compiler simple and nice to work on. Specifically, the compiler exits the whole process whenever there is an error, so I don't need to e.g. return special values that indicate an error has occurred (such as NULL or -1) or free allocated memory to prevent memory leaks (operating system frees it automatically when the process exits).

I think I will instead invoke the compiler as a subprocess (as suggested by Moosems). The idea is to trick the compiler to report multiple errors by running it multiple times. When the langserver runs the compiler for a second time, it tells it to ignore the first error, so that it can get a new error. To do this, it will need to have some sort of minimalistic version of the Jou tokenizer, only enough to figure out where functions start and end.

The compiler still needs to be modified, for example by adding the following command-line flags:

• --check: Compile so that we do not produce an executable. Much faster than normal compiling, because we don't do anything with LLVM. Similar to e.g. black --check in Python or cargo check in Rust.
• --json-output: Instead of printing human-readable compiler warnings and errors, print them as JSON so that it's easy to parse. Somewhat similar to --xml option of valgrind.
• --ignore-lines 123:456: Causes the compiler to ignore the original file contents on lines 123 to 456, and instead see them as blank lines.
• --dummy-function foo: Causes the compiler to imagine that there is a function foo. If it is called, it accepts arguments of any types with no errors.

Next, let's walk through how this would actually work. Suppose file.jou contains the following code (6 lines):

import "./does_not_exist.jou"
def foo() -> itn:
    printf("foo\n")
    reutrn 123
def bar() -> None:
    foo()

User opens the file, so the langserver runs the compiler with --check and --json-output. The compiler says:

compiler error in file "file.jou", line 4: not a valid statement

Or maybe more like this:

{"type": "error", "file": "file.jou", "line": 4, "message": "not a valid statement"}

The langserver knows that lines 2, 3 and 4 belong to function foo, so it recompiles again without the function foo by passing --ignore-lines 2:4 --dummy-function foo. The compiler sees the code as:

import "./does_not_exist.jou"



def bar() -> None:
    foo()

Now there is an error on line 1 because the imported file doesn't exist. The langserver compiles again with --ignore-lines 1:1 and all the flags it had previously. This time, there are no errors because the langserver is still passing --dummy-function foo.

[Akuli]

I'm no longer sure if invoking the compiler multiple times is a good idea. Launching a new process is slow on Windows, so that may get annoying if you have a large file.

A couple days ago, I started refactoring the compiler to emit multiple errors instead of stopping on the first one (nothing pushed to github yet). For most of the code, this has turned out to be surprisingly easy, but not for the parser. If an error occurs, the parser needs to somehow skip the bad tokens before trying again, so that it parses the rest of the file instead of getting stuck on the error.