Precompiling / AOT Compilation

[Turakar]

AFAIK, we are currently taking the java bytecode and compiling it just-in-time. However, I'm hosting a (private) server which lags a bit sometimes, especially once jvml compiles.

It would be really helpful, if I would be able to precompile the complete jar file to be able to run it later really fast.

[ghost]

The problem with this is that it removes the opportunity for runtime optimizations... you lose the advantages of a JIT. I agree that it would be a good option to have, but ... the actual compiled code will not be as optimized as it could be with a JIT... but it will load fast.

It's always a tradeoff right?

[SquidDev]

removes the opportunity for runtime optimizations

I'm not sure if that is correct in all situations. The Lua VM is the same on all computers, there is no difference in instructions. I seem to remember that one of the new versions of Android compiles the bytecode to machine code once on download, so the machine code is optimised for the processor, then never needs to again.

JVML-JIT obviously works differently, so I don't know if the same principal applies.

[ElvishJerricco]

The big thing is the RTI table. During compilation, there's always objects that we want the bytecode to have constant-speed access to. So we put them in a table, give the bytecode the index into that table, and pass RTI as the first argument to the function whenever it's called. I'm not sure it'd be feasible to save the RTI table and recall it correctly.

There's also all the data structures we have. Every class would have to be stored and recalled, annotations, attributes, everything. It all needs easy ways to be stored and recalled. Generally that's what class files are for... But we can't rely on the class files that we're precompiling because those can be changed after precompilation. My only thought is to make the precompiled stuff a zip archive, and to have all the classes used stored in that archive to recall their structures later without having to use a custom save format.

It's definitely possible. I've been thinking of ways to do this for a while now. A lot of things can be reconstructed at load time, which should be very cheap compared to going through the whole compilation process. But it's a super difficult task and it's just not on our radar at the moment.

[Yevano]

I think I'll add to the conversation a bit. Pre-compilation is certainly doable, and there are real-world applications which do this. I think the best way of doing it would be to use an ahead-of-time and just-in-time hybrid, where all the files in the class path are pre-compiled to files containing Lua bytecode chunks, and at runtime we allow the JIT to run like normal in case we load new classes at runtime, do bytecode introspection, or we want to optimize the bytecode at runtime.

The life of the VM would essentially be:

• Load Java classes. We need the bytecode as well in case the user wants to do bytecode introspection at runtime, but we usually won't be recompiling.
• Load Lua bytecode files into memory and assign them to Java class methods.
• Enter the JIT VM.
• From this point on, almost all methods will already be compiled. We can possibly try and do some runtime analysis and optimize methods here. Any new methods that pop up from things like plugin loading will just use the JIT to compile and run them.
• Program ends at some point.

The Lua bytecode files would be formatted something like this:
File name: lua_ClassName.bin
METHOD_NAME
BYTECODE
METHOD_NAME
BYTECODE
...

Another interesting thing to do, yet slightly unrelated, would be to create a program which would generate executable files which would check if JVML is installed, and then unpack itself into a virtual folder and run like normal. This would require us to set some metadata in the user's computer so we can tell whether JVML is installed or not. Perhaps we could use the metadata RFC proposed on the forums. (The forums are unfortunately currently down...)

[ElvishJerricco]

That doesn't really solve the RTI problem. There's definitely a lot of ways to do this. But we have to remember that the entire jvml code base needs to be available and natives need to be available. And again, RTI objects need to be available.

[Yevano]

We could compile a sister function with each method function which builds the RTI on startup. This may make the code for building the RTI a bit less neat though. We might have to do something like this:

Look at line 1170 in newjit.lua.

asmGetObj(rcon, function()
    local ccException = classByName("java.lang.ClassCastException")
    local con = findMethod(ccException, "<init>(Ljava/lang/String;)V")
    return con[1]
end)

[ElvishJerricco]

I'd be worried that the code inside functions like that might access upvalues or other things that wouldn't be available outside of context. We'd need some kind of system that's very restricting. I'm not sure how to go about that...

And yea it also makes RTI code very ugly.

[ElvishJerricco]

Also, this is a very big ticket item. I think there's several smaller items on the issue list at we've been neglecting. First on my list is HashMaps. I need to look at that pull request and get it perfected. And there's a bunch of other things on our issue list that I think we should get to

[dmarcuse]

Just wondering, is there any news on this? It's something I've been interested in as well.

[ElvishJerricco]

Unfortunately, I don't see this as a reasonable addition. There are too many things that get decided at link-time. The AOT compiled objects would end being virtually the same things as class files, but with more specific byte code.

[ardera]

I don't understand. It sometimes takes 15 seconds for a project to JIT compile, how is it not possible to write everything done in this compile time to a file?

[dmarcuse]

@ardera the way I understand it is that it also has to set up memory objects when compiling the code. Saving these with the file would be inefficient and not including them would mean it would still be "slow" to run.

[ElvishJerricco]

Yea that's basically it. RTI is a table of objects that a method needs to run. But they can be anything. Numbers, strings, class tables, method references, functions. All sorts of things that a method needs access to whenever its running. But these things can't be saved to file. We would have to create an intermediate file type, which is silly since that's what the .class files are supposed to be.

[ardera]

I think I understand now, it's very difficult / impossible to save the RTI table to a file. And if it's not saved, it's basically the same as a class file.

[JBYoshi]

Is the RTI table the part that makes compilation so long? If not, then precompilation would save some time.

[ElvishJerricco]

... No. Compilation takes so long because it's a large, difficult task.

[Yevano]

Although this idea isn't directly connected to AOT compilation, it would help solve the same problem. Instead of trying to run the JIT immediately, we could instead begin by running the VM in an interpreter and then do hotspot analysis to see which methods should get compiled to bytecode over time.

[JBYoshi]

@Yevano I think the "real' JVM does something similar. I'm not sure though.

[ElvishJerricco]

Yea it does. It'd be a loooot of effort though