L1VM README 2024-10-28

Running my blog and keep my hardware going costs money! So you can support me on Ko-fi!

This project started on 9. November 2017 as an experimental fast VM. I had some knowledge already from my Nano VM project. But I started from scratch.

L1VM is an incredible tiny virtual machine with RISC (or comparable style) CPU, about 61 opcodes and about 49 KB binary size on X86_64 Linux! The VM has a 64 bit core (256 registers for integer and double float) and can run object code written in Brackets (a high level programming language) or l1asm assembly language.

Code and data are in separated memories for a secure execution. Like in Harvard type CPUs (found now in DSPs or microcontrollers). The opcode set with 61 opcodes is my own opinion how things should work. It does not "copy" other instruction sets known in other "real" CPUs.

The design goals are:

	- be small
	- be fast
	- be simple
	- be modular

Here is the L1VM course on my blog.

In pure console text I/O programs not linked with SDL library, the memory footprint is very low.

The L1VM runs on Linux (x86_64, Arm), BSD OS: OpenBSD, FreeBSD, DragonFly BSD, Windows 10, 11 via WSL, Haiku and macOS. It has an preprocessor, assembler and compiler for my own language Brackets. On the Raspberry Pi the GPIO pins can be used with my GPIO module. Also the serial port can be used.

My language Brackets is a statically typed language. Now with variable type safety! The VM has even a JIT-compiler using "libasmjit" for x86_64 CPUs! Code written in inline assembly can be compiled by the JIT-compiler. Increasing run speed!

NEW: Now you can define objects (OOP) in Brackets. You can write functions inside the object which are using the object data. An example is here: math-circle-oop.

The L1VM can be build as a shared library to embedd it too!

Build the L1VM as a shared library

You need to make a change in "include/settings.h": Set the EMBEDDED flag to "1":

#define L1VM_EMBEDDED 1

Then build the library in "vm/" with one of the zerobuild embedded build scripts. On Linux you need this for example:

$ CC=clang CCPP=clang++ zerobuild zerobuild-nojit-embedded.txt force

Now copy the ".so" file to the "~/l1vm/bin" directory. So it can be found by the system.

As a next step you can build the test program in "vm/test-embedded". To build the example:

$ CC=clang CCPP=clang++ zerobuild force

To run the test program:

$ ./test-l1vm

NEW: I did develop a linter to check if the function call uses the right function arguments variables. You can write this in comments for the linter to check this. See the post on my blog!

Here is the output of my fractalix fractal program using the SDL module:

Here is the pov-edit program GUI for my LED POV kit:

L1VM - pov edit flash video

A more advanced demo program is "prog/people-object.l1com". It shows how to use a memory object and pointers. I did change the program to use a variable end "main" part by using:

#var ~ main

This is more safe because now every variable in main ends with "main". Take a look at this example.

NEW features
Now legal value ranges for variables can be set:

(x x_min x_max range)

If x is outside of the range then it results in a runtime exception.

Now in the infix/RPN parser you can write a short name for the target variable: _ like this:

{target = (_ + x + (_ * y))}

Every _ will be replaced by "target"! See prog/hello-target-shortname.l1com.

How to install "pov-edit" and the internet jargon file database "jargon": Go into the distribution folder and download and install my gpg key:

$ ./get-koder77-key.sh

Then install the programs with:

$ ./download.sh pov-edit

$ ./download.sh jargon

The L1VM has a full SDL graphics/GUI module for writing desktop applications! And you can write your own modules. Here is the full standard modules list:

Modules

Cells - linked neural networks with FANN library
ciphersaber - encrypt/decrypt functions
crypto - libsodium encrypt/decrypt functions
endianess - convert to big endian, or little endian functions
fann - FANN neural networks
file - file module - read/write files
filetools - file functions like copy, create directory, etc.
genann - neural networks module
gpio - Raspberry Pi GPIO module
math - some math functions
math-nofp - math module for use without FPU
math-vect - math on arrays functions
mem - memory allocating for arrays and vectors
mem-obj - memory functions to store different variables into one memory array
mem-vect - C++ vector memory library
mpfr-c++ - MPFR floating point big num library
nanoid - nano ID, create unique IDs
net - TCP/IP sockets module
pigpio - Raspberry Pi GPIO module
process - start a new shell process
rs232-libserialport - RS232 serial port using libserialport
rs232 - RS232 serial port module
sdl 2.0 - graphics primitves module, like pixels, lines..., and GUI with buttons, lists, etc.
string - some string functions
time - get time and date

Credits

Thanks goes to:
Florian Bruder (sportfloh) for helping on the macOS port.
Cedric Kienzler (cedi) for helping setup GitHub Actions .yaml script.
Andreas Weber (Andy1978) for fixing some serious bugs.
The team behind SDL and A. Schiffler (ferzkopp) for their great work!
The author of genann networks library: Lewis Van Winkle (codeplea).
The authors of the libfann library.
The author of MPFR C++: Pavel Holoborodko.
The author of the RS232 library I use: Teunis van Beelen.
The authors of libserialport: the sigrok team.
The authors of mt19937-64.c random number generator :Takuji Nishimura and Makoto Matsumoto.
The author of Wiring Pi, the Raspberry Pi GPIO library: drogon.
The author of libasmjit, JIT-compiler: Petr Kobalicek.
The authors of www.tutorialspoint.com, for their polish math notation parser example!
The author of the fp16 fixed floating point library: https://github.com/kmilo17pet/fp16
The team of the libsodium library. I use for generating random numbers.
The author of the nanoid ID generating library.

Without them this L1VM project would not be possible! Thank you!
-----------------------

New: variable range checks

Now legal value ranges for variables can be set:

(x x_min x_max range)

If x is out of range then an exception is executed to exit the program. This was inspired by Ada! See prog/hello-ranges.l1com!

l1pre - the preprocessor

The new "l1pre" preprocessor can be used to define macros and include files. See the new "include-lib" directory with all header files. The "prog/hello-6.l1com" example shows how to use this!

reversed polish math notation

Math expressions in: { }, are parsed as reversed polish notation:

{a = x y + z x * *}

is the same as: "a = x + y * z * x" This needs no brackets for complex math expressions! See "prog/hello-4.l1com" example!

"(loadreg)" setting

it is set automatically after a function call, or the "stpop" opcodes. Or you can do it in the code.

SDL 2.0!

Finally I ported the SDL gfx/GUI library to SDL 2.0! The source code is in: vm/modules/sdl-2.0/

I did remove the SDL 1.2 code from this repo. So for now the SDL 2.0 libraries are needed to build my L1VM SDL library.

You need to build vm-sdl2/ VM sources to use the new SDL 2.0 gfx/GUI library. This source of the L1VM is linked with the SDL2 libraries.

1. Configuration

NEW Before installation you have to add the following lines to your "~/.bashrc" bash config file:

export PATH="$HOME/l1vm/bin:$PATH"
export LD_LIBRARY_PATH="$HOME/l1vm/bin:$LD_LIBRARY_PATH"

You also can run the set-path.sh script to do this.

Configure the file access for SANDBOX mode or not, and set your /home directory name: include/settings.h:

// SANDBOX FILE ACCESS
#define SANDBOX                 1			// secure file acces: 1 = use secure access, 0 = OFF!!
#define SANDBOX_ROOT			"/l1vm/"	// in /home directory!, get by $HOME env variable

And you can choose if the array variable bounds check and math checks should be run:

// data bounds check exactly
#define BOUNDSCHECK             1

Math limits checks, on/off:

// division by zero checking
#define DIVISIONCHECK           1

// integer and double floating point math functions overflow detection
// See vm/main.c interrupt0:
// 251 sets overflow flag for double floating point number
// 252 returns value of overflow flag
#define MATH_LIMITS				0

// set if only double numbers calculation results are checked (0), or
// arguments and results get checked for full check (1)
#define MATH_LIMITS_DOUBLE_FULL	0

Build with JIT-compiler

Edit the "vm/jit.h" file, set:

#define JIT_COMPILER 1

2. Installation

Note: you must be in the sudoers list to run the installation scripts. To do this just run this as root (on Linux for example):

# /usr/sbin/usermod -aG sudo username

Logout of your current session. And login again. Now you can run the installation scripts with the sudo command.

Debian Linux

Use: install-debian.sh or install-jit-debian.sh

Fedora Linux

Use: install-zerobuild-fedora.sh or install-jit-zerobuild-fedora.sh

Windows 10, 11 WSL

You need to edit: "include/settings.h":

#define WINDOWS_10_WSL 1

#define DIVISIONCHECK  0

You need to add the following lines to your ".bashrc":

export LD_LIBRARY_PATH="$HOME/bin:/lib:/usr/lib:/usr/local/lib"
export XDG_RUNTIME_DIR="$HOME/xdg"
export RUNLEVEL=3
export DISPLAY=$(grep -m 1 nameserver /etc/resolv.conf | awk '{print $2}'):0

Create "xdg" direcory. In Home folder open shell:

$ mkdir xdg

Use: install-wsl-debian.sh or install-wsl-debian-jit.sh

Make the bash install script executable by:

$ chmod +x ./install-wsl-debian.sh

And run it:

$ ./install-wsl-debian.sh

For graphics/GUI programs you need a Xserver installed. I use VcXsrv as the X11 server. Start VcXsrv first then the WSL shell!

Note: the install scripts automatically install clang C compiler and my zerobuild build tool!

Windows MSYS2

You need to edit: "include/settings.h":

#define DIVISIONCHECK  0

#define CPU_SET_AFFINITY   0

And you need to put this at your "/etc/bash.bashrc":

export PATH="$PATH:/mingw64/bin:$HOME/bin"
export LD_LIBRARY_PATH="$HOME/bin:$LD_LIBRARY_PATH"

Create the "~/bin" directory:

$ mkdir bin

Now you can run the installation script:

$ ./install-zerobuild-msys2.sh

Windows Cygwin

You have to install the clang compiler, SDL2 libraries and a window manager like Xfce for graphics output. This can be done with the installer on: https://www.cygwin.com/

Edit the file: "vm/jit.h": and set:

#define JIT_COMPILER 0

The build script is: "install-zerobuild-cygwin.sh". Note: at the moment only this modules can be build: endianess, file, genann, math, mem, process, sdl, time.

You need the "zerobuild" tool to build the VM: https://www.github.com/koder77/zerobuild

And run the "install-jit-zerobuild.sh" script.

Build without JIT-compiler

Set "JIT_COMPILER" to "0"

And run the "install-zerobuild.sh" script.

cli only

To compile for cli (bashs) text in/output only with no SDL gfx support: Set "#define SDL_module 0" in "include/global.h". Then run in "vm/" the bash script: "make-cli.sh".

macOS

The GitHub CI build now uses macOS 11 to build the VM. UPDATE now the modules are working!! I had to build them as "bundle"! NEW: macOS build scripts made together with sportfloh. And cedi did write the .yaml GitHub Actions file. See main directory: "install-zerobuild-macos.sh".

What works right now: l1asm, l1com, l1pre and the l1vm.
And all the following standard modules:
endianess, fann, file, genann, math, mem, mmpfr (high precision math library), net, process, string, time, sdl-2.0
librs232 (serialport by libserialport)

sdl-2.0: prog/lines.l1com now works! I had to call a SDL event function in the delay loop!
(:sdl_get_mouse_state !)
This is needed to avoid errors on macOS.

Before installation you have to add the following lines to your "~/.bashrc" bash config file:

export PATH="$HOME/l1vm/bin:$PATH"
export LD_LIBRARY_PATH="$HOME/l1vm/bin:$LD_LIBRARY_PATH"

Haiku OS

I did write an installer script for the current beta version of Haiku. Programs which use multithreading can't run! This results in a "segmentation fault" in the OS. Maybe this gets fixed by newer Haiku beta releases.

You need to edit the include/settings.h file:

#define DIVISIONCHECK 0

#define CPU_SET_AFFINITY 0

Update: I fixed the installer scripts. Now the modules can be load by the VM too!

Now you are ready to run the install script!

OpenBSD / FreeBSD

You have to edit the settings.h include file:

// division by zero checking
#define DIVISIONCHECK           0

// switch on on Linux
#define CPU_SET_AFFINITY		0

And change the linker flags on the VM zerobuild file: "vm/zerobuild-nojit.txt"

lflags = "-lm -lpthread -Wl,--export-dynamic"

Add the "-L/usr/local/lib" path to the "zerobuild.txt" make files linker line: on: vm/modules/fann, vm/modules/math, vm/modules/crypto, vm/modules/rs232-libserialport and vm/modules/sdl-2.0

lflags = "-shared -lm -L/usr/local/lib -lfann"

Now replace every "SDL_BYTEORDER" in vm/modules/sdl-2.0/sdl.c by "_BYTE_ORDER".

Currently the MPFR module can't be build: "features.h" include is missing! However: there is a workaround:

Copy the "features.h" header in vm/modules/mpfr-c++/openbsd/ to "/usr/include". This header was patched by me to make the MPFR module build.

Now you can start the build with:

OpenBSD:

sh ./install-zerobuild-openbsd.sh

FreeBSD:
Install the "sudo" program as root: pkg install sudo. Then as a normal user:

sh ./install-zerobuild-freebsd.sh

Edit your "~/.bashrc" config file, add the following lines:

export PATH="$HOME/l1vm/bin:$PATH"
export LD_LIBRARY_PATH="$HOME/l1vm/bin:$LD_LIBRARY_PATH"

NetBSD

You have to edit the settings.h include file:

// division by zero checking
#define DIVISIONCHECK           0

// switch on on Linux
#define CPU_SET_AFFINITY		0

Then you can run the install script for NetBSD. Note: the SDL module can't be build. I have to find out what the error is. And the script for building the programs for L1VM fails. It shows a out of memory error message. If you use NetBSD and could take a look after it, this would be nice!

Edit your "~/.bashrc" config file, add the following lines:

export PATH="$HOME/l1vm/bin:$PATH"
export LD_LIBRARY_PATH="$HOME/l1vm/bin:$LD_LIBRARY_PATH"

DragonFly BSD

You have to add a "bin/" directory in your "/home/user" directory:

$ mkdir bin

Then you need to paste the following lines to your ".shrc" bash config in your "/home" directory:

# check if local bin folder exist
# $HOME/bin
# prepend it to $PATH if so
if [ -d $HOME/bin ]; then
    export PATH=$HOME/l1vm/bin:$PATH
fi

And finally run the installation script:

$ sh ./install-DragonFly.sh

That's it! Note the rs232 library is not build yet!

Now there is a bash script to build L1VM without JIT-compiler: "make-nojit.sh" in vm directory. You have to set "JIT_COMPILER" to "0" in the source file vm/main.c to do that. In some cases programs execute faster if they don't need the JIT-compiler to run!

I added a JIT-compiler using asmjit library. At the moment only few opcodes can be translated into code for direct execution.

L1VM ist under active development. As a proof of concept I rewrote the Nano VM fractalix SDL graphics demo in L1VM assembly.

L1VM is 6 - 7 times faster than Nano VM, this comes from the much simpler design and dispatch speedup.

I included a few demo programs.

The source code is released under the GPL.

A simple "Hello world!" in bra(ets (brackets) my language for L1VM:

// hello.l1com
(main func)
	(set int64 1 zero 0)
	(set string 13 hello "Hello world!")
	// print string
	(6 hello 0 0 intr0)
	// print newline
	(7 0 0 0 intr0)
	(255 zero 0 0 intr0)
(funcend)

NEW

MPFR floating point big numbers library added! Now with the MPFR library calculations with high precision can be done. See the example in the lib/ directory: "mpfr-lib-auto.l1com". There are about 87 math functions in the MPFR library.

NOTE

The current version of L1VM only runs on a Linux or other POSIX compatible OS! It works on Cygwin and on Windows 10 WSL too! If you want help to port it to a new OS, then contact me please...

TODO

- make the L1COM compiler a bit more comfortable - write more functions for the modules - more demo programs

USAGE

compile

$ l1com test

compiles program "test.l1com"

run

$ l1vm test

finally executes program "test.l1obj"

compile using preprocessor

Compile "lib/sdl-lib.l1com" example:

$ ./l1vm-build.sh lib/sdl-lib

Always the full name must be used by the preprocessor l1pre!