% Hybrogen Hybrid Code Generation, fast and small as the hydrogen element % HP Charles
-
After ccg, hpbcg, deGoal, the new HybroGen tool to generate efficent binary code
-
General idea : Hybrogen is a tool which helps to generate compilette, compilettes are small code generators able to regenerate a part of an application at run-time. It aims to be fast (~10 cycles to generate 1 instruction), small (KB code generator)
-
How it works : it start from a instruction set database containing instruction encoding, arithmetic, data width and vector len, we create a programming language (HybroLang) which allow to express data set variation & build a compiler which take advantage of (1) data set variation and (2) hardware capability to modify the binary code at runtime.
-
Many publications about dynamic code generation
- 2004 : The invention of the "Compilette" term (based on old ccg tool) : "Efficient data driven run-time code generation" https://dl.acm.org/doi/pdf/10.1145/1066650.1066653
- 2018 : Using binary code generation against side channel attacks (based on deGoal tool) "Automated software protection for the masses against side-channel attacks"
- 2021 : Transprecision example (using HybroLang) : "Dynamic compilation for transprecision applications on heterogeneous platform"
- 2021 : (using HybroLang) "Instruction Set Design Methodology for In-Memory Computing through QEMU-based System Emulator"
-
Grammar use ANTLR4 with python4 backend
pip3 install antlr4-python3-runtime==4.7.2
-
Compilation need a postresql database. To install postgresql and configure it you can use this commands :
sudo apt install postgresql
-
Install a python postgresql connector:
sudo apt install python3-psycopg2
-
Initialize the database
sudo -i -u postgres
createdb hybrogen
createuser --pwprompt hybrogen
# the default password in the code is "hybrogen"- Under psql prompt
grant all privileges on database hybrogen to hybrogen;
It is a good practice to not create the database under your own username. (The database could be multiuser accessible)
-
Qemu build need ninja
sudo apt install ninja-build
-
Clone HybroGen in a directory to extract the source files (with git or fetch / tar)
-
git clone [email protected]:CEA-LIST/HybroGen.git
-
or
wget https://github.com/CEA-LIST/HybroGen/archive/refs/tags/v4.0.tar.gz
tar xf v4.0.tar.gz
-
Choose a target directory to install the release e.g.
/opt/H4.0/
-
For each platforms riscv, aarch64, powerpc, cxram-linux
Run
./GenCrossTools.py -a <platform> -p /opt/H4.0/ -w /opt/H4.0/tmp
This command will generate the cross-compiler environment (gcc, gdb, qemu, linux-headers). This command could take some time to run.
- Run
./GenCrossTools.py -a riscv -p /opt/H4.0/ -w /opt/H4.0/tmp -s
- This will generate the shell environment (csh like or bash like)
- A full installation could be :
- Run
./GenCrossTools.py -a aarch64 -p /opt/H4.0/ -w /opt/H4.0/tmp
./GenCrossTools.py -a aarch64 -p /opt/H4.0/ -w /opt/H4.0/tmp -s
./GenCrossTools.py -a riscv -p /opt/H4.0/ -w /opt/H4.0/tmp
./GenCrossTools.py -a riscv -p /opt/H4.0/ -w /opt/H4.0/tmp -s
./GenCrossTools.py -a powerpc -p /opt/H4.0/ -w /opt/H4.0/tmp
./GenCrossTools.py -a powerpc -p /opt/H4.0/ -w /opt/H4.0/tmp -s
./GenCrossTools.py -a cxram-linux -p /opt/H4.0/ -w /opt/H4.0/tmp
./GenCrossTools.py -a cxram-linux -p /opt/H4.0/ -w /opt/H4.0/tmp -s
This step could take time. Count between 5mn and 20mn for each architecture depending on your computing power and bandwith.
HybroGen is mainly written in with python but need some build
- Run
make DbPopulate
to populate the SQL database with instructions description - Congratulation, HybroGen is ready to work !
If you want to play with grammar / lexer / parser, you'll need some more steps:
- Install
sudo apt install antlr4
make buildGrammar
to build the ANTLR lexer / parser
For this platform we need a qemu plugin which emulate the C-SRAM accelerator and give statistics about executed instructions.
Follow instructions on this repository : https://github.com/CEA-LIST/csram-qemu-plugin
- Some code examples are located in the this sub directory :
CodeExamples
For example to run an demonstration for the power architecture here is the command. Adapt for other architectures / demonstrations.
-
cd CodeExamples/
-
source /opt/H4.0/powerpc/.cshrc
-
./RunDemo.py -a power -i Array-Mult-Specialization
-
Regression can be run in the same directory :
./Regression.py power
GenCrossTools used to generate compiler / debugger and qemu has it's own documentation :
- Read the GenCrossTools documentation