Vortex GPGPU

Vortex is a full-stack open-source RISC-V GPGPU. Vortex supports multiple backend drivers, including our C++ simulator (simx), an RTL simulator, and physical Xilinx and Altera FPGAs-- all controlled by a single driver script. The chosen driver determines the corresponding code invoked to run Vortex. Generally, developers will prototype their intended design in simx, before completing going forward with an RTL implementation. Alternatively, you can get up and running by selecting a driver of your choice and running a demo program.

Website

Vortex news can be found on its website

Citation

@inproceedings{10.1145/3466752.3480128,
	author = {Tine, Blaise and Yalamarthy, Krishna Praveen and Elsabbagh, Fares and Hyesoon, Kim},
	title = {Vortex: Extending the RISC-V ISA for GPGPU and 3D-Graphics},
	year = {2021},
	isbn = {9781450385572},
	publisher = {Association for Computing Machinery},
	address = {New York, NY, USA},
	url = {https://doi.org/10.1145/3466752.3480128},
	doi = {10.1145/3466752.3480128},
	abstract = {The importance of open-source hardware and software has been increasing. However, despite GPUs being one of the more popular accelerators across various applications, there is very little open-source GPU infrastructure in the public domain. We argue that one of the reasons for the lack of open-source infrastructure for GPUs is rooted in the complexity of their ISA and software stacks. In this work, we first propose an ISA extension to RISC-V that supports GPGPUs and graphics. The main goal of the ISA extension proposal is to minimize the ISA changes so that the corresponding changes to the open-source ecosystem are also minimal, which makes for a sustainable development ecosystem. To demonstrate the feasibility of the minimally extended RISC-V ISA, we implemented the complete software and hardware stacks of Vortex on FPGA. Vortex is a PCIe-based soft GPU that supports OpenCL and OpenGL. Vortex can be used in a variety of applications, including machine learning, graph analytics, and graphics rendering. Vortex can scale up to 32 cores on an Altera Stratix 10 FPGA, delivering a peak performance of 25.6 GFlops at 200 Mhz.},
	booktitle = {MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture},
	pages = {754–766},
	numpages = {13},
	keywords = {reconfigurable computing, memory systems., computer graphics},
	location = {Virtual Event, Greece},
	series = {MICRO '21}
}

Specifications

• Support RISC-V RV32IMAF and RV64IMAFD

• Microarchitecture:

  • configurable number of cores, warps, and threads.
  • configurable number of ALU, FPU, LSU, and SFU units per core.
  • configurable pipeline issue width.
  • optional local memory, L1, L2, and L3 caches.

• Software:

  • OpenCL 1.2 Support.

• Supported FPGAs:

  • Altera Arria 10
  • Altera Stratix 10
  • Xilinx Alveo U50, U250, U280
  • Xilinx Versal VCK5000

Directory structure

• doc: Documentation.
• hw: Hardware sources.
• driver: Host drivers repository.
• runtime: Kernel Runtime software.
• sim: Simulators repository.
• tests: Tests repository.
• ci: Continuous integration scripts.
• miscs: Miscellaneous resources.

Quick Start

If you are interested in a stable release of Vortex, you can download the latest release here. Otherwise, you can pull the most recent, but (potentially) unstable version as shown below. The following steps demonstrate how to build and run Vortex with the default driver: SimX. If you are interested in a different backend, look here.

Supported OS Platforms

• Ubuntu 18.04, 20.04, 22.04, 24.04
• Centos 7

Toolchain Dependencies

The following dependencies will be fetched prebuilt by toolchain_install.sh.

• POCL
• LLVM
• RISCV-GNU-TOOLCHAIN
• Verilator
• cvfpu
• SoftFloat
• Ramulator
• Yosys
• Sv2v

Install Vortex codebase

	git clone --depth=1 --recursive https://github.com/vortexgpgpu/vortex.git
	cd vortex

Install system dependencies

# ensure dependent libraries are present
sudo ./ci/install_dependencies.sh

Configure your build folder

    mkdir build
    cd build
    # for 32bit
    ../configure --xlen=32 --tooldir=$HOME/tools
    # for 64bit
    ../configure --xlen=64 --tooldir=$HOME/tools

Install prebuilt toolchain

   ./ci/toolchain_install.sh --all

set environment variables

    # should always run before using the toolchain!
    source ./ci/toolchain_env.sh

Building Vortex

make -s

Quick demo running vecadd OpenCL kernel on 2 cores

./ci/blackbox.sh --cores=2 --app=vecadd

Common Developer Tips

• Installing Vortex kernel and runtime libraries to use with external tools requires passing --prefix= to the configure script.

../configure --xlen=32 --tooldir=$HOME/tools --prefix=<install-path>
make -s
make install

• Building Vortex 64-bit requires setting --xlen=64 configure option.

../configure --xlen=64 --tooldir=$HOME/tools

• Sourcing "./ci/toolchain_env.sh" is required everytime you start a new terminal. we recommend adding "source /ci/toolchain_env.sh" to your ~/.bashrc file to automate the process at login.

echo "source <build-path>/ci/toolchain_env.sh" >> ~/.bashrc

• Making changes to Makefiles in your source tree or adding new folders will require executing the "configure" script again without any options to get changes propagated to your build folder.

../configure

• To debug the GPU, the simulation can generate a runtime trace for analysis. See /docs/debugging.md for more information.

./ci/blackbox.sh --app=demo --debug=3

• For additional information, check out the documentation