Skip to content

Commit

Permalink
reorg docs
Browse files Browse the repository at this point in the history
  • Loading branch information
Udit8348 committed Oct 4, 2024
1 parent 32b0376 commit 208c5b3
Show file tree
Hide file tree
Showing 5 changed files with 60 additions and 87 deletions.
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Vortex GPGPU

Vortex is a full-stack open-source RISC-V GPGPU.
Vortex is a full-stack open-source RISC-V GPGPU. Vortex supports multiple *backend drivers*, including our C++ simulator (simx), an RTL simulator, and physical Xilinx and Altera FPGAs-- all controlled by a single driver script. The chosen driver determines the corresponding code invoked to run Vortex. Generally, developers will prototype their intended design in simx, before completing going forward with an RTL implementation. Alternatively, you can get up and running by selecting a driver of your choice and running a demo program.

## Specifications

Expand Down Expand Up @@ -29,12 +29,14 @@ Vortex is a full-stack open-source RISC-V GPGPU.
- `ci`: Continuous integration scripts.
- `miscs`: Miscellaneous resources.

## Build Instructions
More detailed build instructions can be found [here](docs/install_vortex.md).
## Quick Start
The following steps demonstrate how to run Vortex with the default driver: simx. If you are interested in a different backend, look [here](docs/simulation.md).

### Supported OS Platforms
- Ubuntu 18.04, 20.04
- Centos 7
### Toolchain Dependencies
The following dependencies will be fetched prebuilt by `toolchain_install.sh`.
- [POCL](http://portablecl.org/)
- [LLVM](https://llvm.org/)
- [RISCV-GNU-TOOLCHAIN](https://github.com/riscv-collab/riscv-gnu-toolchain)
Expand Down Expand Up @@ -107,4 +109,4 @@ echo "source <build-path>/ci/toolchain_env.sh" >> ~/.bashrc
```sh
./ci/blackbox.sh --app=demo --debug=3
```
- For additional information, check out the /docs.
- For additional information, check out the [documentation](docs/index.md)
17 changes: 5 additions & 12 deletions docs/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ However, contributions are strongly encouraged and keep the project moving forwa
6. Otherwise, you can go to your fork on Github online and manually create a PR (todo)
(todo): how to name and format your PR, what information you should add to the PR, does not need to be too strict if you are attending the weekly meetings*
7. Github uses the following semantics: `base repository` gets the changes from your `head repository`
8. Therefore, you should set the `base repository` to `vortexgpgpu/vortex` and the `base` branch to `develop` since active development should only be added to this branch
8. Therefore, you should set the `base repository` to `vortexgpgpu/vortex` and the `base` branch to `master` since the master branch is protected by reviewed PRs.
9. And you should assign the `head repository` to `<your-github-username>/vortex` (which represents your fork of vortex) and the `base` branch to the one created in step 2
10. Now that your intended PR has been specified, you should review the status. Check for merge conflicts, if all your commits are present, and all the modified files make sense
11. You can still make a PR if there are issues in step 10, just make sure the structure is correct according to steps 7-9
Expand All @@ -31,14 +31,7 @@ However, contributions are strongly encouraged and keep the project moving forwa
14. As long as the `head repository`'s `base` branch is the one you edited, the PR will automatically get the most recent changes
15. When all merge conflicts are resolved, changes are made, and tests pass you can have an admin merge your PR


- You should create a new branch from develop that is clearly named with the feature that you want to add
- Avoid pushing directly to the `master` branch instead you will need to make a Pull Request (PR)
- There should be protections in place that prevent pushing directly to the main branch, but don't rely on it
- When you make a PR it will be tested against the continuous integration (ci) pipeline (see `continuous_integration.md`)
- It is not sufficient to just write some tests, they need to be incorporated into the ci pipeline to make sure they are run
- During a PR, you might receive feedback regarding your changes and you might need to make further commits to your branch


## Creating and Adding Tests
The CI pipeline the vortex tests. If you are contributing code changes, then review `testing.md` to learn how to integrate your own tests
## What Makes a Good Contribution?
- If you are contributing code changes, then review `testing.md` to ensure your tests are integrated into the CI pipeline
- During a PR, you should consider the advice you are provided by your reviewers. Remember you keep adding commits to an open PR!
- If your change aims to fix an issue opened on Github, please tag that issue in the PR itself
71 changes: 27 additions & 44 deletions docs/fpga_setup.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# FPGA Startup and Configuration Guide

## Gaining Access to FPGA's with CRNCH
If you are associated with Georgia Tech and need remote access to the FPGA's, you can utilize CRNCH's server.
If you are associated with Georgia Tech (or related workshops) you can use CRNCH's server to gain remote access to FPGA's. Otherwise, you can skip to the Xilinx or Intel (Altera) synthesis steps below.

## What is CRNCH?

Expand Down Expand Up @@ -37,19 +37,18 @@ CRNCH resources do not require any VPN access for GT members so you can head to

Alternatively, you can `ssh` into rg with: `ssh <your-gt-acctname>@rg-login.crnch.gatech.edu`

(`ssh usubramanya3@rg-login.crnch.gatech.edu`)
(`ssh gburdell3@rg-login.crnch.gatech.edu`)

Once you’ve logged in, you can use Slurm to request other nodes within the testbed. See more information on Slurm at [this page](https://gt-crnch-rg.readthedocs.io/en/main/general/using-slurm.html).

Note that you can also use VSCode to log into the Rogues Gallery via its terminal functionality. See [this page for more details](https://gt-crnch-rg.readthedocs.io/en/main/general/visual-studio-code.html).
## Synthesis for Xilinx Boards
First, you need to get access to the server with the Xilinx FPGAs.

## **What Machines are Available in the Rogues Gallery?**

Complete list of machines can be found [here](https://gt-crnch-rg.readthedocs.io/en/main/general/rg-hardware.html).

## Which Machine do we Need from RG?

There are three primary nodes you might use. The table below summarizes:
There are three primary nodes you might use for Xilinx FPGAs. The table below summarizes:

| Name | Device | Description |
| --- | --- | --- |
Expand All @@ -62,59 +61,43 @@ There are three primary nodes you might use. The table below summarizes:

## How to Access flubber for Synthesis?

Now that you have the files prepared and available on the FPGA node, you can start the synthesis. To run on hardware we need a rg-xilinx-fpga-hw cluster which includes **flubber[1,4-5]**. First `ssh` into the rouges gallery:
Now that you have the files prepared and available on the FPGA node, you can start the synthesis. To run on hardware we need a rg-xilinx-fpga-hw cluster which includes **flubber[1,4-5]**. First `ssh` into the rouges gallery, if you have not already.

```bash
ssh <username>[@rg-login.crnch.gatech.edu](mailto:[email protected])
```

Then, to access the hardware node you need to `ssh` into flubber:
Once you’ve logged in, you can use Slurm to request an interactive job. First, view the available Slurm Partitions here [here](https://gt-crnch-rg.readthedocs.io/en/main/general/using-slurm.html). Then, the example requests can be found [here](https://gt-crnch-rg.readthedocs.io/en/main/general/using-slurm-examples.html).

In our case we might run:
```bash
ssh flubber1
salloc -p rg-fpga --nodes=1 --ntasks-per-node=1 --nodelist flubber1 --time=01:00:00
```

## Synthesis for Xillinx Boards

XRT Environment Setup
----------------------

$ source /opt/xilinx/Vitis/2023.1/settings64.sh
$ source /opt/xilinx/xrt/setup.sh

## Environment Setup
Once you are logged in, you will need to complete some first time configurations.

Check Installed FPGA Platforms
------------------------------
### Clone Repo

$ platforminfo -l
### Source Configuration Scripts
```
$ source /opt/xilinx/xrt/setup.sh
$ source /opt/xilinx/Vitis/2023.1/settings64.sh
```

### Check Installed FPGA Platforms
`platforminfo -l`

Build FPGA image
----------------

### Build FPGA image
The directory `hw/syn/xilinx/xrt` contains the makefile used to synthesize Vortex.
```
$ cd hw/syn/xilinx/xrt
$ PREFIX=test1 PLATFORM=xilinx_u50_gen3x16_xdma_5_202210_1 TARGET=hw NUM_CORES=4 make

$ PREFIX=test1 PLATFORM=xilinx_u50_gen3x16_xdma_5_202210_1 TARGET=hw NUM_CORES=4 make build_u50_hw_4c.log 2>&1 &
```
Will run the synthesis under new build directory: BUILD_DIR := "\<PREFIX>\_\<PLATFORM>\_\<TARGET>"

The generated bitstream will be located under <BUILD_DIR>/bin/vortex_afu.xclbin

Sample FPGA Run Test
--------------------

Ensure you have the correct opae runtime for the FPGA target

$ make -C runtime/xrt clean
$ TARGET=hw make -C runtime/xrt

Run the following from your Vortex build directory

$ TARGET=hw FPGA_BIN_DIR=<BUILD_DIR>/bin ./ci/blackbox.sh --driver=xrt --app=sgemm --args="-n128"

---

The directory `hw/syn/xilinx/xrt` contains the makefile used to synthesize Vortex.

For long-running jobs, invocation of this makefile can be made of the following form:

`[CONFIGS=<vortex macros>] [PREFIX=<prefix directory name>] [NUM_CORES=<#>] TARGET=hw|hw_emu PLATFORM=<platform baseName> nohup make > <log filename> 2>&1 &`
Expand All @@ -127,17 +110,17 @@ CONFIGS="-DL2_ENABLE -DDCACHE_SIZE=8192" PREFIX=build_4c_u280 NUM_CORES=4 TARGET

The build is complete when the bitstream file `vortex_afu.xclbin` exists in `<prefix directory name><platform baseName>hw|hw_emu/bin`.

## Running a Program on FPGA
### Running a Program on Xilinx FPGA

The blackbox.sh script in `ci` can be used to run a test with Vortex’s xrt driver using the following command:

`FPGA_BIN_DIR=<path to bitstream directory> TARGET=hw|hw_emu PLATFORM=<platform baseName> ./ci/blackbox.sh --driver=xrt --app=<test name>`

For example:

`FPGA_BIN_DIR=`realpath hw/syn/xilinx/xrt/build_4c_u280_xilinx_u280_gen3x16_xdma_1_202211_1_hw/bin` TARGET=hw PLATFORM=xilinx_u280_gen3x16_xdma_1_202211_1 ./ci/blackbox.sh --driver=xrt --app=demo`
```FPGA_BIN_DIR=<realpath> hw/syn/xilinx/xrt/build_4c_u280_xilinx_u280_gen3x16_xdma_1_202211_1_hw/bin TARGET=hw PLATFORM=xilinx_u280_gen3x16_xdma_1_202211_1 ./ci/blackbox.sh --driver=xrt --app=demo```

## Synthesis for Intel (Altera) Boards
### Synthesis for Intel (Altera) Boards

To set up the environment, source the XRT setup.sh and other Xilinx scripts. For example:

Expand Down
25 changes: 1 addition & 24 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,29 +5,6 @@
- [Codebase Layout](codebase.md)
- [Microarchitecture](microarchitecture.md)
- [Cache Subsystem](cache_subsystem.md)
- [Software](software.md)
- [Simulation](simulation.md)
- [Altera FPGA Setup Guide](altera_fpga_guide.md)
- [Xilinx FPGA Setup Guide](xilinx_fpga_guide.md)
- [Contributing](contributing.md)
- [Debugging](debugging.md)
- [Useful Links](references.md)

## Installation

- For the different environments Vortex supports, [read this document](environment_setup.md).
- To install on your own system, [follow this document](install_vortex.md).

## Quick Start Scenarios

Running Vortex simulators with different configurations:
- Run basic driver test with rtlsim driver and Vortex config of 2 clusters, 2 cores, 2 warps, 4 threads

$ ./ci/blackbox.sh --driver=rtlsim --clusters=2 --cores=2 --warps=2 --threads=4 --app=basic

- Run demo driver test with opae driver and Vortex config of 1 clusters, 4 cores, 4 warps, 2 threads

$ ./ci/blackbox.sh --driver=opae --clusters=1 --cores=4 --warps=4 --threads=2 --app=demo

- Run dogfood driver test with simx driver and Vortex config of 4 cluster, 4 cores, 8 warps, 6 threads

$ ./ci/blackbox.sh --driver=simx --clusters=4 --cores=4 --warps=8 --threads=6 --app=dogfood
24 changes: 21 additions & 3 deletions docs/simulation.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,14 @@

### Cycle-Approximate Simulation

SimX is a C++ cycle-level in-house simulator developed for Vortex. The relevant files are located in the `simX` folder.
SimX is a C++ cycle-level in-house simulator developed for Vortex. The relevant files are located in the `simx` folder. The [readme](README.md) has the most detailed instructions for building and running simX.

- To install on your own system, [follow this document](install_vortex.md).
- For the different Georgia Tech environments Vortex supports, [read this document](environment_setup.md).

### FGPA Simulation

The guide to build the fpga with specific configurations is located [here.](fpga_setup.md) You can find instructions for both Xilinx and Intel (Altera) based FPGAs.
The guide to build the fpga with specific configurations is located [here.](fpga_setup.md) You can find instructions for both Xilinx and Altera based FPGAs.

### How to Test

Expand Down Expand Up @@ -47,4 +50,19 @@ PERF: core1: instrs=90693, cycles=53108, IPC=1.707709
PERF: core2: instrs=90849, cycles=53107, IPC=1.710678
PERF: core3: instrs=90836, cycles=50347, IPC=1.804199
PERF: instrs=363180, cycles=53108, IPC=6.838518
```
```

## Additional Quick Start Scenarios

Running Vortex simulators with different configurations:
- Run basic driver test with rtlsim driver and Vortex config of 2 clusters, 2 cores, 2 warps, 4 threads

$ ./ci/blackbox.sh --driver=rtlsim --clusters=2 --cores=2 --warps=2 --threads=4 --app=basic

- Run demo driver test with opae driver and Vortex config of 1 clusters, 4 cores, 4 warps, 2 threads

$ ./ci/blackbox.sh --driver=opae --clusters=1 --cores=4 --warps=4 --threads=2 --app=demo

- Run dogfood driver test with simx driver and Vortex config of 4 cluster, 4 cores, 8 warps, 6 threads

$ ./ci/blackbox.sh --driver=simx --clusters=4 --cores=4 --warps=8 --threads=6 --app=dogfood

0 comments on commit 208c5b3

Please sign in to comment.