Skip to content

Commit

Permalink
[doc] merge cicd into ci section
Browse files Browse the repository at this point in the history
  • Loading branch information
@tangxifan
tangxifan committed Jun 10, 2024
1 parent 5dcaab9 commit e6784fd
Show file tree
Hide file tree
Showing 3 changed files with 89 additions and 225 deletions.
189 changes: 89 additions & 100 deletions docs/source/dev_manual/ci.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -60,135 +60,124 @@ Workflows can categorized in two types

.. warning:: If any validation flow failed, the pull request cannot be merged in general.

.. _developer_ci_workflow_check_tool_version:
CI/CD setup
-----------

Check Tool Version
^^^^^^^^^^^^^^^^^^
OpenFPGA implements CI/CD system using Github actions.
The following figure shows the Actions implements flow.
The source building is skipped if there are changes only in ``openfpga_flow`` or ``docs`` directory,
in which case the docker image compiled for the latest master branch is used for running a regression.

The workflow aims to validate the following:

- All the tools meet the expected versions as documented
.. graphviz::
:align: center

.. warning:: **This workflow is essential!** If it fails, there is a problem in infrastructure.
digraph G {
node [fontname = "Handlee"];
edge [fontname = "Handlee"];
Trigger [
label = "Action triggered"
];

.. _developer_ci_workflow_netlist_generation:
masterCompare [
label = "Diff with current master"
];

Netlist Generation
^^^^^^^^^^^^^^^^^^
Build [
label = "Changes only in\n openfpga_flow/doc?"
shape = diamond
];

As illustrated in Fig. :numref:`fig_ci_workflows_netlist_generation`, the workflow aims to validate the following:
BuildDocker [
label = "Run build regression test\nBuild docker images"
shape = box
];

- RTL netlists are reproduciable by OpenFPGA and architecture files
- Gate-level netlists are reproduciable by OpenFPGA, architecture files and related scripts
PushDockersCond [
label = "Is merge\non master?"
shape = diamond
];

PushDockers [
label = "Push docker Images\n(maintain compiled binary\nin docker + Example tasks)"
shape = box
];

.. _fig_ci_workflows_netlist_generation:
RunRegression [
label = "Run functional regression test"
shape = box
];

.. figure:: ./figures/ci_workflows_netlist_generation.svg
:width: 100%
Trigger ->masterCompare;
masterCompare ->Build;
Build -> BuildDocker [ label = "No" ];
BuildDocker -> PushDockersCond;
edge[weight=0.5] Build -> RunRegression [ label = "Yes" ];
edge[weight=10] PushDockersCond -> RunRegression [ label = "No" ];
PushDockersCond -> PushDockers [ label = "Yes" ];
edge[weight=2] PushDockers -> RunRegression;

Decision tree of netlist generation workflow
{
rank=same;
PushDockersCond PushDockers;
};
}

.. _developer_ci_workflow_bitstream_generation:

Bitstream Generation
^^^^^^^^^^^^^^^^^^^^
|
As illustrated in Fig. :numref:`fig_ci_workflows_bitstream_generation`, the workflow aims to validate the following:

- Bitstream files are reproduciable by OpenFPGA, benchmarks and architecture files
.. option:: Build regression test

.. _fig_ci_workflows_bitstream_generation:
The OpenFPGA source is compiled with the following set of compilers.

.. figure:: ./figures/ci_workflows_bitstream_generation.svg
:width: 100%
#. gcc-7
#. gcc-8
#. gcc-9
#. gcc-10
#. gcc-11
#. clang-6
#. clang-7
#. clang-8
#. clang-10

Decision tree of bitstream generation workflow
The docker images for these build environment are available on `github packages <https://github.com/orgs/lnis-uofu/packages>`_.

.. _developer_ci_workflow_testbench_generation:
.. option:: Functional regression test

Testbench Generation
^^^^^^^^^^^^^^^^^^^^
OpenFPGA maintains a set of functional tests to validate the different functionality.
The test are broadly catagories into ``basic_reg_test``, ``fpga_verilog_reg_test``,
``fpga_bitstream_reg_test``, ``fpga_sdc_reg_test``, and ``fpga_spice_reg_test``.
A functional regression test is run for every commit on every branch.

As illustrated in Fig. :numref:`fig_ci_workflows_testbench_generation`, the workflow aims to validate the following:

- Testbench files are reproduciable by OpenFPGA, benchmarks and architecture files
How to debug failed regression test
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In case the ``functional regression test`` fails,
the actions script will collect all ``.log`` files from
the task directory and upload as a artifacts on github storage.
These artifacts can be downloaded from the github website actions tab, for more reference follow `this <https://docs.github.com/en/actions/managing-workflow-runs/downloading-workflow-artifacts>`_ article.

.. _fig_ci_workflows_testbench_generation:
**NOTE** : The retention time of these artifacts is 1 day,
so in case user want to reserve the failure log for longer duration back it up locally

.. figure:: ./figures/ci_workflows_testbench_generation.svg
:width: 100%
Release Docker Images
^^^^^^^^^^^^^^^^^^^^^^

Decision tree of testbench generation workflow
.. option:: ghcr.io/lnis-uofu/openfpga-master:latest

.. _developer_ci_workflow_rtl_verification:
This is a bleeding-edge release from the current master branch of OpenFPGA.
It is updated automatically whenever there is activity on the master branch.
Due to high development activity, we recommend the user to use the bleeding-edge version to get access to all new features and report an issue in case there are any bugs.

RTL Verification
^^^^^^^^^^^^^^^^

As illustrated in Fig. :numref:`fig_ci_workflows_rtl_verification`, the workflow aims to validate the following:
CI after cloning repository
^^^^^^^^^^^^^^^^^^^^^^^^^^^
If you clone the repository the CI setup will still function, except the based images are still pulled from "lnis-uofu" repository and the master branch
of cloned repo will not push final docker image to any repository .

- RTL netlists can pass all the design verification tests.
**In case you want to host your own copies of OpenFPGA base images** and final release create a github secret variable with name ``DOCKER_REPO`` and set it to ``true``. This will make ci script to download base images from your own repo packages, and upload final release to the same.

.. _fig_ci_workflows_rtl_verification:
**If you don not want to use docker images based regression test** and like to compile all the binaries for each CI run. You can set ``IGNORE_DOCKER_TEST`` secrete variable to ``true``.

.. figure:: ./figures/ci_workflows_rtl_verification.svg
:width: 100%

Decision tree of RTL verification workflow


Useful Labels of Pull Requests
------------------------------

Continous integration is triggered conditionally to avoid high traffic in computing machines.
Users can add the following labels in pull requests, to force running some tests:

.. option:: force_netlist_generation

Force the run of netlist generation workflow. See details in :ref:`developer_ci_workflow_netlist_generation`

.. option:: force_bitstream_generation

Force the run of bitstream generation workflow. See details in :ref:`developer_ci_workflow_bitstream_generation`

.. option:: force_testbench_generation

Force the run of testbench generation workflow. See details in :ref:`developer_ci_workflow_testbench_generation`

.. option:: force_rtl_full_simulation

Force the run of full testbench simulation for RTL netlists. See details in :ref:`developer_ci_workflow_rtl_verification`

.. option:: force_rtl_preconfig_simulation

Force the run of preconfigured testbench simulation for RTL netlists. See details in :ref:`developer_ci_workflow_rtl_verification`

.. option:: force_gl_full_simulation

Force the run of full testbench simulation for gate-level netlists. See details in :ref:`developer_ci_workflow_rtl_verification`

.. option:: force_gl_preconfig_simulation

Force the run of preconfigured testbench simulation for gate-level netlists. See details in :ref:`developer_ci_workflow_rtl_verification`

CI Runners
----------

Workflows are executed on two type of runners (computers)

- Github-hosted runners

- Self-hosted runners

Github-Hosted Runners
^^^^^^^^^^^^^^^^^^^^^

All the detect-changes parts of workflow are executed here because they do not require in-house tools

Self-Hosted Runners
^^^^^^^^^^^^^^^^^^^

Most generation/validation workflow are executed here because they require in-house tools

Currently, the self-hosted runners are on the ``eda01``, ``eda02`` and ``eda03`` workstation
.. note:: Once you add ``DOCKER_REPO`` variable, you need to generate base images. To do this trigger manual workflow ``Build docker CI images``
123 changes: 0 additions & 123 deletions docs/source/dev_manual/cicd_setup.rst
View file

This file was deleted.

2 changes: 0 additions & 2 deletions docs/source/dev_manual/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,6 @@

ci

cicd_setup

regression_tests

tcl_api
Expand Down

0 comments on commit e6784fd

Please sign in to comment.