Merge pull request #1701 from lnis-uofu/xt_ci

Misc Updates on CI and Doc

.github/workflows/build.yml

@@ -7,6 +7,10 @@ on:
   schedule:
     - cron: "0 0 * * 0 " # weekly
 
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
 # Environment variables
 env:
   # Customize the CMake build type here (Release, Debug, RelWithDebInfo, etc.)
@@ -119,8 +123,6 @@ jobs:
 
       - name: Checkout OpenFPGA repo
         uses: actions/checkout@v4
-        with:
-          submodules: recursive
 
       - name: Install dependencies
         run: sudo bash ./.github/workflows/install_dependencies_build_${{ matrix.config.dependency_version }}.sh
@@ -217,8 +219,6 @@ jobs:
 
       - name: Checkout OpenFPGA repo
         uses: actions/checkout@v4
-        with:
-          submodules: recursive
 
       - name: Install dependencies
         run: sudo bash ./.github/workflows/install_dependencies_build_${{ matrix.config.dependency_version }}.sh
@@ -265,8 +265,6 @@ jobs:
 
       - name: Checkout OpenFPGA repo
         uses: actions/checkout@v4
-        with:
-          submodules: recursive
 
       - name: Install dependencies
         run: sudo bash ./.github/workflows/install_dependencies_build_${{ matrix.config.dependency_version }}.sh
@@ -313,8 +311,6 @@ jobs:
 
       - name: Checkout OpenFPGA repo
         uses: actions/checkout@v4
-        with:
-          submodules: recursive
 
       - name: Install dependencies
         run: |
@@ -367,8 +363,6 @@ jobs:
 
       - name: Checkout OpenFPGA repo
         uses: actions/checkout@v4
-        with:
-          submodules: recursive
 
       - name: Install dependencies
         run: |
@@ -406,8 +400,6 @@ jobs:
 
       - name: Checkout OpenFPGA repo
         uses: actions/checkout@v4
-        with:
-          submodules: recursive
       - name: Download a built artifacts
         uses: actions/download-artifact@v2
         with:
@@ -461,8 +453,6 @@ jobs:
 
       - name: Checkout OpenFPGA repo
         uses: actions/checkout@v4
-        with:
-          submodules: recursive
       - name: Download a built artifacts
         uses: actions/download-artifact@v2
         with:
@@ -521,8 +511,6 @@ jobs:
 
       - name: Checkout OpenFPGA repo
         uses: actions/checkout@v4
-        with:
-          submodules: recursive
 
       - name: ${{matrix.config.name}}_GCC-11_(Ubuntu 22.04)
         shell: bash

.github/workflows/cell_lib_test.yml

@@ -7,6 +7,10 @@ on:
   schedule:
     - cron: "0 0 * * 0 " # weekly
 
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
 # Multiple job to tests
 jobs:
   # Test the RTL compilation compatibility

.github/workflows/docker.yml

@@ -7,6 +7,10 @@ env:
   DOCKER_REPO: ${{ secrets.DOCKER_REPO }}
   REPO_OWNER: ${{ github.repository_owner }}
 
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
 jobs:
   change_detect:
     name: "Detect code changes"
@@ -32,8 +36,6 @@ jobs:
     steps:
       - name: Checkout
         uses: actions/checkout@v4
-        with:
-          submodules: recursive
       - name: Set up QEMU
         uses: docker/setup-qemu-action@v1
       - name: Set up Docker Buildx
@@ -77,8 +79,6 @@ jobs:
     steps:
       - name: Checkout
         uses: actions/checkout@v4
-        with:
-          submodules: recursive
       - name: Set up QEMU
         uses: docker/setup-qemu-action@v1
       - name: Set up Docker Buildx

.github/workflows/format.yaml

@@ -7,6 +7,10 @@ on:
   schedule:
     - cron: "0 0 * * 0 " # weekly
 
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
 # Multiple job to tests
 jobs:
   change_detect:

README.md

@@ -11,16 +11,19 @@ Version: see [`VERSION.md`](VERSION.md)
 
 The award-winning OpenFPGA framework is the **first open-source FPGA IP generator with silicon proofs** supporting highly-customizable FPGA architectures. OpenFPGA provides complete EDA support for customized FPGAs, including Verilog-to-bitstream generation and self-testing verification. OpenFPGA opens the door to democratizing FPGA technology and EDA techniques with agile prototyping approaches and constantly evolving EDA tools for chip designers and researchers.
 
-**If this is your first time working with OpenFPGA, we strongly **recommend you watch the** [introduction video about OpenFPGA](https://youtu.be/ocODUGcYGqo)**
+> [!TIP]
+> If this is your first time working with OpenFPGA, we strongly recommend you watch the [introduction video about OpenFPGA](https://youtu.be/ocODUGcYGqo)
 
 A quick overview of OpenFPGA tools can be found [**here**](https://openfpga.readthedocs.io/en/master/tutorials/getting_started/tools/).
 We also recommend potential users check out the summary of [**technical capabilities**](https://openfpga.readthedocs.io/en/master/overview/tech_highlights/#) before compiling.
 
-**Before asking for help, please checkout the** [Frequently Asked Questions](https://github.com/lnis-uofu/OpenFPGA/discussions/937)
+> [!TIP]
+> Before asking for help, please checkout the [Frequently Asked Questions](https://github.com/lnis-uofu/OpenFPGA/discussions/937)
 
 ## Compilation
 
-**A tutorial **video about **how to compile** can be** found [here](https://youtu.be/F9sMRmDewM0)**
+> [!NOTE]
+> A tutorial video about how to compile can be found [here](https://youtu.be/F9sMRmDewM0)
 
 Detailed guidelines are available at [**compilation guidelines**](https://openfpga.readthedocs.io/en/master/tutorials/getting_started/compile/).
 Before starting, we strongly recommend you read the required dependencies and ensure that they are correctly installed.
@@ -36,7 +39,7 @@ You can find a set of [tutorials](https://openfpga.readthedocs.io/en/master/tuto
 
 ## Backward Compatibility
 
-If you were using an old version of OpenFPGA and are now interested to move to the latest version, please check out the [developer guidelines](https://openfpga.readthedocs.io/en/master/dev_manual/back_compatibile).
+If you were using an old version of OpenFPGA and are now interested to move to the latest version, please check out the [developer guidelines](https://openfpga.readthedocs.io/en/master/dev_manual/back_compatible/).
 
 ## License
 
@@ -54,8 +57,8 @@ Bibtex:
 @ARTICLE{9098028,  author={Tang, Xifan and Giacomin, Edouard and Chauviere, Baudouin and Alacchi, Aurélien and Gaillardon, Pierre-Emmanuel},  journal={IEEE Micro},   title={OpenFPGA: An Open-Source Framework for Agile Prototyping Customizable FPGAs},   year={2020},  volume={40},  number={4},  pages={41-48},  doi={10.1109/MM.2020.2995854}}
 ```
 
-A list of related publications can be found [here](https://openfpga.readthedocs.io/en/master/reference/).
+A list of related publications can be found [here](https://openfpga.readthedocs.io/en/master/appendix/reference/).
 
 ## Contributing to OpenFPGA
 
-Please read the [contributor guidelines](https://openfpga.readthedocs.io/en/master/dev_manual/contributor_guide) if you would like to contribute to OpenFPGA.
+Please read the [contributor guidelines](https://openfpga.readthedocs.io/en/master/dev_manual/contributor_guide/) if you would like to contribute to OpenFPGA.

docs/source/dev_manual/cicd_setup.rst → docs/source/dev_manual/ci.rst

@@ -1,4 +1,64 @@
-.. dev_manual_cicd_setup::
+.. _developer_ci:
+
+Continous Integration
+=====================
+
+Motivation
+----------
+
+Continous Integration (CI) systems are built to ensure that input and output files of each teams are
+
+- Correct
+- Reproducable
+- Consistent with other teams
+
+CI system is automatically triggered on 
+
+- Main branch: the master branch of the codebase
+- A pull request on main branch
+
+Workflows
+---------
+
+Principles
+^^^^^^^^^^
+
+Continous Integration system consists a number of workflows, each of which is designed to validate a specific aspect of the codebase.
+For the work of each team, there is at least 1 dedicated workflow.
+
+Workflows can categorized in two types
+
+.. option:: Generation flow
+
+  Such type of workflow is designed to ensure that golden files (netlists, bitstreams, etc.) are reproduciable.
+  A generation workflow consists of three steps:
+
+  - Detect changes on input files, e.g., architecture files, IPs and related scripts. 
+
+    - If no changes detected, the workflow ends, since the golden outputs are not changed in a pull request
+    - If any changes are detected, the workflow will continue to the next steps
+
+  - Regenerate golden files by calling scripts. By the end of this step, it will compare the newly generated files with the golden reference (current branch)
+    - If there are no changes, the workflow ends.
+    - If any changes on golden reference are detected, this will error out. It means that the current golden reference are not reproduciable. 
+
+  .. warning:: If any changes on golden references are detected, code review has to be enforced. Ensure that all the teams impacted agree on the changes.
+
+.. option:: Validation flow
+
+  Such type of workflow is designed to verify the correctness of golden files
+  A validation workflow consists of three steps:
+
+  - Detect changes on golden reference (some pull requests update golden references) 
+
+    - If no changes detected, the workflow ends. There is no need to validate the correctness of the golden reference (previous pull request should already do so).
+    - If any changes are detected, the workflow will continue to the next steps
+
+  - Run validation by calling scripts. For example, verification may call HDL simulations to verify the correctness of netlists.
+    - If the new golden reference passes all the tests, this will end. 
+    - If the new golden reference fails any test, this will error out. It means that the current golden reference can not meet basic requirements. 
+
+  .. warning:: If any validation flow failed, the pull request cannot be merged in general.
 
 CI/CD setup
 -----------

docs/source/dev_manual/contributor_guidelines/general_rules.rst

@@ -0,0 +1,44 @@
+.. _developer_contributor_guidelines_general_rules:
+
+General Rules
+=============
+
+Motivation
+----------
+Github projects involve many parties with different interests.
+It is necessary to establish rules to
+
+- guarantee the quality of each pull request by establishing a standard
+- code review for each pull request is straightforward
+- contributors have confidence when submitting changes
+
+Create Pull requests
+--------------------
+
+- Contributors should state clearly their motivation and the principles of code changes in each pull request
+- Contributors should be active in resolving conflicts with other contributors as well as maintainers. In principle, all the maintainers want every pull request in and are looking for reasons to approve it.
+- Each pull request should pass all the existing tests in CI (See :ref:`developer_contributor_guidelines_checkin_system` for details). Otherwise, it should not be merged unless you get a waiver from all the maintainers.
+- Contributors should not modify any codes/tests which are unrelated to the scope of their pull requests.
+- The size of each pull request should be small. Large pull request takes weeks to be merged. The recommend size of pull request is up to 500 lines of codes changes. If you have one large file, this can be waived. However, the number of files to be changed should be as small as possible.
+
+  .. note:: For large pull requests, it is strongly recommended that contributors should talk to maintainers first or create an issue on the Github. Contributors should clearly define the motivation, detailed technical plan as well as deliverables. Through discussions, the technical plan may be requested to change. Please do not start code changes blindly before the technical plan is approved.
+
+- For any new feature/functionality to be added, there should be
+
+  - Dedicated test cases in CI which validates its correctness
+  - An update on the documentation, if it changes user interface
+  - Provide sufficient code comments to ease the maintenance
+
+.. _developer_contributor_guidelines_checkin_system:
+
+Check-in System
+---------------
+
+.. seealso:: The check-in system is based on continous integration (CI). See details in :ref:`developer_ci` 
+
+The check-in system aims to offer a standardized way to 
+
+- ensure quailty of each contribution
+- resolve conflicts between teams
+
+It is designed for efficient communication between teams.

docs/source/dev_manual/contributor_guidelines/index.rst

@@ -0,0 +1,11 @@
+.. _developer_contributor_guidelines:
+
+Contributor Guidelines
+======================
+
+.. toctree::
+   :maxdepth: 2
+
+   general_rules
+
+   naming_convention