Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[tools,topgen] Split topgen into creating complete topcfg and the rest #25920

Open
matutem opened this issue Jan 17, 2025 · 0 comments
Open

[tools,topgen] Split topgen into creating complete topcfg and the rest #25920

matutem opened this issue Jan 17, 2025 · 0 comments
Assignees
@matutem
Labels
Component:IntegratedFoundational For an issue that is considered general to all Integrated variants Component:MultiTop Component:Tooling Issues related to tooling, e.g. tools/scripts for doc, code generation (docgen, reggen), CSR

Comments

@matutem
Copy link
Contributor

matutem commented Jan 17, 2025

Description

Topgen is a monolithic tool and is called multiple times to generate the full set of artifacts. Each time it generated the complete top configuration, and then performs a some subset of all the actions it is able to do. This is not efficient and is hard to maintain.

This calls for a change in the topgen flow so the complete top configuration is generated and written to disk by one tool. The actions that depend on the complete config can then be performed by smaller separate tools, and most importantly, they can be invoked only if the complete top configuration file has changed. This will allow a more incremental flow, and since most of the separate tasks depending on the complete config are independent, they can be done in parallel.

This is not only more efficient and simpler, but it simplifies the development of different tops, so can be very helpful for multi-top.
@matutem matutem self-assigned this Jan 17, 2025
@matutem matutem added Component:Tooling Issues related to tooling, e.g. tools/scripts for doc, code generation (docgen, reggen), CSR Component:MultiTop Component:IntegratedFoundational For an issue that is considered general to all Integrated variants labels Jan 17, 2025
matutem added a commit to matutem/opentitan that referenced this issue Jan 17, 2025
@matutem
[tools,topgen] Simplify topgen
a9f8277 
This is a big change in topgen. Some salient changes:
- It always regenerates from scratch the ipgen in-memory configurations
  (as IpBlock objects) so it is truly incremental: the previous flow
  reused the pre-existing generated ipgen hjson if found.
- It delays the actual ipgen file generation until the complete top config
  is generated. This is a step towards breaking topgen into smaller and
  mutually independent steps, thus enabling parallelizing them.
- It converges with a single pass of process_top.
In order to accomplish this some important changes are made:
- Call some of the merge_top functions suitable for each specific ipgen
  prior to generating the in-memory hjson config. This means not all
  IpBlock objects are available, so these merge functions are instrumented
  to skip blocks missing.
- The generation of ipgens are ordered according to dependencies. This
  order created from the expected block functionality, so is not derived
  from a constructed graph, but converging in one pass means it is correct.
- Calling merge functions early means some of the objects in the in-memory
  top config are not plain dictionaries but are classes, so the code needs
  to handle either.

Part of lowRISC#25920

Signed-off-by: Guillermo Maturana <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@matutem matutem

Labels
Component:IntegratedFoundational For an issue that is considered general to all Integrated variants Component:MultiTop Component:Tooling Issues related to tooling, e.g. tools/scripts for doc, code generation (docgen, reggen), CSR
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@matutem