Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: python doc script classes #301

Merged
merged 16 commits into from
Oct 9, 2023
Merged

Conversation

jandom
Copy link
Contributor

@jandom jandom commented Aug 20, 2023

This is a slow grind but having the tests there protects me from a lot of simple mistakes.

It's still a bit early, I want to make the TableWriter as dumb as possible. Right now it has too many responsibilities and the code structure is very confusing (the technical term is "inheritance hell")

The worst offender is definitely gen_topologyparser_attrs.py the only file I haven't migrated so far

@jandom jandom self-assigned this Aug 20, 2023
@jandom jandom force-pushed the jandom/refactor/document-classes branch from ba7b387 to 1153df5 Compare August 20, 2023 13:53
@jandom jandom marked this pull request as ready for review August 29, 2023 15:42
@jandom jandom requested a review from lilyminium August 29, 2023 15:42
@jandom jandom changed the base branch from jandom/feat/add-snapshot-tests-with-syrup to develop August 29, 2023 15:48
@jandom jandom changed the title refactor: doc script classes refactor: python doc script classes Aug 29, 2023
@jandom jandom changed the base branch from develop to jandom/feat/add-snapshot-tests-with-syrup August 29, 2023 15:49
@jandom jandom closed this Aug 29, 2023
@jandom jandom reopened this Aug 29, 2023
@jandom jandom force-pushed the jandom/refactor/document-classes branch from 43fc301 to cbe7e4a Compare August 29, 2023 18:39
@jandom jandom changed the base branch from jandom/feat/add-snapshot-tests-with-syrup to develop August 29, 2023 18:48
Copy link
Member

@lilyminium lilyminium left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A partial review to start off with -- this looks fantastic @jandom! Everything I've seen looks a thousand times more elegant and easy to read and maintain than what was there before; I'm pretty sure all of my comments are picking up stuff that you hadn't yet changed from the original code.

doc/source/scripts/base.py Outdated Show resolved Hide resolved
doc/source/scripts/base.py Show resolved Hide resolved
doc/source/scripts/base.py Outdated Show resolved Hide resolved
doc/source/scripts/base.py Show resolved Hide resolved
doc/source/scripts/base.py Outdated Show resolved Hide resolved
doc/source/scripts/gen_format_overview_classes.py Outdated Show resolved Hide resolved
doc/source/scripts/gen_format_overview_classes.py Outdated Show resolved Hide resolved
doc/source/scripts/gen_format_overview_classes.py Outdated Show resolved Hide resolved
doc/source/scripts/gen_format_overview_classes.py Outdated Show resolved Hide resolved
doc/source/scripts/gen_format_overview_classes.py Outdated Show resolved Hide resolved
@jandom jandom requested a review from lilyminium August 30, 2023 17:07
@jandom
Copy link
Contributor Author

jandom commented Aug 30, 2023

The one thing that i'm leaving out of this PR is changes to a few files:

  • conf.py
  • core.py
  • clean_example_notebooks.py
  • update_json_stubs_sitemap.py

Also, any flake8 enforcement will be left for later (but I have used it for local development to weed out a few simple mistakes).

@jandom jandom changed the base branch from develop to jandom/feat/add-snapshot-tests-with-syrup August 30, 2023 17:31
@jandom
Copy link
Contributor Author

jandom commented Aug 30, 2023

Some CI/CD is mysteriously broken... will investigate

Meanwhile, this PR remains broken on the syrup snapshot testing PR: these tests ensure that the refactor doesn't change the behavior of the code, so it's important that we merge that first

#285

@IAlibay
Copy link
Member

IAlibay commented Aug 30, 2023

@jandom

Some CI/CD is mysteriously broken... will investigate

This is already fixed in #304

@IAlibay
Copy link
Member

IAlibay commented Aug 30, 2023

re: syrup snapshot testing PR

Apologies, I realise I'm a blocker here. I'm unlikely to have the chance to look at things until the weekend, it's one of those where if I'm for now still in charge of releases (we will see what happens), then I would like to have a review of how this changes the release process.

@jandom
Copy link
Contributor Author

jandom commented Aug 30, 2023

No worries and let’s take all the time here. These PRs here are just demos, and not urgent at all

@jandom jandom force-pushed the jandom/refactor/document-classes branch from 488692c to 00d2d6d Compare August 31, 2023 09:38
@jandom jandom force-pushed the jandom/feat/add-snapshot-tests-with-syrup branch from 119e680 to b716c8a Compare August 31, 2023 10:26
migrate gen_topology_groupmethods.py

migrate gen_topologyattr_defaults.py

migrate FormatOverview

migrate CoordinateReaders

migrate gen_format_overview_classes

migrate TopologyParsers

migrate TopologyAttrs

migrate ConnectivityAttrs

remove dead code, never executed

remove all class attributes

all non-strict mypy checks pass, mypy in pre-commit

first nice simplification

another nice simplification

more simplifications

further cleanups

further trimmings

continue to refactor

update base.py

harmonize to use private functions

simpler code architecture

more refactoring

mypy strict mode on some files

finish mypy changes

flake8 on everything

Update doc/source/scripts/base.py

Co-authored-by: Lily Wang <[email protected]>

Update doc/source/scripts/gen_format_overview_classes.py

Co-authored-by: Lily Wang <[email protected]>

Update doc/source/scripts/gen_format_overview_classes.py

Co-authored-by: Lily Wang <[email protected]>

Update doc/source/scripts/gen_format_overview_classes.py

Co-authored-by: Lily Wang <[email protected]>

Update doc/source/scripts/gen_format_overview_classes.py

Co-authored-by: Lily Wang <[email protected]>

review comments from Lily

fixing mypy
@jandom jandom force-pushed the jandom/refactor/document-classes branch from 00d2d6d to 3269982 Compare August 31, 2023 10:26
Copy link
Member

@lilyminium lilyminium left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finished looking over the other scripts -- just some minor comments, but otherwise everything looks great!! Thank you @jandom :)

doc/source/scripts/gen_topologyparser_attrs.py Outdated Show resolved Hide resolved
doc/source/scripts/gen_topologyparser_attrs.py Outdated Show resolved Hide resolved
doc/source/scripts/gen_topologyparser_attrs.py Outdated Show resolved Hide resolved
doc/source/scripts/gen_topologyparser_attrs.py Outdated Show resolved Hide resolved
doc/source/scripts/gen_topologyparser_attrs.py Outdated Show resolved Hide resolved
@jandom
Copy link
Contributor Author

jandom commented Sep 1, 2023

Thank you @lilyminium – those were some really great suggestions, merged them all and tweaked the tests to match!

Copy link
Member

@lilyminium lilyminium left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jandom -- this overhaul looks fantastic! Looking forward to seeing them in the main repo :D

@jandom
Copy link
Contributor Author

jandom commented Sep 3, 2023

Thank you for the kind words @lilyminium that's really nice – I'm still not completely happy with it, it could be much simpler. But yeah my plan was to get some folks familiar with this toolchain and then slowly start hammering away at the core repo ;-) Maybe I'll add flake8 here and wrap up – also Irfan will probably have more feedback

Base automatically changed from jandom/feat/add-snapshot-tests-with-syrup to develop October 8, 2023 19:42
@jandom
Copy link
Contributor Author

jandom commented Oct 9, 2023

This should see the light of day soon, merged develop into this feature branch and hopefully everything will pass.

@jandom jandom merged commit a952cb2 into develop Oct 9, 2023
3 checks passed
@jandom jandom deleted the jandom/refactor/document-classes branch October 9, 2023 19:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants