Thoughts on the workflows

[zain-sohail]

Currently, the tests workflow takes very long to run, approx. 10 mins. The main slowdown seems to be in processor.py but I didn't look at length.

Secondly, we only have limited number of action minutes. Considering it takes 10 mins for each run and there are 4 currently, we can only do a total of 50 pushes (PR merge or push to a branch) a month. I think, the testing can only be restricted for 3.8, and we can do one test on the other python version either manually, on push to main or when we tag a commit.

The common parts can maybe be merged into one workflow file, like checking out the code and setting up the environment and then we can mention the different jobs later on in the same yml (or potentially linked with others).

@rettigl what do you think?

[rettigl]

Much time will be from downloading the data. Maybe this is done multiple times, needs to be checked. Even more reason to have the code in a separate file to check manually.
What is suggested was to put this into the work flos that runs only on pushes to main, but maybe that is also too much, fair point.

[rettigl]

Ah, I see, I misinterpreted this discussion then.
yes, the work takes about 10-11 min, out of which installation is 2 min, pylint 1 min, and tests ~7 min. The main reason tests take so long is the generation of the inverse deformation field, which is done ~7-8 times for the different branches of the pose correction and meta data tests. Potentially, we can speed this up by using a coarser binning for these tests, I can look into this.
To save computation time, we can also define the version-dependent tests to only run at pushes to main, or at creation of tags as you suggested @zain-sohail

[zain-sohail]

Is it maybe possible to generate the inverse deformation field once, save it to a file and read it? Even if it's data dependant, if it is not created after several transformations to the data, it can be saved as is (as the data is consistently same for tests).

Let's try to also do the version-dependent tests on merges to main for now, then.

[zain-sohail]

Might be useful: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows
https://github.com/actions/cache

[steinnymir]

To me testing extensively only at pushed to main sounds like a good idea. Testing a single "primary" version often should catch most of the problems anyway. At each release might be too little though.

[rettigl]

"GitHub Actions usage is free for standard GitHub-hosted runners in public repositories, and for self-hosted runners. "
https://docs.github.com/en/billing/managing-billing-for-github-actions/about-billing-for-github-actions

[rettigl]

I made PR #166 for this. However, if we don't do tests on pull requests, the coveralls functionality does not seem to properly summarize what has changed compared to the reference.
@zain-sohail where did you see that we are running low on action minutes? For me, it always shows 0/2000 used, both for OpenCompes, and my personal account. I thought they are free for public repositories.

[zain-sohail]

ok I wasn't knowing of that. I just saw in opencompes about limited minutes but didn't know they were free for public repos.

[zain-sohail]

I believe, with PR #182, this discussion can be closed.