CI: make integration tests run faster

[zaro0508]

Subject of the issue

The integration tests took about 20mins to complete when it was run on circleci. Now that Sceptre has been moved to github actions it is taking nearly 40mins to complete. We should try to make it faster on github actions.

On circleci we ran the tests in parallel by splitting them up using the native split test feature. Github actions doesn't seem to have an equivalent feature.

[zaro0508]

example of splitting ruby tests with a custom script, https://rubyyagi.com/how-to-run-tests-in-parallel-in-github-actions/

or maybe run tests in parallel with behave-parallel or behavex, behave/behave#930

[alex-harvey-z3q]

@zaro0508 you have added a comment in the code here suggesting that the tests are not designed to run in parallel. I had a bit of a look at this and seems like parallelising each feature file in a matrix could be a way forward, but not if the tests themselves can't run in parallel.

[zaro0508]

parallelising each feature file in a matrix could be a way forward, but not if the tests themselves can't run in parallel.

I'm not sure what that would look like but i think it would be worth investigating.

I'm not even sure parallelization is the best solution anymore. I think a better solution is to investigate running integration tests with something like localstack so that any contributor can execute the integration tests on their own machine.

[zaro0508]

I took an initial stab at running integration tests using localstack however I quickly ran into problems where localstack cloudformation and AWS cloudforamtion do not behave the same which causes a bunch of the Sceptre integration tests to fail. The issue has been reported at localstack/localstack#10674. I also asked about this issue in the localstack slack community however I did not get any replies, https://localstack-community.slack.com/archives/CMAFN2KSP/p1712955872693799

[HarshCasper]

Hi @zaro0508 — I am Harsh from the LocalStack team. Sorry about the experience! One of our team members will soon check in with you.