Convert test runners into build steps

[Vexu]

For reference look at Cases.zig.

[eightfilms]

would like to try to tackle this, if this is not being worked on already

[Vexu]

Not that I know. I'd recommend starting with the integration tests and converting a few different types of tests first and then making a PR.

[eightfilms]

@Vexu sorry, was away for a while and didn't have time to look at this - finally got a good look just now.

Have a quick question to understand how I can approach this: What is the end goal of this change? Is this to be able to eventually move towards being able to classify tests in categories (eg. errors, expected types, etc.)? Right now from my (very amateur) understanding the advantage of the approach in the zig repo is being able to have test manifests and to be able to test according to category (error, compile, translate-c). This contrasts with arocc, which can only do the integration tests wholesale now. I'm imagining the goal here is to move towards a more modular test runner?

[Vexu]

No need to apologize, you're volunteering and any help is appreciated!

Being able to more explicitly classify tests is one goal. The tests are already classified but it's based on different flags and macros being set. Other benefits from the change will be the build systems built-in parallelism and caching in some cases.