RPS mode with cold/warm ratio

[cvetkovic]

• Support for RPS mode
• Reimplementation of individual trace driver
• Specification generator bugfixing
• Code stability improvement (a lot of new unit and integration tests)
• Workload code deduplication
• Knative deployment parallelization
• Code moving and beautification

[cvetkovic]

@leokondrashov: This is the last PR from the big one. Please have a look thoroughly at this once, since it affects the core functionality of the loader. After this one, I will follow up with a new one that will extract Dirigent-specific features/settings.

[leokondrashov]

We should clarify the interpretation of the IAT array; only then can I check the driver and spec generation code. Right now, it seems inconsistent in the tests for RPS and trace modes.

[leokondrashov]

Some minor issues in the tests for the generator; otherwise, looks good. But I have concerns about the driver now. Since we are not necessarily hit the minute border, we should be careful how it is handled (especially, startOfMinute, previousIATSum, and minuteIndex for empty minutes)

[leokondrashov]

Some minor fixes in specification tests again. There is a change in the driver that is inconsistent with the rest of the code: experiment duration with second granularity. In the front-end we interpret experiment duration as seconds, but in driver, now, we try to interpret it as number of minutes. See the comment for functionsDriver.

[leokondrashov]

Please fix warmup handling. Looks fine to me, but I want to test-run it on several experiments to be sure that nothing goes wrong.

[leokondrashov]

Unfortunately, I see a problem with warmup and RPS mode: (I modified the config file to include 1 minute of warmup)

$ go run cmd/loader.go -config cmd/config_knative_rps.json -verbosity debug
WARN[Dec  4 06:14:18.474] It is recommended that the first 10% of cold starts are discarded from the experiment results for low cold start RPS.
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x999be7]

goroutine 1 [running]:
github.com/vhive-serverless/loader/pkg/trace.profileConcurrency(...)
        /users/lkondras/loader/pkg/trace/trace_profiler.go:87
github.com/vhive-serverless/loader/pkg/trace.DoStaticTraceProfiling({0xc0000908f0, 0x1, 0xc0001ffba8?})
        /users/lkondras/loader/pkg/trace/trace_profiler.go:38 +0x107
github.com/vhive-serverless/loader/pkg/driver.(*Driver).RunExperiment(0xc0000b9980)
        /users/lkondras/loader/pkg/driver/trace_driver.go:488 +0x38
main.runRPSMode(0xc00019cc60, 0x0, 0x0)
        /users/lkondras/loader/cmd/loader.go:236 +0x29e
main.main()
        /users/lkondras/loader/cmd/loader.go:111 +0x479
exit status 2

After closer inspection, I see that there are problems with RPS mode and warmup together: we generate only the experiment duration, but run for experiment duration + warmup.

I think the whole warmup might be broken now: previously, we were skipping one minute for profiling during driver execution, and now we do the invocation even for the minute that we were using for static profiling.

[cvetkovic]

@leokondrashov: I think we should also change names of ExperimentDuration and WarmupDuration parameters and add Minute suffix, since it becomes unclear how this works if Granularity is second. It might be written in the doc, but we might have some bugs around this in the code now or in the future. What do you think?

[leokondrashov]

I agree that it is extremely confusing. We should take care of that.

Right now, I've checked the loader once more, and it works as expected. I think we can merge this one and add an issue about the granularity. And work on the existing issues one-by-one so that the review process would be fast.

[cvetkovic]

Ok. I have created an issue. I will proceed with merging then.