Use wait4 and measure memory usage of children in the runner

[filipeom]

Should Close #478

The PR is not ready yet, I still have to format the code, remove some comments, and I'm going to split this into two commits: one which focuses on introducing wait4 and replacing the usage of waitpid and another which counts and displays maxrss.

Also, I'm benchmarking this locally to see if the results are consistent with main.

But feel free to comment already :)

[filipeom]

I also made Marvin give us the memory statistics:

Btw, I know that you've told me you don't like that we're always iterating through the results list to accumulate the results we want to print/compare. Do you think we should make an issue to address it?

We could try to use owl to create a results data frame and then do some statistics/graphs?

[filipeom]

On a side note, do you think that adding wait4 to an existing Unix library might be more appropriate than explicitly including it here?

I was thinking of ocaml-rusage or, more appropriately, extunix, since they are already dependencies.

[zapashcanon]

Btw, I know that you've told me you don't like that we're always iterating through the results list to accumulate the results we want to print/compare. Do you think we should make an issue to address it?

I don't understand what you're refering to 😅

[zapashcanon]

On a side note, do you think that adding wait4 to an existing Unix library might be more appropriate than explicitly including it here?

Yes, I'd be very happy not to have to maintain this piece of C code haha. IIRC @chambart initially wrote some C bindings for wait4 but they were much more lightweight, they should still be in the commit history if you want to have a look

[zapashcanon]

I also made Marvin give us the memory statistics:

This looks good but why is the min much bigger than max ? haha

[zapashcanon]

We could try to use owl to create a results data frame and then do some statistics/graphs?

I tried that at some point but I finally ended-up using the gnuplot ocaml bindings (see the code in bench/report/time_distribution.ml for instance)

[filipeom]

Btw, I know that you've told me you don't like that we're always iterating through the results list to accumulate the results we want to print/compare. Do you think we should make an issue to address it?

I don't understand what you're refering to 😅

I was talking the added functions in bench/report/runs.ml

On a side note, do you think that adding wait4 to an existing Unix library might be more appropriate than explicitly including it here?

Yes, I'd be very happy not to have to maintain this piece of C code haha. IIRC @chambart initially wrote some C bindings for wait4 but they were much more lightweight, they should still be in the commit history if you want to have a look

I will look into these and try to get a pr on extunix 🤞

I also made Marvin give us the memory statistics:

This looks good but why is the min much bigger than max ? haha

It's because, in the min_max_maxrss function, the neutral element for the min operator is Int64.max_int, and since I ran the testcomp runner without files, it returned Int64.max_int. But it should be fine unless you want me to change it for this particular case.

We could try to use owl to create a results data frame and then do some statistics/graphs?

I tried that at some point but I finally ended-up using the gnuplot ocaml bindings (see the code in bench/report/time_distribution.ml for instance)

Ah yes, I forgot we already had graphs 😅 my bad

[zapashcanon]

I was talking the added functions in bench/report/runs.ml

It seems OK! :)

I will look into these and try to get a pr on extunix 🤞

Would be great, thanks for taking the time!

It's because, in the min_max_maxrss function, the neutral element for the min operator is Int64.max_int, and since I ran the testcomp runner without files, it returned Int64.max_int. But it should be fine unless you want me to change it for this particular case.

Haha OK, I was just wondering if you inverted two numbers or something. No I don't mind about it haha, it's just some benchmarking code, it does not matter.

Ah yes, I forgot we already had graphs 😅 my bad

Yes, I'm not sure how easily we can post them to Zulip. Maybe we can post them in a base64 encoding or something? To avoid having to upload them etc.

[filipeom]

Made it depend on ygrek/extunix#58 directly. So we need to wait for a new release or pin it.

Yes, I'm not sure how easily we can post them to Zulip. Maybe we can post them in a base64 encoding or something? To avoid having to upload them etc

This is a nice feature! I might experiment with this in a separate PR 😃