Allow combination of parameter scans

[christianhujer]

A parameter scan works similar to a parameter list.
Every parameter scan can, in fact, be expressed with a parameter list instead.

For example, the command

hyperfine -P jobs 1 8 'make -j {jobs}

could instead (albeit slightly less comfortably) be written as

hyperfine -L jobs $(seq -s, 1 8) 'make -j {jobs}'

It is thus surprising that a single command can have multiple -L arguments but only a single -P argument.

Expected behavior: Both of the following commands work and are equivalent.

hyperfine -P a 1 5 -P b 1 5 'echo {a} {b}'
hyperfine -L a $(seq -s, 1 5) -L b $(seq -s, 1 5) 'echo {a} {b}'

Actual behavior:
The command version with two -P options does not work.

This is the error message:

$ hyperfine -P a 1 5 -P b 1 5 'echo {a} {b}'
error: The following required arguments were not provided:
    <command>...

USAGE:
    hyperfine <command>... --parameter-scan <VAR> <MIN> <MAX>

For more information try --help

As a side note, the error message seems to be wrong as well.

Workaround:
No big deal, just use -L <VAR> $(seq -s, <MIN> <MAX>) instead of -P <VAR> <MIN> <MAX>.

Version of hyperfine used: hyperfine 1.12.0 (Kubuntu 22.04.1 LTS)

[sharkdp]

That sounds very reasonable, thank you for the request.

Before someone goes ahead and implements this, we should evaluate if we can generally share more code between parameter scans and parameter lists.

[teofr]

Hey, I've been wishing for a way to combine -P and -L myself, would like to give this feature a go if that's ok (and if there's no hurry on it).

Been thinking a bit about the design:

• -L stays the same
• -P takes a fourth optional parameter, the step size, which is assumed to be 1 if not specified (or maybe -1 if MAX < MIN)
• -D is deprecated (maybe with a warning) and only allowed when there's exactly one appearance of -P
• Internally I still need to look a bit more at the code, but ideally having the same interface for either parameter scans or parameters lists, just allowing to iterate over them.
  • Ideally this common interface should be flexible enough to allow other ways to specify parameters in the future

I think deprecating -D is better than asking that each -P is matched with one -D, and then having to do all the juggling of matching them when parsing the arguments.

I've read the request asking for zipping parameters (#637) I think it would make the CLI overly complex (which is a thing I like about hyperfine, its simplicity) and usually (at least for me) it's ok when a CLI tool gives you product by default, since it's easy enough to sum different commands at shell level (hyperfine ... && hyperfine ...) . The bit about auto detecting unused parameters could be helpful, but I'm still not convinced.