time could be improved

[jpco]

Some brainstorming about ways es' time builtin could be improved:

1. More precise -- 0.1s is way too large a time quantum to be useful on modern machines, and 1s (for the real-time measurement) is absurd. I find myself running bash -c 'time es -c ''blah''' just to get meaningful timing output, which seems wrong. Milliseconds is probably most appropriate for a shell.

2. Custom formatting -- It could be nice for $&time to simply return a list of (real user system), and have time decide how to format it. We could also have %time in the middle to get a "three-layer" setup like var or whatis. time could be changed to change how times are printed, and %time could be changed to change how times are collected. Then, with a hookable time, you could keep the current whole-seconds and tenth-of-seconds printed formatting despite a more precise $&time if you wanted to maintain the current behavior :)

3. Forkless -- Speaking of changing how times are collected, it seems to me that with both getrusage() and times(), we don't actually need to fork a new process to get a measure of the running time of a block. This is very interesting; whole scripts could be broken up into chunks wrapped in time calls to find slow sections, without changing the functionality of the script at all. This would also, for example, allow the example of hooking time into %pipe in the old es paper to be extended to other control flow constructs. Heck, you could hook time into %dispatch, %seq, or forever without breaking anything! With a hookable time and %time as described above, you could also wrap the timed command in a fork {} if you wanted to maintain the current behavior :)

[memreflect]

1. More precise -- 0.1s is way too large a time quantum to be useful on modern machines, and 1s (for the real-time measurement) is absurd. I find myself running bash -c 'time es -c ''blah''' just to get meaningful timing output, which seems wrong. Milliseconds is probably most appropriate for a shell.

Milliseconds are definitely the way to go, when possible.
clock_gettime(CLOCK_REALTIME) or gettimeofday(), if either are available, could also be used in place of time() for higher-resolution timing when possible, though clock_gettime() would require linking with -lrt.

There are also potential issues with times() for long-running commands.
POSIX requires CLOCKS_PER_SEC to be 1000000 on XSI systems, which means a 32-bit clock_t can overflow in less than 36 minutes or less than 72 minutes, depending on whether clock_t is signed or unsigned.
Ideally, nobody would time anything for that long, and hopefully no system is using a 32-bit clock_t with such a fine-grained resolution, but obviously there's no way to guarantee that.

On non-XSI systems, CLOCKS_PER_SEC can vary, perhaps even being 10, meaning the system clock only provides 0.1s precision but a 32-bit clock_t can time things for a bit longer than 13 years and 7 months.
On FreeBSD 14 (amd64) for example, the values of CLOCKS_PER_SEC and sysconf(_SC_CLK_TCK) are both 128, and clock_t is indeed a signed 32-bit integer, which means approximately 1 year and 23 days can elapse before clock_t overflows.

times(2) on Linux additionally recommends avoiding the return value of times() due to variations across Linux kernel versions and overflow issues.
Instead, time() or gettimeofday() or whatever could be used as with the getrusage() implementation for measuring real time.

Or BUILTIN_TIME could simply be unavailable on systems without getrusage(), and the issues with times() simply disappear.

2. Custom formatting -- It could be nice for $&time to simply return a list of (real user system), and have time decide how to format it. We could also have %time in the middle to get a "three-layer" setup like var or whatis. time could be changed to change how times are printed, and %time could be changed to change how times are collected. Then, with a hookable time, you could keep the current whole-seconds and tenth-of-seconds printed formatting despite a more precise $&time if you wanted to maintain the current behavior :)

I'm not sure how you get from (1 0.000000 0.003459) with $&time to 1r 0.0u 0.0s {cmd} with time using es alone, unless you are suggesting one could use printf(1) or bc(1) or whatever to accomplish this sort of thing?

%time cmd would need to return the times followed by the results of cmd at minimum; this way, time cmd would be able to print the times returned and provide the correct result to avoid the script breaking.

It's unclear how something like time {throw error foo bar} should behave, however.
Such behavioral customization is what makes %time a good idea, even if it has nothing to do with formatting.

3. Forkless

This looks easy enough to do, and i definitely like this idea, but it needs a bit more thought with respect to exception handling.
That said, the current times() implementation forks twice when it only needs to fork once, like the getrusage() implementation.
Of course, if it is made forkless, then this doesn't matter.

Keep in mind that tms_utime/ru_utime and tms_stime/ru_stime has nothing to do with child processes, so you would need (self_end - self_start) + (child_end - child_start) for proper statistics, and even then it might not be entirely accurate, but it would be "good enough", and it would mean one can do as you said, timing specific blocks without interrupting how they work.

[jpco]

After a little time spent hacking up a version of this (see jpco/es-shell@bugfixes...jpco:es-shell:timier -- please excuse my horrible timeval math), I think you're basically right on all counts.

clock_gettime(CLOCK_MONOTONIC) and variants such asCLOCK_MONOTONIC_RAW seems like the real winner in terms of measuring durations without, for example, getting negative runtimes during leap seconds. Unfortunately CLOCK_MONOTONIC isn't totally portable, and it seems like there isn't a clean-and-easy static way to check if it works on a machine, so there's some autoconf logic that may be required to handle that. Also, as you point out, on some systems -lrt is required for clock_gettime, so that's a little more autoconfing to do. Plus, all of this raises the question of fallbacks; I guess we just fall back to clock_gettime(CLOCK_REALTIME), which is standard, for systems that don't support the monotonic clock, and say "sorry you got a negative number, patches welcome"?

%time cmd would need to return the times followed by the results of cmd at minimum

Yup, that's exactly what I did. Excerpted from my hacky version's initial.es:

fn-%time = $&time
fn time cmd {
	let ((r u s result) = <={%time $cmd}) {
		echo <={%flatten \t $r^r $u^u $s^s $^cmd}
		result $result
	}
}

Formatting, as you point out, is a challenge here; pattern extraction and matching can be used to round from xx.yyy to xx.y (for people who want that), but I don't have any good way in mind to replicate the %6ld format strings that are used currently in $&time. On my machine at least printf can do it, but I don't know if it's standard, and I don't think I'd want to put printf in initial.es.

exception handling

There's some design work to be done with that, yeah. Some thoughts off the top of my head:

• A "transparent" $&time which lets you replace any {...} with a {$&time {...}} without side effects would need to make sure any interior exceptions are passed through.
• A $&time which communicates timing data in its return value would not be able to do so if it throws an exception.
• It would be theoretically possible to "wrap" the exception with timing data in $&time and "unwrap" it for use in %time and/or time, but that kind of exception-wrapping isn't a pattern currently used in the shell, so that would require some thinking-through to introduce. (I have something similar proposed in Minor overhaul of history writing #65 now, but queasy feelings over that exception wrapping thing is one of the main reasons I haven't merged that PR yet!)
• Alternatively, and more simply, we could just drop the timing data from $&time if the body throws an exception. Users can configure a catcher ($&time catch @ e {echo $e} {$body}) or just do a $&time fork {$body} if they want to time exception-throwing behavior. This is a little compromised (you always lose at least one of: "pure" timing of the body, forkless operation, or getting timing data on exceptions), but it's a much simpler change to make than introducing a whole new exception-wrapping pattern. On the other hand, there's another PR already in the works that "wants" to introduce exception-wrapping already, so maybe it's worth giving that a real consideration now.
• Even more simply we could just let $&time keep printing its timing data to stderr.

even then it might not be entirely accurate

Do you have any more detail on this? Is it just a matter of compounding errors while doing math (which I think would all get shaved off anyway converting from us or ns to ms), or is there some risk of the system double-counting or failing to count parent and child resources in some way?

[memreflect]

After a little time spent hacking up a version of this (see jpco/[email protected]:es-shell:timier -- please excuse my horrible timeval math), I think you're basically right on all counts.

clock_gettime(CLOCK_MONOTONIC) and variants such asCLOCK_MONOTONIC_RAW seems like the real winner in terms of measuring durations without, for example, getting negative runtimes during leap seconds. Unfortunately CLOCK_MONOTONIC isn't totally portable, and it seems like there isn't a clean-and-easy static way to check if it works on a machine, so there's some autoconf logic that may be required to handle that. Also, as you point out, on some systems -lrt is required for clock_gettime, so that's a little more autoconfing to do. Plus, all of this raises the question of fallbacks; I guess we just fall back to clock_gettime(CLOCK_REALTIME), which is standard, for systems that don't support the monotonic clock, and say "sorry you got a negative number, patches welcome"?

clock_gettime() itself is optional in POSIX.1-2001, so a configure test is needed for that.
If it succeeds, then a test for CLOCK_MONOTONIC with a fallback to CLOCK_REALTIME is ideal.
If clock_gettime() isn't available, gettimeofday() is a good fallback, with the final fallback being time() as it is currently used.

Formatting, as you point out, is a challenge here; pattern extraction and matching can be used to round from xx.yyy to xx.y (for people who want that), but I don't have any good way in mind to replicate the %6ld format strings that are used currently in $&time. On my machine at least printf can do it, but I don't know if it's standard, and I don't think I'd want to put printf in initial.es.

printf should definitely stay out of initial.es; if someone really wants to do this, it can be done using awk or sed or whatever.

exception handling

• A "transparent" $&time which lets you replace any {...} with a {$&time {...}} without side effects would need to make sure any interior exceptions are passed through.

That should be easy enough to do since it's only the implementation of $&catch that handles retry exceptions, instead of the ExceptionHandler...EndException code itself—save the exception and throw it after printing the timing data, else return the result.

• A $&time which communicates timing data in its return value would not be able to do so if it throws an exception.

This is why, after some reflection, i'm not in favor of $&time changing its behavior too much.
Unlike a "real" programming language, printing the timing data even when an exception occurs is what i would expect from a shell.
I don't think returning timing data, or even %time, would serve any truly useful purpose anyway aside from omitting one or more fields, and that could be done with a flag, as could changing the timing resolution.

• It would be theoretically possible to "wrap" the exception with timing data in $&time and "unwrap" it for use in %time and/or time, but that kind of exception-wrapping isn't a pattern currently used in the shell, so that would require some thinking-through to introduce. (I have something similar proposed in Minor overhaul of history writing #65 now, but queasy feelings over that exception wrapping thing is one of the main reasons I haven't merged that PR yet!)

I think your exception-wrapping variation in #65 is suitable in the context of $&parse, but doing it for $&time as well seems like an argument in favor of returning errors instead of throwing exceptions, and es having flattened lists makes returning results and errors together a bad idea in the general case.
I also don't believe a %time hook is worth the added complication of doing things this way.
Clearly i'm not a fan of this idea.

• Alternatively, and more simply, we could just drop the timing data from $&time if the body throws an exception. Users can configure a catcher ($&time catch @ e {echo $e} {$body}) or just do a $&time fork {$body} if they want to time exception-throwing behavior. This is a little compromised (you always lose at least one of: "pure" timing of the body, forkless operation, or getting timing data on exceptions), but it's a much simpler change to make than introducing a whole new exception-wrapping pattern. On the other hand, there's another PR already in the works that "wants" to introduce exception-wrapping already, so maybe it's worth giving that a real consideration now.

Requiring a catcher to avoid interrupting $&time is, of course, an option, and it's an established pattern already.
It's a good alternative if it's desirable to return timing data, but if $&time continues to print timing data, then a catcher is unnecessary, and the command printed by $&time is all the shorter without it.

• Even more simply we could just let $&time keep printing its timing data to stderr.

I definitely favor this idea.
It feels like the best behavior for a shell.

even then it might not be entirely accurate

Do you have any more detail on this? Is it just a matter of compounding errors while doing math (which I think would all get shaved off anyway converting from us or ns to ms), or is there some risk of the system double-counting or failing to count parent and child resources in some way?

The invocation order of the various timing commands could have an effect upon the result of getrusage(RUSAGE_SELF).
For example, getrusage(RUSAGE_CHILDREN) is always unaffected by getrusage(RUSAGE_SELF), but getrusage(RUSAGE_SELF) might be affected by getrusage(RUSAGE_CHILDREN).
A stack-like invocation order of REAL -> CHILDREN -> SELF -> eval() -> SELF -> CHILDREN -> REAL would be the most accurate, with eval() executed between the start and end times, but the difference, if any, is probably too insignificant to matter (<10ms, likely <1ms), so i wouldn't worry too much about this.

[jpco]

Alright, I think it's reasonable to toss out the "returned timing data" idea. There are two things we'd lose (or would need to get another way if we didn't want to lose them):

• customizable formatting
• the ability to save timing info to a variable forklessly

but I don't feel terribly strongly about either of those, so it's probably fine to leave them. The only concrete use case I can imagine for the second one would be a bash-like times builtin, and I don't even really know the use-case for that builtin! It's also conceptually straightforward to switch to, if in the future we wish we had it.

That does still leave open the question of how we'd want to format the output. My first instinct would be to leave formatting as it currently is, just with more significant digits.

clock_gettime() itself is optional in POSIX.1-2001, so a configure test is needed for that.

Ugh, what a mess! This is what I get trying to mess around with a standard for which I don't actually have the text. You know, bash (the only shell with a time builtin whose source I have handy) just uses gettimeofday() with a fallback to time(). That seems reasonable to me at least as a first pass.

Because I can't help myself, I kind of want to have the printed precision depend on which function is used. Awkward and misleading to have a time like x.000 printed for every command if we're using time(). Similar for times(), though times() is complicated even further by the existence of sysconf(_SC_CLK_TCK). We'll see how all that goes.

A stack-like invocation order of REAL -> CHILDREN -> SELF -> eval() -> SELF -> CHILDREN -> REAL would be the most accurate, with eval() executed between the start and end times, but the difference, if any, is probably too insignificant to matter (<10ms, likely <1ms), so i wouldn't worry too much about this.

Aha, yes, you're right the ordering matters. But do you mean CHILDREN -> SELF -> REAL -> eval() -> REAL -> SELF -> CHILDREN? The real-time durations would be most sensitive to extra code being executed, wouldn't they?

[memreflect]

Alright, I think it's reasonable to toss out the "returned timing data" idea. There are two things we'd lose (or would need to get another way if we didn't want to lose them):

• customizable formatting
• the ability to save timing info to a variable forklessly

but I don't feel terribly strongly about either of those, so it's probably fine to leave them. The only concrete use case I can imagine for the second one would be a bash-like times builtin, and I don't even really know the use-case for that builtin! It's also conceptually straightforward to switch to, if in the future we wish we had it.

That does still leave open the question of how we'd want to format the output. My first instinct would be to leave formatting as it currently is, just with more significant digits.

Yes, i think milliseconds is precise enough for timing things.
A flag could be added later to allow customization of the formatting, with -f '%6Rr %5.1Uu %5.1Ss'\t'%C' corresponding to the current behavior.
Saving timing info could similarly be done using something like -v VARNAME, but i don't think that is an immediate concern.

clock_gettime() itself is optional in POSIX.1-2001, so a configure test is needed for that.

Ugh, what a mess! This is what I get trying to mess around with a standard for which I don't actually have the text.

You can use the Single Unix Specification starting with version 3.
SUSv3 is simultaneously

• a revision of the IEEE 1003.1-1996 (POSIX.1-1996) and IEEE 1003.2-1992 (POSIX.2-1992) standards,
• version 3 of the Single Unix Specification, and
• issue 6 of the X/Open Portability Guidelines (XPG).

Unfortunately, i don't know of any links to the original 2001 edition without the two TCs that make up POSIX.1-2004 (IEEE 1003.1-2001/Cor 2-2004), but i would expect most operating systems from 20 years ago to have been patched at some point, including changes to align with the updated standard, or upgraded to a newer release that supports it or even POSIX.1-2008.

Anything without notation, like sysconf(), or with a [CX]> ... <[CX] notation, like the Errors section of malloc(), just means it's an extension to the C standard and is required by POSIX.
Any other notations like [XSI]> ... <[XSI] or [TMR]> ... <[TMR] are optional and not required by POSIX.

You know, bash (the only shell with a time builtin whose source I have handy) just uses gettimeofday() with a fallback to time(). That seems reasonable to me at least as a first pass.

I find it reasonable as well, especially since clock_gettime() requires linking with -lrt.
gettimeofday() on the other hand is still optional because it's marked with the XSI notation, so it might need a configure test like getrusage()—if getrusage() is available, then gettimeofday() is almost certainly available as well.

A stack-like invocation order of REAL -> CHILDREN -> SELF -> eval() -> SELF -> CHILDREN -> REAL would be the most accurate, with eval() executed between the start and end times, but the difference, if any, is probably too insignificant to matter (<10ms, likely <1ms), so i wouldn't worry too much about this.

Aha, yes, you're right the ordering matters. But do you mean CHILDREN -> SELF -> REAL -> eval() -> REAL -> SELF -> CHILDREN? The real-time durations would be most sensitive to extra code being executed, wouldn't they?

I believe the real-time duration should include CHILDREN and SELF because fetching those is a part of timing things (i.e. the total time to execute and measure the user and system times).
es already does that with time() when using getrusage() anyway, and both Bash and ksh93 appear to do the same thing.

[jpco]

A flag could be added later to allow customization of the formatting, with -f '%6Rr %5.1Uu %5.1Ss'\t'%C' corresponding to the current behavior.
Saving timing info could similarly be done using something like -v VARNAME, but i don't think that is an immediate concern.

For the record, I don't like either of these (they remind me too much of the constructs from other shells that drove me to es in the first place!)

I believe the real-time duration should include CHILDREN and SELF because fetching those is a part of timing things (i.e. the total time to execute and measure the user and system times).
es already does that with time() when using getrusage() anyway, and both Bash and ksh93 appear to do the same thing.

I find this counterintuitive (my thinking is that if I wanted to time the timing machinery, I could just use something like time time ...), but it does seem to be the common behavior, so maybe it's me that's off. I presume the difference is completely negligible anyway.

[memreflect]

A flag could be added later to allow customization of the formatting, with -f '%6Rr %5.1Uu %5.1Ss'\t'%C' corresponding to the current behavior.
Saving timing info could similarly be done using something like -v VARNAME, but i don't think that is an immediate concern.

For the record, I don't like either of these (they remind me too much of the constructs from other shells that drove me to es in the first place!)

Completely understandable.
I don't see any particular reason for either, so that was a sort of compromise between the idea of providing a way to modify or capture timing info, even when an exception occurs.

I believe the real-time duration should include CHILDREN and SELF because fetching those is a part of timing things (i.e. the total time to execute and measure the user and system times).
es already does that with time() when using getrusage() anyway, and both Bash and ksh93 appear to do the same thing.

I find this counterintuitive (my thinking is that if I wanted to time the timing machinery, I could just use something like time time ...), but it does seem to be the common behavior, so maybe it's me that's off. I presume the difference is completely negligible anyway.

Historically, the real time has always been greater than the user and system times, but your idea seems to be that the real time is the amount of time spent executing the task itself, which is reasonable and is likely to lead to the same outcome anyway.
It's possible that gettimeofday() or whatever affects RUSAGE_SELF, but as you presumed, the difference is negligible at best, so i don't feel strongly one way or another.