Syntax extensions

[jpco]

I've been thinking about user-extensible syntax lately and I have one more wacky idea to round out 2024.

I've been becoming less and less sure that a fully-general user-extensible syntax is possible to do in es, at least using a syntax that actually resembles es itself.

You could do something based on matching the most common patterns of syntax, like "SYMBOL word => {%hook $word}", "cmd1 SYMBOL cmd2 => {%hook {$cmd1} {$cmd2}}", but that concept has a lot of open questions still.

But that's all just context for what I'm actually concretely proposing here, which is simply a user-extensible syntax on $var words. This would build on $#var and $^var syntax, allowing other dollar-non-word characters to directly follow the $ and translate into hook functions as well. Using the syntax from the proof-of-concept I've written, the existing two forms appear as:

$&varsyn '#' x {%count $x}
$&varsyn '^' x {%flatten ' ' $x}

This could be used for a few different uses. For example, to do something like the ${foo?} in posix shells, one could have

$&varsyn '?' x {%nonempty x $x}

and have

fn %nonempty name value {
	if {~ $#value 0} {
		throw error %nonempty 'variable '^$name^' is not set'
	} {
		result $value
	}
}

With this you can use $?blah without needing to worry about if it's "unset". (Note that this $&varsyn setup allows one to pass the variable name itself (and not just the value of the variable) to the hook function... actually, since this works by munging the AST, you can even do funky things like

; $&varsyn '>' n {result $path(n)}
; echo $>(3 ... 5)
/usr/local/sbin /usr/local/bin /usr/bin

Neat!)

Another practical use-case could be something similar to csh-style history substitution with things like $!c, $!e, $!(s string), and a function that takes these single-character commands and fetches the appropriate history entry/sub-entry.

In theory this could also be used for job control specifications like the csh-style %, if/when job control is finally added.

Now, I'm actually not sure whether this would be a good idea to add to the shell or not. Some pros and cons:

Pros

• Fits neatly within the grammar and parser as-is
• Easy to implement compared to other syntax extension systems
• Unlocks a few pretty useful specific cases for user-supplied syntax

Cons

• Very limited -- only extends this kind of "modified variable reference" syntax
• Because it's hard to imagine more general syntax extension, it's hard to design this in a "forward-compatible" way
• Adds more abstruse symbology to the shell. $?(blah foo) is kinda gross, just like ${blah?foo}
• A challenge (not a con per se) is how to externalize these syntax extensions. My proof-of-concept hacks it in with a settor variable, but it's a kludge
• Another challenge is how syntax extensions interact with things like lexical bindings -- it seems ideal to have lexically-scoped syntax extensions, but that would require the extension "primitive" to run at parse time! Maybe that's the best choice in general?

Anyway, this is just an idea I had and was pleasantly surprised that I was able to hack it up.