docs(book): Add test set chapters

docs/book/src/SUMMARY.md

@@ -9,17 +9,17 @@
 
 # Guides
 - [Automation]()
-- [Using Test Sets]()
+- [Using Test Sets](./guides/test-sets.md)
 - [Setting Up CI]()
 
 # Reference
 - [Tests]()
   - [Reference Kinds]()
   - [Annotations]()
   - [Directory Structure]()
-- [Test Set Language]()
-  - [Grammar]()
-  - [Built-in Test Sets]()
+- [Test Set Language](./test-sets/README.md)
+  - [Grammar](./test-sets/grammar.md)
+  - [Built-in Test Sets](./test-sets/built-in.md)
 - [Configuration Schema]()
   - [Template]()
   - [Hooks]()

docs/book/src/guides/test-sets.md

@@ -0,0 +1,100 @@
+# Using Test Sets
+
+## Why Tests Sets
+Many operations such as running, comparing, removing or updating tests all need to somehow select which tests to operate on.
+To avoid having lots of hard-to-remember options, which may or may not interact well, `typst-test` offers an expression based set language which is used to select tests.
+Instead of writing
+
+```bash
+tt run --regex --mod 'foo-.*' --name 'bar/baz' --no-ignored
+```
+
+`typst-test` can be invoked like
+
+```bash
+tt run --expression '(mod(/foo-.*/) & !ignored) | name(=bar/baz)'
+```
+
+This comes with quite a few advantages:
+- it's easier to compose multiple identifier filters like `mod` and `name`
+- options are ambiguous whether they apply to the next option only or to all options like `--regex`
+- with options it's unclear how to compose complex relations like `and` vs `or` of other options
+- test set expressions are visually close to the filter expressions they describe, their operators are deiberately chosen to feel like witing a predicate which is applied over all tests
+
+Let's first disect what this expression actually means:
+`(mod(/foo-.*/) & !ignored) | id(=bar/baz)`
+
+1. We have a top-level binary expression like so `a | b`, this is a union expression, it includes all tests found in either `a` or `b`.
+1. The right expression is `id(=bar/baz)`, this includes all tests who's full identifier matches the given pattern `=bar/baz`.
+   That's an exact matcher (indicated by `=`) for the test identifier `bar/baz`.
+   This means that whatever is on the left of your union, we also include the test `bar/baz`.
+1. The left expression is itself a binary expression again, this time an intersection.
+   It consists of another matcher test set and a complement.
+   1. The name matcher is only applied to modules this time, indiicated by `mod` and uses a regex matcher (delimited by `/`).
+      It includes all tests who's module identifier matches the given regex.
+   1. The complement `!ignored` includes all tests which are not marked as ignored.
+
+Tying it all together, we can describe what this expression matches in a sentence:
+
+> Select all tests which are not marked ignore and are inside a module starting with `foo-`, include also the test `bar/baz`.
+
+Trying to describe this relationship using options on the command line would be cumbersome, error prone and, depending on the options present, impossible.
+
+## Default Test Sets
+Many operations take either a set of tests as positional arguments, which are matched exactly, or a test set expression.
+If neither are given the `default` test set is used, which is itself a shorthand for `!ignored`.
+
+<div class="warning">
+
+This may change in the future, commands my get their own, or even configurable default test sets.
+See [#40](https://github.com/tingerrr/typst-test/issues/40).
+
+</div>
+
+More concretely given the invocation
+```bash
+tt list test1 test2 ...
+```
+
+is equivalent to the following invocation
+
+```txt
+tt list --expression 'default & (id(=test1) | id(=test2) | ...)'
+```
+
+## An Iterative Example
+Suppose you had a project with the following tests:
+```
+mod/sub/foo ephemeral  ignored
+mod/sub/bar ephemeral
+mod/sub/baz persistent
+mod/foo     persistent
+bar         ephemeral
+baz         persistent ignored
+```
+
+and you wanted run only ephemeral tests in `mod/sub`.
+You could construct a expression with the following steps:
+
+1. Firstly, filter out all ignored tests, `typst-test` does by default, but once we use our own expression we must include this restriction ourselves.
+   Both of the following would work.
+   - `default & ...`
+   - `!ignored & ...`
+   Let's go with `default ` to keep it simple.
+1. Now include only those tests which are ephemeral, to do this, add the ephemeral test set.
+   - `default & ephemeral`
+1. Now finally, restrict it to be only tests which are in `mod/sub` or it's sub modules.
+   You can do so by adding any of the following identifier matchers:
+   - `default & ephemeral & mod(~sub)`
+   - `default & ephemeral & mod(=mod/sub)`
+   - `default & ephemeral & id(/^mod\/sub/)`
+
+You can iteratively test your results with `typst-test list -e '...'` until you're satisfied.
+Then you can run whatever operation you want with the same expression. IF it is a destructive operation, i.e. one that writes chaanges to non-temporary files, then you must also pass `--all` if your test set contains more than one test.
+
+## Scripting
+If you build up test set expressions programmatically, consider taking a look at the built-in test set constants.
+Specifically the `all` and `none` test sets can be used as identity sets for certain operators, possibly simplifying the code generating the test sets.
+
+Some of the syntax used in test sets may interfere with your shell, especially the use of whitespace.
+Use non-interpreting quotes around the test set expression (commonly single quotes `'...'`) to avoid interpreting them as shell specific sequences.

docs/book/src/test-sets/README.md

@@ -0,0 +1,32 @@
+# Test Set Language
+The test set language is an expression based language, top level expression can be built up form smaller expressions consisting of binary and unary operators and built-in functions and constants.
+
+## Evaluation
+Test set expressions restrict the set of all tests which are contained in the expression and are compiled to an AST which check against all tests.
+A test set such as `!ignored` would be checked against each test that is found by reading its annotations and filtering all tests out which do have an ignored annotation.
+While the order of some operations like union and intersection doesn't matter semantically, the left operand is checked first for those where short circuiting can be applied.
+The expression `!ignored & id(/complicated regex/)` is more efficient than `id(/complicated regex/) & !ignored`, since it will avoid the regex check for ignored tests entirely.
+This may change in the future if optimizations are added for test set expressions.
+
+## Operators
+Test set expressions can be composed using binary and unary operators.
+
+|Type|Prec.|Name|Symbols|Explanation|
+|---|---|---|---|---|
+|infix|1|union|`∪`, <code>&vert;</code> , `+`, `or`|Includes all tests which are in either the left OR right test set expression.|
+|infix|1|difference|`\`, `-` [^diff-lit]|Includes all tests which are in the left but NOT in the right test set expression.|
+|infix|2|intersection|`∩`, `&`, `and`|Includes all tests which are in both the left AND right test set expression.|
+|infix|3|symmetric difference|`Δ`, `^`, `xor`|Includes all tests which are in either the left OR right test set expression, but NOT in both.|
+|prefix|4|complement|`¬`, `!`, `not`|Includes all tests which are NOT in the test set expression.|
+
+Be aware of precedence when combining different operators, higher precedence means operators bind more strongly, e.g. `not a and b` is `(not a) and b`, not `not (a and b)` because `not` has a higher precedence than `and`.
+Binary operators are left associative, e.g. `a - b - c` is `(a - b) - c`, not `a - (b - c)`.
+When in doubt, use parenthesis to force precedence.
+
+
+## Sections
+- [Grammar](grammar.md) defines the formal grammar using EBNF [^ebnf].
+- [Built-in Test Sets](built-in.md) lists built-in test sets and functions.
+
+[^ebnf]: Extended Backus-Naur-Form
+[^diff-lit]: There is currently no literal difference operator such as `and` or `not`.

docs/book/src/test-sets/built-in.md

@@ -0,0 +1,35 @@
+# Built-in Test Sets
+
+
+## Constants
+The following constants are available, they can be written out in place of any expression.
+
+|Name|Explanation|
+|---|---|
+|`none`|Includes no tests.|
+|`all`|Includes all tests.|
+|`ignored`|Includes tests with an ignored annotation|
+|`compile-only`|Includes tests without references.|
+|`ephemeral`|Includes tests with ephemeral references.|
+|`persistent`|Includes tests with persistent references.|
+|`default`|A shorthand for `!ignored`, this is used as a default if no test set is passed.|
+
+## Functions
+The following functions operate on identifiers using matchers.
+
+|Name|Example|Explanation|
+|---|---|---|
+|`id`|`id(=mod/name)`|Includes tests who's full identifier matches the pattern.|
+|`mod`|`mod(/regex/)`|Includes tests who's module matches the pattern.|
+|`name`|`name(~foo)`|Includes tests who's name matches the pattern.|
+|`custom`|`custom(foo)`|Includes tests with have a `custom` annotation and the given identifier.|
+
+## Matchers
+Matchers are patterns which are checked against identifiers.
+
+|Name|Example|Explanation|
+|---|---|---|
+|`=exact`|`=mod/name`|Matches by comparing the identifier exactly to the given term.|
+|`~contains`|`~plot`|Matches by checking if the given term is contained in the identifier.|
+|`/regex/`|`/mod-[234]\/.*/`|Matches using the given regex, literal slashes `/` and backslashes `\` must be escaped using a back slash `\\`.|
+

docs/book/src/test-sets/grammar.md

@@ -0,0 +1,82 @@
+# Grammar
+The test set expression entrypoint rule is the `main` node.
+The nodes `SOI` and `EOI` stand for start of input and end of input respectively.
+
+```ebnf
+main ::=
+  SOI
+  WHITESPACE*
+  expr
+  WHITESPACE*
+  EOI
+  ;
+
+expr ::=
+  unary_operator*
+  term
+  (
+    WHITESPACE*
+    binary_operator
+    WHITESPACE*
+    unary_operator*
+    term
+  )*
+  ;
+
+atom ::= func | val ;
+term ::= atom | "(" expr ")" ;
+
+unary_operator ::= complement ;
+
+complement ::= "¬" | "!" | "not" ;
+
+binary_operator ::=
+  intersect
+  | union
+  | difference
+  | symmetric_difference
+  ;
+
+intersect ::= "∩" | "&" | "and" ;
+union ::= "∪" | "|" | "or" | "+" ;
+difference ::= "\\" | "-" ;
+symmetric_difference ::= "Δ" | "^" | "xor" ;
+
+val ::= id ;
+func ::= id args ;
+
+args ::= "(" arg ")" ;
+arg ::= matcher ;
+
+matcher =
+  exact_matcher
+  | contains_matcher
+  | regex_matcher
+  | plain_matcher
+  ;
+
+exact_matcher ::= "=" name
+contains_matcher ::= "~" name
+regex_matcher ::= "/" regex "/"
+plain_matcher ::= name
+
+
+id ::=
+  ASCII_ALPHA
+  (ASCII_ALPHANUMERIC | "-" | "_")*
+  ;
+
+name ::=
+  ASCII_ALPHA
+  (ASCII_ALPHANUMERIC | "-" | "_" | "/")*
+  ;
+
+regex ::=
+  (
+    ( "\\" "/")
+    | (!"/" ANY)
+  )+
+  ;
+
+WHITESPACE ::= " " | "\t" | "\r" | "\n" ;
+```

docs/test-set-dsl.typ

@@ -1,99 +1 @@
-#set table(stroke: none)
-#show table: block.with(stroke: (top: 1pt, bottom: 1pt))
 
-= Test sets
-The test set expression DSL is used to get fine control over selecting tests for operations without requiring a lot of complicated options.
-Test sets binary or unary nested expressions of sets using set operators.
-
-If no test set is specified by the user, but individual test identifiers are passed they are used as overrides for the `default` test set.
-More concretely given the invocation `typst-test list test1 test2 ...` the following test set is constructed: `default & (id(=test1) | id(=test2) | ...)`.
-
-= Special test sets
-The following built in constants are given and can be used as regular test sets in compound expressions:
-#table(
-  columns: 2,
-  table.header[Name][Explanation],
-  table.hline(stroke: 0.5pt),
-  [`none`],         [Includes no tests.],
-  [`all`],          [Includes all tests.],
-  [`ignored`],      [Includes all tests with an ignored annotation],
-  [`compile-only`], [Includes all tests without references.],
-  [`ephemeral`],    [Includes all tests with ephemeral references.],
-  [`persistent`],   [Includes all tests with persistent references.],
-  [`default`],      [A shorthand for `!ignored`, this is used as a default if no test set is passed.],
-)
-
-= Matcher test sets
-The following matchers are given and are used within `id(...)`, `mod(...)` and `name(...)` to match on the identifiers of tests:
-#let ft = footnote[
-  This may change in the future to allow different defaults for different matchers.
-]
-#table(
-  columns: 3,
-  table.header[Name][Example][Explanation],
-  table.hline(stroke: 0.5pt),
-  [`=exact`],    [`=mod/name`],
-  [Matches exactly one test who's identifier is exactly the contained term.],
-
-  [`~contains`], [`~plot`],
-  [Matches any test which contains the given term in its identifier.],
-
-  [`/regex/`],   [`/mod-[234]\/.*/`],
-  [Matches an tests who's identifier matches the given regex, literal `/` must be escaped using `\`.],
-)
-
-= Operators
-The following binary operators exist and can operatate on any other test set expression:
-#let ft = footnote[There is currently no literal operator for set difference.]
-#table(
-  columns: 6,
-  table.header[Type][Prec.][Name][Symbols][Literal][Explanation],
-  table.hline(stroke: 0.5pt),
-  [infix],  [1], [union],                [`∪`, `|` or `+`], [`or`],
-  [Includes all tests which are in either the left OR right test set expression.],
-
-  [infix],  [1], [difference],           [`\` or `-`],      [#text(red)[---]#ft],
-  [Includes all tests which are in the left but NOT in the right test set expression.],
-
-  [infix],  [2], [intersection],         [`∩` or `&`],      [`and`],
-  [Includes all tests which are in both the left AND right test set expression.],
-
-  [infix],  [3], [symmetric difference], [`Δ` or `^`],      [`xor`],
-  [Includes all tests which are in either the left OR right test set expression, but NOT in both.],
-
-  [prefix], [4], [complement],           [`¬` or `!`],      [`not`],
-  [Includes all tests which are NOT in the test set expression.],
-)
-
-Be aware of precedence when combining different operators, higher precedence means operators bind more strongly, e.g. `not a and b` is `(not a) and b`, not `not (a and b)` because `complement` has a higher precedence than `intersection`.
-Binary operators are left associative, e.g. `a - b - c` is `(a - b) - c`, not `a - (b - c)`.
-
-= Examples
-Suppose you had a project with the following tests:
-```
-...
-mod/sub/foo ephemeral
-mod/sub/bar ephemeral
-mod/sub/baz persistent
-mod/foo     persistent
-mod/bar     ephemeral
-...
-```
-
-And you wanted to make your ephemeral tests in `mod/sub` persistent, you could construct a expression with the following steps:
-
-+ Let's filter out all ignored test as typst-test does by default, this could be done with `!ignored`, but there is the also handy default test set for this.
-  - `default`
-+ We only want ephemeral tests so we add annother intersection with the ephemeral test set.
-  - `default & ephemeral`
-+ Now we finally restrict it to be only test which are in `mod/sub` by adding an identifier matcher test set. Each of the following would work:
-  - `default & ephemeral & mod(~sub)`
-  - `default & ephemeral & mod(=mod/sub)`
-  - `default & ephemeral & id(/^mod\/sub/)`
-
-You can iteratively test your results with `typst-test list -e '...'` until you're satisfied and then do `typst-test update --all -e '...'` with the given expression, the `--all` option is required if you're operating destructively (editing, updating, removing) on more than one test.
-
-= Notes on scripting
-When building expressions programmatically it may serve simplicity to assign a default value to one operand of an n-ary expression. The `all` and `none` test sets can be used as identity sets for certain set operations.
-
-Make sure to use your shell's non-interpreting quotes (often single quotes `'...'`) around the expression to avoid accidentally running.