Skip to content

Commit

Permalink
Remove mention of Outlines in the README
Browse files Browse the repository at this point in the history
  • Loading branch information
rlouf committed Oct 25, 2024
1 parent 51dd6be commit 2c4337e
Showing 1 changed file with 1 addition and 19 deletions.
20 changes: 1 addition & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,24 +19,6 @@ Discussions around the performance of different structured generation methods te
Different methods make different trade-offs, and it is important to know when a method is faster than another. We will highlight differences, ideally using minimum pathological examples.


## Explanations
## How benchmarks are run

We do not use models to run the benchmarks, as it would lead to increased runtime, more complex code, and unpredictable generation lengths. We instead take a string in the language of the regular expressions / JSON Schemas, tokenize it and iterate over it pretending these were generated tokens.

### Outlines

If you look at the [benchmarking suite for Outlines](https://github.com/outlines-dev/benchmarks/blob/main/src/outlines.py) you will notice that we execute:

``` python
Regexguide("a", tokenizer)
```

in the initialization phase of the benchmark. This serves two purposes:

1. JIT-compile the functions decorated with `@numba.njit`;
2. Convert vocabulary strings to Numba types.

This only ever needs to be done once, possibly while loading the model, and could be made to disappear using Ahead Of Time compilation. In this benchmarking suite we thus measure:

1. The time it takes to compile the index corresponding to a regular expression;
2. The time it takes to look for valid tokens when generating text.

0 comments on commit 2c4337e

Please sign in to comment.