ggml: avoid rebuild of GGML graph for each token (#7456) #15884
| Job | Run time |
|---|---|
| 11m 56s | |
| 4m 48s | |
| 2m 38s | |
| 3m 42s | |
| 3m 13s | |
| 3m 14s | |
| 2m 37s | |
| 2m 51s | |
| 11m 23s | |
| 3m 7s | |
| 1m 38s | |
| 5m 27s | |
| 1m 41s | |
| 4m 30s | |
| 1m 56s | |
| 2m 52s | |
| 3m 29s | |
| 2m 41s | |
| 18m 16s | |
| 2m 38s | |
| 3m 53s | |
| 4m 25s | |
| 4m 35s | |
| 4m 49s | |
| 5m 26s | |
| 12m 56s | |
| 51m 13s | |
| 28m 4s | |
| 5m 7s | |
| 9m 0s | |
| 43m 33s | |
| 6m 15s | |
| 5m 29s | |
| 12m 47s | |
| 6m 30s | |
| 4m 25s | |
| 1m 24s | |
| 4m 43s | |
| 0s | |
| 3m 20s | |
| 2m 39s | |
| 0s | |
| 5h 15m 10s |