ggml: avoid rebuild of GGML graph for each token (#7456) #199
| Job | Run time |
|---|---|
| 3m 19s | |
| 8m 41s | |
| 14m 48s | |
| 10m 18s | |
| 2m 36s | |
| 1m 41s | |
| 2m 51s | |
| 2m 58s | |
| 2m 38s | |
| 3m 10s | |
| 3m 33s | |
| 6m 38s | |
| 2m 12s | |
| 5m 36s | |
| 2m 34s | |
| 12s | |
| 2m 50s | |
| 23s | |
| 2m 2s | |
| 11s | |
| 10m 15s | |
| 12s | |
| 10m 12s | |
| 1s | |
| 9m 15s | |
| 36m 42s | |
| 8m 36s | |
| 9m 31s | |
| 9m 24s | |
| 40m 22s | |
| 8m 42s | |
| 9m 37s | |
| 8m 59s | |
| 9m 22s | |
| 8m 34s | |
| 16m 18s | |
| 2m 50s | |
| 2m 22s | |
| 1m 33s | |
| 12m 13s | |
| 9m 57s | |
| 0s | |
| 5h 4m 8s |