You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, the only method in insert_evals relies on the search method. That means that the synthetic dataset generated is dependent on the search and data processing that was used at the time of generation.
This can be an issue since in many cases you want to benchmark different embedding / data processing methods, and right now the dataset will favour methods that were more in line with whatever method was used to generate the evals, rather than the best ones.
One way we could avoid this is by using a vision model to look at a page and generate some questions pertaining to it.
The text was updated successfully, but these errors were encountered:
Currently, the only method in insert_evals relies on the search method. That means that the synthetic dataset generated is dependent on the search and data processing that was used at the time of generation.
This can be an issue since in many cases you want to benchmark different embedding / data processing methods, and right now the dataset will favour methods that were more in line with whatever method was used to generate the evals, rather than the best ones.
One way we could avoid this is by using a vision model to look at a page and generate some questions pertaining to it.
The text was updated successfully, but these errors were encountered: