Compare full request to source of cache instead of comparing commit date

[Wituareard]

Anthony and I came to the conclusion that it probably makes sense to store the whole HTTP request that lead to a localized document instead of comparing commit dates, such that switching models etc. leads to the creation of new localizations.

Does anyone have a good idea for a file structure for this?

[anthonybailey]

Should be clear, I'm not *sure* exact HTTP request is the right abstraction, since it may be fragile to changes that don't invalidate results. (Individual LLM vendor changes some implementation detail, ditto in URL libraries, irrelevant local dev differences add noise.) But curating our own representation of the things we care about seems to have cost too. And I'm not aware of an emergent standard for "call this model, with this prompting, these common parameters and these bespoke parameters." Some attempts at that to enable cross-comparison but not sure any has stuck? Happy to be delightedly wrong on that. My default would be storing requests, but applying a curated equivalence function to judge whether they match. Starts as plain equality, but can be weakened as needed.

…

On Mon, 25 Nov 2024, 11:35 Wituareard, ***@***.***> wrote: Anthony and I came to the conclusion that it probably makes sense to store the whole HTTP request that lead to a localized document instead of comparing commit dates, such that switching models etc. leads to the creation of new localizations. Does anyone have a good idea for a file structure for this? — Reply to this email directly, view it on GitHub <#237>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAXXDPNLPKHMPAG7SYXMYD2CMDQDAVCNFSM6AAAAABSNYTVBOVHI2DSMVQWIX3LMV43ASLTON2WKOZSGY4TANBTGYZTAOI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>