Introduce a bit of randomness in the search results output

[kgilpin]

Consider the two log files that result from identical Navie questions:

search.log

search.log

The search results are identical. Consider adding an option to introduce a bit of randomness to the search results. This way, when a request is being retried, the LLM will see data that's a little different, resulting in a higher probability that the retried answer is different.

Increasing temperature could be another way to get the same effect.