add example for Llama3 vllm server

[cameron-chen]

Hello, I am creating this PR to share the example of evaluating by local model using API call (vllm server).

I find this approach can be quite useful when:

• server the local annotator on the cluster so that other nodes only need to call API for evaluating.
• the user wants to use "weighted"-style annotator similar with weighted_alpaca_eval_gpt4_turbo.

Please let me know if this looks good. I am happy to add more detailed instructions.

---

To use the API, add the client config local_configs.yaml and activate it:

default:
    - api_key: "token-abc123"
      base_url: "http://localhost:8000/v1"

[YannDubs]

LGTM, can you add a README.md in src/alpaca_eval/evaluators_configs/weighted_alpaca_eval_vllm_llama3_70b/ that explains the setup and goal? thanks!

[cameron-chen]

LGTM, can you add a README.md in src/alpaca_eval/evaluators_configs/weighted_alpaca_eval_vllm_llama3_70b/ that explains the setup and goal? thanks!

Hi @YannDubs , thanks for looking into the PR! I have added README to provide more info regarding setup.

[cameron-chen]

Hi @YannDubs , I wanted to check in on the PR after adding the README as you suggested. If there’s anything more I can clarify or adjust, please let me know. Thanks again for your feedback so far!

[YannDubs]

Look great, thanks @cameron-chen
sorry for the delay on that one