diff --git a/README.md b/README.md index 51023ba..6cba033 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,8 @@ Python Library for Evaluation ## MT-Bench / MT-Bench-Branch Testing Steps +> **⚠️ Note:** Must use Python version 3.11 or later. + ```shell # Optional: Use cloud-instance.sh (https://github.com/instructlab/instructlab/tree/main/scripts/infra) to launch and setup the instance scripts/infra/cloud-instance.sh ec2 launch -t g5.4xlarge @@ -27,6 +29,13 @@ pip install -r requirements.txt pip install -r requirements-dev.txt pip install -e . pip install vllm + +# Ensure the nvidia-toolkit-setup-service is running +sudo systemctl status nvidia-toolkit-setup.service +# If it's not running +sudo systemctl start nvidia-toolkit-setup.service + +# Now start vllm python -m vllm.entrypoints.openai.api_server --model instructlab/granite-7b-lab --tensor-parallel-size 1 ``` @@ -34,6 +43,7 @@ In another shell window ```shell export INSTRUCTLAB_EVAL_FIRST_N_QUESTIONS=10 # Optional if you want to shorten run times +cd eval python3 tests/test_gen_answers.py python3 tests/test_branch_gen_answers.py ``` @@ -104,4 +114,4 @@ eval_output/ └── reference_answer └── instructlab └── granite-7b-lab.jsonl -``` \ No newline at end of file +```