Skip to content

Commit

Permalink
update README
Browse files Browse the repository at this point in the history
Signed-off-by: sallyom <[email protected]>
  • Loading branch information
sallyom committed Aug 26, 2024
1 parent ec709c7 commit c72fc79
Showing 1 changed file with 11 additions and 1 deletion.
12 changes: 11 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ Python Library for Evaluation

## MT-Bench / MT-Bench-Branch Testing Steps

> **⚠️ Note:** Must use Python version 3.11 or later.
```shell
# Optional: Use cloud-instance.sh (https://github.com/instructlab/instructlab/tree/main/scripts/infra) to launch and setup the instance
scripts/infra/cloud-instance.sh ec2 launch -t g5.4xlarge
Expand All @@ -27,13 +29,21 @@ pip install -r requirements.txt
pip install -r requirements-dev.txt
pip install -e .
pip install vllm

# Ensure the nvidia-toolkit-setup-service is running
sudo systemctl status nvidia-toolkit-setup.service
# If it's not running
sudo systemctl start nvidia-toolkit-setup.service

# Now start vllm
python -m vllm.entrypoints.openai.api_server --model instructlab/granite-7b-lab --tensor-parallel-size 1
```

In another shell window

```shell
export INSTRUCTLAB_EVAL_FIRST_N_QUESTIONS=10 # Optional if you want to shorten run times
cd eval
python3 tests/test_gen_answers.py
python3 tests/test_branch_gen_answers.py
```
Expand Down Expand Up @@ -104,4 +114,4 @@ eval_output/
└── reference_answer
└── instructlab
└── granite-7b-lab.jsonl
```
```

0 comments on commit c72fc79

Please sign in to comment.