[WIP] Add new GPU runner for E2E job, incorporate unit tests into existing runner #71
Conversation
nathan-weinberg
force-pushed
the
gpu-runner
branch
3 times, most recently
from
7b5a2d5 to
dfd6430
Compare
nathan-weinberg
force-pushed
the
gpu-runner
branch
3 times, most recently
from
055be45 to
9707f67
Compare
nathan-weinberg
changed the title
nathan-weinberg
force-pushed
the
gpu-runner
branch
3 times, most recently
from
ab0af26 to
4f4be69
Compare
| @@ -1,6 +1,7 @@ | |||
| # eval | |||
|
|
|||
|  | |||
|  | |||
There was a problem hiding this comment.
@nathan-weinberg on line 38, 39 of this file can you mention how to run the tests with pytest individually and as a group?
| python3 -m pip install . | ||
| python3.11 -m pip install . | ||
| # start llama-cpp server | ||
| ilab model download --repository instructlab/granite-7b-lab-GGUF --filename granite-7b-lab-Q4_K_M.gguf |
There was a problem hiding this comment.
Since what we want to do with ilab is pretty basic, I'd suggest we just install instructlab from pypi.
There was a problem hiding this comment.
I would agree except the current pypi package is pretty out-of-date - using it now is just going to require us to change a bunch of stuff when the next release comes out - but once 0.18.0 lands that'll make sense
There was a problem hiding this comment.
I agree we should install from pypi, installing instructlab from pypi won't change the ilab model download command
There was a problem hiding this comment.
@nathan-weinberg we can install the beta releases with --pre and then remove that once 0.18.0 lands
| python3.11 -m pip install . | ||
| # start llama-cpp server | ||
| ilab model download --repository instructlab/granite-7b-lab-GGUF --filename granite-7b-lab-Q4_K_M.gguf | ||
| ilab model serve --model-path /home/runner/.local/share/instructlab/models/granite-7b-lab-Q4_K_M.gguf |
There was a problem hiding this comment.
In the functional tests scripts we shut down the server to clean up. I wonder if we should split all of this into a shell script that manages ilab installation, server startup, pytest running, and server shutdown.
There was a problem hiding this comment.
Really, we shouldn't need ilab at all to run unit or functional tests for this library - it should operate independently of the CLI
There was a problem hiding this comment.
I'm gonna look into some other possible approaches
There was a problem hiding this comment.
Really, we shouldn't need ilab at all to run unit or functional tests for this library - it should operate independently of the CLI
I think this makes sense for unit tests, but functional ones should prob still use a server from ilab etc
| python3 -m pip install . | ||
| python3.11 -m pip install . | ||
| # start llama-cpp server | ||
| ilab model download --repository instructlab/granite-7b-lab-GGUF --filename granite-7b-lab-Q4_K_M.gguf |
There was a problem hiding this comment.
I agree we should install from pypi, installing instructlab from pypi won't change the ilab model download command
| python3.11 -m pip install . | ||
| # start llama-cpp server | ||
| ilab model download --repository instructlab/granite-7b-lab-GGUF --filename granite-7b-lab-Q4_K_M.gguf | ||
| ilab model serve --model-path /home/runner/.local/share/instructlab/models/granite-7b-lab-Q4_K_M.gguf |
There was a problem hiding this comment.
Really, we shouldn't need ilab at all to run unit or functional tests for this library - it should operate independently of the CLI
I think this makes sense for unit tests, but functional ones should prob still use a server from ilab etc
| @@ -53,6 +53,7 @@ coverage.xml | |||
| .hypothesis/ | |||
| .pytest_cache/ | |||
| cover/ | |||
| eval_output/ | |||
| - name: Install ilab | ||
| run: | | ||
| export CUDA_HOME="/usr/local/cuda" | ||
| export LD_LIBRRY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64" |
| # Install the local version of eval before installing the CLI so PR changes are included | ||
| python3.11 -m pip install . | ||
|
|
||
| python3.11 -m pip install instructlab |
There was a problem hiding this comment.
(Question) AFAIU this assumes that instructlab package never caps / pins the eval library in an incompatible way. Otherwise, this line could revert the eval package to the one from pypi. Is it an acceptable assumption?
Use previous GPU runner for unit tests Signed-off-by: Nathan Weinberg <[email protected]>
nathan-weinberg
force-pushed
the
gpu-runner
branch
from
4f4be69 to
772bc74
Compare
mergify
bot
added
CI/CD
|
closing in favor of #155 |
This PR does the following:
Resolves #12