amd build improvements #156
Conversation
|
Skipping CI for Draft Pull Request. |
|
/test all |
dtrifiro
force-pushed
the
amd-build-improvements
branch
from
cf62620 to
64a8df2
Compare
dtrifiro
force-pushed
the
amd-build-improvements
branch
from
64a8df2 to
d7a74df
Compare
| RUN rpm -ivh https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm && \ | ||
| rpm -ql epel-release && \ |
There was a problem hiding this comment.
We need epel for ccache, the -ql listing is not required
Dockerfile.rocm.ubi
Outdated
| ENV CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:/libtorch/include:/libtorch/include/torch/csrc/api/include:/opt/rocm/include | ||
| ENV PYTORCH_ROCM_ARCH="gfx908;gfx90a;gfx942;gfx1100" | ||
| ENV CCACHE_DIR=/root/.cache/ccache | ||
| ENV PYTORCH_ROCM_ARCH=${PYTORCH_ROCM_ARCH} |
There was a problem hiding this comment.
isn't this var used by vllm only later on..?
Dockerfile.rocm.ubi
Outdated
| torch==2.5.0.dev20240726+rocm6.1 \ | ||
| torchvision==0.20.0.dev20240726+rocm6.1 && \ |
There was a problem hiding this comment.
we already installed torch at line 77, do we just copy files from mounted cache?
There was a problem hiding this comment.
Moved the torch install to the rocm_base layer instead
dtrifiro
force-pushed
the
amd-build-improvements
branch
3 times, most recently
from
e5f6c41 to
0f2d1ef
Compare
dtrifiro
force-pushed
the
amd-build-improvements
branch
from
0f2d1ef to
a25f69f
Compare
dtrifiro
force-pushed
the
amd-build-improvements
branch
from
a25f69f to
8f1fcff
Compare
|
@dtrifiro: The following test failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
Smoke test failure is unrelated |
dtrifiro
force-pushed
the
amd-build-improvements
branch
from
8f1fcff to
1e8d6df
Compare
- get rid of non-essential dependencies
- consolidate package installs
- do not copy wheels in final stage
- fix ccache usage
- use flashattention with triton backend by default:
- clone main_perf branch
- build rocm target
- set up triton rocm env var
- configure numba, outlines and triton cache directory
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: dtrifiro, NickLucche The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Dockerfile.rocm.ubi:
flash-attentionwith triton backend by default:https://issues.redhat.com/browse/RHOAIENG-12611