julian/amd-strix-halo-toolboxes
1
0
Fork 0
Code Issues Pull Requests 2 Actions Packages Projects Releases Wiki Activity

feat: upgrade ROCm to 7.1.1 and update associated tooling and documentation

Browse Source
This commit is contained in:
Donato Capitella
2025-12-07 09:30:14 +00:00
parent 7584a31548
commit 9ba6812003
9 changed files with 177 additions and 177 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/build_and_publish.yml
+1 -1
View File
@@ -28,7 +28,7 @@ jobs:
IN='${{ inputs.backends }}' IN='${{ inputs.backends }}'
if [[ "$IN" == "all" || -z "$IN" ]]; then if [[ "$IN" == "all" || -z "$IN" ]]; then
JSON='["rocm-6.4.4","rocm-6.4.4-rocwmma","rocm-7.1","rocm-7.1-rocwmma","rocm-7alpha","rocm-7alpha-rocwmma","rocm-7alpha-rocwmma-improved","rocm-7rc","rocm-7rc-rocwmma","vulkan-amdvlk","vulkan-radv"]' JSON='["rocm-6.4.4","rocm-6.4.4-rocwmma","rocm-7.1.1","rocm-7.1.1-rocwmma","rocm-7alpha","rocm-7alpha-rocwmma","rocm-7alpha-rocwmma-improved","rocm-7rc","rocm-7rc-rocwmma","vulkan-amdvlk","vulkan-radv"]'
else else
# Remove spaces and build JSON array from comma list # Remove spaces and build JSON array from comma list
IN_CLEAN=$(echo "$IN" | tr -d '[:space:]') IN_CLEAN=$(echo "$IN" | tr -d '[:space:]')
README.md
+10 -10
View File
@@ -43,11 +43,11 @@ toolbox create llama-vulkan-radv \
-- --device /dev/dri --group-add video --security-opt seccomp=unconfined -- --device /dev/dri --group-add video --security-opt seccomp=unconfined
``` ```
**Command — Create ROCm toolbox (6.4.4/7.1/7rc/7alpha)** **Command — Create ROCm toolbox (6.4.4/7.1.1/7rc/7alpha)**
```sh ```sh
toolbox create llama-rocm-7.1-rocwmma \ toolbox create llama-rocm-7.1.1-rocwmma \
--image docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-7.1-rocwmma \ --image docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-7.1.1-rocwmma \
-- --device /dev/dri --device /dev/kfd \ -- --device /dev/dri --device /dev/kfd \
--group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined
``` ```
@@ -105,7 +105,7 @@ llama-cli --no-mmap -ngl 999 -fa 1 -m models/qwen3-coder-30B-A3B/BF16/Qwen3-Code
**Command — Refresh specific toolboxes** **Command — Refresh specific toolboxes**
```bash ```bash
./refresh-toolboxes.sh llama-vulkan-radv llama-rocm-7.1-rocwmma ./refresh-toolboxes.sh llama-vulkan-radv llama-rocm-7.1.1-rocwmma
``` ```
## 1. Llama.cpp Compiled for Every Backend ## 1. Llama.cpp Compiled for Every Backend
@@ -125,8 +125,8 @@ You can check the containers on DockerHub: https://hub.docker.com/r/kyuz0/amd-st
| `vulkan-radv` | Vulkan (Mesa RADV) | Most stable and compatible. Recommended for most users and all models. | | `vulkan-radv` | Vulkan (Mesa RADV) | Most stable and compatible. Recommended for most users and all models. |
| `rocm-6.4.4` | ROCm 6.4.4 (HIP) + hipBLASLt* | Latest stable build for ROCm 6.4.4, performs very well with most model architectures/quants. | | `rocm-6.4.4` | ROCm 6.4.4 (HIP) + hipBLASLt* | Latest stable build for ROCm 6.4.4, performs very well with most model architectures/quants. |
| `rocm-6.4.4-rocwmma` | ROCm 6.4.4 + ROCWMMA + hipBLASLt* | 6.4.4 with ROCWMMA enabled for better flash attention on RDNA3+/CDNA. | | `rocm-6.4.4-rocwmma` | ROCm 6.4.4 + ROCWMMA + hipBLASLt* | 6.4.4 with ROCWMMA enabled for better flash attention on RDNA3+/CDNA. |
| `rocm-7.1` | ROCm 7.1 GA (HIP) + hipBLASLt* | Current GA release for ROCm 7.x; improved scheduler and hipBLASLt kernels. | | `rocm-7.1.1` | ROCm 7.1.1 GA (HIP) + hipBLASLt* | Current GA release for ROCm 7.x; improved scheduler and hipBLASLt kernels. |
| `rocm-7.1-rocwmma` | ROCm 7.1 GA + ROCWMMA + hipBLASLt* | 7.1 with ROCWMMA for maximum flash-attention throughput. | | `rocm-7.1.1-rocwmma` | ROCm 7.1.1 GA + ROCWMMA + hipBLASLt* | 7.1.1 with ROCWMMA for maximum flash-attention throughput. |
| `rocm-7rc` | ROCm 7.9 (HIP) + hipBLASLt* | Used to be the release candidate for ROCm 7.9.0 (hence the `rc` tag in the name), now released. | | `rocm-7rc` | ROCm 7.9 (HIP) + hipBLASLt* | Used to be the release candidate for ROCm 7.9.0 (hence the `rc` tag in the name), now released. |
| `rocm-7rc-rocwmma` | ROCm 7.9 + ROCWMMA + hipBLASLt* | 7.9.0 build with ROCWMMA—useful for early flash-attention validation. | | `rocm-7rc-rocwmma` | ROCm 7.9 + ROCWMMA + hipBLASLt* | 7.9.0 build with ROCWMMA—useful for early flash-attention validation. |
| `rocm-7alpha` | ROCm 7 Nightly (“7rc-alpha”) + hipBLASLt* | Tracks ROCm 7 nightly (alpha) preview with bleeding-edge patches. | | `rocm-7alpha` | ROCm 7 Nightly (“7rc-alpha”) + hipBLASLt* | Tracks ROCm 7 nightly (alpha) preview with bleeding-edge patches. |
@@ -180,10 +180,10 @@ toolbox create llama-rocm-7.1-rocwmma \
Ubuntu’s `toolbox` package still breaks GPU access, so follow gyhor’s [issue comment](https://github.com/kyuz0/amd-strix-halo-toolboxes/issues/16#issuecomment-3582028864) and use [Distrobox](https://github.com/89luca89/distrobox) instead: Ubuntu’s `toolbox` package still breaks GPU access, so follow gyhor’s [issue comment](https://github.com/kyuz0/amd-strix-halo-toolboxes/issues/16#issuecomment-3582028864) and use [Distrobox](https://github.com/89luca89/distrobox) instead:
```sh ```sh
distrobox create -n llama-rocm-7.1 \ distrobox create -n llama-rocm-7.1.1 \
--image docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-7.1-rocwmma \ --image docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-7.1.1-rocwmma \
--additional-flags "--device /dev/kfd --device /dev/dri --group-add video --group-add render --security-opt seccomp=unconfined" --additional-flags "--device /dev/kfd --device /dev/dri --group-add video --group-add render --security-opt seccomp=unconfined"
distrobox enter llama-rocm-7.1 distrobox enter llama-rocm-7.1.1
llama-cli --list-devices llama-cli --list-devices
``` ```
@@ -206,7 +206,7 @@ This will:
You can also refresh just one or more toolboxes: You can also refresh just one or more toolboxes:
```bash ```bash
./refresh-toolboxes.sh llama-vulkan-radv llama-rocm-7.1-rocwmma ./refresh-toolboxes.sh llama-vulkan-radv llama-rocm-7.1.1-rocwmma
``` ```
### 2.2 Running models inside the toolboxes ### 2.2 Running models inside the toolboxes
benchmark/generate_results.json.py
+1 -1
View File
@@ -43,7 +43,7 @@ LONGCTX_RE = re.compile(r"longctx(\d+)", re.IGNORECASE)
# --- Helpers --------------------------------------------------------------- # --- Helpers ---------------------------------------------------------------
ENV_CANON = { ENV_CANON = {
"rocm7_1": "rocm7.1", "rocm7_1_1": "rocm7.1.1",
"rocm7_alpha": "rocm-7alpha", "rocm7_alpha": "rocm-7alpha",
} }
benchmark/run_benchmarks.sh
+2 -2
View File
@@ -28,8 +28,8 @@ echo
declare -A CMDS=( declare -A CMDS=(
[rocm6_4_4]="toolbox run -c llama-rocm-6.4.4 -- /usr/local/bin/llama-bench" [rocm6_4_4]="toolbox run -c llama-rocm-6.4.4 -- /usr/local/bin/llama-bench"
[rocm6_4_4-rocwmma]="toolbox run -c llama-rocm-6.4.4-rocwmma -- /usr/local/bin/llama-bench" [rocm6_4_4-rocwmma]="toolbox run -c llama-rocm-6.4.4-rocwmma -- /usr/local/bin/llama-bench"
[rocm7.1]="toolbox run -c llama-rocm-7.1 -- /usr/local/bin/llama-bench" [rocm7.1.1]="toolbox run -c llama-rocm-7.1.1 -- /usr/local/bin/llama-bench"
[rocm7.1-rocwmma]="toolbox run -c llama-rocm-7.1-rocwmma -- /usr/local/bin/llama-bench" [rocm7.1.1-rocwmma]="toolbox run -c llama-rocm-7.1.1-rocwmma -- /usr/local/bin/llama-bench"
[rocm-7alpha-rocwmma-improved]="toolbox run -c llama-rocm-7alpha-rocwmma-improved -- /usr/local/bin/llama-bench" [rocm-7alpha-rocwmma-improved]="toolbox run -c llama-rocm-7alpha-rocwmma-improved -- /usr/local/bin/llama-bench"
[rocm-7alpha]="toolbox run -c llama-rocm-7alpha -- /usr/local/bin/llama-bench" [rocm-7alpha]="toolbox run -c llama-rocm-7alpha -- /usr/local/bin/llama-bench"
[rocm-7alpha-rocwmma]="toolbox run -c llama-rocm-7alpha-rocwmma -- /usr/local/bin/llama-bench" [rocm-7alpha-rocwmma]="toolbox run -c llama-rocm-7alpha-rocwmma -- /usr/local/bin/llama-bench"
benchmark/run_rpc_benchmarks.sh
+6 -6
View File
@@ -28,8 +28,8 @@ fi
declare -A TOOLBOX_IMAGES=( declare -A TOOLBOX_IMAGES=(
[rocm6_4_4]="llama-rocm-6.4.4" [rocm6_4_4]="llama-rocm-6.4.4"
[rocm6_4_4-rocwmma]="llama-rocm-6.4.4-rocwmma" [rocm6_4_4-rocwmma]="llama-rocm-6.4.4-rocwmma"
[rocm7_1]="llama-rocm-7.1" [rocm7_1_1]="llama-rocm-7.1.1"
[rocm7_1-rocwmma]="llama-rocm-7.1-rocwmma" [rocm7_1_1-rocwmma]="llama-rocm-7.1.1-rocwmma"
[rocm7_rc]="llama-rocm-7rc" [rocm7_rc]="llama-rocm-7rc"
[rocm7_rc-rocwmma]="llama-rocm-7rc-rocwmma" [rocm7_rc-rocwmma]="llama-rocm-7rc-rocwmma"
[rocm7_alpha]="llama-rocm-7alpha" [rocm7_alpha]="llama-rocm-7alpha"
@@ -42,8 +42,8 @@ declare -A TOOLBOX_IMAGES=(
declare -A CLIENT_CMDS=( declare -A CLIENT_CMDS=(
[rocm6_4_4]="toolbox run -c llama-rocm-6.4.4 -- /usr/local/bin/llama-bench" [rocm6_4_4]="toolbox run -c llama-rocm-6.4.4 -- /usr/local/bin/llama-bench"
[rocm6_4_4-rocwmma]="toolbox run -c llama-rocm-6.4.4-rocwmma -- /usr/local/bin/llama-bench" [rocm6_4_4-rocwmma]="toolbox run -c llama-rocm-6.4.4-rocwmma -- /usr/local/bin/llama-bench"
[rocm7_1]="toolbox run -c llama-rocm-7.1 -- /usr/local/bin/llama-bench" [rocm7_1_1]="toolbox run -c llama-rocm-7.1.1 -- /usr/local/bin/llama-bench"
[rocm7_1-rocwmma]="toolbox run -c llama-rocm-7.1-rocwmma -- /usr/local/bin/llama-bench" [rocm7_1_1-rocwmma]="toolbox run -c llama-rocm-7.1.1-rocwmma -- /usr/local/bin/llama-bench"
[rocm7_rc]="toolbox run -c llama-rocm-7rc -- /usr/local/bin/llama-bench" [rocm7_rc]="toolbox run -c llama-rocm-7rc -- /usr/local/bin/llama-bench"
[rocm7_rc-rocwmma]="toolbox run -c llama-rocm-7rc-rocwmma -- /usr/local/bin/llama-bench" [rocm7_rc-rocwmma]="toolbox run -c llama-rocm-7rc-rocwmma -- /usr/local/bin/llama-bench"
[rocm7_alpha]="toolbox run -c llama-rocm-7alpha -- /usr/local/bin/llama-bench" [rocm7_alpha]="toolbox run -c llama-rocm-7alpha -- /usr/local/bin/llama-bench"
@@ -56,8 +56,8 @@ declare -A CLIENT_CMDS=(
ENVIRONMENTS=( ENVIRONMENTS=(
rocm6_4_4 rocm6_4_4
rocm6_4_4-rocwmma rocm6_4_4-rocwmma
rocm7_1 rocm7_1_1
rocm7_1-rocwmma rocm7_1_1-rocwmma
rocm7_rc rocm7_rc
rocm7_rc-rocwmma rocm7_rc-rocwmma
rocm7_alpha rocm7_alpha
refresh-toolboxes.sh
+2 -2
View File
@@ -9,8 +9,8 @@ TOOLBOXES["llama-vulkan-amdvlk"]="docker.io/kyuz0/amd-strix-halo-toolboxes:vulka
TOOLBOXES["llama-vulkan-radv"]="docker.io/kyuz0/amd-strix-halo-toolboxes:vulkan-radv --device /dev/dri --group-add video --security-opt seccomp=unconfined" TOOLBOXES["llama-vulkan-radv"]="docker.io/kyuz0/amd-strix-halo-toolboxes:vulkan-radv --device /dev/dri --group-add video --security-opt seccomp=unconfined"
TOOLBOXES["llama-rocm-6.4.4"]="docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-6.4.4 --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined" TOOLBOXES["llama-rocm-6.4.4"]="docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-6.4.4 --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"
TOOLBOXES["llama-rocm-6.4.4-rocwmma"]="docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-6.4.4-rocwmma --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined" TOOLBOXES["llama-rocm-6.4.4-rocwmma"]="docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-6.4.4-rocwmma --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"
TOOLBOXES["llama-rocm-7.1"]="docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-7.1 --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined" TOOLBOXES["llama-rocm-7.1.1"]="docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-7.1.1 --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"
TOOLBOXES["llama-rocm-7.1-rocwmma"]="docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-7.1-rocwmma --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined" TOOLBOXES["llama-rocm-7.1.1-rocwmma"]="docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-7.1.1-rocwmma --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"
TOOLBOXES["llama-rocm-7rc"]="docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-7rc --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined" TOOLBOXES["llama-rocm-7rc"]="docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-7rc --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"
TOOLBOXES["llama-rocm-7rc-rocwmma"]="docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-7rc-rocwmma --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined" TOOLBOXES["llama-rocm-7rc-rocwmma"]="docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-7rc-rocwmma --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"
TOOLBOXES["llama-rocm-7alpha"]="docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-7alpha --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined" TOOLBOXES["llama-rocm-7alpha"]="docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-7alpha --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"
toolboxes/Dockerfile.rocm-7.1
-111
View File
@@ -1,111 +0,0 @@
# build stage
FROM registry.fedoraproject.org/fedora:43 AS builder
# rocm 7.1 repo
RUN <<'EOF'
tee /etc/yum.repos.d/rocm.repo <<REPO
[ROCm-7.1]
name=ROCm7.1
baseurl=https://repo.radeon.com/rocm/el9/7.1/main
enabled=1
priority=50
gpgcheck=1
gpgkey=https://repo.radeon.com/rocm/rocm.gpg.key
REPO
EOF
# deps
RUN dnf -y --nodocs --setopt=install_weak_deps=False \
--exclude='*sdk*' --exclude='*samples*' --exclude='*-doc*' --exclude='*-docs*' \
install \
make gcc cmake lld clang clang-devel compiler-rt libcurl-devel ninja-build \
rocm-llvm rocm-device-libs hip-runtime-amd hip-devel \
rocblas rocblas-devel hipblas hipblas-devel rocm-cmake libomp-devel libomp \
rocminfo radeontop \
git-core vim sudo rsync \
&& dnf clean all && rm -rf /var/cache/dnf/*
# rocm env
ENV ROCM_PATH=/opt/rocm \
HIP_PATH=/opt/rocm \
HIP_CLANG_PATH=/opt/rocm/llvm/bin \
HIP_DEVICE_LIB_PATH=/opt/rocm/amdgcn/bitcode \
PATH=/opt/rocm/bin:/opt/rocm/llvm/bin:$PATH
# llama.cpp
WORKDIR /opt/llama.cpp
RUN git clone --recursive https://github.com/ggerganov/llama.cpp.git .
# build
RUN git clean -xdf \
&& git submodule update --recursive \
&& cmake -S . -B build \
-DGGML_HIP=ON \
-DAMDGPU_TARGETS=gfx1151 \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_RPC=ON \
-DLLAMA_HIP_UMA=ON \
-DGGML_CUDA_ENABLE_UNIFIED_MEMORY=ON \
-DROCM_PATH=/opt/rocm \
-DHIP_PATH=/opt/rocm \
-DHIP_PLATFORM=amd \
-DCMAKE_HIP_FLAGS="--rocm-path=/opt/rocm" \
&& cmake --build build --config Release -- -j$(nproc) \
&& cmake --install build --config Release
# libs
RUN find /opt/llama.cpp/build -type f -name 'lib*.so*' -exec cp {} /usr/lib64/ \; \
&& ldconfig
# helper
COPY gguf-vram-estimator.py /usr/local/bin/gguf-vram-estimator.py
RUN chmod +x /usr/local/bin/gguf-vram-estimator.py
# runtime stage
FROM registry.fedoraproject.org/fedora-minimal:43
# rocm 7.1 repo
RUN <<'EOF'
tee /etc/yum.repos.d/rocm.repo <<REPO
[ROCm-7.1]
name=ROCm7.1
baseurl=https://repo.radeon.com/rocm/el9/7.1/main
enabled=1
priority=50
gpgcheck=1
gpgkey=https://repo.radeon.com/rocm/rocm.gpg.key
REPO
EOF
# runtime deps
RUN microdnf -y --nodocs --setopt=install_weak_deps=0 \
--exclude='*sdk*' --exclude='*samples*' --exclude='*-doc*' --exclude='*-docs*' \
install \
bash ca-certificates libatomic libstdc++ libgcc libgomp sudo \
hip-runtime-amd rocblas hipblas \
rocminfo radeontop \
&& microdnf clean all && rm -rf /var/cache/dnf/*
# copy
COPY --from=builder /usr/local/ /usr/local/
COPY --from=builder /opt/llama.cpp/build/bin/rpc-* /usr/local/bin/
# ld
RUN echo "/usr/local/lib" > /etc/ld.so.conf.d/local.conf \
&& echo "/usr/local/lib64" >> /etc/ld.so.conf.d/local.conf \
&& ldconfig \
&& cp -n /usr/local/lib/libllama*.so* /usr/lib64/ 2>/dev/null || true \
&& ldconfig
# helper
COPY gguf-vram-estimator.py /usr/local/bin/gguf-vram-estimator.py
RUN chmod +x /usr/local/bin/gguf-vram-estimator.py
# profile
RUN printf '%s\n' \
'export ROCBLAS_USE_HIPBLASLT=1' \
> /etc/profile.d/rocm.sh && chmod +x /etc/profile.d/rocm.sh \
&& echo 'source /etc/profile.d/rocm.sh' >> /etc/bashrc
# shell
CMD ["/bin/bash"]
toolboxes/Dockerfile.rocm-7.1.1
+111
View File
@@ -0,0 +1,111 @@
# build stage
FROM registry.fedoraproject.org/fedora:43 AS builder
# rocm 7.1 repo
RUN <<'EOF'
tee /etc/yum.repos.d/rocm.repo <<REPO
[ROCm-7.1.1]
name=ROCm7.1.1
baseurl=https://repo.radeon.com/rocm/el9/7.1.1/main
enabled=1
priority=50
gpgcheck=1
gpgkey=https://repo.radeon.com/rocm/rocm.gpg.key
REPO
EOF
# deps
RUN dnf -y --nodocs --setopt=install_weak_deps=False \
--exclude='*sdk*' --exclude='*samples*' --exclude='*-doc*' --exclude='*-docs*' \
install \
make gcc cmake lld clang clang-devel compiler-rt libcurl-devel ninja-build \
rocm-llvm rocm-device-libs hip-runtime-amd hip-devel \
rocblas rocblas-devel hipblas hipblas-devel rocm-cmake libomp-devel libomp \
rocminfo radeontop \
git-core vim sudo rsync \
&& dnf clean all && rm -rf /var/cache/dnf/*
# rocm env
ENV ROCM_PATH=/opt/rocm \
HIP_PATH=/opt/rocm \
HIP_CLANG_PATH=/opt/rocm/llvm/bin \
HIP_DEVICE_LIB_PATH=/opt/rocm/amdgcn/bitcode \
PATH=/opt/rocm/bin:/opt/rocm/llvm/bin:$PATH
# llama.cpp
WORKDIR /opt/llama.cpp
RUN git clone --recursive https://github.com/ggerganov/llama.cpp.git .
# build
RUN git clean -xdf \
&& git submodule update --recursive \
&& cmake -S . -B build \
-DGGML_HIP=ON \
-DAMDGPU_TARGETS=gfx1151 \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_RPC=ON \
-DLLAMA_HIP_UMA=ON \
-DGGML_CUDA_ENABLE_UNIFIED_MEMORY=ON \
-DROCM_PATH=/opt/rocm \
-DHIP_PATH=/opt/rocm \
-DHIP_PLATFORM=amd \
-DCMAKE_HIP_FLAGS="--rocm-path=/opt/rocm" \
&& cmake --build build --config Release -- -j$(nproc) \
&& cmake --install build --config Release
# libs
RUN find /opt/llama.cpp/build -type f -name 'lib*.so*' -exec cp {} /usr/lib64/ \; \
&& ldconfig
# helper
COPY gguf-vram-estimator.py /usr/local/bin/gguf-vram-estimator.py
RUN chmod +x /usr/local/bin/gguf-vram-estimator.py
# runtime stage
FROM registry.fedoraproject.org/fedora-minimal:43
# rocm 7.1 repo
RUN <<'EOF'
tee /etc/yum.repos.d/rocm.repo <<REPO
[ROCm-7.1.1]
name=ROCm7.1.1
baseurl=https://repo.radeon.com/rocm/el9/7.1.1/main
enabled=1
priority=50
gpgcheck=1
gpgkey=https://repo.radeon.com/rocm/rocm.gpg.key
REPO
EOF
# runtime deps
RUN microdnf -y --nodocs --setopt=install_weak_deps=0 \
--exclude='*sdk*' --exclude='*samples*' --exclude='*-doc*' --exclude='*-docs*' \
install \
bash ca-certificates libatomic libstdc++ libgcc libgomp sudo \
hip-runtime-amd rocblas hipblas \
rocminfo radeontop \
&& microdnf clean all && rm -rf /var/cache/dnf/*
# copy
COPY --from=builder /usr/local/ /usr/local/
COPY --from=builder /opt/llama.cpp/build/bin/rpc-* /usr/local/bin/
# ld
RUN echo "/usr/local/lib" > /etc/ld.so.conf.d/local.conf \
&& echo "/usr/local/lib64" >> /etc/ld.so.conf.d/local.conf \
&& ldconfig \
&& cp -n /usr/local/lib/libllama*.so* /usr/lib64/ 2>/dev/null || true \
&& ldconfig
# helper
COPY gguf-vram-estimator.py /usr/local/bin/gguf-vram-estimator.py
RUN chmod +x /usr/local/bin/gguf-vram-estimator.py
# profile
RUN printf '%s\n' \
'export ROCBLAS_USE_HIPBLASLT=1' \
> /etc/profile.d/rocm.sh && chmod +x /etc/profile.d/rocm.sh \
&& echo 'source /etc/profile.d/rocm.sh' >> /etc/bashrc
# shell
CMD ["/bin/bash"]
toolboxes/Dockerfile.rocm-7.1-rocwmma → toolboxes/Dockerfile.rocm-7.1.1-rocwmma
+44 -44
View File
@@ -4,9 +4,9 @@ FROM registry.fedoraproject.org/fedora:43 AS builder
# rocm 7.1 repo # rocm 7.1 repo
RUN <<'EOF' RUN <<'EOF'
tee /etc/yum.repos.d/rocm.repo <<REPO tee /etc/yum.repos.d/rocm.repo <<REPO
[ROCm-7.1] [ROCm-7.1.1]
name=ROCm7.1 name=ROCm7.1.1
baseurl=https://repo.radeon.com/rocm/el9/7.1/main baseurl=https://repo.radeon.com/rocm/el9/7.1.1/main
enabled=1 enabled=1
priority=50 priority=50
gpgcheck=1 gpgcheck=1
@@ -16,21 +16,21 @@ EOF
# deps # deps
RUN dnf -y --nodocs --setopt=install_weak_deps=False \ RUN dnf -y --nodocs --setopt=install_weak_deps=False \
--exclude='*sdk*' --exclude='*samples*' --exclude='*-doc*' --exclude='*-docs*' \ --exclude='*sdk*' --exclude='*samples*' --exclude='*-doc*' --exclude='*-docs*' \
install \ install \
make gcc cmake lld clang clang-devel compiler-rt libcurl-devel ninja-build \ make gcc cmake lld clang clang-devel compiler-rt libcurl-devel ninja-build \
rocm-llvm rocm-device-libs hip-runtime-amd hip-devel \ rocm-llvm rocm-device-libs hip-runtime-amd hip-devel \
rocblas rocblas-devel hipblas hipblas-devel rocm-cmake libomp-devel libomp \ rocblas rocblas-devel hipblas hipblas-devel rocm-cmake libomp-devel libomp \
rocminfo radeontop \ rocminfo radeontop \
git-core vim sudo rsync \ git-core vim sudo rsync \
&& dnf clean all && rm -rf /var/cache/dnf/* && dnf clean all && rm -rf /var/cache/dnf/*
# rocm env # rocm env
ENV ROCM_PATH=/opt/rocm \ ENV ROCM_PATH=/opt/rocm \
HIP_PATH=/opt/rocm \ HIP_PATH=/opt/rocm \
HIP_CLANG_PATH=/opt/rocm/llvm/bin \ HIP_CLANG_PATH=/opt/rocm/llvm/bin \
HIP_DEVICE_LIB_PATH=/opt/rocm/amdgcn/bitcode \ HIP_DEVICE_LIB_PATH=/opt/rocm/amdgcn/bitcode \
PATH=/opt/rocm/bin:/opt/rocm/llvm/bin:$PATH PATH=/opt/rocm/bin:/opt/rocm/llvm/bin:$PATH
# rocWMMA # rocWMMA
WORKDIR /opt WORKDIR /opt
@@ -47,25 +47,25 @@ RUN chmod +x /opt/apply-rocwmma-fix.sh && /opt/apply-rocwmma-fix.sh /opt/llama.c
# build # build
RUN git clean -xdf \ RUN git clean -xdf \
&& git submodule update --recursive \ && git submodule update --recursive \
&& cmake -S . -B build \ && cmake -S . -B build \
-DGGML_HIP=ON \ -DGGML_HIP=ON \
-DAMDGPU_TARGETS=gfx1151 \ -DAMDGPU_TARGETS=gfx1151 \
-DCMAKE_BUILD_TYPE=Release \ -DCMAKE_BUILD_TYPE=Release \
-DGGML_RPC=ON \ -DGGML_RPC=ON \
-DLLAMA_HIP_UMA=ON \ -DLLAMA_HIP_UMA=ON \
-DGGML_CUDA_ENABLE_UNIFIED_MEMORY=ON \ -DGGML_CUDA_ENABLE_UNIFIED_MEMORY=ON \
-DGGML_HIP_ROCWMMA_FATTN=ON \ -DGGML_HIP_ROCWMMA_FATTN=ON \
-DROCM_PATH=/opt/rocm \ -DROCM_PATH=/opt/rocm \
-DHIP_PATH=/opt/rocm \ -DHIP_PATH=/opt/rocm \
-DHIP_PLATFORM=amd \ -DHIP_PLATFORM=amd \
-DCMAKE_HIP_FLAGS="--rocm-path=/opt/rocm" \ -DCMAKE_HIP_FLAGS="--rocm-path=/opt/rocm" \
&& cmake --build build --config Release -- -j$(nproc) \ && cmake --build build --config Release -- -j$(nproc) \
&& cmake --install build --config Release && cmake --install build --config Release
# libs # libs
RUN find /opt/llama.cpp/build -type f -name 'lib*.so*' -exec cp {} /usr/lib64/ \; \ RUN find /opt/llama.cpp/build -type f -name 'lib*.so*' -exec cp {} /usr/lib64/ \; \
&& ldconfig && ldconfig
# helper # helper
COPY gguf-vram-estimator.py /usr/local/bin/gguf-vram-estimator.py COPY gguf-vram-estimator.py /usr/local/bin/gguf-vram-estimator.py
@@ -78,9 +78,9 @@ FROM registry.fedoraproject.org/fedora-minimal:43
# rocm 7.1 repo # rocm 7.1 repo
RUN <<'EOF' RUN <<'EOF'
tee /etc/yum.repos.d/rocm.repo <<REPO tee /etc/yum.repos.d/rocm.repo <<REPO
[ROCm-7.1] [ROCm-7.1.1]
name=ROCm7.1 name=ROCm7.1.1
baseurl=https://repo.radeon.com/rocm/el9/7.1/main baseurl=https://repo.radeon.com/rocm/el9/7.1.1/main
enabled=1 enabled=1
priority=50 priority=50
gpgcheck=1 gpgcheck=1
@@ -90,12 +90,12 @@ EOF
# runtime deps # runtime deps
RUN microdnf -y --nodocs --setopt=install_weak_deps=0 \ RUN microdnf -y --nodocs --setopt=install_weak_deps=0 \
--exclude='*sdk*' --exclude='*samples*' --exclude='*-doc*' --exclude='*-docs*' \ --exclude='*sdk*' --exclude='*samples*' --exclude='*-doc*' --exclude='*-docs*' \
install \ install \
bash ca-certificates libatomic libstdc++ libgcc libgomp sudo \ bash ca-certificates libatomic libstdc++ libgcc libgomp sudo \
hip-runtime-amd rocblas hipblas \ hip-runtime-amd rocblas hipblas \
rocminfo radeontop \ rocminfo radeontop \
&& microdnf clean all && rm -rf /var/cache/dnf/* && microdnf clean all && rm -rf /var/cache/dnf/*
# copy # copy
COPY --from=builder /usr/local/ /usr/local/ COPY --from=builder /usr/local/ /usr/local/
@@ -103,10 +103,10 @@ COPY --from=builder /opt/llama.cpp/build/bin/rpc-* /usr/local/bin/
# ld # ld
RUN echo "/usr/local/lib" > /etc/ld.so.conf.d/local.conf \ RUN echo "/usr/local/lib" > /etc/ld.so.conf.d/local.conf \
&& echo "/usr/local/lib64" >> /etc/ld.so.conf.d/local.conf \ && echo "/usr/local/lib64" >> /etc/ld.so.conf.d/local.conf \
&& ldconfig \ && ldconfig \
&& cp -n /usr/local/lib/libllama*.so* /usr/lib64/ 2>/dev/null || true \ && cp -n /usr/local/lib/libllama*.so* /usr/lib64/ 2>/dev/null || true \
&& ldconfig && ldconfig
# helper # helper
COPY gguf-vram-estimator.py /usr/local/bin/gguf-vram-estimator.py COPY gguf-vram-estimator.py /usr/local/bin/gguf-vram-estimator.py