Trevor Starick
be936d6b59
- Introduce ARG REPO and ARG BRANCH to replace the hardcoded git clone with: `git clone -b ${BRANCH} --single-branch --recursive ${REPO}` . This allows overriding the llama.cpp repository and branch at build time via `--build-arg`.
- Update `docs/building.md` to recommend using `--build-arg` instead of updating the file
2.1 KiB
2.1 KiB
Building Containers Locally
If you want to build or customize the toolbox containers yourself (rather than using the pre-built Docker Hub images), this guide explains the process. Local builds are useful if you want to:
- Use a patched or forked version of llama.cpp
- Add additional tools or libraries
- Change the Fedora base image (Rawhide vs. stable)
- Audit every installed dependency
1. Prerequisites
- Podman (recommended on Fedora) or Docker (also fine)
2. Build an Image
Each backend has its own subdirectory and Dockerfile in toolboxes/.
Example: Build the Vulkan RADV toolbox image
cd toolboxes
podman build --no-cache -t llama-vulkan-radv -f Dockerfile.vulkan-radv .
Example: Build the ROCm 6.4.2 toolbox image
cd toolboxes
podman build --no-cache -t llama-rocm-6.4.2 -f Dockerfile.rocm-6.4.2 .
You can use
docker buildif you prefer Docker.
3. Customizing the Build
- llama.cpp version: Use the
--build-arg REPO=...and--build-arg BRANCH=...options to specify a different repository or branch. - Extra dependencies: Add them to the Dockerfile as needed.
- Other customizations: Install tools, patch scripts, or swap to a different base image.
4. Using the Custom Image with Toolbx
Create a new toolbox using your freshly built image:
toolbox create llama-vulkan-radv --image localhost/llama-vulkan-radv \
-- --device /dev/dri --group-add video --security-opt seccomp=unconfined
Replace the backend/image name and device/group options as needed (see main README Section 2.1).
5. Troubleshooting
- Build fails (ROCm images especially): Try building with more memory or swap.
- Toolbox can't access GPU: Make sure you pass the correct device/group options.