NVIDIA in a non-GNU environment

NVIDIA proprietary drivers on Linux expect a GNU userland and glibc. With some work they can be made to work under musl on Alpine Linux. Other distros may be possible as well, though I have not personally attempted this; glibc can be compiled manually if there is no package or framework to support it already. There is already plenty of documentation on getting this working so I will not dwell on it too much.

Getting drivers working in Docker

The NVIDIA Container Toolkit is a shim Docker runtime that provides basic functionality for the Docker daemon and Docker Compose files to manage how NVIDIA resources are allocated to containers, and it is not actually required to get NVIDIA cards working in containers. This is especially beneficial in non-GNU environments where the Container Toolkit is much more difficult to get running than the drivers.

With a working driver on the host, you can map the required libraries as volumes. This is how Docker alternative Singularity does it.

docker-compose.yml example

This example may not match what is actually installed on your system. Copy paths as shown in nvliblist.conf and feel free to remove any that are not present for your install.

      - /dev/nvidia0
      # https://github.com/sylabs/singularity/blob/main/etc/nvliblist.conf
      # binaries, only mount what's needed
      - /usr/bin/nvidia-smi:/usr/bin/nvidia-smi:ro
      - /usr/bin/nvidia-debugdump:/usr/bin/nvidia-debugdump:ro
      - /usr/bin/nvidia-persistenced:/usr/bin/nvidia-persistenced:ro
      - /usr/bin/nvidia-cuda-mps-control:/usr/bin/nvidia-cuda-mps-control:ro
      - /usr/bin/nvidia-cuda-mps-server:/usr/bin/nvidia-cuda-mps-server:ro
      # libs, only mount what exists
      - /usr/lib/libcuda.so:/usr/lib/libcuda.so.1:ro
      - /usr/lib/libEGL.so:/usr/lib/libEGL.so.1:ro
      - /usr/lib/libGLESv1_CM.so:/usr/lib/libGLESv1_CM.so.1:ro
      - /usr/lib/libGLESv2.so:/usr/lib/libGLESv2.so.1:ro
      - /usr/lib/libGL.so:/usr/lib/libGL.so.1:ro
      - /usr/lib/libGLX.so:/usr/lib/libGLX.so.1:ro
      - /usr/lib/libnvcuvid.so:/usr/lib/libnvcuvid.so.1:ro
      - /usr/lib/libnvidia-cfg.so:/usr/lib/libnvidia-cfg.so.1:ro
      - /usr/lib/libnvidia-encode.so:/usr/lib/libnvidia-encode.so.1:ro
      - /usr/lib/libnvidia-fbc.so:/usr/lib/libnvidia-fbc.so.1:ro
      - /usr/lib/libnvidia-ifr.so:/usr/lib/libnvidia-ifr.so.1:ro
      - /usr/lib/libnvidia-ml.so:/usr/lib/libnvidia-ml.so.1:ro
      - /usr/lib/libnvidia-ptxjitcompiler.so:/usr/lib/libnvidia-ptxjitcompiler.so.1:ro
      - /usr/lib/libOpenCL.so:/usr/lib/libOpenCL.so.1:ro
      - /usr/lib/libOpenGL.so:/usr/lib/libOpenGL.so.1:ro
      - /usr/lib/libvdpau_nvidia.so:/usr/lib/libvdpau_nvidia.so.1:ro

Run ldconfig after mounting

You may need to run ldconfig so that the new libraries are discovered. This can be a problem in Docker containers, where container initialization is often frequent, automated, and unmonitored. The quickest but least flexible way to fix this is by injecting a command before the container entrypoint.

Official documentation and files

See also

This document was inspired by my adventures trying to get an old Fermi card working with Frigate, and the lack of documentation on the process.

Left-click: follow link, Right-click: select node, Scroll: zoom