error while loading shared libraries: libcudart.so.13: cannot open shared object file: no such file or directory

Short answer: find where libcudart.so.13 actually lives on the machine, then put that directory on the linker search path, or install the matching CUDA 13 runtime. Do not symlink a different major version in its place. `` error while loading shared libraries: libcudart.so.13: cannot open shared object file: No such file or directory 1. Locate the library: find / -name "libcudart.so*" 2>/dev/null. Two useful outcomes: a path under a CUDA install (for example the cuda lib64 directory) or a path inside your Python env (site-packages/nvidia/cu13/lib). If the search returns nothing, the runtime is genuinely absent; go to step 3. 2. If it exists, expose it to the linker for the current shell: export LD_LIBRARY_PATH with that directory prepended, then rerun the program. Expected: the program starts, or ldd YOURBINARY | grep cudart now resolves to a real path instead of not found. For system CUDA installs, adding the cuda lib64 directory to a file under /etc/ld.so.conf.d and running sudo ldconfig makes it permanent. 3. If it does not exist, install the CUDA 13 runtime: the cuda-toolkit-13 packages from NVIDIA's repo, or inside a Python-only stack python -m pip install nvidia-cuda-runtime-cu13 (then still apply step 2 to its lib directory). Expected: the find from step 1 now returns a libcudart.so.13 path. 4. Verify versions line up: nvidia-smi` shows the driver's maximum CUDA version (13.0 or higher is needed for a CUDA 13 runtime) and the failing program's expected runtime. A driver too old for CUDA 13 cannot be fixed by paths; update the driver or rebuild/reinstall the program against CUDA 12. Root cause: the linker only searches its configured paths (ld config, LDLIBRARY_PATH, standard lib dirs). CUDA runtimes installed by pip wheels live outside all of them, and a runtime of the wrong major version is invisible in a different way: present, but never a match for the .so.13 name the binary requests. When to use: Linux binaries or Python GPU packages (vLLM, PyTorch extensions, TensorRT tools) failing at startup with this exact loader error. When NOT to use: if ldd resolves cudart but the program then fails on a GLIBC version message, that is a libc mismatch, not a search-path problem. In Docker, fix the image (use a cuda runtime base image or install the runtime in the Dockerfile) instead of exporting paths at container start. Variant phrasings: "importerror libcudart.so.13 cannot open shared object file", "vllm libcudart.so.13 not found", "libcudart.so.13 missing ubuntu". Edge cases: conda environments put the runtime under the env's lib directory; the same export works with that path. Never copy or symlink a libcudart from another CUDA major version: the loader will accept the name and the program will fail later with undefined symbols.

Context: A binary linked against the CUDA 13 runtime starts, the dynamic linker looks for libcudart.so.13, and it is either not installed or not in any directory the linker searches. The .so.13 suffix is a major-version contract: a CUDA 12 runtime does not satisfy it. The current twist is that pip-installed GPU stacks (vLLM, PyTorch CUDA wheels) often ship this library inside the Python environment under site-packages/nvidia, a directory the system linker never searches by default, so the file exists on disk while the error insists it cannot be found.

Source: A binary linked against the CUDA 13 runtime starts, the dynamic linker looks for libcudart.so.13, and it is either not installed or not in any directory the linker searches. The .so.13 suffix is a major-version contract: a CUDA 12 runtime does not satisfy it. The current twist is that pip-installed GPU stacks (vLLM, PyTorch CUDA wheels) often ship this library inside the Python environment under site-packages/nvidia, a directory the system linker never searches by default, so the file exists on disk while the error insists it cannot be found.

Verified resolution: Short answer: find where libcudart.so.13 actually lives on the machine, then put that directory on the linker search path, or install the matching CUDA 13 runtime. Do not symlink a different major version in its place. `` error while loading shared libraries: libcudart.so.13: cannot open shared object file: No such file or directory 1. Locate the library: find / -name "libcudart.so*" 2>/dev/null. Two useful outcomes: a path under a CUDA install (for example the cuda lib64 directory) or a path inside your Python env (site-packages/nvidia/cu13/lib). If the search returns nothing, the runtime is genuinely absent; go to step 3. 2. If it exists, expose it to the linker for the current shell: export LD_LIBRARY_PATH with that directory prepended, then rerun the program. Expected: the program starts, or ldd YOURBINARY | grep cudart now resolves to a real path instead of not found. For system CUDA installs, adding the cuda lib64 directory to a file under /etc/ld.so.conf.d and running sudo ldconfig makes it permanent. 3. If it does not exist, install the CUDA 13 runtime: the cuda-toolkit-13 packages from NVIDIA's repo, or inside a Python-only stack python -m pip install nvidia-cuda-runtime-cu13 (then still apply step 2 to its lib directory). Expected: the find from step 1 now returns a libcudart.so.13 path. 4. Verify versions line up: nvidia-smi` shows the driver's maximum CUDA version (13.0 or higher is needed for a CUDA 13 runtime) and the failing program's expected runtime. A driver too old for CUDA 13 cannot be fixed by paths; update the driver or rebuild/reinstall the program against CUDA 12. Root cause: the linker only searches its configured paths (ld config, LDLIBRARY_PATH, standard lib dirs). CUDA runtimes installed by pip wheels live outside all of them, and a runtime of the wrong major version is invisible in a different way: present, but never a match for the .so.13 name the binary requests. When to use: Linux binaries or Python GPU packages (vLLM, PyTorch extensions, TensorRT tools) failing at startup with this exact loader error. When NOT to use: if ldd resolves cudart but the program then fails on a GLIBC version message, that is a libc mismatch, not a search-path problem. In Docker, fix the image (use a cuda runtime base image or install the runtime in the Dockerfile) instead of exporting paths at container start. Variant phrasings: "importerror libcudart.so.13 cannot open shared object file", "vllm libcudart.so.13 not found", "libcudart.so.13 missing ubuntu". Edge cases: conda environments put the runtime under the env's lib directory; the same export works with that path. Never copy or symlink a libcudart from another CUDA major version: the loader will accept the name and the program will fail later with undefined symbols.

Source URL: https://codechefvaibhavkashyap.medium.com/deploying-vllm-on-a-gcp-deep-learning-vm-my-real-world-journey-with-nvidia-l4-cuda-and-gemma-31b-464c94019f5a