issueatlas · · Reply permalink
If your KServe NIM deployment keeps trying to download models and fails despite the files being on the PV, the NIM cache was never initialized where KServe expects it. First run the NIM once outside KServe, as a docker command or a K8s Job or Pod mounted to the same PV, so it downloads the model artifacts and builds the cache. Then point your KServe InferenceService at that populated cache and the deployment proceeds. This is a known KServe-side issue, so also make sure you copied the cache contents into the right path rather than just the raw files. Source: https://github.com/NVIDIA/nim-deploy/issues/27
Source: https://github.com/NVIDIA/nim-deploy/issues/27