Immutable solution version
Issue with Pod Termination in InferenceService using nvcr.io/nim/meta/llama-3.1-8b-instruct:1.1.2 - No Terminate...
Complete solution
NIM InferenceService pods hanging on deletion (no SIGTERM reaching the NIM server, pod only force-deleted after terminationGracePeriodSeconds): the bug was reported to engineering and a fix is in place for NIM 1.12, tested by the maintainer to catch SIGTERM as expected. Upgrade to NIM >= 1.12.
What was reported: Issue NVIDIA/nim-deploy#91 (closed, 10 comments): Currently, I am using the nvcr.io/nim/meta/llama-3.1-8b-instruct:1.1.2 image to create an InferenceService following the provided guide, and the service is running properly. Pod creation and API calls are working fine, but I am encountering an issue when trying to delete the Pod. It seems that the Terminate command is not being sent to the Nim server when I request the deletion of the InferenceService or Pod. There are no KILL signals in the internal logs either, and the Pod is only forcefully deleted when it reaches the terminationGracePeriodSeconds: 300.
What works: NIM InferenceService pods hanging on deletion (no SIGTERM reaching the NIM server, pod only force-deleted after terminationGracePeriodSeconds): the bug was reported to engineering and a fix is in place for NIM 1.12, tested by the maintainer to catch SIGTERM as expected. Upgrade to NIM >= 1.12.
Evidence
In the GitHub issue, the reporter asked: Issue with Pod Termination in InferenceService using nvcr.io/nim/meta/llama-3.1-8b-instruct:1.1.2 - No Terminate.... The working answer that came out of the thread: NIM InferenceService pods hanging on deletion (no SIGTERM reaching the NIM server, pod only force-deleted after terminationGracePeriodSeconds): the bug was reported to engineering and a fix is in place for NIM 1.12, tested by the maintainer to catch SIGTERM as expected. Upgrade to NIM >= 1.12. Source: https://github.com/NVIDIA/nim-deploy/issues/91
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_3qO-5e6Vq3D1i-OT5Ct6IA" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Issue with Pod Termination in InferenceService using nvcr.io/nim/meta/llama-3.1-8b-instruct:1.1.2 - No Terminate... <span>— Vectle</span></a>