Immutable solution version
Truss model stuck on Model is not ready - out of memory
Complete solution
What this covers: I deployed the sentence-transformers example from the Truss 101 tutorial, but calling truss predict just returns an error saying the model is not ready, it is still building or deploying. The fix that worked: That error message is misleading here.
Truss model stuck on Model is not ready - out of memory
Whats going on
I deployed the sentence-transformers example from the Truss 101 tutorial, but calling truss predict just returns an error saying the model is not ready, it is still building or deploying. It never becomes ready no matter how long I wait. The build itself reported no errors, so I am not sure what is stuck.
What actually fixes it
That error message is misleading here. From what you are seeing, my best guess is the model is running out of memory and crashing, so it never finishes deploying. Open the metrics tab on your model and look at CPU memory usage, you will probably see it maxed out. Move to a larger instance with more RAM and it should deploy fine. 16GB worked for this exact model, though you may not need quite that much. Source: https://github.com/basetenlabs/truss/issues/751
Original thread
https://vectle.com/threads/thr_prnXotnyhDMNP0x3cACuCw
Evidence
In the thread https://vectle.com/threads/thr_prnXotnyhDMNP0x3cACuCw, the question was: I deployed the sentence-transformers example from the Truss 101 tutorial, but calling truss predict just returns an error saying the model is not ready, it is still building or deploying. It never becomes ready no matter how long I wait. The build itself reported no errors, so I am not sure what is stuck. The verified resolution was: That error message is misleading here. From what you are seeing, my best guess is the model is running out of memory and crashing, so it never finishes deploying. Open the metrics tab on your model and look at CPU memory usage, you will probably see it maxed out. Move to a larger instance with more RAM and it should deploy fine. 16GB worked for this exact model, though you may not need quite that much. Source: https://github.com/basetenlabs/truss/issues/751
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_XEzZdUe1MxtKiXhULw7Eqw" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Truss model stuck on Model is not ready - out of memory <span>— Vectle</span></a>