Skip to content
← Back to activity

Immutable solution version

Truss model stuck on Model is not ready - out of memory

Version 1Sign in to save

Complete solution

What this covers: I deployed the sentence-transformers example from the Truss 101 tutorial, but calling truss predict just returns an error saying the model is not ready, it is still building or deploying. The fix that worked: That error message is misleading here.

Truss model stuck on Model is not ready - out of memory

Whats going on

I deployed the sentence-transformers example from the Truss 101 tutorial, but calling truss predict just returns an error saying the model is not ready, it is still building or deploying. It never becomes ready no matter how long I wait. The build itself reported no errors, so I am not sure what is stuck.

What actually fixes it

That error message is misleading here. From what you are seeing, my best guess is the model is running out of memory and crashing, so it never finishes deploying. Open the metrics tab on your model and look at CPU memory usage, you will probably see it maxed out. Move to a larger instance with more RAM and it should deploy fine. 16GB worked for this exact model, though you may not need quite that much. Source: https://github.com/basetenlabs/truss/issues/751

Original thread

https://vectle.com/threads/thr_prnXotnyhDMNP0x3cACuCw

Evidence

In the thread https://vectle.com/threads/thr_prnXotnyhDMNP0x3cACuCw, the question was: I deployed the sentence-transformers example from the Truss 101 tutorial, but calling truss predict just returns an error saying the model is not ready, it is still building or deploying. It never becomes ready no matter how long I wait. The build itself reported no errors, so I am not sure what is stuck. The verified resolution was: That error message is misleading here. From what you are seeing, my best guess is the model is running out of memory and crashing, so it never finishes deploying. Open the metrics tab on your model and look at CPU memory usage, you will probably see it maxed out. Move to a larger instance with more RAM and it should deploy fine. 16GB worked for this exact model, though you may not need quite that much. Source: https://github.com/basetenlabs/truss/issues/751

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_XEzZdUe1MxtKiXhULw7Eqw" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Truss model stuck on Model is not ready - out of memory <span>— Vectle</span></a>