Problem when using pipeline serialization
Haystack: problem when using pipeline serialization. The verified fix is Workaround until the fix: disable telemetry with export HAYSTACK_TELEMETRY_ENABLED [False]. Use this when you hit the symptoms above in haystack; not for unrelated haystack errors or general setup questions.
TL;DR
Workaround until the fix: disable telemetry with export HAYSTACKTELEMETRYENABLED [False].
The error
Issue deepset-ai/haystack#7277 (closed, 12 comments): Hello, I woul like to use pipeline serialization. Below is my code for the llm ``` llm: init_parameters: huggingface_pipeline_kwargs: model: mistralai/Mixtral-8x7B-Instruct-v0.1 device_map: auto model_kwargs: bnb_4bit_compute_dtype: torch.bfloat16 bnb_4bit_quant_type: nf4 bnb_4bit_use_double_quant: true load_in_4bit: true #task: text-generation stop_words: null type: haystack.components.generators.hugging_face_local.HuggingFaceLocalGenerator ``` this code will generate the below error: ~/.local/lib/python3.9/site-packages/bitsandbytes/nn/modules.py in forward(self, x) 251 inp_dtype = x.dtype 2Fix
- Workaround until the fix: disable telemetry with export HAYSTACKTELEMETRYENABLED [False].
Success check: the symptom above is gone and the original operation completes.
When to use
- You hit this exact symptom in haystack.
- You want the verified fix without digging through the thread.
When not to use
- A different error in haystack, even a similar-looking one.
- General haystack setup or credential questions.
Compatibility
- Haystack (deepset-ai/haystack)
Variant phrasings
- How to fix "problem when using pipeline serialization" in haystack
- haystack: problem when using pipeline serialization
Root cause
Same root cause as issue #7255.
Source
- https://github.com/deepset-ai/haystack/issues/7277