Problem when using pipeline serialization

TL;DR

Workaround until the fix: disable telemetry with export HAYSTACKTELEMETRYENABLED [False].

The error

Issue deepset-ai/haystack#7277 (closed, 12 comments): Hello, I woul like to use pipeline serialization. Below is my code for the llm ``` llm: init_parameters: huggingface_pipeline_kwargs: model: mistralai/Mixtral-8x7B-Instruct-v0.1 device_map: auto model_kwargs: bnb_4bit_compute_dtype: torch.bfloat16 bnb_4bit_quant_type: nf4 bnb_4bit_use_double_quant: true load_in_4bit: true #task: text-generation stop_words: null type: haystack.components.generators.hugging_face_local.HuggingFaceLocalGenerator ``` this code will generate the below error: ~/.local/lib/python3.9/site-packages/bitsandbytes/nn/modules.py in forward(self, x) 251 inp_dtype = x.dtype 2

Fix

1. Workaround until the fix: disable telemetry with export HAYSTACKTELEMETRYENABLED [False].

Success check: the symptom above is gone and the original operation completes.

When to use

• You hit this exact symptom in haystack.
• You want the verified fix without digging through the thread.

When not to use

• A different error in haystack, even a similar-looking one.
• General haystack setup or credential questions.

Compatibility

• Haystack (deepset-ai/haystack)

Variant phrasings

• How to fix "problem when using pipeline serialization" in haystack
• haystack: problem when using pipeline serialization

Root cause

Same root cause as issue #7255.

Source

• https://github.com/deepset-ai/haystack/issues/7277

In gh:deepset-ai/haystack#7277 (https://github.com/deepset-ai/haystack/issues/7277), someone reported: Issue deepset-ai/haystack#7277 (closed, 12 comments): Hello, I woul like to use pipeline serialization. Below is my code for the llm `` llm: init_parameters: huggingface_pipeline_kwargs: model: mistralai/Mixtral-8x7B-Instruct-v0.1 device_map: auto model_kwargs: bnb_4bit_compute_dtype: torch.bfloat16 bnb_4bit_quant_type: nf4 bnb_4bit_use_double_quant: true load_in_4bit: true #task: text-generation stop_words: null type: haystack.components.generators.hugging_face_local.HuggingFaceLocalGenerator `` this code will generate the below error: ~/.local/lib/python3.9/site-packages/bitsandbytes/nn/modules.py in forward(self, x) 251 inpdtype = x.dtype 2 The verified resolution from the thread: Maintainer anakin87 confirmed this is a real bug: incorrect serialization in telemetry breaks pipeline deserialization with HuggingFaceLocalGenerator kwargs (e.g. bnb4bitcomputedtype: torch.bfloat16 needs quotes). Workaround until the fix: disable telemetry with export HAYSTACKTELEMETRYENABLED [False]. Same root cause as issue #7255.