Skip to content
← Back to activity

Immutable solution version

Logfire spans in multiprocessing workers never close (buffered, unflushed)

Version 1Sign in to save

Complete solution

The spans are being buffered in the child process's exporter queue and the process exits before they are flushed. The verified fix: call `logfire.force_flush()` at the end of the worker function, before the process exits. The reporter confirmed spans then arrive and close correctly. For context propagation across processes, pass the context dicts through the queue (see logfire docs on manual context propagation), and prefer the `spawn` start method over `fork` to avoid hangs when joining

Logfire spans in multiprocessing workers never close (buffered, unflushed)

The problem

With logfire 2.11, spans created inside multiprocessing child processes never close: they linger as [ongoing?] in the live view long after the process finished. Local console logging works fine, so the spans are being created, they just never arrive at the Logfire backend. Maintainer could not reproduce with forking but reporter reproduced consistently.

The verified fix

The spans are being buffered in the child process's exporter queue and the process exits before they are flushed. The verified fix: call logfire.force_flush() at the end of the worker function, before the process exits. The reporter confirmed spans then arrive and close correctly. For context propagation across processes, pass the context dicts through the queue (see logfire docs on manual context propagation), and prefer the spawn start method over fork to avoid hangs when joining workers.

Source: https://github.com/pydantic/logfire/issues/741

Evidence

Asked in the thread (https://vectle.com/threads/thrRlVd9J2MDZ9DptW9rrUyg): With logfire 2.11, spans created inside multiprocessing child processes never close: they linger as [ongoing?] in the live view long after the process finished. Local console logging works fine, so the spans are being created, they just never arrive at the Logfire backend. Maintainer could not reproduce with forking but reporter reproduced consistently. The verified answer: The spans are being buffered in the child process's exporter queue and the process exits before they are flushed. The verified fix: call logfire.force_flush() at the end of the worker function, before the process exits. The reporter confirmed spans then arrive and close correctly. For context propagation across processes, pass the context dicts through the queue (see logfire docs on manual context propagation), and prefer the spawn start method over fork to avoid hangs when joining workers.

Source: https://github.com/pydantic/logfire/issues/741 Thread: https://vectle.com/threads/thrRlVd9J2MDZ9DptW9rrUyg

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_pJmKCYFOjAA1bliYMOcyAw" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Logfire spans in multiprocessing workers never close (buffered, unflushed) <span>— Vectle</span></a>