Fix confluent-kafka build error: librdkafka/rdkafka.h file not found

TL;DR: if installing confluent-kafka from source fails with fatal error: 'librdkafka/rdkafka.h' file not found, the build needs the librdkafka C library headers first. Fastest fix on a modern release: pip install -U confluent-kafka to a version with bundled binary wheels (0.11.4+), which never compiles anything. If you must build from source, install librdkafka first (brew install librdkafka on macOS, your system package elsewhere) so the headers are on the include path.

fatal error: 'librdkafka/rdkafka.h' file not found

Steps

1. Try the wheels first:

pip install -U confluent-kafka

Expected: a binary wheel installs, no compilation. Success check: import confluent_kafka works.

2. If you must compile, install librdkafka via your system package manager first.

When to use this

• Source build fails on the missing rdkafka.h.

When NOT to use this

• Other compile errors; read the message.
• Wheels already install fine; nothing to do.

Compatibility

• confluent-kafka 0.11.4+ (bundled wheels).

Variant phrasings

• "confluent-kafka rdkafka.h not found"
• "confluent kafka build error librdkafka"

Root cause

The Python package wraps a C library; without its headers the C extension cannot build.

Edge cases

• The maintainers added bundled wheels precisely because this kept biting people; prefer them.

Source: https://github.com/confluentinc/confluent-kafka-python/issues/184

In the thread, the question was: Installing confluent-kafka from source failed with fatal error: 'librdkafka/rdkafka.h' file not found because the build needs the librdkafka C library headers first. Two options: install librdkafka before pip (brew install librdkafka on macOS, your system package elsewhere) so the headers exist, or use a release that bundles it. Since v0.11.4 the project ships binary wheels containing librdkafka, so pip install -U confluent-kafka on a supported platform avoids the source build and this error entirely. The verified resolution: My pip install confluent-kafka died compiling because the C headers for librdkafka were missing. Fastest fix on a modern release: pip install -U confluent-kafka to a version with bundled binary wheels (0.11.4+), which never compiles anything. If you must build from source, install librdkafka first (brew install librdkafka on macOS) so the headers are on the include path. The maintainers added the bundled wheels precisely because this build step kept biting people. Source: https://github.com/confluentinc/confluent-kafka-python/issues/184 Thread: https://vectle.com/threads/thr0n8FFrTDtnM1W130JQ93g