Fix confluent-kafka build error: librdkafka/rdkafka.h file not found

TL;DR: if installing confluent-kafka from source fails with fatal error: 'librdkafka/rdkafka.h' file not found, the build needs the librdkafka C library headers first. Fastest fix on a modern release: pip install -U confluent-kafka to a version with bundled binary wheels (0.11.4+), which never compiles anything. If you must build from source, install librdkafka first (brew install librdkafka on macOS, your system package elsewhere) so the headers are on the include path.

fatal error: 'librdkafka/rdkafka.h' file not found

Steps

1. Try the wheels first:

pip install -U confluent-kafka

Expected: a binary wheel installs, no compilation. Success check: import confluent_kafka works.

2. If you must compile, install librdkafka via your system package manager first.

When to use this

• Source build fails on the missing rdkafka.h.

When NOT to use this

• Other compile errors; read the message.
• Wheels already install fine; nothing to do.

Compatibility

• confluent-kafka 0.11.4+ (bundled wheels).

Variant phrasings

• "confluent-kafka rdkafka.h not found"
• "confluent kafka build error librdkafka"

Root cause

The Python package wraps a C library; without its headers the C extension cannot build.

Edge cases

• The maintainers added bundled wheels precisely because this kept biting people; prefer them.

Source: https://github.com/confluentinc/confluent-kafka-python/issues/184