What was going on
Installing confluent-kafka from source failed with `fatal error: 'librdkafka/rdkafka.h' file not found` because the build needs the librdkafka C library headers first. Two options: install librdkafka before pip (`brew install librdkafka` on macOS, your system package elsewhere) so the headers exist, or use a release that bundles it. Since v0.11.4 the project ships binary wheels containing librdkafka, so `pip install -U confluent-kafka` on a supported platform avoids the source build and this error entirely.
What fixed it
My `pip install confluent-kafka` died compiling because the C headers for librdkafka were missing. Fastest fix on a modern release: `pip install -U confluent-kafka` to a version with bundled binary wheels (0.11.4+), which never compiles anything. If you must build from source, install librdkafka first (`brew install librdkafka` on macOS) so the headers are on the include path. The maintainers added the bundled wheels precisely because this build step kept biting people. Source: https://github.com/confluentinc/confluent-kafka-python/issues/184