## TL;DR
`UNKNOWN_TOPIC_OR_PARTITION` in automation means the producer or consumer asked for a topic that does not exist on the broker, or asked for it before metadata refreshed. Stop the code from auto-creating topics by accident, confirm the topic exists with the admin client, and make the automation wait for metadata. The fix is usually in the deploy config, not in Kafka.

```text
UNKNOWN_TOPIC_OR_PARTITION
```

## Steps

1. Check what topics the broker actually knows about.

```bash
kafka-topics --bootstrap-server [BROKER_LIST] --list
```

Expected: a plain list of topic names. If your topic is missing, the automation raced creation or targeted the wrong cluster.

2. Describe the topic to check partitions and replication.

```bash
kafka-topics --bootstrap-server [BROKER_LIST] --describe --topic [TOPIC_NAME]
```

Expected: partition rows with leaders assigned. `Leader: none` rows mean partitions without a leader, which clients report as unknown.

3. Confirm your client is pointed at the right cluster and has the exact topic name.

```bash
kafka-console-producer --bootstrap-server [BROKER_LIST] --topic [TOPIC_NAME]
```

Expected: the producer starts and accepts input. A name mismatch (dash vs underscore, missing environment prefix) is the most common cause in automation.

4. In the automation, create the topic explicitly before the producer or consumer starts, and enable metadata retries.

Expected: the pipeline creates `[TOPIC_NAME]` with the intended partition count as a separate step, then starts the workload. Clients configured with a metadata refresh interval and a few retries survive a topic that appears seconds after startup.

## When to use

- A producer or consumer fails with `UNKNOWN_TOPIC_OR_PARTITION` in CI, a cron job, or a freshly deployed service.
- The error appears right after a deploy, topic migration, or cluster failover.
- You are writing automation that creates topics and immediately uses them.

## When not to use

- Every topic is unknown (that is a connectivity or broker-down problem, not a topic problem).
- The error is `LEADER_NOT_AVAILABLE` or `NOT_LEADER_FOR_PARTITION` (those are leader-election issues with different fixes).
- You are debugging consumer lag on an existing, working topic.

## Why it happens

Clients cache broker metadata, and a brand-new topic takes time to propagate. Automation that creates a topic and instantly produces to it loses the race. Typos and environment prefixes (`prod-orders` vs `staging-orders`) are the human version of the same race. Some teams disable auto-creation, so a producer that used to self-heal now fails loudly, which is actually the desired behavior once you see the error.

## Edge cases

- Topic names are case-sensitive; `[Topic]` and `[topic]` are different topics.
- If the topic exists but some partitions show no leader, wait out the election or check for under-replicated partitions rather than recreating the topic.
- Deleting and recreating a topic with the same name while old metadata is cached produces this error until metadata expires; restart the client or wait out the refresh interval.
- Cross-cluster mirroring setups can report the topic on the wrong cluster; verify which broker list the automation is really using.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_aedW3ck44hxpbHoYXYMxYA