kubectl "the server could not find the requested resource": CRD missing after upgrade

TL;DR

After upgrades, custom resources fail with "could not find the requested resource" when their CRDs are gone: not reinstalled after the upgrade, removed by an operator uninstall, or never applied to this cluster. The API server only knows built-in types plus installed CRDs. Reinstall the CRD from the operator's manifests, then verify the custom resources become visible again.

The query

kubectl "the server could not find the requested resource": CRD missing after upgrade

Use this when

• kubectl fails for custom resources after an upgrade
• Operators were uninstalled or upgraded
• CRDs exist in one cluster but not another
• Custom resources suddenly unknown

Not for when

• Built-in resources (pods, services) not found
• RBAC forbidden errors
• Wrong-namespace issues

Steps

Step 1: Confirm the resource is custom

Check whether the failing kind is a custom resource (defined by a CRD) or built-in. Built-in resources do not vanish on upgrade; custom ones do when their CRD goes missing. Expected output: the kind classified as custom (CRD-backed).

Step 2: Check installed CRDs

List CRDs and look for the one defining the kind. If it is absent, that is the whole problem. If it is present but the version differs, the stored version may need migration. Expected output: the CRD found missing, or present with a version issue.

Step 3: Reinstall the CRD from the operator

Apply the CRD manifests from the operator or chart version you run. Use the matching version: a newer CRD with an older operator (or vice versa) creates subtler problems than a missing one. Expected output: the CRD installed at the correct version.

Step 4: Verify custom resources reappear

After CRD installation, the existing custom resource objects become visible again (they were never deleted, just unservable). Get them and confirm their state is intact. Expected output: custom resources listed with their data preserved.

Step 5: Include CRDs in upgrade runbooks

Add CRD verification to cluster upgrade checklists: list expected CRDs before and after, and reinstall operators as part of the upgrade. Missing CRDs should be caught by the runbook, not by the first failing kubectl. Expected output: upgrades that preserve the full API surface, verified.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_WgAKTIhCgM2dWD2MypF80g