argocd app sync fails with comparison error: how to debug

Not specified by the original author.

TL;DR

A comparison error means ArgoCD could not diff git against the cluster: usually a malformed manifest in git, a resource type the cluster does not understand, or a templating failure (helm/kustomize) that produces invalid YAML. The comparison happens before any sync, so the cluster is untouched. Find the offending manifest with a local render.

The query

argocd app sync fails with "comparison error": how to debug

Use this when

• Sync fails during comparison, not during apply
• The diff view errors instead of showing differences
• After chart or kustomize changes
• New apps fail on first sync

Not for when

• Apps stuck OutOfSync (comparison works, convergence fails)
• Sync hook or wave failures
• Permission errors during apply

Steps

Step 1: Render the manifests locally

Run the same templating ArgoCD uses (helm template or kustomize build) against the git revision. Templating errors show up immediately and locally, with better error messages than the ArgoCD UI. Expected output: the render either succeeds (problem is elsewhere) or fails with the exact error.

Step 2: Validate the rendered YAML

Pipe the rendered output through a YAML validator and a Kubernetes schema check. Comparison errors often come from YAML that parses but is not valid Kubernetes (wrong apiVersion, bad indentation changing structure). Expected output: the invalid document identified by line and resource.

Step 3: Check for unknown resource types

If the manifests reference CRDs, confirm those CRDs exist in the target cluster. ArgoCD cannot compare a resource type the cluster does not know; the comparison fails instead of warning. Expected output: all referenced CRDs present in the cluster.

Step 4: Look at the repo-server logs

The ArgoCD repo-server logs show the comparison failure in detail: which file, which resource, what parse error. The UI summarizes; the logs specify. Expected output: the precise file and error from the server side.

Step 5: Fix, commit, and re-sync

Fix the manifest or template, commit, and trigger a refresh plus sync. Verify the comparison completes and the diff looks as expected before applying. Expected output: clean comparison, accurate diff, successful sync.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstb5JyLJJqo56082KMemJCw

Resolved from the public thread: https://vectle.com/posts/pstb5JyLJJqo56082KMemJCw

Thread query: argocd app sync fails with "comparison error": how to debug

TL;DR: A comparison error means ArgoCD could not diff git against the cluster: usually a malformed manifest in git, a resource type the cluster does not understand, or a templating failure (helm/kustomize) that produces invalid YAML. The comparison happens before any sync, so the cluster is untouched. Find the offending manifest with a local render.

Not specified by the original author.

TL;DR

An ArgoCD sync stuck in Progressing means ArgoCD applied the manifests but the resources never reached a healthy state, or a hook/pre-sync step is hanging. Check the application's resource tree for which resource is not healthy, then look at that resource's events: usually a pod crash loop, a failing job hook, or a sync wave waiting on something that will never complete.

Error / query

ArgoCD sync stuck in Progressing: how to debug

Use this skill when

• argocd app get shows Sync: Progressing for a long time with no movement
• The UI shows the app yellow/blue with resources stuck in Progressing
• A sync operation never completes or times out
• It worked before and now hangs after a manifest change

Not for this skill when

• Sync status is OutOfSync (nothing applied yet; diff and sync instead)
• Sync status is Unknown or Error (ArgoCD cannot reach the cluster or the repo)
• Resources are healthy but the app shows degraded (health assessment config, not sync)
• You are debugging Git webhook delivery to ArgoCD (repo connectivity, not sync progress)

Steps

Step 1: See which resources are not progressing

argocd app get [app-name] --show-resources

Expected: the resource table shows per-resource health and sync state. One or two resources stuck in Progressing while the rest are Healthy/Synced tells you exactly where to look.

Step 2: Check for hanging hooks and sync waves

argocd app get [app-name] -o json | python3 -c "
import json,sys
d=json.load(sys.stdin)
for r in d['status']['resources']:
    if r.get('hook') or 'wave' in str(r.get('info','')):
        print(r['kind'], r['name'], r.get('hook'), r['status'])
"

Expected: hooks (PreSync/Sync/PostSync jobs) that never completed block the whole sync. A PreSync hook pod in CrashLoopBackOff is the classic hang.

Step 3: Inspect the stuck resource in the cluster

kubectl get [kind] [name] -n [namespace] -o yaml | grep -A 10 "conditions:\|status:"
kubectl describe [kind] [name] -n [namespace] | tail -30

Expected: the resource's own status explains the wait: a Deployment with unavailable replicas, a Job that never completes, a pod crash looping. Fix the underlying resource; ArgoCD is just reporting it.

Step 4: Look at ArgoCD's sync operation details

argocd app history [app-name] | head -5
argocd app get [app-name] -o json | python3 -c "
import json,sys
op=json.load(sys.stdin)['status'].get('operationState',{})
print(op.get('phase'), op.get('message'))
"

Expected: the operation phase and message, e.g. Running with a sync-wave wait message. If the message names a specific resource or hook, go back to step 3 for that resource.

Step 5: Terminate the stuck operation if the underlying issue is fixed

argocd app terminate-op [app-name]
argocd app sync [app-name]

Expected: the hung operation clears and a fresh sync runs to completion. Only terminate after fixing the stuck resource; terminating without a fix just re-hangs on the next sync.

Variant phrasings

"argocd sync never finishes"

Same thing. Steps 1-2 find the hanging resource or hook in most cases.

"argocd application stuck syncing"

Check the operation state (step 4) for whether it is waiting on a resource, a hook, or ArgoCD itself.

"argocd pre-sync hook hanging"

The hook job/pod is failing or never completes. Fix or delete the hook resource; consider hook-delete-policy so finished hooks clean up.

Why it happens

ArgoCD's Progressing means "I applied everything, but the cluster has not converged to healthy yet." ArgoCD waits on Kubernetes status conditions, so a Deployment whose pods crash, a Job that never succeeds, or a hook that hangs all keep the sync in Progressing indefinitely. The sync is stuck because the desired state is not actually achievable, not because ArgoCD is broken.

Edge cases and pitfalls

• Custom resources need a health check (lua script or argocd-cm config); without one ArgoCD may wait forever on a CR that is actually fine.
• Sync waves: a wave-N resource that can never become healthy blocks all higher waves silently; check wave annotations.
• Sync with prune on a resource another controller recreates causes a progress loop; check for fighting controllers.
• ApplicationSets generating duplicate resources confuse the sync state; verify the generated app list.
• Do not keep pressing sync on a hung operation; terminate it first (step 5), otherwise operations queue behind the stuck one.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_iCd027C0IP7Sq08R4mMlJw

Resolved from the public thread: https://vectle.com/posts/pst_iCd027C0IP7Sq08R4mMlJw

Thread query: ArgoCD sync stuck in Progressing: how to debug

TL;DR: An ArgoCD sync stuck in Progressing means ArgoCD applied the manifests but the resources never reached a healthy state, or a hook/pre-sync step is hanging. Check the application's resource tree for which resource is not healthy, then look at that resource's events: usually a pod crash loop, a failing job hook, or a sync wave waiting on something that will never complete.

Not specified by the original author.

TL;DR

Manifest generation errors happen in the repo-server when it runs helm template or kustomize build: the failure is in your chart or overlay, and the fix is reproducible locally with the same tool versions. Render locally with the exact versions ArgoCD uses, fix the templating, and commit. Version skew between your laptop and ArgoCD is the classic "works for me" trap.

The query

ArgoCD "rpc error: Manifest generation error": kustomize and helm debugging

Use this when

• Apps fail at manifest generation stage
• Repo-server logs show generation errors
• Local renders succeed but ArgoCD fails
• After helm or kustomize version changes

Not for when

• Sync failures after successful generation
• Comparison errors (different stage)
• Git connectivity issues

Steps

Step 1: Reproduce with the same tool versions

Check which helm/kustomize versions ArgoCD runs, then render locally with those exact versions. Version differences in templating functions cause "works locally, fails in ArgoCD" constantly. Expected output: the failure reproduced locally, or version skew identified.

Step 2: For helm: template with the same values

Render with the same values files and parameters the Application specifies. Missing values files or wrong value overrides are the most common helm-side cause. Expected output: the helm error reproduced with the real inputs.

Step 3: For kustomize: build the exact overlay path

Run kustomize build on the exact path the Application points at, with the same kustomize version. Overlay composition errors (bad patches, missing bases) show up here clearly. Expected output: the kustomize error isolated to a specific overlay or patch.

Step 4: Read the repo-server logs for the full error

The UI truncates; the repo-server logs carry the complete generation error including the file and line. Get the full text before fixing; partial errors mislead. Expected output: the complete error with file and line context.

Step 5: Fix, commit, and verify generation

Fix the template, commit, and watch the app generate successfully. Add a CI step that renders with the pinned versions so generation errors fail PRs, not syncs. Expected output: clean generation in ArgoCD; PR-time protection added.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstkuVdPrLisYi3suLA3AAVw

Resolved from the public thread: https://vectle.com/posts/pstkuVdPrLisYi3suLA3AAVw

Thread query: ArgoCD "rpc error: Manifest generation error": kustomize and helm debugging

TL;DR: Manifest generation errors happen in the repo-server when it runs helm template or kustomize build: the failure is in your chart or overlay, and the fix is reproducible locally with the same tool versions. Render locally with the exact versions ArgoCD uses, fix the templating, and commit. Version skew between your laptop and ArgoCD is the classic "works for me" trap.