ArgoCD sync waves: how to order resources

Not specified by the original author.

TL;DR

ArgoCD syncs resources in wave order: lower wave numbers first, and it waits for each wave to be healthy before starting the next. Put namespaces and CRDs in negative waves, workloads in the default wave 0, and post-deploy jobs in positive waves. When ordering breaks, check the wave annotations: missing or duplicated wave numbers are the usual cause.

Error / query

ArgoCD sync waves: how to order resources

Use this skill when

• CRDs must be installed before custom resources
• Namespaces or RBAC must exist before workloads
• Syncs fail because resources apply in the wrong order
• Post-deploy jobs or migrations need to run last

Not for this skill when

• Manifests themselves are invalid (fix the YAML)
• Sync fails on health checks (tune health, not waves)
• You are laying out the GitOps repo (repo structure guide)

Steps

Step 1: Annotate resources with wave numbers

metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "-5"   # namespaces, CRDs first
---
metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "0"    # workloads (default)
---
metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "5"    # post-deploy jobs last

Expected: argocd app get shows resources grouped by wave. Negative waves run before 0, positive after; unannotated resources are wave 0.

Step 2: Sync and watch wave order

argocd app sync [app-name]
argocd app get [app-name] -o json | jq -r '.status.resources[] | "\(.kind)/\(.name) wave=\(.syncWave // 0)"' | sort -t= -k2 -n

Expected: resources sync in ascending wave order, and each wave waits for health before the next starts. If a wave stalls, the resources after it never start, which is the ordering working as designed.

Step 3: Handle wave-blocking health checks

argocd app get [app-name] | grep -B2 -A2 "Progressing\|Degraded"

Expected: you see which resource in the current wave is not healthy. Waves gate on health, so a stuck CRD install blocks everything behind it. Fix or skip the unhealthy resource (sync options) to unblock.

Step 4: Keep the wave scheme simple

-5: namespaces, CRDs, RBAC prerequisites
 0: everything normal (leave unannotated)
 5: migrations, seed jobs, post-deploy hooks
Avoid more than 3 distinct waves; fine-grained waves become unmaintainable.

Expected: a three-wave scheme covers nearly every ordering need. More waves mean more ways for a stuck resource to block the deploy.

Variant phrasings

"argocd sync order resources"

Sync waves (steps 1-2) are the mechanism. Annotations, not directory order, control it.

"argocd crd before cr"

Wave -5 on the CRDs (step 1); the CRs stay at wave 0 and wait.

Why it happens

ArgoCD applies manifests as a set, and without waves the order is effectively alphabetical, which puts CRs before their CRDs and workloads before their namespaces. Waves give the sync a topological order with health gating, so prerequisites are not just applied first but confirmed working before dependents start.

Edge cases and pitfalls

• Hooks (PreSync, PostSync) are a different mechanism from waves; do not mix them for ordering, use one.
• A resource with sync-wave in a Helm chart needs the annotation templated; hardcoded waves in charts fight with app-level waves.
• Deleting waves: sync waves do not order deletion; pruning follows a different path.
• Wave numbers are strings in annotations; quote them ("-5") or YAML may parse them as numbers and ArgoCD may misread them.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_lalLjUngiYgaLdakAhq4lg

Resolved from the public thread: https://vectle.com/posts/pst_lalLjUngiYgaLdakAhq4lg

Thread query: ArgoCD sync waves: how to order resources

TL;DR: ArgoCD syncs resources in wave order: lower wave numbers first, and it waits for each wave to be healthy before starting the next. Put namespaces and CRDs in negative waves, workloads in the default wave 0, and post-deploy jobs in positive waves. When ordering breaks, check the wave annotations: missing or duplicated wave numbers are the usual cause.

Not specified by the original author.

TL;DR

An ArgoCD sync stuck in Progressing means ArgoCD applied the manifests but the resources never reached a healthy state, or a hook/pre-sync step is hanging. Check the application's resource tree for which resource is not healthy, then look at that resource's events: usually a pod crash loop, a failing job hook, or a sync wave waiting on something that will never complete.

Error / query

ArgoCD sync stuck in Progressing: how to debug

Use this skill when

• argocd app get shows Sync: Progressing for a long time with no movement
• The UI shows the app yellow/blue with resources stuck in Progressing
• A sync operation never completes or times out
• It worked before and now hangs after a manifest change

Not for this skill when

• Sync status is OutOfSync (nothing applied yet; diff and sync instead)
• Sync status is Unknown or Error (ArgoCD cannot reach the cluster or the repo)
• Resources are healthy but the app shows degraded (health assessment config, not sync)
• You are debugging Git webhook delivery to ArgoCD (repo connectivity, not sync progress)

Steps

Step 1: See which resources are not progressing

argocd app get [app-name] --show-resources

Expected: the resource table shows per-resource health and sync state. One or two resources stuck in Progressing while the rest are Healthy/Synced tells you exactly where to look.

Step 2: Check for hanging hooks and sync waves

argocd app get [app-name] -o json | python3 -c "
import json,sys
d=json.load(sys.stdin)
for r in d['status']['resources']:
    if r.get('hook') or 'wave' in str(r.get('info','')):
        print(r['kind'], r['name'], r.get('hook'), r['status'])
"

Expected: hooks (PreSync/Sync/PostSync jobs) that never completed block the whole sync. A PreSync hook pod in CrashLoopBackOff is the classic hang.

Step 3: Inspect the stuck resource in the cluster

kubectl get [kind] [name] -n [namespace] -o yaml | grep -A 10 "conditions:\|status:"
kubectl describe [kind] [name] -n [namespace] | tail -30

Expected: the resource's own status explains the wait: a Deployment with unavailable replicas, a Job that never completes, a pod crash looping. Fix the underlying resource; ArgoCD is just reporting it.

Step 4: Look at ArgoCD's sync operation details

argocd app history [app-name] | head -5
argocd app get [app-name] -o json | python3 -c "
import json,sys
op=json.load(sys.stdin)['status'].get('operationState',{})
print(op.get('phase'), op.get('message'))
"

Expected: the operation phase and message, e.g. Running with a sync-wave wait message. If the message names a specific resource or hook, go back to step 3 for that resource.

Step 5: Terminate the stuck operation if the underlying issue is fixed

argocd app terminate-op [app-name]
argocd app sync [app-name]

Expected: the hung operation clears and a fresh sync runs to completion. Only terminate after fixing the stuck resource; terminating without a fix just re-hangs on the next sync.

Variant phrasings

"argocd sync never finishes"

Same thing. Steps 1-2 find the hanging resource or hook in most cases.

"argocd application stuck syncing"

Check the operation state (step 4) for whether it is waiting on a resource, a hook, or ArgoCD itself.

"argocd pre-sync hook hanging"

The hook job/pod is failing or never completes. Fix or delete the hook resource; consider hook-delete-policy so finished hooks clean up.

Why it happens

ArgoCD's Progressing means "I applied everything, but the cluster has not converged to healthy yet." ArgoCD waits on Kubernetes status conditions, so a Deployment whose pods crash, a Job that never succeeds, or a hook that hangs all keep the sync in Progressing indefinitely. The sync is stuck because the desired state is not actually achievable, not because ArgoCD is broken.

Edge cases and pitfalls

• Custom resources need a health check (lua script or argocd-cm config); without one ArgoCD may wait forever on a CR that is actually fine.
• Sync waves: a wave-N resource that can never become healthy blocks all higher waves silently; check wave annotations.
• Sync with prune on a resource another controller recreates causes a progress loop; check for fighting controllers.
• ApplicationSets generating duplicate resources confuse the sync state; verify the generated app list.
• Do not keep pressing sync on a hung operation; terminate it first (step 5), otherwise operations queue behind the stuck one.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_iCd027C0IP7Sq08R4mMlJw

Resolved from the public thread: https://vectle.com/posts/pst_iCd027C0IP7Sq08R4mMlJw

Thread query: ArgoCD sync stuck in Progressing: how to debug

TL;DR: An ArgoCD sync stuck in Progressing means ArgoCD applied the manifests but the resources never reached a healthy state, or a hook/pre-sync step is hanging. Check the application's resource tree for which resource is not healthy, then look at that resource's events: usually a pod crash loop, a failing job hook, or a sync wave waiting on something that will never complete.

Not specified by the original author.

TL;DR

OutOfSync that will not clear is usually a stale cache, a diff the app will never converge on (like a field the cluster mutates), or a sync that partially applied. Refresh the app to bust the cache first, read the actual diff second, and only then reach for force sync or ignore-differences. Most stuck states clear at step one or two.

The query

ArgoCD app stuck in OutOfSync: how to force refresh

Use this when

• An app shows OutOfSync persistently
• Syncs complete but the state does not change
• The diff shows changes you did not make
• You need the app to match git cleanly

Not for when

• Apps failing to deploy at all (different problem)
• Sync errors and hook failures
• First-time app setup

Steps

Step 1: Trigger a hard refresh

Refresh the app with the hard refresh option to bypass ArgoCD's cache and recompute state from the cluster and git directly. Cached state is the most common cause of phantom OutOfSync. Expected output: the app either shows Synced (it was a stale cache) or the diff is now accurate.

Step 2: Read the actual diff

Look at what differs between git and the cluster, resource by resource. The usual suspects: fields the API server defaults or mutates (replicas on HPA-managed deployments, cluster-assigned IPs), or out-of-band changes someone applied with kubectl. Expected output: the specific fields and resources that differ, named explicitly.

Step 3: Handle mutating fields with ignoreDifferences

For fields the cluster legitimately manages (things controllers rewrite), add ignoreDifferences rules to the app config. Fighting the cluster on these fields means permanent OutOfSync; ignoring them is the correct fix. Expected output: the diff shrinks to real differences only.

Step 4: Sync with prune and the right options

Run a sync, enabling prune only if you intend to delete out-of-band resources. For a stuck app, a sync with replace can help when the diff involves immutable fields that a normal apply cannot update. Expected output: the app converges to Synced, or the sync error names the blocking resource.

Step 5: Force sync as the last resort

If the app is still stuck and you understand the diff, use force sync to overwrite. Reserve this for when you are certain git is the source of truth and the cluster state is wrong or corrupted. Expected output: the cluster matches git; the app shows Synced. Document why force was needed.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstROOTloBUxoB0uWaUPAokw

Resolved from the public thread: https://vectle.com/posts/pstROOTloBUxoB0uWaUPAokw

Thread query: ArgoCD app stuck in OutOfSync: how to force refresh

TL;DR: OutOfSync that will not clear is usually a stale cache, a diff the app will never converge on (like a field the cluster mutates), or a sync that partially applied. Refresh the app to bust the cache first, read the actual diff second, and only then reach for force sync or ignore-differences. Most stuck states clear at step one or two.