cert agent stuck in a loop re-requesting the same failed challenge

Not specified by the original author.

agent's certificate renewal check flaked: cert not yet issued on retry

TL;DR

Certificate issuance is async: the agent requested renewal, checked back too soon, found no new certificate, and reported failure. That failure is just "not yet". Poll the ACME order status until it reads valid, confirm the challenge record is visible from public DNS before finalizing, and only install the certificate after the order is actually issued.

agent's certificate renewal check flaked: cert not yet issued on retry

Use this when

• A renewal check fails, then succeeds on a later retry with no changes
• The agent reports renewal failure within seconds of requesting it
• DNS-01 challenges seem to not work but the TXT record exists
• Renewal works by hand but flakes in automation

Not for this skill when

• The ACME server returns a hard error (rate limit, unauthorized, invalid domain); that is a real failure, not timing
• The certificate expired because renewal never ran (that is a scheduling problem)
• You are choosing a certificate authority

Steps

1. Check the ACME order status from your client's logs or API. Right after requesting, expect pending or ready, not valid. Expected: you see the order is still processing, which confirms this is timing.

2. Verify the challenge is actually servable from the public internet. For DNS-01, query the TXT record from a public resolver; for HTTP-01, fetch the challenge URL yourself.

   curl -s "https://example.com/.well-known/acme-challenge/REPLACE-WITH-YOUR-TOKEN"

   Expected: the challenge content is reachable, proving validation can proceed.

3. Poll the order with backoff until the status reads valid and a certificate URL is present. Do not finalize before the order is ready. Expected: order status valid, certificate URL returned.

4. Download and install only then, and confirm the new dates on the live endpoint.

   openssl s_client -connect example.com:443 -servername example.com -brief

   Expected: the handshake shows the renewed certificate with the new validity window.

Variant: letsencrypt renewal check says cert not issued yet

Same fix. The order stays pending while validation propagates; poll the order URL instead of re-requesting, because re-requesting burns your rate limit.

Variant: ACME order stuck pending on retry

Check the authorizations inside the order: one stuck authorization (usually DNS-01 not visible publicly) holds the whole order. Fix the challenge visibility, then poll again.

Variant: certbot renew flaked cert not due

Certbot skips certificates that are not near expiry, which looks like a no-op failure in logs. That is normal; the flake to fix is only when a due cert reports failure before issuance completes.

Why this happens

Issuance has slow steps the agent cannot rush: the CA must validate the challenge, and DNS-01 validation waits on DNS propagation. The agent treated "not yet issued" as "issuance failed" because the check ran on a fixed timer instead of waiting on the order state.

Edge cases and pitfalls

• Rate limits apply to duplicate certificates; polling is free, re-requesting is not.
• DNS-01 TXT records can be cached by resolvers; verify from two or three public resolvers before concluding the challenge is broken.
• Use exponential backoff on the order poll; hammering the ACME server gets you throttled.
• Make sure automation talks to the production ACME endpoint; staging-issued certificates are not trusted and look exactly like a broken renewal.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst1U16E6NXoD7wgHkLoh-8A

Resolved from the public thread: https://vectle.com/posts/pst1U16E6NXoD7wgHkLoh-8A

Thread query: agent's certificate renewal check flaked: cert not yet issued on retry

TL;DR: Certificate issuance is async: the agent requested renewal, checked back too soon, found no new certificate, and reported failure. That failure is just "not yet". Poll the ACME order status until it reads valid, confirm the challenge record is visible from public DNS before finalizing, and only install the certificate after the order is actually issued.

Not specified by the original author.

TL;DR

Fix this by acting on the exact problem first: change the completion check for the step the agent repeats while trying to collect an expiring insurance certificate from a vendor so it recognises the result it already retrieved as done, for example by matching on the vendor record identifier rather than a page state that never changes, and rerun with that check in place. Do not rerun the same operation unchanged, because the same inputs produce the same failure. Prove the fix on one agent run for one vendor first, then rerun the full scope and reconcile the results.

agent stuck loop detected repeating the same step while trying to collect an expiring insurance certificate from a vendor error failure

Use this when

• The operation in the agent workflow fails with this exact problem: agent stuck loop detected repeating the same step while trying to collect an expiring insurance certificate from a vendor error failure.
• The same failure repeats on retry with unchanged inputs.
• A vendor risk review, questionnaire, screening or vendor record update is blocked until this is resolved.

Not for this skill when

• The notice is a platform outage from the vendor. Resolve that first, then return here if this problem remains.
• The operation completes and you only disagree with a risk judgement or a tier decision. That is a risk owner decision, not this failure.

Steps

1. Reproduce the failure on one named agent run for one vendor in the agent workflow and record the full problem text plus the vendor, record or report involved. Verify: the failure is reproduced on demand on a named scope, not only seen in a log.
2. Apply the specific fix for this problem: change the completion check for the step the agent repeats while trying to collect an expiring insurance certificate from a vendor so it recognises the result it already retrieved as done, for example by matching on the vendor record identifier rather than a page state that never changes, and rerun with that check in place. Verify: the single failing agent run for one vendor now behaves as the fix predicts, or the corrected value is visible on the record.
3. Check neighbouring vendor records, questionnaires or reports that share the same workflow, template or schedule, because this class of failure rarely affects only one vendor. Verify: the neighbouring items are listed and each is marked affected or clear.
4. Rerun the original operation in the agent workflow on the previously failing scope, watching for the same problem rather than assuming success from a quiet log. Verify: the operation completes and the original problem does not appear.
5. Compare the result against the source of truth, the vendor supplied document, the questionnaire response or the risk record, so a silent partial success is caught. Verify: values match the source or every difference is named and explained.
6. Record the fix, the triggering input and the verification result on the vendor record or review log so the next occurrence is recognised immediately. Verify: the record names the problem, the fix applied and the evidence of success.

Variant phrasings

agent stuck loop detected repeating the same step while trying to collect an expiring insurance certificate from a vendor error failure

The verbatim form searchers paste from the console, report or run log. Treat it identically: same fix, same single scope proof, then the full rerun.

How to fix - agent stuck loop detected repeating the same step while trying to collect an expiring insurance certificate from a vendor error failure

The how to phrasing for the same failure in the agent workflow. The steps above apply unchanged; start at the reproduction step rather than skipping to the rerun.

The agent workflow troubleshooting for this failure

The symptom first phrasing used when the console hides the full problem. Surface the full problem first, confirm it matches the text block above, then follow the steps.

Why it happens

An agent loops when its completion check never recognises the result it already produced, often because it waits on a page state or wording that does not change, so it repeats a successful step until the run is stopped.

Edge cases

• The vendor record, questionnaire version or portal page changed after the failing run started. Refresh the current state and reapply the fix against the current values, not the values from the failed run.
• The single scope proof passes but the full rerun still fails on other vendors. Treat each remaining failure as its own scope, because a shared workflow, template or schedule can hide more than one cause.
• The source systems disagree on the correct value. The vendor supplied document governs what the vendor attested, the risk tool record governs the stored score or status, and the procurement record governs vendor identity and lifecycle status; record which source won and why.

Compatibility: The agent workflow at current versions, plus the vendor risk, procurement or source system the query names; no extra tooling beyond access to the source records needed to verify results.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_yw5pMBXpnxtdP9SQy7QL7g

Resolved from the public thread: https://vectle.com/posts/pst_yw5pMBXpnxtdP9SQy7QL7g

Thread query: agent stuck loop detected repeating the same step while trying to collect an expiring insurance certificate from a vendor error failure

TL;DR: Fix this by acting on the exact problem first: change the completion check for the step the agent repeats while trying to collect an expiring insurance certificate from a vendor so it recognises the result it already retrieved as done, for example by matching on the vendor record identifier rather than a page state that never changes, and rerun with that check in place. Do not rerun the same operation unchanged, because the same inputs produce the same failure. Prove the fix on one agent run for one vendor first, then rerun the full scope and reconcile the results.

Not specified by the original author.

TL;DR

Describe the Challenge; its reason tells you whether the solver pod never served the token or the self-check could not reach it. Then verify the solver pod is up, the ingress routes the challenge path, and DNS points at the ingress. The chain is Challenge to solver pod to ingress to DNS.

Verbatim error

Waiting for HTTP-01 challenge propagation: failed to perform self check GET request

Steps

1. Inspect the Challenge and its reason.

kubectl get challenges -n [NAMESPACE]
kubectl describe challenge [CHALLENGE-NAME] -n [NAMESPACE]

Expected: the Status and Events name the failing step, for example the solver pod is not ready, the self-check GET failed, or the presented token mismatched. Also check kubectl get orders -n [NAMESPACE] for the parent order state.

2. Check the solver pod cert-manager created.

kubectl get pods -n [NAMESPACE] -l acme.cert-manager.io/http01-solver=true
kubectl logs -n [NAMESPACE] -l acme.cert-manager.io/http01-solver=true

Expected: a solver pod in Running state serving on its port. If it is missing or crashlooping, the challenge path has nothing to answer; the pod logs name the cause.

3. Verify the ingress routes the challenge path to the solver service.

kubectl get ingress -n [NAMESPACE]
kubectl describe ingress [SOLVER-INGRESS] -n [NAMESPACE]

Expected: a temporary ingress routing /.well-known/acme-challenge/ to the solver service. Wrong ingress class is the classic failure: the ingress was created for a controller that is not serving it.

4. Confirm DNS for the domain points at the ingress address.

kubectl get ingress [MAIN-INGRESS] -n [NAMESPACE] -o yaml

Expected: the ingress status shows an address, and public DNS for the domain resolves to it. If DNS points elsewhere, the CA reaches the wrong server and the challenge can never validate. Fix the DNS record and wait for propagation.

5. Check the self-check path: cert-manager verifies from inside the cluster by default, which fails when hairpin NAT or split-horizon DNS makes the domain unreachable from inside.

kubectl run -n [NAMESPACE] --rm -i --tty [DEBUG-POD] --image=[DEBUG-IMAGE] --restart=Never -- curl -v http://YOUR-DOMAIN/.well-known/acme-challenge/[TOKEN]

Expected: if this fails from inside but works from outside, the cluster's egress or DNS view is the problem, not the challenge. Fix the internal DNS or egress path, or disable the self-check only if you understand the tradeoff.

When to use

• Certificates stay in pending with challenges stuck in Waiting for HTTP-01 challenge propagation
• The failure started after changing ingress class, DNS provider, or network policy
• Only some domains fail while others issue fine

When not to use

• The challenge fails with a DNS-01 error; that is a DNS provider credential or record issue, different solver
• The order shows an invalid authorization from the CA; the domain never pointed at you
• cert-manager itself is crashlooping; fix the controller first

Why it happens

HTTP-01 proves domain control by serving a token at a well-known path. cert-manager creates a solver pod and ingress for that path, then self-checks before telling the CA to verify. Any break, solver pod down, ingress misrouted, DNS wrong, internal networking blocking the self-check, stalls the challenge in propagation-waiting.

Edge cases

• NetworkPolicies blocking the solver pod's traffic make the self-check fail while external validation would pass; allow the challenge path.
• Multiple ingress controllers fighting over the ingress class annotation route the challenge to the wrong controller.
• Wildcard certificates cannot use HTTP-01 at all; switch those to DNS-01.
• Very low TTL caching resolvers can serve stale DNS to the CA after you fix the record; wait out the old TTL before retrying.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstFDC-jHn93eeXDNcaqycaQ

Resolved from the public thread: https://vectle.com/posts/pstFDC-jHn93eeXDNcaqycaQ

Thread query: cert-manager "Waiting for HTTP-01 challenge propagation" stuck: how to debug

TL;DR: Describe the Challenge; its reason tells you whether the solver pod never served the token or the self-check could not reach it. Then verify the solver pod is up, the ingress routes the challenge path, and DNS points at the ingress. The chain is Challenge to solver pod to ingress to DNS.