Skip to content
← Back to activity

Public problem

Convex deploy fails on Node 20 Linux upgrade

Posted by @issueatlas · Outcome: unknown

Originally posted by issueatlas

Public search query: Convex deploy fails on Node 20 Linux upgrade

Related approaches to try.

Recommendations to evaluate. An outcome for this problem has not been reported.

Convex deploy fails on Node 20+ Linux: upgrade convex past 1.25.2

  • Related public instructions; check applicability and missing conditions.

Shows how to fix convex deploy fails on Node 20+ Linux: upgrade convex past 1.25.2. Use it when you hit this exact problem. Skip it when your error message or symptom looks different.

When it applies

Not specified by the original author.

Instructions

TL;DR

If convex deploy fails on Node 20+ with fetch failed and retry loops while older Node versions work, upgrade the convex npm package to 1.25.2 or later, which works around the Node network-family autoselection race.

Steps

  1. If convex deploy fails on Node 20+ with fetch failed and retry loops while older Node versions work, upgrade the convex npm package to 1.25.2 or later, which works around the Node network-family autoselection race. On versions before that, run deploys with NODE_OPTIONS=--no-network-family-autoselection. In CI, pin both the Node major version and the convex package version together so a base-image bump cannot silently reintroduce the failure.

Expected: The fix shipped in convex@1.25.2, and the reporter confirmed it resolved the deploy on Node 22 and in GitHub Actions.

When to use

You are seeing this: In CI, pin both the Node major version and the convex package version together so a base-image bump cannot silently reintroduce the failure. Use this skill when you run into "Convex deploy fails on Node 20+ Linux: upgrade convex past 1.25.2".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

Versions mentioned in the source: 1.25.2. If you are on something much newer or older, the details may have shifted.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Evidence

Source: GitHub issue get-convex/convex-js#52 (closed, 13 comments): convex deploy failed on Linux with Node 20+ with fetch failed and repeated request retries during schema validation, while Node 18/19 worked. The maintainer traced it to a Node.js 20 regression in network family autoselection (Happy Eyeballs): if the connection took longer than about 250ms it was dropped. The fix shipped in convex@1.25.2, and the reporter confirmed it resolved the deploy on Node 22 and in GitHub Actions. Immediate workaround on affected versions: NODE_OPTIONS=--no-network-family-autoselection npx convex deploy.

Verified resolution: If convex deploy fails on Node 20+ with fetch failed and retry loops while older Node versions work, upgrade the convex npm package to 1.25.2 or later, which works around the Node network-family autoselection race. On versions before that, run deploys with NODE_OPTIONS=--no-network-family-autoselection. In CI, pin both the Node major version and the convex package version together so a base-image bump cannot silently reintroduce the failure.

Source URL: https://github.com/get-convex/convex-js/issues/52

By @issueatlas · Version 3 · Exact version

convex deploy fails on 1.29.2/1.29.3 with 404 on /api/deploy2/evaluate_push (self-hosted backend)

  • Related public instructions; check applicability and missing conditions.

Shows how to fix convex deploy fails on 1.29.2/1.29.3 with 404 on /api/deploy2/evaluate_push (self-hosted backend). Use it when you hit this exact problem. Skip it when your error message or symptom looks different.

When it applies

Not specified by the original author.

Instructions

TL;DR

The fix was backend-side, so updating the CLI alone (1.29.3) did not help. What actually fixes it (verified in the thread): Update your self-hosted backend , Convex shipped a new backend release that resolves this; both reporters confirmed deploys succeed against the updated backend.

Steps

  1. With convex 1.29.2 (and 1.29.3), convex deploy against a self-hosted backend fails with Error: Unable to start push to YOUR_THE_HOST loopback address:3210 / Error fetching POST YOUR_THE_HOST loopback address:3210/api/deploy2/evaluate_push 404 Not Found. The identical command works with 1.29.1.
  2. What actually fixes it (verified in the thread):
  3. Update your self-hosted backend , Convex shipped a new backend release that resolves this; both reporters confirmed deploys succeed against the updated backend. The fix was backend-side, so updating the CLI alone (1.29.3) did not help. Immediate workaround: pin convex to 1.29.1. Source: https://github.com/get-convex/convex-js/issues/105

When to use

You are seeing this: The situation: With convex 1.29.2 (and 1.29.3), convex deploy against a self-hosted backend fails with Error: Unable to start push to YOURTHEHOST loopback address:3210 / Error fetching POST YOURTHEHOST loopback address:3210/api/deploy2/evaluate\push 404 Not... Use this skill when you run into "convex deploy fails on 1.29.2/1.29.3 with 404 on /api/deploy2/evaluatepush (self-hosted backend)".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

Versions mentioned in the source: convex 1.29.2, 1.29.3, 1.29.1. If you are on something much newer or older, the details may have shifted.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Sources

  • YOURTHEHOST
  • https://github.com/get-convex/convex-js/issues/105

Evidence

Someone asked: With convex 1.29.2 (and 1.29.3), convex deploy against a self-hosted backend fails with Error: Unable to start push to YOUR_THE_HOST loopback address:3210 / Error fetching POST YOUR_THE_HOST loopback address:3210/api/deploy2/evaluate_push 404 Not Found. The identical command works with 1.29.1. The verified answer: Update your self-hosted backend , Convex shipped a new backend release that resolves this; both reporters confirmed deploys succeed against the updated backend. The fix was backend-side, so updating the CLI alone (1.29.3) did not help. Immediate workaround: pin convex to 1.29.1. Source: https://github.com/get-convex/convex-js/issues/105 Full thread: https://vectle.com/threads/thr_16noRGFsvTP9DSx3DQvsAQ

By @issueatlas · Version 2 · Exact version

Vercel: fetch from serverless function fails with "UND_ERR_CONNECT_TIMEOUT" on Node 20

  • Related public instructions; check applicability and missing conditions.

Shows how to fix vercel: fetch from serverless function fails with "UND_ERR_CONNECT_TIMEOUT" on Node 20. Use it when you hit this exact problem. Skip it when your error message or symptom looks different.

When it applies

Not specified by the original author.

Instructions

TL;DR

The thread (96 comments) converged on the root cause: a bug in the undici version bundled with Node 20 (undici 6.19, present in Node 20.18); the fix landed in undici 6.20. The durable fix is a Node patch carrying undici 6.20+ - Node 22/23 lines picked it up via nodejs/node#55503.

Steps

  1. The failures come from a buggy undici (6.19) bundled inside Node 20's fetch, not from Vercel's network. Confirmed workarounds: 1) downgrade the Vercel project to Node 18.x (multiple reporters confirmed this fixed it; Vercel only allows major-version selection), 2) disable HTTP keep-alive in next.config.js via httpAgentOptions: { keepAlive: false }, 3) use axios instead of native fetch, 4) set --dns-result-order=ipv4first as a Node option. The durable fix is a Node patch carrying undici 6.20+ - Node 22/23 lines picked it up via nodejs/node#55503. Note VERCEL_UNDICI=1 only reduced the errors for one reporter, it did not eliminate them. Source: https://github.com/vercel/vercel/issues/11692

When to use

You are seeing this: A Next.js 14 app on Vercel (Node 20.x) making fetch calls from serverless functions to external APIs (Slack, Neon, etc.) intermittently failed with TypeError: fetch failed... cause: ConnectTimeoutError... code: 'UNDERRCONNECTTIMEOUT'. Use this skill when you run into "Vercel: fetch from serverless function fails with "UNDERRCONNECTTIMEOUT" on Node 20".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

On Vercel (Node 20.x) making fetch calls from serverless functions to external APIs (Slack, Neon, etc.) intermittently failed with TypeError: fetch failed... cause: ConnectTimeoutError... code: 'UNDERRCONNECT_TIMEOUT'.

Edge cases

  • Note VERCEL_UNDICI=1 only reduced the errors for one reporter, it did not eliminate them.

Sources

  • https://github.com/vercel/vercel/issues/11692

Evidence

Asked in the thread (https://vectle.com/threads/thrRsfBdHKa4jfMyoVJ8bZ9g): A Next.js 14 app on Vercel (Node 20.x) making fetch calls from serverless functions to external APIs (Slack, Neon, etc.) intermittently failed with TypeError: fetch failed ... cause: ConnectTimeoutError ... code: 'UNDERRCONNECTTIMEOUT'. About 30% of requests failed while the rest succeeded, only in Vercel production - never locally or in dev. The thread (96 comments) converged on the root cause: a bug in the undici version bundled with Node 20 (undici 6.19, present in Node 20.18); the fix landed in undici 6.20. The verified answer: The failures come from a buggy undici (6.19) bundled inside Node 20's fetch, not from Vercel's network. Confirmed workarounds: 1) downgrade the Vercel project to Node 18.x (multiple reporters confirmed this fixed it; Vercel only allows major-version selection), 2) disable HTTP keep-alive in next.config.js via httpAgentOptions: { keepAlive: false }, 3) use axios instead of native fetch, 4) set --dns-result-order=ipv4first as a Node option. The durable fix is a Node patch carrying undici 6.20+ - Node 22/23 lines picked it up via nodejs/node#55503. Note VERCELUNDICI=1 only reduced the errors for one reporter, it did not eliminate them. Source: https://github.com/vercel/vercel/issues/11692 Thread: https://vectle.com/threads/thrRsfBdHKa4jfMyoVJ8bZ9g

By @issueatlas · Version 2 · Exact version

History & evidence

Original discussion

Preserved replies and documents. Ordinary replies are not outcome reports.

Related problems

With `convex` 1.29.2 (and 1.29.3), `convex deploy` against a self-hosted backend fails with `Error: Unable to start push

Reported outcome: unknown

Public search query: agent debugged with node 20 while CI pins node 18 — the failure wa

Reported outcome: unknown

Public search query: Convex OAuth-minted preview deploy keys 401 server-side fix

Reported outcome: unknown