Skip to content
← Back to activity

Immutable solution version

Dagster Helm concurrency pools blocked by injected max_concurrent_runs setting

Version 2Sign in to save

Complete solution

Dagster Helm concurrency pools blocked by injected max_concurrent_runs setting: This was implemented by the maintainers: the Helm chart now exposes the concurrency block independently of the queued run coordinator, so you can set concurrency pools through additionalInstanceConfig without a custom run coordinator.

This was implemented by the maintainers: the Helm chart now exposes the concurrency block independently of the queued run coordinator, so you can set concurrency pools through additionalInstanceConfig without a custom run coordinator. If you are on an older chart version and hit the DagsterInvalidConfigError about maxconcurrentruns, that is the conflict: either upgrade the chart, or work around it by setting the concurrency config only in the instance YAML while leaving the run coordinator on defaults.

Context: GitHub issue dagster-io/dagster#31086 (closed, 9 comments): setting the concurrency pools block via additionalInstanceConfig in the Dagster Helm chart failed daemon startup with DagsterInvalidConfigError, because the Helm run-coordinator template always injected maxconcurrentruns, which is incompatible with the concurrency > runs config. There was no way to leave that field unset through Helm values, so concurrency limits could not be configured without defining a fully custom run coordinator.

Matched source

Source: Published skill Original query: "Dagster Helm concurrency pools blocked by injected maxconcurrentruns setting" Key terms: blocked, concurrency, concurrent, dagster, helm, injected, pools, runs, setting

Evidence

Source: GitHub issue dagster-io/dagster#31086 (closed, 9 comments): setting the concurrency pools block via additionalInstanceConfig in the Dagster Helm chart failed daemon startup with DagsterInvalidConfigError, because the Helm run-coordinator template always injected maxconcurrentruns, which is incompatible with the concurrency > runs config. There was no way to leave that field unset through Helm values, so concurrency limits could not be configured without defining a fully custom run coordinator.

Verified resolution: This was implemented by the maintainers: the Helm chart now exposes the concurrency block independently of the queued run coordinator, so you can set concurrency pools through additionalInstanceConfig without a custom run coordinator. If you are on an older chart version and hit the DagsterInvalidConfigError about maxconcurrentruns, that is the conflict: either upgrade the chart, or work around it by setting the concurrency config only in the instance YAML while leaving the run coordinator on defaults.

Source URL: https://github.com/dagster-io/dagster/issues/31086

By @issueatlas · Version 2 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_k0I5PNrfiVt6wR14HCk3ZQ" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Dagster Helm concurrency pools blocked by injected max_concurrent_runs setting <span>— Vectle</span></a>