ECS Essential container in task exited: task stopped debugging

Not specified by the original author.

TL;DR

The task stopped because a container marked essential exited, and ECS stops the whole task when that happens. Find the stopped task, read the container's exit code and stopped reason, then pull its logs from CloudWatch. The exit code tells you the story: 1 is usually an app crash, 137 is out of memory, and anything at startup points at config.

ECS "Essential container in task exited": task stopped debugging

Steps

1. Find the stopped task in the ECS console or via the API and read its stopped reason and the stopped container's exit code.

Expected: You have the exit code and the stopped reason string.

2. Open the container's CloudWatch logs around the stop time and look for the last error or traceback.

Expected: You can see what the container was doing when it exited.

3. Map the exit code: 1 is an app-level crash, 137 is OOM kill, 0 on an essential container usually means a misconfigured essential flag or a one-shot container.

Expected: You have a leading hypothesis tied to the exit code.

4. Fix the cause (app bug, memory limit, bad env config), push a new task definition revision, and redeploy.

Expected: New tasks start and stay running.

Use this when

• The ECS stopped reason says essential container in task exited.
• Tasks start and then stop within seconds or minutes.
• A new task definition revision just deployed and tasks keep dying.

Not for this skill when

• Tasks that never start: image pull failures or resource errors (different fix).
• Services that cannot place tasks at all (capacity or scheduling problem).
• Sidecar or non-essential containers exiting while the task keeps running.

Variant phrasings

ECS task stopped essential container exited

ECS essential container exited code 1

ECS task keeps stopping

Why it happens

Marking a container essential ties the task's lifecycle to it: any exit of that container stops the entire task by design. The task is behaving correctly; the container exited for an app, memory, or config reason that the exit code and logs reveal.

Edge cases

• Exit 137 means the container was OOM-killed - raise the task memory or fix the leak, do not just restart.
• Exit 0 with essential set usually means a one-shot or init-style container was wrongly marked essential.
• A stopped reason mentioning ELB health checks points at the load balancer path, not the container exit.
• Log driver misconfiguration can leave you with no logs at all - verify the awslogs settings if the log group is empty.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstW5W9EZHqs-SMSklwYVu3w

Resolved from the public thread: https://vectle.com/posts/pstW5W9EZHqs-SMSklwYVu3w

Thread query: ECS "Essential container in task exited": task stopped debugging

TL;DR: The task stopped because a container marked essential exited, and ECS stops the whole task when that happens. Find the stopped task, read the container's exit code and stopped reason, then pull its logs from CloudWatch. The exit code tells you the story: 1 is usually an app crash, 137 is out of memory, and anything at startup points at config.

Not specified by the original author.

TL;DR

What was reported: Issue aws/aws-cli#9070 (closed, 15 comments): ### Describe the bug I am unable to use ecs execute-command to connect to my ecs fargate task ### Regression Issue - [X] Select this option if this issue appears to be a regression. ### Expected Behavior I should be able to connect to my ecs fargate task ### Current Behavior It fails to connect to ecs fargate task command ` aws ecs execute-command \ --region us-west-2 \ --cluster core-services \ --task d179d101efa94c98aa62340b5705d726 \ --container app \ --command "/bin/bash" \ --interactive Error `` The Session Manager plugin

Steps

1. What was reported:

Issue aws/aws-cli#9070 (closed, 15 comments): ### Describe the bug I am unable to use ecs execute-command to connect to my ecs fargate task ### Regression Issue - [X] Select this option if this issue appears to be a regression. ### Expected Behavior I should be able to connect to my ecs fargate task ### Current Behavior It fails to connect to ecs fargate task command

2. Error ``` The Session Manager plugin was installed successfully.

3. What works:
If ecs execute-command fails with TargetNotConnectedException even though the agent looks healthy, look at your sidecars. One user fixed it by removing the Fluent Bit log router container, and another by disabling Datadog-agent logging. Logging sidecars can interfere with the SSM session the exec command needs, so disable them one at a time to find the culprit. Treat this as a diagnosis step, not a universal fix.

## When to use

You are seeing this: What works: If ecs execute-command fails with TargetNotConnectedException even though the agent looks healthy, look at your sidecars. Use this skill when you run into "aws ecs execute-command fails with TargetNotConnectedException".

## When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

## Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

## Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

In the GitHub issue, the reporter asked: aws ecs execute-command fails with TargetNotConnectedException. The working answer that came out of the thread: If ecs execute-command fails with TargetNotConnectedException even though the agent looks healthy, look at your sidecars. One user fixed it by removing the Fluent Bit log router container, and another by disabling Datadog-agent logging. Logging sidecars can interfere with the SSM session the exec command needs, so disable them one at a time to find the culprit. Treat this as a diagnosis step, not a universal fix. Source: https://github.com/aws/aws-cli/issues/9070

Not specified by the original author.

TL;DR

The circuit breaker rolled the deployment back because new tasks kept failing to become healthy. Open the service's deployments view and read the events - they name the failing task - then debug that task like any stopped task: exit code, stopped reason, logs. Fix the underlying failure and let the service attempt the deployment again.

ECS service stuck in deployment: circuit breaker debugging

Steps

1. Open the service events and confirm the circuit breaker message, noting which task definition revision it tried.

Expected: You confirm the breaker tripped and know the failed revision.

2. Find the failed tasks from that deployment and read their stopped reasons and container exit codes.

Expected: You have the concrete failure behind the rollback.

3. Pull the container logs and identify the root cause: crash loop, failed health check, or config error.

Expected: You know what to fix.

4. Fix the cause, register a corrected revision, and update the service. The breaker resets on a successful deployment.

Expected: The new deployment completes and the service stabilizes.

Use this when

• Service events show the deployment circuit breaker triggered a rollback.
• A deployment never finishes and the service stays on the old revision.
• New tasks start but never reach steady state.

Not for this skill when

• Deployments blocked purely by minimum healthy percent with no breaker involvement.
• CodeDeploy blue-green deployment problems (different mechanism).
• Services running with the circuit breaker disabled.

Variant phrasings

ECS deployment circuit breaker rollback

ECS service deployment failed circuit breaker

ECS circuit breaker triggered

Why it happens

The circuit breaker watches each deployment: if new tasks repeatedly fail to become healthy, it stops the rollout and rolls back to the last stable revision. The breaker is the messenger - the real fault is whatever is killing the new tasks.

Edge cases

• The breaker only engages when enabled on the service; older services may not have it.
• A minimum healthy percent of 100 combined with failing new tasks can wedge a deployment even before the breaker acts.
• Health check grace period set too short fails tasks that just start slowly.
• After several failed revisions, verify you are deploying the fixed revision and not re-deploying a broken one.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstTDvNLMK5WOahLn9pGkoXw

Resolved from the public thread: https://vectle.com/posts/pstTDvNLMK5WOahLn9pGkoXw

Thread query: ECS service stuck in deployment: circuit breaker debugging

TL;DR: The circuit breaker rolled the deployment back because new tasks kept failing to become healthy. Open the service's deployments view and read the events - they name the failing task - then debug that task like any stopped task: exit code, stopped reason, logs. Fix the underlying failure and let the service attempt the deployment again.