Datadog Agent does not start with read-only root filesystem

Datadog Agent does not start with read-only root filesystem

Whats going on

Running the Datadog Agent container with a read-only root filesystem: the agent dies on startup with ln: failed to create symbolic link '/etc/datadog-agent/datadog.yaml': Read-only file system and rm: cannot remove ... Read-only file system errors. Same on ECS Fargate (readonlyRootFilesystem: true) and Kubernetes.

What actually fixes it

Datadog has now shipped official read-only root filesystem (ROFS) support, confirmed by a maintainer on the thread. On Kubernetes: datadog helm chart v3.150.0+ enables readOnlyRootFilesystem by default for all agent containers (override with securityContext.readOnlyRootFilesystem if you need it off); the Datadog Operator v1.22 does the same. On ECS Fargate: the Datadog Terraform module exposes ddreadonlyrootfilesystem and the CDK construct exposes DatadogECSFargateProps#readOnlyRootFilesystem, use those instead of hand-rolling, because mounting your own writable volume at /etc/datadog-agent shadows the image's default conf.d files and loses the ecsfargate check. For other setups, the official guide walks through the manual mounts: the Agent writes to /etc/datadog-agent/, /tmp, /var/log/datadog and /var/run/datadog, so mount writable volumes there.

Original thread

https://vectle.com/threads/thr_FeEbeLbIDD88aATKG6FWvg

In the thread https://vectle.com/threads/thrFeEbeLbIDD88aATKG6FWvg, the question was: Running the Datadog Agent container with a read-only root filesystem: the agent dies on startup with ln: failed to create symbolic link '/etc/datadog-agent/datadog.yaml': Read-only file system and rm: cannot remove ... Read-only file system errors. Same on ECS Fargate (readonlyRootFilesystem: true) and Kubernetes. The verified resolution was: Datadog has now shipped official read-only root filesystem (ROFS) support, confirmed by a maintainer on the thread. On Kubernetes: datadog helm chart v3.150.0+ enables readOnlyRootFilesystem by default for all agent containers (override with securityContext.readOnlyRootFilesystem if you need it off); the Datadog Operator v1.22 does the same. On ECS Fargate: the Datadog Terraform module exposes ddreadonlyrootfilesystem and the CDK construct exposes DatadogECSFargateProps#readOnlyRootFilesystem, use those instead of hand-rolling, because mounting your own writable volume at /etc/datadog-agent shadows the image's default conf.d files and loses the ecs_fargate check. For other setups, the official guide walks through the manual mounts: the Agent writes to /etc/datadog-agent/, /tmp, /var/log/datadog and /var/run/datadog, so mount writable volumes there.