dbt environment variables not resolving in profiles.yml

dbt environment variables not resolving in profiles.yml

TL;DR: when dbt cannot see an environment variable you believe you set, the variable is almost always missing from the exact process that runs dbt, not from your machine. Verify with dbt debug, then confirm the variable in the same shell or job that launches dbt (printenv VAR). Then fix the delivery: export it, pass it into the container or CI step, or point dbt at the profiles.yml you actually edited.

Env var required but not provided

Steps

1. Find out which profiles.yml dbt is actually reading. Run dbt debug and look at the "Profile" and "Using profiles.yml file at" lines. Expected: the path matches the file you edited.

1. Check for an override. If DBT_PROFILES_DIR is set, or you pass --profiles-dir, dbt ignores ~/.dbt/profiles.yml. Expected: printenv DBT_PROFILES_DIR prints nothing unless you meant to set it.

1. In the SAME shell, job step, or container that runs dbt, verify the variable is visible: printenv DBT_PASSWORD. Expected: the real value prints. If it prints nothing here, dbt cannot see it either. Skip to step 5.

1. Check the variable is non-empty. An empty value behaves like an unset one at parse time. Expected: printenv DBT_PASSWORD | wc -c shows a nonzero length.

1. Export it, don't just assign it. In a shell script, DBT_PASSWORD [your value] without export is invisible to child processes including dbt. Use export DBTPASSWORD [your value] (or `export DBTPASSWORD` after assigning), then rerun. Expected: the error moves on or disappears.

1. If dbt runs inside Docker, the host's environment does not cross the boundary automatically. Pass it explicitly with -e DBT_PASSWORD or an env file (--env-file). Expected: docker exec [container] printenv DBT_PASSWORD shows the value.

1. If dbt runs in CI, add the variable to the same job step's env block as the dbt command. Secrets defined at the workflow level but scoped to another job are invisible. Expected: a printenv in the dbt step shows the variable.

1. If dbt was launched from an IDE, scheduler, or agent harness, restart that parent process after setting the variable. Processes inherit the environment once at startup; exporting later in a terminal does not update an already-running parent. Expected: after restart, dbt debug resolves the profile.

1. Confirm the right target. dbt debug prints the active target. If you set the variable for dev but dbt runs prod (or vice versa), you are editing the wrong output block. Expected: the target line matches the output block whose env_var calls you changed.

1. For integer-typed fields (port, threads), env vars arrive as strings. Use a filter: port: "{{ env_var('DBT_PORT') | int }}". Without it dbt raises a type error instead of resolving. Expected: dbt debug validates the profile.

1. Quote the whole Jinja expression in profiles.yml: password: "{{ env_var('DBT_PASSWORD') }}". Unquoted curly braces confuse the YAML parser and the profile fails to load at all. Expected: dbt debug --profiles-dir [dir] parses without a YAML error.

1. If values look stale after changing them, bypass the partial-parse cache once: dbt --no-partial-parse debug. Expected: the profile reflects the current environment.

1. In dbt Cloud, shell exports on your laptop are irrelevant. Set the variable in the dbt Cloud environment settings for the environment you run. Expected: the dbt Cloud run logs show the profile resolved.

When this applies

• the Env var required but not provided error even though the variable looks set
• dbt debug fails on profile resolution while printenv in your terminal shows the value
• env vars work locally but not in Docker, CI, dbt Cloud, or an IDE-launched run
• profiles.yml uses {{ env_var('NAME') }} or {{ env_var('NAME', 'default') }} and the value never lands

When it doesn't

• the variable was never set anywhere: that is the missing-variable case, covered by the skill "dbt: Env var required but not provided"
• --vars CLI values, which never go through env_var()
• values rendered from dbt_project.yml vars: rather than the process environment

Compatibility

dbt Core (1.x) and the Fusion CLI; env_var() in profiles.yml. Values prefixed with DBT_ENV_SECRET_ are masked in dbt logs since dbt 1.5, which can look like the variable did not resolve when it actually did.

Variant phrasings

• dbt env var set but not working
• dbt profiles.yml env_var not picking up environment variable
• dbt debug env var required but not provided even though exported
• environment variable not visible to dbt in docker / CI

Root cause

env_var() reads the environment of the dbt process at parse time. Shells, containers, CI jobs, IDEs, and agent harnesses each build their own environment, and none of them see a variable that was only assigned (not exported) or only set in a different shell. dbt is almost never wrong about the variable being absent; it is reading a different environment than the one you checked.

Edge cases

• on Windows, set in one terminal does not carry to another; set the variable in system environment settings or the same session
• a variable set in .bashrc is invisible to GUI-launched IDEs and cron, which do not run login shells
• dbt Cloud CLI (local) reads real shell env vars; dbt Cloud (managed) reads the environment settings in the UI
• fallback defaults in env_var('NAME', 'default') can silently mask a delivery problem, so remove the default while diagnosing