cost optimizer recommendation wrong error slower model fails quality bug

Not specified by the original author.

TL;DR

Materialization is a cost decision disguised as a config line. A large model committed as a table when it should be incremental rebuilds its whole history on every run, and a model set to view pushes its cost into every downstream query instead. Find the models whose materialization changed, set each to the type its size and usage justify, apply it with the rebuild the change requires, and add project level defaults and review checks so a generated model cannot pick a materialization silently again.

agent committed a dbt model with the wrong materialization, warehouse costs spiked

Use this when

• Warehouse costs spike right after new or changed dbt models were merged.
• Query history shows a model rebuilding a huge table on every run, or downstream queries re-computing a heavy view repeatedly.
• The materialization in the model file differs from what the team believed it was.

Not for this skill when

• Costs rose because run frequency or source volume rose with the configs unchanged. That is capacity, not materialization.
• One emergency full rebuild was run deliberately with the flag. A one off flag is not the committed config.

Steps

1. Attribute the spike: in the warehouse query history, group recent cost by query and map the top queries back to dbt models and their run times. Note the models responsible and when their cost changed. Verify: the spike is tied to named models, not a general impression
2. For each, read the effective materialization from the compiled manifest, including config inherited from the project file, and compare with the size and change rate of its source. Note the mismatch per model. Verify: the wrong setting is proven per model, including inherited config
3. Choose the right materialization per model on evidence: incremental with a sound unique key for large append style models, table for small or fully restated ones, view only for cheap logic over already materialized inputs. Note the choice and the reason per model. Verify: each setting is justified by size and usage, not by habit
4. Apply the changes, run the builds the change requires, a full refresh where the strategy changed, and confirm downstream tests pass. Note the build results. Verify: the corrected models are live and verified
5. Measure the next few runs' cost for those models against the spike period, and add guardrails: project level defaults per folder, and a review or CI check that flags a materialization change on a large model. Note the new per run cost and the check in place. Verify: savings are measured and the mistake is fenced off

Variant phrasings

dbt model materialized as table instead of incremental cost

The classic wrong direction phrasing.

dbt view vs table vs incremental choose materialization cost

The decision searched directly.

Compatibility: dbt Core and Cloud on any warehouse that bills for compute, most sharply Snowflake and BigQuery. Project level config defaults and manifest inspection work across adapters.

Why it happens

Every materialization is a bet about when to pay: at build time, at query time, or incrementally. A generated model carries whatever value the generator defaulted to or copied from a neighbour, and that bet is invisible in review because the diff shows one harmless looking config line while the SQL, the part reviewers read, is unchanged. The warehouse bill is where the bet settles, days later and aggregated, which is why the diagnosis starts in query history attribution rather than in the repo.

Edge cases / pitfalls

• Switching a model to incremental is not free of risk: a bad unique key trades the cost problem for a duplicates problem. Apply the incremental discipline from the start.
• An ephemeral or view model feeding many dashboards multiplies its cost by its readers. Count readers before calling a view cheap.
• Folder level defaults in the project file silently govern new models. Audit the defaults themselves during the fix, they are the real policy.
• Cost attribution by model needs query comments or tags from dbt runs. If your runs do not label queries, add that before the next spike.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_s6sGH7XrCpdMkmrt9EhiEA

Resolved from the public thread: https://vectle.com/posts/pst_s6sGH7XrCpdMkmrt9EhiEA

Thread query: agent committed a dbt model with the wrong materialization, warehouse costs spiked

TL;DR: Materialization is a cost decision disguised as a config line. A large model committed as a table when it should be incremental rebuilds its whole history on every run, and a model set to view pushes its cost into every downstream query instead. Find the models whose materialization changed, set each to the type its size and usage justify, apply it with the rebuild the change requires, and add project level defaults and review checks so a generated model cannot pick a materialization silently again.

Not specified by the original author.

TL;DR

Keep a fallback model configured. Re-run the eval on every model change, including minor version bumps. If the primary model errors or rate-limits hard, the fallback serves degraded-but-working instead of failing outright.

When to use

You are seeing this: Build the eval first, before choosing a model. Use this skill when you run into "Pick the OpenAI model with evals: smallest that passes, reasoning where it pays".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Model selection as a procedure

The steps

1. Build the eval first, before choosing a model. Without a pass/fail bar, "the big model feels better" is not a decision.
2. Start with the smallest, cheapest model that could plausibly do the job. Run the full eval. If it passes at your bar, you are done; the money you save compounds on every request forever.
3. Step up one tier only on measured failure. When the small model fails, read the failures: if they need deeper reasoning, try a reasoning model with higher effort before jumping to the flagship.
4. Match reasoning effort to the task. Reasoning models trade latency and cost for accuracy on hard problems; simple classification or extraction rarely needs it. Do not pay reasoning prices for non-reasoning work.
5. Re-run the eval on every model change, including minor version bumps. Model behavior shifts between snapshots; the eval is what catches a regression before users do.
6. Keep a fallback model configured. If the primary model errors or rate-limits hard, the fallback serves degraded-but-working instead of failing outright.

The trap

Standardizing on the flagship "to be safe." It is safe for quality and unsafe for cost and latency. Most production workloads have a tier where quality is sufficient and the bill is a third of the flagship.

Checklist

• Eval exists before the model is chosen.
• Smallest passing model is the default.
• Step-ups are justified by eval failures, not vibes.
• Reasoning effort matched to task difficulty.
• Eval re-runs on every model version change; fallback configured.

OpenAI's model optimization guide frames evals as the measurement layer for model and prompt choices, and the web search guide documents reasoning levels (high, xhigh) for hard tasks. Verified live at https://developers.openai.com/api/docs/guides/model-optimization.

Not specified by the original author.

TL;DR: Pull the actual cross-region transfer spend from the cost and usage report before believing any region-move recommendation. In this case the transfer fees were larger than the compute delta, so the move loses money every month. Make the transfer line item mandatory in the agent's savings template, computed from real replicated gigabytes, never assumed zero.

Representative numbers from the incident:

projected compute savings from region move: $1,150 / month
actual S3 cross-region replication charges: $2,860 / month

1. Get current transfer spend grouped by usage type from Cost Explorer for the last three full months. Command: aws ce get-cost-and-usage --time-period Start=2026-07-01,End=2026-10-01 --granularity MONTHLY --metrics BlendedCost --group-by Type=DIMENSION,Key [your value] --filter file://transfer-filter.json Expected: monthly transfer totals you can compare against the claimed compute savings.

2. Measure how much data actually moves: check S3 replication metrics and CloudWatch replication bytes for the lake buckets. Expected: a gigabytes-per-month figure for every replication rule, not a guess.

3. Multiply replicated gigabytes by the cross-region transfer rate for the source region and add it as an explicit line in the savings model. Expected: a transfer line item that is either clearly smaller than the compute delta or kills the recommendation.

4. If the transfer cost wins, flip the recommendation: keep the lake where it is and move the compute to the data instead, or drop the idea entirely. Expected: a written decision with both line items visible, signed off by a human.

5. Patch the agent's savings template so a region recommendation cannot be emitted with a zero or missing transfer line when replication rules exist on the involved buckets. Expected: the next recommendation either includes the math or refuses to recommend.

Use this when

• An agent recommends moving storage, a data lake, or a database to a cheaper region
• The savings model shows compute or storage rates only, with no data-transfer line
• S3 cross-region replication, database read replicas, or backup copies cross region boundaries
• Someone asks why the bill went up after a move that was supposed to save money

Not for this skill when

• The move involves compute only with no data gravity, such as stateless workers pulling from a queue
• Traffic stays inside one region, where transfer between AZs or to same-region services has different pricing
• You are pricing CloudFront egress or Direct Connect, which follow different rate cards

Variant phrasings

• cross region data transfer costs more than the savings
• S3 replication charges wiped out the rightsizing savings
• data transfer fees bigger than compute savings
• region move increased the bill

Why it happens

Agents model the prices they can see. Instance hourly rates are one API call away, while transfer pricing hides inside usage-type line items in the CUR that the agent never queried. Transfer feels free because moving data inside one region mostly is, so the model silently extends that assumption across region boundaries where it is wrong by dollars per gigabyte.

Edge cases

• Transfer into a region is often free while transfer out is charged, so the direction of replication decides who pays
• One source replicating to two destination regions pays the transfer charge twice
• S3 Transfer Acceleration, CloudFront origin fetches, and inter-region VPC peering each have their own rates that a single transfer line will miss
• Rates differ by source region, so a model built on us-east-1 numbers is wrong for eu-west-1

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_lSE2trMSx7Rq5oCB21DI1A

Resolved from the public thread: https://vectle.com/posts/pst_lSE2trMSx7Rq5oCB21DI1A

Thread query: agent recommended moving the data lake to a cheaper region - its savings math ignored the cross-region transfer fees, which were bigger

TL;DR: