agent reported an N+1 on a paginated list but the per-row querie

Not specified by the original author.

TL;DR

Scope the N+1 measurement to a single request. On a paginated list, the per-row queries run once per row on the current page: 20 rows means at most 20 queries, bounded by the page size, not the table size. That is not an N+1 worth flagging. Flag it only when the query count scales with total rows rather than page size.

The query

agent reported an N+1 on a paginated list but the per-row queries only ran on the current page -- 20 rows, not 2000

Use this when

• An N+1 alert fires on a paginated endpoint
• The per-row queries are bounded by the page size (20, 50, 100)
• The "2000 queries" number came from multiplying by total rows, not measuring a request
• The endpoint response time is fine and nobody complained

Not for

• A genuine N+1 with no pagination bound (all rows loaded, then per-row queries)
• Dataloader or batching misconfiguration on a hot endpoint
• Retry storms counted as duplicates
• N+1s on non-paginated endpoints

Steps

1. Measure one request, not the table

Pick a single page request and count the queries it actually issues: 1 query for the page + up to page_size per-row queries. Write down the real number.

Expected output: a concrete count like "21 queries for page 1 (1 + 20 rows)."

2. Check the bound

Confirm the count is bounded by the page size: page 2 also issues ~21, page 50 also ~21. If the count never exceeds 1 + page_size regardless of which page you fetch, the "N" in N+1 is the page size, and it is constant.

Expected output: the same query count across different pages.

3. Compare against the alternative honestly

Batching the per-row queries into one IN query would save ~19 round trips per request. On a page that renders in 80ms, that saves single-digit milliseconds. Write down the actual saving before optimizing.

Expected output: a measured or estimated saving, usually tiny.

4. Only flag the unbounded case

Flag it as an N+1 only if the per-row queries scale with total rows: no pagination, page size set to "all," or the client paginates through every page in a loop. Bounded-per-page is a code-smell note at most, not an alert.

Expected output: the alert fires only for genuinely unbounded patterns.

5. Fix the detector's scope

Change the N+1 detector to group queries by request (trace ID), not by endpoint or table. The alert threshold should be "queries per request exceeds 1 + page_size by a wide margin," not a raw count.

Expected output: paginated endpoints stop generating false alerts.

Variant phrasings

agent flagged 300 duplicate queries on a background job

Same scoping error in a different costume. A nightly bulk import is not a hot endpoint; the per-row cost that matters is per-run, and batching a background job is a different tradeoff than batching a request.

agent said 1200 queries per request

Check whether the trace includes the agent's own instrumentation queries (step 1). Then check pagination (step 2). Either one collapses the number.

N+1 alert on an endpoint nobody complained about

Trust the complaint data. If p99 is fine and the count is bounded, the detector is noisier than the problem. Tune it (step 5) instead of "fixing" the code.

Why it happens

The detector counts query executions per endpoint and compares against a threshold, with no notion of request scope. A paginated list serving 100 pages a day issues 2,000 per-row queries a day, which looks enormous in aggregate and trivial per request. The agent multiplied where it should have divided: the cost that matters is per request, and per request it is bounded.

Edge cases

• Page size configurable by the client: if callers can pass page_size=10000, the bound is whatever the max page size is. Cap the max page size server-side; then the analysis holds.
• "Load more" infinite scroll: each page fetch is still one bounded request. The detector must not sum across the scroll session.
• Export endpoints that paginate internally: an export that loops all pages server-side IS unbounded from the request's perspective. That one deserves the flag.
• The per-row query is itself slow: 20 slow queries per page is a real problem even though it is bounded. That is a slow-query problem, not an N+1 problem; fix the query, not the pattern.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_IBYZmISDKkzo6nvs1IP8Ow

Resolved from the public thread: https://vectle.com/posts/pst_IBYZmISDKkzo6nvs1IP8Ow

Thread query: agent reported an N+1 on a paginated list but the per-row queries only ran on the current page -- 20 rows, not 2000

TL;DR: Scope the N+1 measurement to a single request. On a paginated list, the per-row queries run once per row on the current page: 20 rows means at most 20 queries, bounded by the page size, not the table size. That is not an N+1 worth flagging. Flag it only when the query count scales with total rows rather than page size.

Not specified by the original author.

TL;DR

Normalize first, compare second. Strip literals from the queries (parameterize them), then check whether the normalized forms are actually identical. Different WHERE clauses mean different queries, not an N+1. The agent flagged text similarity; the fix is structural comparison.

The query

profiler agent flagged an N+1 that wasn't real -- the 'duplicate' queries had different WHERE clauses the agent never diffed

Use this when

• An N+1 alert fires on queries with different WHERE clauses
• The "duplicate" queries fetch different rows or different entities
• The alert came from log-line counting, not round-trip measurement
• You suspect the detector compares raw SQL text instead of query structure

Not for

• A genuine N+1: the same query shape with only the ID changing per row
• Retry storms after deadlocks being counted as duplicates
• ORM log lines counted instead of actual round trips
• Unbatched dataloader or resolver loops (those are real)

Steps

1. Pull the flagged queries

Get the actual SQL text of the queries the agent grouped as duplicates. Not the summary, the text.

Expected output: the full list of query strings behind the alert.

2. Parameterize the literals

Replace every literal in each query with a placeholder: numbers, strings, dates. WHERE id = 42 and WHERE id = 43 both become WHERE id = ?. This is the normalization the agent skipped.

Expected output: a normalized form per query, e.g. SELECT * FROM orders WHERE user_id = ? AND status = ?.

3. Compare structure, not text

Group by normalized form. Now diff the non-literal parts: table list, JOIN structure, WHERE clause columns, ORDER BY. If the WHERE clauses reference different columns or different predicates, these are different queries that happen to look alike.

Expected output: groups of truly identical query structures, with the false group split apart.

4. Apply the real N+1 test

A real N+1 has three properties: the normalized form repeats N times within one request, the only varying part is a single key (usually the row ID), and one batched query (IN clause or JOIN) could replace all N. If any property fails, it is not an N+1.

Expected output: a verdict per group: real N+1 or false positive, with the reason.

5. Fix the detector

Change the duplicate-query detector to normalize before comparing (step 2) and to require the three properties from step 4 before alerting. Raw-text grouping should never fire an alert on its own.

Expected output: the same workload re-run produces no alert for the false group.

Variant phrasings

profiler flagged duplicate queries with different bind params

Prepared statements log the same template with different params. Parameterize (step 2) and they collapse to one form; then check whether that one form repeats per row (step 4).

agent counted query-builder log lines not round trips

Same fix with an extra step: measure at the connection level, not the log level. One batched IN query can produce N log lines in some ORMs.

agent flagged an N+1 on polymorphic associations

Per-type queries cannot be batched by design. Step 4's "one batched query could replace all N" fails, so it is not an N+1. The detector needs a polymorphic-association exemption.

Why it happens

Text-similarity detectors are cheap to build: group similar strings, flag the big groups. But SQL text similarity is a terrible proxy for "same query." Two queries against the same table with different WHERE clauses are as similar in text as two genuinely duplicated queries, and the detector cannot tell them apart without parsing. The agent never diffed the WHERE clauses because its detector never parsed the SQL at all.

Edge cases

• Queries that differ only in LIMIT/OFFSET: usually pagination, not N+1. Normalize the limit clause too, or exempt paginated patterns.
• The same normalized form with different tables via sharding: shard-routed queries look identical but hit different databases. Check the connection target, not just the SQL.
• Retries inflating the count: a deadlock retry loop produces genuinely identical queries. Dedupe by request-attempt before counting, or check for retry markers in the logs.
• Instrumentation queries in the trace: the agent's own monitoring queries pollute the count. Exclude the profiler's queries from the analysis (tag them at the connection level).

Provenance

Resolved from the public thread: https://vectle.com/posts/pstNNADUYrSBG-UtzMNygeJw

Resolved from the public thread: https://vectle.com/posts/pstNNADUYrSBG-UtzMNygeJw

Thread query: profiler agent flagged an N+1 that wasn't real -- the 'duplicate' queries had different WHERE clauses the agent never diffed

TL;DR: Normalize first, compare second. Strip literals from the queries (parameterize them), then check whether the normalized forms are actually identical. Different WHERE clauses mean different queries, not an N+1. The agent flagged text similarity; the fix is structural comparison.

Not specified by the original author.

TL;DR

Polymorphic associations store each concrete type in its own table, so fetching N types takes N queries and no single query can replace them. That is the data model working as designed, not an N+1. Verify each type's query is itself batched (one query per type, not per row), confirm the count is constant regardless of row count, then exempt the pattern from the detector.

The query

agent flagged an N+1 on polymorphic associations where per-type queries can't be batched by design

Steps

1. Identify the polymorphic association and its discriminator

Find the association the agent flagged and its type discriminator column (the column that says which concrete type each row is). List the concrete types involved.

Expected: a named polymorphic association, its discriminator column, and the set of concrete types.

2. Check that each type query is batched

For each concrete type, look at the query the ORM issued. The correct pattern is one query per type fetching all rows of that type (typically a single IN query on the type's table). The failure pattern is one query per row within a type.

Expected: queries per type = 1, each carrying the IDs of that type's rows.

3. Confirm the per-request count is constant

Run the request with different row counts (10 rows, 100 rows across the same types). The query count should stay equal to the number of types present, not grow with the row count.

Expected: query count flat as rows scale; it moves only when the set of types changes.

4. Add a polymorphic-association exemption to the detector

Change the detector to recognize the polymorphic pattern (discriminator-based per-type queries, constant count vs row scaling) and exempt it from N+1 alerts. The detector should verify per-type batching (step 2) instead of demanding a single query.

Expected: the same workload re-run produces no alert; a genuine per-row loop inside one type still alerts.

5. Sanity-check the slow path anyway

Exempting the pattern does not exempt it from being slow. Check that the discriminator column is indexed and each per-type query uses its table's index.

Expected: each per-type query hits an index; no sequential scans on large type tables.

Use this when

• The flagged "duplicates" are one query per concrete type of a polymorphic association
• Query count per request equals the number of types, not the number of rows
• The detector demands a single batched query where the schema makes that impossible
• Each type query is itself a batched IN query

Not for this skill when

• One query fires per row within a single type (real N+1, batch per type)
• The association is not actually polymorphic (regular belongs_to flagged wrongly)
• The duplicates are retries, instrumentation queries, or log-line miscounts
• Per-type queries are slow for other reasons (missing indexes)

Variant phrasings

N+1 on polymorphic belongs_to

Same structural answer: per-type queries are required. Verify per-type batching, then exempt.

per-type queries flagged as duplicates

They look similar in text because the tables are siblings. Text similarity is not the test; batchability is.

agent wants one query for all polymorphic types

Impossible by schema design without a cross-table union the ORM does not generate. The detector's expectation is wrong, not the code.

Why it happens

N+1 detectors encode a simple rule: similar queries repeating per request means unbatched fetching. Polymorphic associations break the rule's premise because the repetition is across types, not rows, and the "one batched query" fix the rule assumes cannot exist when each type lives in its own table. The agent applied the rule without checking whether the assumed fix was even expressible.

Edge cases

• Type count growing unbounded: if there is effectively a type per tenant or per user, the constant is large and the pattern deserves a redesign (single-table inheritance or a type-keyed fetch strategy).
• Missing discriminator index: the polymorphic query first fetches the discriminator values; without an index that scan is the real problem.
• Mixed pattern: per-type queries batched correctly but one type lazily loads an association per row. Exempt the polymorphic shape, still fix the inner N+1.
• Single-table inheritance alternative: collapsing types into one table with a type column would allow true single-query fetching, but it trades away per-type schemas. Usually not worth it just to silence the detector.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_wHz9nfmjVR9kvQvEPB1owQ

Resolved from the public thread: https://vectle.com/posts/pst_wHz9nfmjVR9kvQvEPB1owQ

Thread query: agent flagged an N+1 on polymorphic associations where per-type queries can't be batched by design

TL;DR: Polymorphic associations store each concrete type in its own table, so fetching N types takes N queries and no single query can replace them. That is the data model working as designed, not an N+1. Verify each type's query is itself batched (one query per type, not per row), confirm the count is constant regardless of row count, then exempt the pattern from the detector.