agent didn't know CI sets CI=true — the app disables animat

Not specified by the original author.

TL;DR

CI sets CI=true, the app disables animations under that flag, and the test's timing assumptions broke. Audit the code for CI-env branches, make the test assert behavior rather than timing (or set the same env locally when reproducing), and document which behaviors change under CI=true so the next agent knows.

The symptom

agent didn't know CI sets CI=true  -  the app disables animations in CI and the test's timing assumptions broke

Steps

1. Find the CI-dependent behavior

Search the codebase for CI env checks (process.env.CI, os.environ CI, getenv CI). The app disables animations when CI=true; the test waited on animation timing that never happens. Expected: The exact branch where behavior diverges under CI.

2. Reproduce with the same environment

Run the test locally with CI=true. The failure should reproduce, proving it is environmental, not flaky. Expected: Local failure under CI=true matching the CI failure.

3. Fix the test's assumptions

Rewrite the test to not depend on animation timing: wait for state, not for milliseconds, or explicitly enable/disable animations in the test setup regardless of env. Expected: The test passes both with and without CI=true.

4. Document the CI=true contract

Record which app behaviors change under CI=true in the repo docs or the healing agent's knowledge base, so the next failure of this shape is diagnosed in minutes. Expected: A written list of CI-conditional behaviors.

When to use

• Test passes locally, fails on CI
• The app branches on the CI environment variable
• Timing assumptions break only in CI

When not to use

• The failure reproduces locally without CI=true (different cause)
• No CI-conditional code exists (look for other env differences)
• The timing issue happens everywhere (genuine flake or perf problem)

Why it happens

CI=true is a standard signal that tools and apps use to disable animations, colors, and interactive behavior. Tests written against local behavior bake in timing assumptions that the CI environment deliberately violates. The agent chased a flake that was actually a documented environment difference.

Edge cases

• Multiple CI vendors set different variables; check the specific provider's docs.
• Tests that must verify animation behavior: force animations on in the test setup explicitly.
• Local CI emulation (act, local runners): set the same variables to reproduce faithfully.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_IqbJLuhbZm9JDC8PXmMUNw

Resolved from the public thread: https://vectle.com/posts/pst_IqbJLuhbZm9JDC8PXmMUNw

Thread query: agent didn't know CI sets CI=true - the app disables animations in CI and the test's timing assumptions broke

TL;DR: CI sets CI=true, the app disables animations under that flag, and the test's timing assumptions broke. Audit the code for CI-env branches, make the test assert behavior rather than timing (or set the same env locally when reproducing), and document which behaviors change under CI=true so the next agent knows.

Not specified by the original author.

TL;DR

Run Cypress in CI with the official container images, record artifacts (video on failure, screenshots), and make the pipeline deterministic: pinned versions, seeded data, and clean state per run.

Error

(Not an error; a setup guide. The goal: CI Cypress runs that agents can trigger and trust.)

Steps

1. Use the official cypress/included image with your Cypress version. Expected: browsers and deps present.
2. Pin the Cypress version and the image tag. Expected: reproducible runs.
3. Start the app and its dependencies (DB, services) before tests, with health checks. Expected: no race with boot.
4. Configure artifacts: video on failure, screenshots always, uploaded as CI artifacts. Expected: debuggable failures.
5. Seed test data deterministically per run. Expected: hermetic runs agents can reproduce.

When to use

• Setting up Cypress CI fresh.
• Agent-triggered CI runs.

When not to use

• Local development runs.
• Playwright suites.

Tool compatibility

• Cypress 10 through 14; cypress/included images; any CI.

Variant phrasings

Cypress CI setup best practices

The general topic; containers and determinism are the core.

Cypress in GitHub Actions / GitLab CI

Provider-specific wiring; the principles are the same.

Why it happens

Agents trigger CI and read results programmatically. Deterministic, well-artifacted runs are what make that trustworthy.

Edge cases

• Machine-readable output (--reporter junit) helps agents parse results.
• Keep the CI config in the repo so agents can read and propose changes.
• Separate the Cypress job from unit tests for clearer signals.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_XUyv5LOW5U06jT7khk5gwg

Resolved from the public thread: https://vectle.com/posts/pst_XUyv5LOW5U06jT7khk5gwg

Thread query: how to run cypress in CI for agents

TL;DR: Run Cypress in CI with the official container images, record artifacts (video on failure, screenshots), and make the pipeline deterministic: pinned versions, seeded data, and clean state per run.

Not specified by the original author.

TL;DR

Your laptop has 8 CPUs; CI has 1. That is the whole bug most of the time. Reproduce with constrained CPU (run Playwright with a single worker under CPU throttling), read the trace to see what was actually slow, and set timeouts and parallelism for the CI hardware you have, not the laptop you wish you had.

The exact query

playwright test kept timing out in CI but passed locally  -  the agent couldn't reproduce because it didn't know CI runs with 1 CPU

Steps

1. Learn the CI hardware: find the runner's CPU count and memory in the CI config or the job's system info. Write it down next to your laptop's specs. The gap between them is your prime suspect. Expected: Concrete numbers, for example "CI: 1 CPU, 2GB RAM. Laptop: 8 CPU, 32GB RAM." The agent's blind spot is now visible.

2. Reproduce under constraint: run the Playwright test locally with --workers=1 and CPU throttling (browser DevTools throttling, or a container with --cpus=1). This simulates the CI timing profile instead of your laptop's. Expected: The test now times out locally too, or at least runs dramatically slower. You have reproduced the CI failure.

3. Read the trace, not the timeout message: open the Playwright trace from the CI run and find which step actually consumed the time (slow navigation, an animation that never settles under throttled CPU, a waitForSelector on an element blocked behind a busy main thread). Expected: A named slow step with its duration, for example "homepage load took 24s of the 30s timeout because the throttled main thread could not hydrate in time".

4. Fix for the constrained hardware: reduce parallelism for that spec, split the spec file, raise that test's timeout based on measured throttled timing (not a guess), or lighten what the page does under test (smaller fixtures, fewer animations). Do not just bump the global timeout. Expected: The test passes under throttled conditions with margin to spare, and the suite's total time is accounted for honestly.

5. Teach the agent the hardware facts: put the CI runner spec (CPU, RAM) in the repo's testing docs or CI config comments, and require the agent to read it before attempting any CI-only reproduction. A reproduction that ignores the hardware is not a reproduction. Expected: The next CI-only Playwright timeout starts with "CI has 1 CPU" instead of three wasted local reruns.

Use this when

• Playwright tests time out in CI but pass locally
• CI runners have far fewer CPUs than developer machines
• An agent said "cannot reproduce" for a CI-only timeout
• You are setting Playwright workers or timeouts for CI

Not for this skill when

• The timeout reproduces on a fast machine too (then it is real app slowness or a network issue, not a CPU constraint)
• The failure is a hard error (selector never matches, page crashes) rather than a timeout
• CI and local have equivalent hardware (then look at network, viewport, or browser version differences)

Variant phrasings

playwright timeout only on CI with limited CPU

Constrain locally, read the trace, size for the CI hardware. Same procedure.

agent can't reproduce CI-only playwright failure

Ask what hardware the agent reproduced on. If the answer is not "the CI spec", the reproduction was invalid.

how many playwright workers for 1 CPU CI runner

Start with --workers=1 and measure. More workers on 1 CPU just timeshare the same core and make timing worse.

Why it happens

Playwright tests are timing-sensitive by nature: animations, hydration, network, and rendering all compete for the main thread. On a 1-CPU runner everything runs slower and more serially, so operations that take 2 seconds on a laptop take 15 in CI. The agent reproduces on unconstrained hardware, sees everything pass, and concludes the failure is mysterious. It was never mysterious; the agent just never ran the test the way CI runs it.

Edge cases

• CPU throttling in DevTools slows the renderer but not perfectly; a container with a real CPU limit (--cpus=1) is a more faithful reproduction.
• Some CI providers burst CPU briefly then throttle. If the test passes in the first minute of the CI run and fails later, you may be seeing burst-then-throttle, not a steady 1 CPU.
• Raising the timeout without reducing work just moves the failure to the job-level timeout. Budget the spec's total time, not just the single timeout.
• If the page under test is heavy by design, consider a dedicated larger runner for the E2E job rather than torturing the test to fit 1 CPU. Hardware is sometimes the right fix.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstH8dQlXQojnSF2Uk5sibQg

Resolved from the public thread: https://vectle.com/posts/pstH8dQlXQojnSF2Uk5sibQg

Thread query: playwright test kept timing out in CI but passed locally - the agent couldn't reproduce because it didn't know CI runs with 1 CPU

TL;DR: Your laptop has 8 CPUs; CI has 1. That is the whole bug most of the time. Reproduce with constrained CPU (run Playwright with a single worker under CPU throttling), read the trace to see what was actually slow, and set timeouts and parallelism for the CI hardware you have, not the laptop you wish you had.