CI: drop the browser job, stop the CSS gate flaking, keep the chores off the runner, cut the install noise

kristofferopsahl commented

2026-09-18 23:22:55 +02:00

Collaborator

CI on this repo has been red far more often than the code has been broken. Between 13 and 18 September, 14 of 23 browser runs failed. Almost none of those failures were about the change under test.

Failure	Runs	Cause	Where it is fixed
`test_css_characterization` (determinism proof and gate)	9	the Matches tab's loading skeleton pulses forever; two pages read it at different points of the curve (`0.354607` vs `0.354603`)	this PR
`test_strengths_follow_results::…real_fit…` (#31 only)	9	reads a snapshot from gitignored `data/pit/`	#39 (merged into `feat/captaincy-board`)
`test_asset_version` (tests job, `main`)	3	digest over raw bytes, so CRLF (Windows) and LF (runners) disagree	#28 (merged)
`test_header_narrow[400]` (#31 only)	1	real regression: the page is 410px wide at 400px, and it reproduces locally every time	left alone, since that's the test doing its job

The test run also started the app's real chore scheduler on the runner. Every with TestClient(app) entered the lifespan, which fetched the live FPL API and shelled out to scripts/…. On a clean checkout the [test] suite rewrote the tracked data/current_2026_27/rounds.json / results.json with GW1–4 and wrote a data/chores.json ledger.

Changes

The browser job is gone from CI. Browser tests are now a local suite: the job cost 13–17 minutes of Chromium on the shared runner per push. The tests job still runs every non-browser test and skips the Playwright ones, as before. README says the browser suite runs locally only, and installs it from the declared browser extra with constraints.txt instead of an unpinned pip install playwright.
CSS gate settles animations before reading styles, in the same evaluate that reads them. This matters for local runs now. Transitions and finite animations are finished; infinite ones are paused at t=0. Nothing is disabled, so animation-* / transition-* still compare and the first keyframe is still a stylesheet value. A new proof test drives a real .loading-block and a 30s linear transition, and requires two snapshots 300ms apart to agree. With the settle removed it fails every time.
Chores off under pytest. This cherry-picks 69e2d02 from feat/captaincy-board, the author's own fix with its guard test, unchanged, so that it doesn't wait for #31. It will merge cleanly when #31 lands.
Quiet on success, whole on failure.
- pip install -q still prints the full error and backend traceback when it fails; I checked this with a broken pyproject.
- python -m build is captured to a file that the step prints only if it fails. I avoided build -q because it hides the backend's own error.
- The tests job's full pip freeze is unchanged.
pytest config.
- slow is registered, and --strict-markers is on; slow is the only custom mark on any branch.
- The two Starlette TestClient deprecations are filtered by exact message. The warnings summary is empty, so the next new warning stands out.

No test is removed, skipped, retried, or loosened. The browser tests are unchanged and simply run locally instead of in CI.

Log size, green run

Job	Before	After
tests	~110 pip lines + 3 warnings	freeze + dots + summary
wheel	~400 lines after checkout	2 lines (`fpl_app-0.0.1-py3-none-any.whl`, `wheel imports the serving layer`)
browser	~600 lines, 13–17 min	removed

Verification

Runners, run #51, before #28 was merged: tests job 1 failed, 1235 passed, 83 skipped in 39.37s. No warnings summary and no pip output; the only failure was test_asset_version, which is fixed by #28.
Local, Python 3.14, Playwright 1.62 / Chromium 151:
- Full tests/web with the browser: 995 passed, 3 skipped, no warnings. The only failure was the same asset test, since that run was before #28.
- test_css_characterization.py: 6 passed, including the full 48-combination gate and the 77px mutation proof.
- New settle proof: red with _SETTLE removed, green with it.
- Full [test] suite on a clean checkout: no tracked file modified, no data/chores.json. Before, the same run rewrote both data files.

🤖 Generated with Claude Code

CI on this repo has been red far more often than the code has been broken. Between 13 and 18 September, 14 of 23 browser runs failed. Almost none of those failures were about the change under test. | Failure | Runs | Cause | Where it is fixed | |---|---|---|---| | `test_css_characterization` (determinism proof and gate) | 9 | the Matches tab's loading skeleton pulses forever; two pages read it at different points of the curve (`0.354607` vs `0.354603`) | **this PR** | | `test_strengths_follow_results::…real_fit…` (#31 only) | 9 | reads a snapshot from gitignored `data/pit/` | #39 (merged into `feat/captaincy-board`) | | `test_asset_version` (tests job, `main`) | 3 | digest over raw bytes, so CRLF (Windows) and LF (runners) disagree | #28 (merged) | | `test_header_narrow[400]` (#31 only) | 1 | **real regression**: the page is 410px wide at 400px, and it reproduces locally every time | left alone, since that's the test doing its job | The test run also started the app's **real chore scheduler** on the runner. Every `with TestClient(app)` entered the lifespan, which fetched the live FPL API and shelled out to `scripts/…`. On a clean checkout the `[test]` suite rewrote the tracked `data/current_2026_27/rounds.json` / `results.json` with GW1–4 and wrote a `data/chores.json` ledger. ## Changes 1. **The browser job is gone from CI.** Browser tests are now a local suite: the job cost 13–17 minutes of Chromium on the shared runner per push. The `tests` job still runs every non-browser test and skips the Playwright ones, as before. README says the browser suite runs locally only, and installs it from the declared `browser` extra with `constraints.txt` instead of an unpinned `pip install playwright`. 2. **CSS gate settles animations before reading styles**, in the same `evaluate` that reads them. This matters for local runs now. Transitions and finite animations are finished; infinite ones are paused at t=0. Nothing is disabled, so `animation-*` / `transition-*` still compare and the first keyframe is still a stylesheet value. A new proof test drives a real `.loading-block` and a 30s linear transition, and requires two snapshots 300ms apart to agree. **With the settle removed it fails every time.** 3. **Chores off under pytest.** This cherry-picks 69e2d02 from `feat/captaincy-board`, the author's own fix with its guard test, unchanged, so that it doesn't wait for #31. It will merge cleanly when #31 lands. 4. **Quiet on success, whole on failure.** - `pip install -q` still prints the full error and backend traceback when it fails; I checked this with a broken `pyproject`. - `python -m build` is captured to a file that the step prints only if it fails. I avoided `build -q` because it hides the backend's own error. - The tests job's full `pip freeze` is unchanged. 5. **pytest config.** - `slow` is registered, and `--strict-markers` is on; `slow` is the only custom mark on any branch. - The two Starlette TestClient deprecations are filtered by exact message. The warnings summary is empty, so the next new warning stands out. No test is removed, skipped, retried, or loosened. The browser tests are unchanged and simply run locally instead of in CI. ## Log size, green run | Job | Before | After | |---|---|---| | tests | ~110 pip lines + 3 warnings | freeze + dots + summary | | wheel | ~400 lines after checkout | 2 lines (`fpl_app-0.0.1-py3-none-any.whl`, `wheel imports the serving layer`) | | browser | ~600 lines, 13–17 min | removed | ## Verification - **Runners, run #51, before #28 was merged:** `tests` job **1 failed, 1235 passed, 83 skipped in 39.37s**. No warnings summary and no pip output; the only failure was `test_asset_version`, which is fixed by #28. - **Local, Python 3.14, Playwright 1.62 / Chromium 151:** - Full `tests/web` with the browser: **995 passed, 3 skipped**, no warnings. The only failure was the same asset test, since that run was before #28. - `test_css_characterization.py`: 6 passed, including the full 48-combination gate and the 77px mutation proof. - New settle proof: red with `_SETTLE` removed, green with it. - Full `[test]` suite on a clean checkout: **no tracked file modified, no `data/chores.json`**. Before, the same run rewrote both data files. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

kristofferopsahl added 2 commits

2026-09-18 23:22:55 +02:00

fix: the CSS gate read a pulse mid-flight, and CI logs buried their signal 7fa487702d

The render gate and its determinism proof compared computed styles while
the Matches tab's loading skeleton was still pulsing (`loading-pulse`,
opacity .35 -> .8, infinite). Each page's pulse started when that page
loaded, so two snapshots read two points on the curve -- 0.354607 against
0.354603 -- and the gate went red for a reason that was not the
stylesheet: nine of the 23 browser runs between 2026-09-13 and 09-18.

Every snapshot now brings the page's animations to a fixed point in the
same evaluate that reads the styles: transitions and finite animations
are finished, infinite ones paused at t=0. Nothing is disabled, so
animation-* and transition-* still compare, and the first keyframe is
still a value the stylesheet decides. A new proof test drives a real
.loading-block and a 30s linear transition and requires two snapshots
300ms apart to agree; with the settle removed it fails every time.

The rest is log volume. pip installs run -q, which still prints the full
error and the backend traceback on failure. The wheel build and the
Chromium fetch are captured to a file and printed only if they fail
(`build -q` would hide the backend's own error). The browser job keeps a
one-line pip freeze so a failure is still attributable to a version.
pytest registers the `slow` mark under --strict-markers and filters the
two Starlette TestClient deprecations by exact message, so the warnings
summary is empty and the next new warning stands out.

No test is removed, skipped, retried or loosened.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

fix: the test suite never starts the chores scheduler

CI / syntax (pull_request) Successful in 8s

Details

CI / tests (pull_request) Failing after 1m6s

Details

CI / wheel (pull_request) Has been skipped

Details

CI / browser (pull_request) Has been skipped

Details

f52ea03c39

tests/web/test_compression.py enters the app's lifespan (`with
TestClient(app)`), and the lifespan starts chores.run_forever. A chore is a
subprocess: after_gameweek.py fetches the live FPL API, rewrites the
TRACKED data/current_2026_27/rounds.json and results.json, and rebuilds the
projection cache — and it outlives pytest, which only kills the thread.

Measured 2026-09-12 in a fresh checkout whose data stopped at GW1: the full
suite ingested GW2 and GW3 into those files and wrote data/chores.json,
timestamped as the run finished. Rerun with FPL_CHORES=off: both files
byte-identical, no ledger. In an up-to-date checkout nothing is due, so it
was invisible on the machine anyone works on.

The switch already existed and nothing in the suite set it. It is set in
pytest_configure rather than a fixture because test_compression enters the
lifespan from a MODULE-scoped fixture, which runs before any function-scoped
autouse fixture. The guard enters the lifespan from a module-scoped fixture
too, with run_forever replaced by a recorder, and is proven red both by
removing the switch and by moving it into a function-scoped fixture.

Not changed: the lifespan's autorefresh thread still fetches the live API
during that one test; it writes no tracked file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 69e2d02e3b)

kristofferopsahl referenced this pull request

2026-09-18 23:23:05 +02:00

fix: the real-fit strengths test read a gitignored snapshot #39

kristofferopsahl added 1 commit

2026-09-18 23:31:53 +02:00

Merge remote-tracking branch 'origin/main' into ci/quiet-test-suite

CI / syntax (pull_request) Successful in 8s

Details

CI / tests (pull_request) Successful in 1m7s

Details

CI / wheel (pull_request) Successful in 29s

Details

CI / browser (pull_request) Has been cancelled

Details

6065231bf3

kristofferopsahl added 1 commit

2026-09-18 23:46:44 +02:00

ci: browser tests are a local suite, not a CI job

CI / syntax (pull_request) Successful in 9s

Details

CI / tests (pull_request) Successful in 1m5s

Details

CI / wheel (pull_request) Successful in 28s

Details

05530f7544

The browser job cost 13-17 minutes of Chromium on the shared runner per
push. It is dropped; the tests job already runs every non-browser test and
skips the Playwright ones, which is unchanged. README now says the browser
suite runs locally only, and installs it from the declared `browser` extra
under constraints.txt instead of an unpinned `pip install playwright`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

kristofferopsahl changed title from ~~CI: stop the CSS gate flaking, keep the chores off the runner, cut the install noise~~ to CI: drop the browser job, stop the CSS gate flaking, keep the chores off the runner, cut the install noise

2026-09-18 23:47:05 +02:00

kristofferopsahl merged commit 56a5026d7b into main

2026-09-18 23:48:58 +02:00

kristofferopsahl deleted branch ci/quiet-test-suite

2026-09-18 23:48:58 +02:00

kristofferopsahl referenced this pull request from a commit

2026-09-18 23:48:59 +02:00

Merge PR #38: drop the browser job, stop the CSS gate flaking, keep the chores off the runner

kristofferopsahl referenced this pull request from a commit

2026-09-19 00:00:34 +02:00

Merge branch 'main' into feat/nt-only-board

kristofferopsahl referenced this pull request from a commit