fixtures_raw is frozen at server startup while refresh-live refreshes only the bootstrap #17

New issue

Closed

opened 2026-08-29 16:15:14 +02:00 by solvreven · 2 comments

solvreven commented

2026-08-29 16:15:14 +02:00

Owner

Found 2026-08-29 from a user report: the Matches tab showed gameweek 3's
fixtures while nine gameweek 2 matches were being played, and kept showing them
for over an hour of debugging. The code was correct. The data it read was
fetched the previous morning.

/api/refresh-live was built for exactly this and covers the wrong half.
Its own docstring says: "The bootstrap was loaded once at server startup and
never again, so a [stale flag] stayed on screen." It is called by loadAll()
on every page load, so prices, ownership and availability flags are current.

But it sets one key:

set_deps(bootstrap=bs, fetched_at=...)     # main.py, refresh_live()

fixtures_raw is not in that call. It is written once, by build_deps_only()
in the startup lifespan, and never again until someone presses the full
/api/refresh button (which rebuilds the whole dependency graph and takes
minutes). So the fixture list — and every flag on it — is frozen at the moment
the process started.

Those flags are the ones that decide what round it is. started,
finished and finished_provisional flip continuously through a weekend; a
boot-time snapshot never does. On the reported day the server had been up since
28.08 09:14, before any GW2 fixture kicked off, so every fixture it held
reported started: false. Measured directly:

live fixtures from the API            -> gameweek_in_play = 2
same fixtures, every flag forced False -> gameweek_in_play = None   (before fix)

What reads the frozen list:

main.py:1064  gameweek_in_play(...)          which round the Matches tab shows
main.py:1101  kickoff[...] = kickoff_time    the times printed on every card
main.py:1182  props_load / nt_load fixtures  which banked prices belong to this round
main.py:614   cache_integrity_warnings(...)  freshness warnings, from stale input
main.py:1330  fixtures_grid()                every fixture cell in the app
absence.py:117 gw_dates_from_fixtures(...)   the dates injuries are mapped onto

Partially mitigated, deliberately not fixed. gameweek_in_play now keys on
kickoff_time rather than the flags, because a kickoff time is fixed when the
fixture is scheduled and is the one field on the record a stale snapshot cannot
get wrong. That fixes the reported symptom and none of the rest: scorer/assist
prices are still matched against a frozen fixture pairing, and a fixture
rescheduled after boot is invisible until restart.

Suggested fix. Add fixtures_raw to refresh_live(). It is one more free
FPL call beside the bootstrap fetch already happening there, on a page load
that already pays for one round trip. If that is judged too eager, give
fixtures_raw an explicit age and refuse to answer "is this round in play"
from a snapshot older than it — a stated refusal rather than a confident wrong
answer, which is the pattern this repo uses everywhere else.

Worth recording separately: the symptom was indistinguishable from a code
bug. An hour went into diagnosing it as browser caching, then as a stale
projection cache, then as a stale fixtures snapshot, with three requests to the
user to restart the server. One process listing settled it — a uvicorn started
28.08 09:14:43 and never replaced. Check what is RUNNING before theorising
about what is written; the process list is one command and it is evidence,
where "have you restarted" is a question neither side can verify. (The app also
serves /static/* through a no-cache middleware, so the hard-refresh advice was
wrong on its own terms.)

Related: #13 (no scheduler; every recurring ingest is a manual chore) and #12
(the app reports cache freshness but never its own code version) — the same
shape, an artefact whose age nothing on screen states.

**Found 2026-08-29 from a user report:** the Matches tab showed gameweek 3's fixtures while nine gameweek 2 matches were being played, and kept showing them for over an hour of debugging. The code was correct. The data it read was fetched the previous morning. **`/api/refresh-live` was built for exactly this and covers the wrong half.** Its own docstring says: *"The bootstrap was loaded once at server startup and never again, so a [stale flag] stayed on screen."* It is called by `loadAll()` on every page load, so prices, ownership and availability flags are current. But it sets one key: ```python set_deps(bootstrap=bs, fetched_at=...) # main.py, refresh_live() ``` `fixtures_raw` is not in that call. It is written once, by `build_deps_only()` in the startup lifespan, and never again until someone presses the full `/api/refresh` button (which rebuilds the whole dependency graph and takes minutes). So the fixture list — and every flag on it — is frozen at the moment the process started. **Those flags are the ones that decide what round it is.** `started`, `finished` and `finished_provisional` flip continuously through a weekend; a boot-time snapshot never does. On the reported day the server had been up since `28.08 09:14`, before any GW2 fixture kicked off, so every fixture it held reported `started: false`. Measured directly: ``` live fixtures from the API -> gameweek_in_play = 2 same fixtures, every flag forced False -> gameweek_in_play = None (before fix) ``` **What reads the frozen list:** ``` main.py:1064 gameweek_in_play(...) which round the Matches tab shows main.py:1101 kickoff[...] = kickoff_time the times printed on every card main.py:1182 props_load / nt_load fixtures which banked prices belong to this round main.py:614 cache_integrity_warnings(...) freshness warnings, from stale input main.py:1330 fixtures_grid() every fixture cell in the app absence.py:117 gw_dates_from_fixtures(...) the dates injuries are mapped onto ``` **Partially mitigated, deliberately not fixed.** `gameweek_in_play` now keys on `kickoff_time` rather than the flags, because a kickoff time is fixed when the fixture is scheduled and is the one field on the record a stale snapshot cannot get wrong. That fixes the reported symptom and none of the rest: scorer/assist prices are still matched against a frozen fixture pairing, and a fixture rescheduled after boot is invisible until restart. **Suggested fix.** Add `fixtures_raw` to `refresh_live()`. It is one more free FPL call beside the bootstrap fetch already happening there, on a page load that already pays for one round trip. If that is judged too eager, give `fixtures_raw` an explicit age and refuse to answer "is this round in play" from a snapshot older than it — a stated refusal rather than a confident wrong answer, which is the pattern this repo uses everywhere else. **Worth recording separately: the symptom was indistinguishable from a code bug.** An hour went into diagnosing it as browser caching, then as a stale projection cache, then as a stale fixtures snapshot, with three requests to the user to restart the server. One process listing settled it — a uvicorn started `28.08 09:14:43` and never replaced. Check what is RUNNING before theorising about what is written; the process list is one command and it is evidence, where "have you restarted" is a question neither side can verify. (The app also serves `/static/*` through a no-cache middleware, so the hard-refresh advice was wrong on its own terms.) Related: #13 (no scheduler; every recurring ingest is a manual chore) and #12 (the app reports cache freshness but never its own code version) — the same shape, an artefact whose age nothing on screen states.

solvreven referenced this issue from a commit

2026-08-30 19:36:48 +02:00

feat: the app follows the season by itself, and your money is read not derived

solvreven commented

2026-08-30 19:37:53 +02:00

Author

Owner

Fixed on fix/gw2-window-and-provenance (9d9bd4d, pushed just now).

backend/web/autorefresh.py re-fetches bootstrap and fixtures_raw together every 300s and applies them in one step, on a daemon thread started in the lifespan. Four properties, each a way an auto-refresher is worse than none, and each with a test:

Atomic or nothing — today's prices beside yesterday's fixtures is a state no single fetch could produce and nothing downstream expects. apply is called once or not at all.
A failure leaves the old data standing and says so — last_error and consecutive_failures on /api/status.
The age is published, not implied — age_seconds, None before the first success.
It refuses to overlap itself — a fetch slower than the interval must not have a second start on top.

The reported symptom is gone: the Matches tab keyed on a snapshot that never updated, so a server started before the first kickoff reported no round in play for the whole round. gameweek_in_play was also moved onto the clock rather than only the flags (kickoff_time is the one field in a fixture record a stale snapshot cannot get wrong), with the flags still sharpening it. Measured both ways after the fix: live fixtures → 2, and the same fixtures with every flag forced False → 2. Before it, the second case returned None.

Worth recording for anyone reading this later: the original diagnosis cost an hour because the reader was asked to restart three times while the cause was theorised as caching, then stale deps, then a stale snapshot. One Get-CimInstance Win32_Process showed a uvicorn started 28.08 09:14:43 and never replaced — holding code from before every change and a fixtures snapshot from before any GW2 match kicked off. Both symptoms, one fact. Check what is running before theorising about what is written.

**Fixed on `fix/gw2-window-and-provenance` (`9d9bd4d`, pushed just now).** `backend/web/autorefresh.py` re-fetches bootstrap **and** `fixtures_raw` together every 300s and applies them in one step, on a daemon thread started in the lifespan. Four properties, each a way an auto-refresher is worse than none, and each with a test: - **Atomic or nothing** — today's prices beside yesterday's fixtures is a state no single fetch could produce and nothing downstream expects. `apply` is called once or not at all. - **A failure leaves the old data standing and says so** — `last_error` and `consecutive_failures` on `/api/status`. - **The age is published, not implied** — `age_seconds`, None before the first success. - **It refuses to overlap itself** — a fetch slower than the interval must not have a second start on top. The reported symptom is gone: the Matches tab keyed on a snapshot that never updated, so a server started before the first kickoff reported no round in play for the whole round. `gameweek_in_play` was also moved onto the **clock** rather than only the flags (`kickoff_time` is the one field in a fixture record a stale snapshot cannot get wrong), with the flags still sharpening it. Measured both ways after the fix: live fixtures → 2, and the same fixtures with every flag forced False → 2. Before it, the second case returned None. Worth recording for anyone reading this later: the original diagnosis cost an hour because the reader was asked to restart three times while the cause was theorised as caching, then stale deps, then a stale snapshot. One `Get-CimInstance Win32_Process` showed a uvicorn started 28.08 09:14:43 and never replaced — holding code from before every change *and* a fixtures snapshot from before any GW2 match kicked off. Both symptoms, one fact. **Check what is running before theorising about what is written.**

solvreven referenced this issue from a pull request that will close it,

2026-08-30 19:53:57 +02:00

The season stops being hardcoded, the chores run themselves, and Norsk Tipping drives the market #19