End-to-end baseline — 2026-04-28
Snapshot of the full ingest → dbt → frontend pipeline taken right after PLAN-003 merged. This is the known-good starting point for further development. Re-run the same sweep before any major refactor to catch regressions.
Repo state at snapshot:
- HEAD:
c51d02c(PLAN-003 phase 1 squash merge — body covers all 6 phases) - Branch:
main - Local Postgres reachable;
dbt debugclean - All
node_modules,dbt/.venv,ingest/.envpresent
Phase A — Ingest sweep
Result: 20 / 20 sources succeeded (after the ssb-06913 fix below).
cd atlas-data/ingest
for s in ssb-klass-kommuner ssb-klass-fylker ssb-08764 ssb-06083 ssb-06913 \
ssb-06944 ssb-06947 ssb-07459 ssb-09429 ssb-12063 ssb-12131 ssb-12132 \
ssb-12292 ssb-12944 ssb-13995 fhi-bor-alene fhi-mobbing fhi-trangbodd \
fhi-vgs-gjennomforing redcross-branches; do
npm run --silent "ingest:$s" 2>&1 | tail -1
done
| Source | Rows | Status |
|---|---|---|
| ssb-klass-kommuner | 1 327 | ✓ |
| ssb-klass-fylker | 41 | ✓ |
| ssb-08764 | 1 790 | ✓ |
| ssb-06083 | 9 459 | ✓ |
| ssb-06913 | 783 104 | ✓ (after explicit-valuecodes fix; see below) |
| ssb-06944 | 49 245 | ✓ |
| ssb-06947 | 5 180 | ✓ |
| ssb-07459 | 210 728 | ✓ |
| ssb-09429 | 41 118 | ✓ |
| ssb-12063 | 13 365 | ✓ |
| ssb-12131 | 6 237 | ✓ |
| ssb-12132 | 13 365 | ✓ |
| ssb-12292 | 43 659 | ✓ |
| ssb-12944 | 4 296 | ✓ |
| ssb-13995 | 30 294 | ✓ |
| fhi-bor-alene | 12 270 | ✓ |
| fhi-mobbing | 2 454 | ✓ |
| fhi-trangbodd | 36 810 | ✓ |
| fhi-vgs-gjennomforing | 24 540 | ✓ |
| redcross-branches | 391 + 2 143 | ✓ |
ssb-06913 fix — explicit valuecodes required
During the baseline run ssb-06913 failed with PxWebAPI 400 "Parameter error: Non-existent value". Root cause: SSB's PxWebAPI v2-beta now rejects unfiltered /data calls for large tables (06913 is 1 288 × 8 × 76 ≈ 783 k cells, just under the 800 k cap). Smaller tables like 08764 (1 790 cells) still accept the no-filter default.
Fix landed (same PR as this baseline): the ingest module passes filters: { Region: "*", ContentsCode: "*", Tid: "*" } explicitly. Ingest now succeeds with 783 104 rows; downstream indicators__ssb_06913 and fact_kommune_indicators rebuild clean (51/51 tests pass).
If a similar 400 "Non-existent value" error appears on another source in the future, the same fix likely applies — add explicit * filters for every dimension.
Phase B — dbt build + test
Result: clean build, baseline match.
cd atlas-data/dbt
uv run --env-file ../ingest/.env dbt build
- PASS: 654
- WARN: 21 (matches the post-PLAN-002 baseline — 20 pre-existing data-quality warns + 1 expected new for Svalbard
2100onmart_kommune_local_chapters) - ERROR: 0
- TOTAL: 675
./check-osmosis.sh strict ✓ (every column in every schema.yml has a description, repo-wide).
Phase C — Frontend
Result: all 14 routes render with real data.
cd atlas-frontend
npm run typecheck # clean
npm run build # clean
npm run start # serves on :4000 (configured in package.json)
Note: Atlas's frontend serves on port 4000, not the Next.js default 3000 (port 3000 is held by the urbalurba-infrastructure Docusaurus site in this dev setup).
| Route | bytes | Notes |
|---|---|---|
/ | 8 648 | Home |
/admin/supply/redcross-branches | 49 059 | Admin / supply audit |
/coverage-gap/barnefattigdom | 60 192 | Map page |
/data | 353 917 | Indicator index (all sources) |
/data/ssb-08764/EUskala60 | 56 342 | Per-indicator detail |
/kommuner/0301 | 220 145 | Oslo detail |
/ngo | 54 925 | NGO landing |
/ngo/folkehjelp | 12 921 | NGO detail (slug-based) |
/ngo/redcross | 20 611 | Red Cross overview |
/ngo/redcross/aktiviteter | 87 629 | Activity catalog |
/ngo/redcross/chapters | 1 187 427 | All chapters (large response) |
/ngo/redcross/chapters/redcross-S008 | 17 415 | Chapter detail |
/ngo/redcross/distrikt/redcross-D003 | 15 967 | Distrikt detail |
/ngo/redcross/distrikter | 37 831 | Distrikter index |
Spot-checked content: /coverage-gap/barnefattigdom contains "Barnefattigdom", kommune_nr, fylke strings. /data/ssb-08764/EUskala60 contains "EUskala60", "kommune". Real renders, not error pages.
How to re-run this baseline
# Phase A — ingest (~5 minutes)
cd atlas-data/ingest
for s in ssb-klass-kommuner ssb-klass-fylker ssb-08764 ssb-06083 ssb-06913 \
ssb-06944 ssb-06947 ssb-07459 ssb-09429 ssb-12063 ssb-12131 ssb-12132 \
ssb-12292 ssb-12944 ssb-13995 fhi-bor-alene fhi-mobbing fhi-trangbodd \
fhi-vgs-gjennomforing redcross-branches; do
echo "=== $s ==="
npm run --silent "ingest:$s" 2>&1 | tail -1
done
# Phase B — dbt (~1 minute)
cd ../dbt
uv run --env-file ../ingest/.env dbt build
# Phase C — frontend (~1 minute)
cd ../../atlas-frontend
npm run typecheck && npm run build
npm run start & # serves on :4000
sleep 4
for r in "/" "/admin/supply/redcross-branches" "/coverage-gap/barnefattigdom" \
"/data" "/data/ssb-08764/EUskala60" "/kommuner/0301" \
"/ngo" "/ngo/redcross" "/ngo/redcross/aktiviteter" \
"/ngo/redcross/chapters" "/ngo/redcross/distrikter"; do
printf " %-50s %s\n" "$r" "$(curl -s -o /dev/null -w '%{http_code}' http://localhost:4000$r)"
done
kill %1
Numbers will drift over time as upstream sources update; what matters for regression-checking is that every cell in the table above stays at "✓" / "200" / non-trivial body size.