api_v1.meta_sources
Atlas view
api_v1.meta_sources
api_v1.meta_sourcesOne row per ingested source: publisher, licence, coverage, freshness and what it feeds.
First 5 rows ·
api_v1.meta_sourcessnapshot 2026-05-12
| source_id | upstream_id | upstream_url | upstream_landing_page | upstream_title | description | publisher | license | license_url | periodicity | eu_theme | attribution | tags | last_ingested_at | last_upstream_update_at | latest_row_count | total_runs | downstream_model_count |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| bufdir-barnefattigdom | barnefattigdom-monitor | https://www.bufdir.no/statistikk-og-analyse/monitor/barnefattigdom/ | https://www.bufdir.no/statistikk-og-analyse/monitor/barnefattigdom/ | Barnefattigdom kommunemonitor | Barne-, ungdoms- og familiedirektoratet (Bufdir) publishes the Barnefattigdom kommunemonitor on bufdir.no and ships a single bulk ZIP of `Indikator_*.xlsx` workbooks ("last ned hele datasettet"). Atlas downloads that ZIP (URL discovered from the monitor page), parses sheet `Data` in each workbook, and loads kommune- and sub-kommune-level time series into `raw.bufdir_barnefattigdom` so analysts can compare Bufdir's poverty lens with register-based sources such as SSB low-income statistics. | Barne-, ungdoms- og familiedirektoratet | NLOD | https://data.norge.no/nlod/no/2.0 | P1Y | SOCI | Kilde: Barne-, ungdoms- og familiedirektoratet, Barnefattigdom kommunemonitor | ["provider:bufdir","topic:income","geo:kommune","cadence:annual","eu_theme:SO… | 2026-05-06T14:32:41.546258+00:00 | 2025-07-31T12:50:25+00:00 | 395420 | 5 | 1 |
| fhi-alkohol | 332 | https://statistikk-data.fhi.no/api/open/v1/nokkel/table/332 | https://statistikk.fhi.no/nokkel/129pesZRVuNuNrXngx3ao4bF8odgrZClPRFCbZEJnxs?GEO=0,03&AAR=2020_2020,2021_2021,2022_2022,2023_2023,2024_2024,2025_2025&KJONN=0&ALDER=1_6&ANTALL_GANGER=engangellerflere&SOES=0&MEASURE_TYPE=SMR | Alkohol_Ungdata_KH | FHI Folkehelsestatistikk table 332 — share of Ungdata respondents reporting alcohol use one or more times in the past year. Risk-direction youth-substance-use indicator on the Ungdata cohort base. | Folkehelseinstituttet | NLOD | https://data.norge.no/nlod/no/2.0 | P1Y | HEAL | Kilde: Folkehelseinstituttet, Folkehelsestatistikk tabell 332 (Ungdata) | ["provider:fhi","topic:health","geo:kommune","cadence:annual","eu_theme:HEAL"] | 2026-05-06T14:32:48.195641+00:00 | 2026-01-20T21:37:20.273+00:00 | 34356 | 1 | 0 |
| fhi-befolkning | 338 | https://statistikk-data.fhi.no/api/open/v1/nokkel/table/338 | https://statistikk.fhi.no/nokkel/iIALHYpjwwxCYECV9iAnanUAYXxFBhFZdQW1M7T5EeA?GEO=0&AAR=2020_2020,2021_2021,2022_2022,2023_2023,2024_2024,2025_2025&KJONN=0&ALDER=0_120,0_17,80_120&MEASURE_TYPE=TELLER | Befolkningssammensetning_antall_andel | FHI Folkehelsestatistikk table 338 — Befolkningssammensetning. Population counts by region × sex × age band, used as the demographic denominator across other indicators (per-capita rates, age structure). | Folkehelseinstituttet | NLOD | https://data.norge.no/nlod/no/2.0 | P1Y | SOCI | Kilde: Folkehelseinstituttet, Folkehelsestatistikk tabell 338 | ["provider:fhi","topic:demographics","geo:kommune","cadence:annual","eu_theme… | 2026-05-06T14:32:55.767721+00:00 | 2025-10-21T08:50:38.184+00:00 | 42945 | 1 | 0 |
| fhi-befolkningsvekst | 185 | https://statistikk-data.fhi.no/api/open/v1/nokkel/table/185 | https://statistikk.fhi.no/nokkel/_SvW9Ag1SYWnNEu0EHxrI_00QJUulcZKUvTgpPNqiyw?GEO=0,03,31,32,33,34,39,40,42,11,46,15,50,18,55,56&AAR=2019_2019,2020_2020,2021_2021,2022_2022,2023_2023,2024_2024&KJONN=0&ALDER=0_120&MEASURE_TYPE=TELLER | Befolkningsvekst | FHI Folkehelsestatistikk table 185 — year-over-year population growth per region (counts and percent rates). Pairs with fhi-befolkning (observed) and fhi-prognose (projected) for full historical-and-forecast trajectories. | Folkehelseinstituttet | NLOD | https://data.norge.no/nlod/no/2.0 | P1Y | SOCI | Kilde: Folkehelseinstituttet, Folkehelsestatistikk tabell 185 | ["provider:fhi","topic:demographics","geo:kommune","cadence:annual","eu_theme… | 2026-05-06T14:32:59.879202+00:00 | 2025-10-21T08:56:39.806+00:00 | 18814 | 1 | 0 |
| fhi-bor-alene | 187 | https://statistikk-data.fhi.no/api/open/v1/nokkel/table/187 | https://statistikk.fhi.no/nokkel/E1gPZ5le1oMsIccxUCPlzvSxxb-8u3gtb7h1tg_zGwg?GEO=0&AAR=2020_2020,2021_2021,2022_2022,2023_2023,2024_2024,2025_2025&ALDER=30_44,45_120,75_120&MEASURE_TYPE=TELLER | Personer som bor alene | FHI Folkehelsestatistikk table 187 — "Personer som bor alene". Share of adults (16+) living alone, per region, annual. | Folkehelseinstituttet | NLOD | https://data.norge.no/nlod/no/2.0 | P1Y | SOCI | Kilde: Folkehelseinstituttet, Folkehelsestatistikk — tabell 187 | ["provider:fhi","topic:demographics","geo:kommune","cadence:annual","eu_theme… | 2026-05-06T14:33:03.172822+00:00 | 2025-11-18T09:20:23.692+00:00 | 12270 | 1 | 6 |
Built from
This Atlas view joins data from the following upstream sources. Click through for provenance, methodology, and citation.
Showing direct dbt parents. See full lineage in /lineage/ →
Columns
| Column | Description |
|---|---|
| source_id | Atlas catalogue id; primary key. Matches `_sources_manifest.source_id`. |
| upstream_id | Upstream's own identifier (SSB table number, FHI dataset slug, etc.). |
| upstream_url | Canonical link to the data on the upstream's site (often the API endpoint). |
| upstream_landing_page | Human-browsable web page describing the dataset (DCAT-AP `dcat:landingPage`). Empty when `upstream_url` is already browsable; clients fall back to that. |
| upstream_title | Source's authoritative title at the upstream (usually Norwegian). |
| description | One-paragraph framing for the customer-facing catalogue. |
| publisher | Institution that publishes the data. |
| license | License token (e.g. `NLOD`); see `license_url` for terms. |
| license_url | URL of the license terms. |
| periodicity | Update cadence as ISO 8601 (`P1Y` / `P3M` / `P1M` / `P1D`) or `irregular`. More precise than the `cadence:` tag. |
| eu_theme | EU Publications Office Data Theme code (one of 13: `AGRI`, `ECON`, `EDUC`, `ENER`, `ENVI`, `GOVE`, `HEAL`, `INTR`, `JUST`, `REGI`, `SOCI`, `TECH`, `TRAN`). Aligns with Felles datakatalog / DCAT-AP. |
| attribution | Citation string for academic / legal compliance (e.g. "Kilde: Statistisk sentralbyrå, tabell 08764"). |
| tags | `text[]` of namespaced tags: `provider:<X>`, `topic:<X>`, `geo:<X>`, `cadence:<X>`, `eu_theme:<X>`. Filter via PostgREST's array-contains operator, QUOTING the value because it contains a colon: `?tags=cs.{"topic:ngo-supply"}`. Unquoted it returns 22P02 malformed array literal. Always 5 entries (one per declared namespace). |
| last_ingested_at | `MAX(finished_at)` from `raw.ingest_runs` filtered to successful runs (`exit_code = 0`). NULL until the source has at least one successful run on the current cluster. ⚠️ AS OF THE LAST TRANSFORM, NOT LIVE. mart_meta_sources is materialized as a TABLE, so this is a snapshot taken when transform_and_publish last ran. An ingest that succeeded since then is invisible here until the next one. Do not poll this field to decide whether an ingest has completed — it can lag by hours and reads as though it were current. |
| last_upstream_update_at | `MAX(upstream_updated_at)` from successful ingest runs. NULL when the source hasn't run yet OR its ingest module doesn't capture the upstream's "updated" field (Red Cross web scrape, Brreg per-entity fetch). ⚠️ AS OF THE LAST TRANSFORM, NOT LIVE. mart_meta_sources is materialized as a TABLE, so this is a snapshot taken when transform_and_publish last ran. An ingest that succeeded since then is invisible here until the next one. Do not poll this field to decide whether an ingest has completed — it can lag by hours and reads as though it were current. |
| latest_row_count | `rows_parsed` from the most recent successful run. NULL when the source has zero successful runs on the current cluster. ⚠️ AS OF THE LAST TRANSFORM, NOT LIVE. mart_meta_sources is materialized as a TABLE, so this is a snapshot taken when transform_and_publish last ran. An ingest that succeeded since then is invisible here until the next one. Do not poll this field to decide whether an ingest has completed — it can lag by hours and reads as though it were current. |
| total_runs | Count of successful runs ever recorded for this source. 0 when no successful run exists. ⚠️ AS OF THE LAST TRANSFORM, NOT LIVE. mart_meta_sources is materialized as a TABLE, so this is a snapshot taken when transform_and_publish last ran. An ingest that succeeded since then is invisible here until the next one. Do not poll this field to decide whether an ingest has completed — it can lag by hours and reads as though it were current. |
| downstream_model_count | Number of distinct downstream dbt models that derive from this source via the `lineage` seed. 0 for sources not yet wired into a mart. 🔴 DO NOT READ THIS AS "IS IT SERVED". It counts models, and a model is not a consumer surface. A source feeding 357 rows into a published relation and a source whose only model reaches nothing both report a positive count. Use `served_as`. ⚠️ On 2026-09-21 this field produced a statement to a consumer that "the eight FHI sources" were served, when it was seven — `fhi-innvandrere` has one model, 32,720 rows and no published series. Four parties had each built a different partial proxy for the question this field looks like it answers, and three were wrong the same day (urb-agents #1344). 🔴 AND THIS NUMBER CAN CHANGE FOR TWO REASONS THAT LOOK IDENTICAL. Either a model genuinely shipped, or the lineage it is computed from was CORRECTED. A consumer reading it twice cannot tell an honest value change from an accuracy fix: on 2026-09-21 the demo consumer read it morning and evening, got 9 and 1, and had no way to know which it was seeing (urb-agents #1348). 🔵 `lineage` is generated from the dbt manifest and committed, so it can be stale — 103 edges were missing that day, some for weeks. `check-lineage-is-current.sh` now fails CI on drift, so the accuracy kind of change should stop happening. It cannot be detected retroactively in a value you already read. |
| served_as | The `api_v1` relations a consumer can actually reach this source through, as a Postgres `text[]`. **Empty means nothing published depends on it** — that is the gap, and it is the one question `downstream_model_count` cannot answer. Filter for the gap with `?served_as=eq.{}`, or for a specific surface with `?served_as=cs.{indicator_summary}`. 🔵 NOT A PLAIN LINEAGE JOIN, DELIBERATELY. A relation derived from `fact_kommune_indicators` is only claimed when the source actually has rows in that fact. The case that forced that design: `ssb-06913` fed the fact and so reached four published relations on paper while contributing ZERO rows to them — its `kommune_nr` regex never matched, so 783,104 rows were discarded downstream. A structural boolean would have called it served and become the fifth wrong proxy. It was fixed the same day (urb-agents #1345) and now reports those four relations, which is the column working rather than the example expiring. ⚠️ THE CATALOGUE RELATIONS ARE EXCLUDED. `meta_sources`, `meta_endpoints` and `meta_dimensions` hold a row for every source by construction, so naming them would be the same value on every row. They describe a source; they do not serve it. 🔴 IT DOES NOT ASSERT THAT ANY NAMED RELATION CONTAINS ROWS. For a relation derived from the fact, the source is checked to have rows there. For every other relation the claim is LINEAGE ONLY — what the source WOULD serve if it had data. ⚠️ THE CASE THAT PROVES IT: `redcross-branches` has `total_runs` 0 and has never ingested, and it still lists five relations, three of which hold zero rows. A source that has never run looks served. 🔵 THE COMPANION CHECK IS ON THIS SAME ROW. Read `served_as` with `total_runs` and `latest_row_count`: lineage says what could arrive, those say whether anything did. ⚠️ AN EARLIER VERSION OF THIS PARAGRAPH SAID THE SAME THING AND DID NOT WORK. It read "for non-fact relations this is reachability, not volume", naming `brreg_enhet` and `dim_kommune` — relations that are never empty, so the warning was abstract. ops-dev read it and built a gate on the field within the hour (urb-agents #1346). A caveat that names no failing case is a caveat nobody applies. 🔵 Excluding never-ingested sources outright was considered and rejected: it would collapse "has never run" into "is not modelled", losing the distinction between `redcross-branches` (five relations waiting for data) and the sources deferred in BACKLOG (no relation at all). Say the word if that trade is wrong. 🔵 Derived from the same artefact the CI gate reads, so the two cannot drift: `check-every-source-is-served.sh` and this column both resolve the source against the generated `api_v1` relation list. ⚠️ THE GATE AND THIS COLUMN CAN DISAGREE, AND THAT IS CORRECT. The gate is static and cannot see a source that is wired up and delivers nothing, so it will report fewer gaps than this column does. The gate answers "is it wired"; this answers "does it arrive". Neither count is written here on purpose — a number in a published description goes stale silently, which is most of why this column had to be written at all. |
Sample query
Live PostgREST query that returns the first 5 rows. URL adapts to the current host — local dev shows api-atlas.localhost; the deployed site shows the configured API base.
https://api-atlas.urbalurba.com/meta_sources?limit=5
About this view
One row per ingested source: publisher, licence, coverage, freshness and what it feeds.
Per-source catalogue row — one per ingest source in
`_sources_manifest`, joined to `raw.ingest_runs` aggregates so
consumers see freshness alongside the static metadata. Wraps to
`api_v1.meta_sources` and backs `/data/sources` in the customer
frontend (PLAN-007 phase 4).
🔴 THIS RELATION DESCRIBES A SOURCE, NOT ITS COLUMNS. For what an
upstream dimension MEANS — what `AAR` is, whether a code is a
kommune, whether a year is a single year or a window — read
`meta_dimensions`, one row per source × dimension with the
upstream's own words and the value format.
⚠️ THIS POINTER EXISTS BECAUSE ITS ABSENCE COST A DAY. On
2026-09-21 a consumer read all 21 descriptions here, concluded
"exactly two sources use multi-year windows", and shipped a wrong
year on a front page. `meta_dimensions` said `"3-year rolling
cohort"` for a third the whole time. Reading every row of this
relation is thorough and structurally incapable of answering a
question about a dimension (urb-agents #1335).
Tags column is a Postgres `text[]` carrying the four declared
namespaces (`provider:`, `topic:`, `geo:`, `cadence:`) plus the
`eu_theme:` namespace. The customer frontend renders each as a
filter pill; PostgREST consumers filter via `?tags=cs.{"provider:ssb"}`
— the quotes are required, because the value contains a colon.
`last_ingested_at` and `last_upstream_update_at` come from
`raw.ingest_runs` aggregated over successful (`exit_code = 0`)
runs. Sources whose ingest module hasn't yet captured the
upstream's "updated" field leave `last_upstream_update_at` NULL —
that's the design (`raw.ingest_runs.upstream_updated_at` is
nullable per migration 028).
`downstream_model_count` is the number of distinct dbt models
that derive from this source via the `lineage` seed (Phase 3.3).
0 for sources whose data hasn't yet been wired into a mart.