Skip to main content

api_v1: the public API surface

Atlas's external HTTP API is a PostgREST instance projecting the api_v1.* schema as REST endpoints with auto-generated OpenAPI docs. The schema is auto-generated by Atlas; PostgREST is deployed by UIS (separate repo, separate ownership). This page covers Atlas's side: what api_v1 is, why it's separate from marts.*, how the generator works, and what to do when you add a new mart.

For PostgREST itself (the deployment, role pair, configure semantics), see UIS's PostgREST service docs. For the design rationale, see INVESTIGATE-postgrest-api-v1-wrapper.md. For the build itself, see PLAN-004-postgrest-api-v1-wrapper.md.

What api_v1 is​

A Postgres schema containing one wrapper view per mart_* view in models/marts/api/. Each wrapper does almost nothing:

CREATE OR REPLACE VIEW api_v1.indicator_summary AS
  SELECT * FROM marts.mart_indicator_summary;

Three lines per dataset. The view doesn't transform — it re-names and re-locates the underlying mart so PostgREST projects a clean public surface. The mart_ prefix is dropped in the api_v1 name (mart_indicator_summary → api_v1.indicator_summary) because every view in api_v1 is API by definition; the prefix is noise.

Why a separate schema?​

Three reasons:

  1. Internal vs external boundary. marts.* is what dbt produces; api_v1.* is the public contract. They're not the same thing — a refactor that changes a mart_* column shape needs to be invisible to external API consumers, which means the public name must be stable independently. Without api_v1, every internal change is an external break.
  2. Versioning anchor. api_v1 literally signals v1 of the contract. When a future breaking change lands, api_v2 exists alongside as a separate schema; consumers migrate at their own pace. marts.* has no equivalent.
  3. Multi-app coherence. UIS deploys one PostgREST instance per consuming app (atlas, eventually customers, etc.). The api_v1 schema name is the convention every UIS-deployed app follows. Atlas accepting it sets the pattern.

What's in api_v1 today​

9 wrapper views, one per PLAN-001 mart:

api_v1.activity_catalog              api_v1.kommune_local_chapters
api_v1.coverage_gap_barnefattigdom   api_v1.ngo_index
api_v1.distrikt_summary              api_v1.ngo_overview
api_v1.indicator_latest_values
api_v1.indicator_missing_kommuner
api_v1.indicator_summary

71 columns total, each with a Postgres COMMENT ON COLUMN (the OpenAPI spec PostgREST projects sources its description fields from these comments).

What's NOT in api_v1​

  • mart_* views in models/marts/ but NOT under models/marts/api/ — these are internal-only (consumed by Atlas's frontend or by other dbt models). Example: mart_ingest_health. They stay in marts.* and don't get a wrapper.
  • dim_* / fact_* tables — these aren't intended for direct external consumption. The mart_* views handle the join + filter + shape work.
  • FK embeds. PostgREST's ?select=*,kommune(*) embedding feature requires actual Postgres FOREIGN KEY constraints on the underlying tables. Atlas's marts.* doesn't have those (dbt's relationships: tests are SQL assertions, not DDL). v1 of the API offers the wrapper views with all needed columns inline (fat rows by design); embeds are deferred. See INVESTIGATE-postgrest-api-v1-wrapper.md § Q10.

How the generator works​

atlas-data/dbt/scripts/generate_api_v1.py reads dbt's target/manifest.json after dbt parse, walks every model under models/marts/api/, and emits a single SQL artefact:

atlas-data/dbt/api_v1_generated.sql   ← the SQL to apply
atlas-data/dbt/api_v1_state.json      ← snapshot of view names; drives Q17 drop emission

The SQL contains:

  • CREATE SCHEMA IF NOT EXISTS api_v1
  • DROP VIEW IF EXISTS api_v1.X CASCADE for any view in the prior state-JSON but not in the current manifest (a view being removed)
  • CREATE OR REPLACE VIEW api_v1.<name> AS SELECT * FROM marts.mart_<name> per current api/ model
  • COMMENT ON COLUMN api_v1.<view>.<col> IS '<description from manifest>' per described column
  • A guarded DO-block that grants SELECT to <app>_web_anon if that role exists (created by UIS's ./uis configure postgrest)
  • NOTIFY pgrst, 'reload schema' so a running PostgREST refreshes its schema cache

Idempotent throughout (CREATE OR REPLACE / IF NOT EXISTS / DROP IF EXISTS). Re-applying is a no-op.

The shell wrapper atlas-data/dbt/regenerate-api-v1.sh just runs dbt parse then the Python script.

Why not in migrations/?​

The wrapper views reference marts.mart_* tables that only exist after dbt run. Atlas's migration runner (atlas-data/ingest/scripts/migrate.ts) applies SQL files on every npm run migrate — running before dbt run would fail because the underlying tables don't exist. So the api_v1 SQL lives at atlas-data/dbt/api_v1_generated.sql and is applied separately via apply-api-v1.sh after dbt run. The order:

npm run migrate         # creates raw schemas
npm run ingest:*        # populates raw.*
dbt seed                # loads ref_*.csv + dim_postnummer.csv (required on fresh db)
dbt run                 # builds marts.*
./apply-api-v1.sh       # creates api_v1.* wrappers

apply-api-v1.sh uses docker run postgres:16-alpine psql ... -f so contributors don't need a host-installed psql.

Day-to-day workflow​

Adding a new mart_* (with public-API wrapper)​

The full ingest+dbt workflow lives in adding-a-source.md. The api_v1-relevant additions:

  1. After your dbt model lands under models/marts/api/<name>.sql and its schema.yml entry has descriptions on every column (per check-osmosis.md) — run: 
    cd atlas-data/dbt
    ./regenerate-api-v1.sh
  2. Inspect the diff in atlas-data/dbt/api_v1_generated.sql and api_v1_state.json — should contain a new CREATE OR REPLACE VIEW api_v1.<your_view> block + per-column COMMENTs.
  3. Apply to your local DB so the runtime tests pass: 
    ./apply-api-v1.sh
    uv run --env-file ../ingest/.env dbt test --select api_v1_descriptions_complete api_v1_rowcount_matches_marts
  4. Update tests/api_v1_rowcount_matches_marts.sql — add a union all line for the new view pair (the test is hand-maintained today).
  5. Commit: dbt model + schema.yml + the regenerated artefacts + the test edit.

The drift gate (./check-api-v1.sh) catches step 1 if you forget; the row-count test catches step 4. Naming-conventions rule #9 enforces this in PR review.

Deprecating then removing a mart_*​

A two-phase process:

Phase A — Deprecate (signal, but keep serving traffic). Add a meta: block to the model's schema.yml entry indicating deprecation:

- name: mart_old_view
  meta:
    deprecated_until: '2026-12-01'
    deprecated_reason: 'Replaced by mart_new_view; see PLAN-XXX'

(Future enhancement: the generator will surface this as a deprecation comment on the api_v1 view. Today, communicate via release notes / NOTE files.)

The wrapper still serves traffic; consumers see the deprecation in release notes / commit history. Grace period: at least one consumer-notice cycle.

Phase B — Remove (after grace period, no traffic). Move the model out of models/marts/api/ (delete the .sql + schema.yml entry, or move it to models/marts/ if it's still used internally). Run:

./regenerate-api-v1.sh

The generator notices the view is in api_v1_state.json but not in the current manifest, and emits DROP VIEW IF EXISTS api_v1.<name> CASCADE at the top of the new SQL. The diff of api_v1_state.json is the audit trail. Apply via ./apply-api-v1.sh. Don't forget to remove the corresponding line from tests/api_v1_rowcount_matches_marts.sql.

Changing a column description​

schema.yml description in dbt → regenerator → COMMENT ON COLUMN in api_v1_generated.sql → applies to the live view → propagates into the next OpenAPI spec PostgREST serves. Same workflow as adding a new mart: regenerate-api-v1.sh, commit, apply.

The five validation gates​

./check-api-v1.sh runs three static gates locally + in CI (no Postgres connection needed):

  1. Drift — regenerator output matches checked-in artefacts. Catches "you changed the model but forgot to regenerate" and "someone hand-edited the generated SQL".
  2. Coverage — every models/marts/api/ model has a wrapper; no orphan wrappers.
  3. Static description coverage — number of COMMENT ON COLUMN api_v1.* lines = total described columns in api/ models.

dbt test runs two singular tests against an applied database:

  1. Runtime description coverage (tests/api_v1_descriptions_complete.sql) — pg_catalog.pg_description returns no NULLs for api_v1.* columns.
  2. Row-count parity (tests/api_v1_rowcount_matches_marts.sql) — every api_v1.X row count matches marts.mart_X.

All five must be green to merge a PR touching api_v1.

Rollback​

If something goes wrong with the api_v1 layer:

  1. Coordinate with UIS first. PostgREST is reading from api_v1 once UIS deploys. The order matters: 
    # On the UIS side first:
    ./uis undeploy postgrest --app atlas             # stop the deployment
    ./uis configure postgrest --app atlas --purge    # drop atlas_authenticator + atlas_web_anon roles + secret
  2. Then on Atlas side: 
    psql "$DATABASE_URL" -c 'DROP SCHEMA api_v1 CASCADE;'
    This drops all api_v1.* views. The underlying marts.mart_* tables are untouched.
  3. To restore: re-run ./apply-api-v1.sh (creates the schema and views again). UIS's configure-then-deploy then re-creates the role pair + redeploys PostgREST.

The generated SQL is idempotent so re-applying is always safe. Don't hand-edit api_v1_generated.sql; the drift gate will fail and the next regenerate will overwrite your edits.

Cross-references​