SV-4 — Data Flow

Viewpoint: how data moves from external sources through curation to the visitor.

flowchart LR
  subgraph sources [External sources]
    osm[OpenStreetMap]
    wikidata[Wikidata]
    wikiquote[Wikiquote]
    commons[Wikimedia Commons]
    iggraph[Instagram Graph API]
  end

  subgraph curation [Offline curation - scripts/]
    importers[Importer scripts - candidates only]
    review{Operator review}
    seedsql[Seed/migration SQL builders]
    imgsync[Image sync + QA scripts]
  end

  subgraph store [Data plane]
    migrations[migrations/*.sql]
    d1[(D1)]
    r2[(R2)]
  end

  subgraph serve [Serving]
    workerapi[Worker API]
    astro[Astro build - static pages]
    cron[Cron: scoring + feed cache]
  end

  visitor([Visitor])

  osm --> importers
  wikidata --> importers
  wikiquote --> importers
  importers --> review --> seedsql --> migrations -->|wrangler d1 migrations apply| d1
  commons --> imgsync --> r2
  imgsync -->|image rows + qa_tier| migrations
  iggraph --> cron --> d1

  d1 --> workerapi --> visitor
  astro --> visitor
  r2 -->|hero images, clips| visitor

Data-flow rules:

• Additive migrations only — history is never rewritten; schema and seed data both travel as numbered SQL files (duplicate numbers exist; apply order is name order).
• Importers emit candidates, not writes — a human gate sits between every external source and D1.
• Instagram data is cache, not record — instagram_feed_cache and instagram_account_scores are refreshed by cron and safely disposable.
• The generated ER view lives at generated/db-schema.md; the route inventory at generated/routes.md.