Data Structure Reorganisation

Target Applications Inventory

Audit of every target application's current data layout against endpoint-doctrine.

  AppFolderRoleCurrent L1 storage

    hinata-studioapplications/hinata-studio/Multi-tab UI (24 commanders × surfaces)Mixed: D1 + public/data JSON + jsonl
    zepile-collectorapplications/zepile-collector/Housing scraperproperties.json (flat file)
    shikamaru-geographyapplications/shikamaru-geography/Geography data layerstatic JSON (canon)
    trunks-scoutapplications/trunks-scout/Instagram + TikTok researchersflat folder outputs (Whisper)
    Football pollers~/Sandpit/hinata/scripts/football-*.pyFootball dataPostgres
    Eventsjimmy-vps /eventsActivity logPostgres

Doctrine Compliance Matrix

  AppL1L2 queryL3 snapshotEventsSchemaSelf-describeScore

    eventsPostgresFastAPIn/ayespartialno4/6
    footballPostgresFastAPIn/anonono2/6
    bulma-ledgerjsonl + HTMLstatic filerebuiltpartialnono1/6
    zepile-collectorproperties.jsonVite devnononono0/6
    trunks-scoutflat foldernonenononono0/6
    shikamaru-geographystatic JSONVite serveddata is L3nonono3/6

Per-App Reorg Plan

A — bulma-ledger

Current sin: HTML files that Studio iframes. Migration: ship Wave 2-3 of bulma-endpoints (#840063, #840064) — extract report-generation logic into a query layer with Postgres + FastAPI router. Status: staged, awaiting #600010 OAuth verify.

B — zepile-collector

Current sin: monolithic properties.json. Migration: ship Wave 4 of zepile-endpoints (#840070). Postgres zepile.properties table. FastAPI /zepile/housing with proper filtering.

C — trunks-scout

Current sin: Whisper transcripts as flat .md files. Migration: NEW Postgres tenant scout.*. FastAPI /scout/transcripts/search with full-text search via Postgres tsvector. Task: emit #840073.

D — hinata-studio (non-D1 tabs)

Migration: consolidate per-Commander tab onto its tenant pattern. Bulma → /bulma/* (Wave 3), Zepile → /zepile/* (Wave 4), and 6 more. Each Commander gets {commander}-snapshot.json (≤10KB). Task: umbrella #840074.

Cross-Cutting Moves

G — Snapshot Rebuilder Pattern

Every Commander gets update-{commander}-snapshot.py LaunchAgent that reads L1, computes L3 snapshot, writes to public/data/{commander}-snapshot.json, emits {commander}.snapshot_updated event, re-fires on producer events. Eight rebuilders — tasks #840075.

H — Schema YAML Harvest

Each tenant gets the-government/information_reference/reference_api-schemas/{tenant}.yaml — single source of truth for Studio TypeScript codegen, bot system prompt injection, contract testing. Task: #840076.

I — /grammar Self-Describe

Each tenant exposes /{tenant}/grammar GET returning categories, periods, buckets, endpoints. Bots learn their own query vocabulary from the same source the FastAPI router uses.

Migration Sequence

  OrderWaveWhy

    1Doctrine + audit (this doc)Lock pattern before mass migration
    2Bulma endpoints Wave 2-3 (#840063, #840064)First doctrine-compliant tenant; proves pattern
    3Simba endpoints Wave 2-4 (#840068-#840070)Second tenant; validates pattern
    4Snapshot rebuilders for remaining 6 Commanders (#840075)Cheap, high observability gain
    5Trunks-scout /scout/* tenant (#840073)Unlocks full-text search
    6Remaining Commander tenants (Zoro, AllMight, Calendar, Brook, Sanji, Squidward extension)Heaviest cumulative — defer until 2-3 prove value

◆ hinata · projects/data-structure-reorganisation.html · phase-18 flatten