LogoThreatmatic
Database

Local DB Playbook

Restoring a stale local database, syncing schema drift after a pull, and moving a real org's data between two local dev machines.

Local development only

Everything on this page targets localhost Postgres, driven by the root .env file's DATABASE_URL. None of it should ever be pointed at .env.production — that database has a public IP and is live. Always double-check which DATABASE_URL a command is actually using before running anything here.

Syncing after a pull

Pulling main (or any branch with new migrations) can leave your local schema behind. Symptom: you can log in, but pages relying on newer columns/tables error out, or an org/data that should be there just isn't.

cd packages/db
bun run push

drizzle-kit push diffs your live local schema against schema.ts directly and applies the difference — no migration files involved. It needs a real terminal (see below), and if it detects a genuinely destructive change (a dropped/renamed column with real rows in it), it'll show a data-loss warning and ask for confirmation before proceeding. Read that prompt; don't reflexively force through it.

If org/membership data is missing after the schema is current:

bun run seed

This recreates the seeded org and the super-admin user (BETTER_AUTH_ADMIN_EMAIL / BETTER_AUTH_ADMIN_PASSWORD in .env, org id SHARED_ORGANIZATION_ID). If you can log in but see no orgs, you're very likely logged in with the wrong account, not looking at a broken database — check you're using the seeded admin email before assuming something's wrong.

Needs a real terminal

drizzle-kit push renders an interactive confirmation prompt and will fail with Interactive prompts require a TTY terminal if run through anything that pipes its output (CI, an agent's sandboxed shell, etc.). Run it directly in your own terminal.

Known issue: drizzle-kit generate is unreliable here

bun run generate (or the drizzle-kit generate binary directly) currently picks the next migration filename by counting _journal.json entries, not by parsing the highest existing numeric filename prefix. The journal's idx sequence has drifted from the filename numbers — bootstrap_extensions.sql occupies idx 0 with no numeric prefix, and several early migrations were squashed out without the journal being reconciled — so generate can produce a colliding filename and, worse, a full from-scratch schema dump instead of a real diff.

Use push instead to apply schema changes to a local database until this is fixed. If you do run generate and it looks wrong — check the printed filename against ls drizzle/*.sql | tail, and skim whether the SQL body is a few lines (expected) or hundreds (full dump, wrong) — abort: delete the bad .sql file, then git checkout both drizzle/meta/_journal.json and whichever drizzle/meta/00XX_snapshot.json it touched.

Moving a real org between two local machines

Useful when you've built up real test data (devices, zones, policies) on one machine and want it on another, rather than starting from the bare seed. This replaces the destination's entire local database — it's a full swap, not a merge, since per-org data spans ~20+ tables and reliably splicing just one org's rows across all of them (while respecting foreign keys into non-org-scoped tables like pi_worker) is far more fragile than just replacing the whole thing.

Prerequisite: both machines on the same network, and the source machine's Docker Postgres port reachable from the destination (check with docker port postgres on the source — it should show 0.0.0.0:5432, not just bound to the container's internal network).

On the destination machine, find its LAN IP:

ipconfig getifaddr en0   # macOS
# or: ipconfig (Windows) / ip addr (Linux) — find the LAN-facing adapter's IPv4

On the source machine, dump and restore in one pipe — no intermediate file:

docker exec -e PGPASSWORD=postgres postgres \
  pg_dump -h localhost -U postgres -d postgres --clean --if-exists \
  --exclude-table-data='_timescaledb_catalog.*' \
  --exclude-table-data='_timescaledb_internal.*' \
  --exclude-table-data='_timescaledb_config.*' \
  | docker exec -i -e PGPASSWORD=postgres postgres \
    psql -h <destination-lan-ip> -p 5432 -U postgres -d postgres -1 -v ON_ERROR_STOP=1

Exclude TimescaleDB's internal catalog data

The timescaledb extension marks its own internal bookkeeping tables (_timescaledb_catalog.*, _timescaledb_internal.*, _timescaledb_config.*) for inclusion in any pg_dump — this happens regardless of whether you're actually using hypertables (check with SELECT hypertable_name FROM timescaledb_information.hypertables; — an empty result means none are in use, but the extension still ships this internal state). That state is version-specific and not portable across machines with different TimescaleDB versions, producing errors like column "relid" of relation "chunk" does not exist. The three --exclude-table-data flags leave the destination's own extension-internal state untouched while still fully dumping/restoring everything in public — your actual application data.

Don't try to work around this by excluding specific application tables instead (e.g. --exclude-table=public.payload_inspection_events) — that just trades one problem for another: --clean then can't DROP TYPE/CREATE TYPE on any enum that table's leftover columns still reference, since the excluded table survives as a dependent. Excluding the extension's own internal data is the actual fix; application tables should go through the restore normally.

-1 -v ON_ERROR_STOP=1 is not optional

A plain psql run without these executes the dump as a series of separate statements, not one transaction. If anything errors partway through — a constraint, an ordering issue, anything — psql logs it and keeps going by default, silently leaving some tables fully replaced and others still holding old data from before the restore. That produces a genuinely broken, half-migrated database that looks fine until something crashes on a stale foreign key days later. -1 wraps the whole script in one transaction; ON_ERROR_STOP=1 aborts and rolls the whole thing back the instant anything fails, so a restore either fully succeeds or fails loudly and leaves the destination untouched — never stuck in between.

Notes on that command:

  • --clean --if-exists makes pg_dump emit DROP ... IF EXISTS before every CREATE, so the destination gets fully replaced without you needing to manually wipe its schema first — and it's safe against extensions already installed there (CREATE EXTENSION IF NOT EXISTS is a no-op if present).
  • docker exec does not inherit your shell's env vars into the container. Setting PGPASSWORD in your own terminal before running this does nothing — you have to pass -e PGPASSWORD=... on each docker exec call individually, since the two legs of the pipe may need different passwords if the two machines' local Postgres containers weren't provisioned identically. This is the cause of the FATAL: password auth failed error if you hit it. Check the actual configured password on a given machine with:
    docker inspect postgres --format '{{range .Config.Env}}{{println .}}{{end}}' | grep POSTGRES_PASSWORD
  • Swap the container name (postgres here) on either side if yours differs — docker ps to check.

Verify it landed — check every table, not a sample. A spot check on 2-3 tables can look fine while others silently missed a statement. Since the restore ran under ON_ERROR_STOP=1, a clean finish (no error printed) is actually strong evidence everything applied — but confirm it directly rather than trusting that alone:

psql -h localhost -p 5432 -U postgres -d postgres -t -c "
SELECT table_name FROM information_schema.tables
WHERE table_schema = 'public' AND table_type = 'BASE TABLE'
ORDER BY table_name;
" | while read -r t; do
  t=$(echo "$t" | xargs); [ -z "$t" ] && continue
  count=$(psql -h localhost -p 5432 -U postgres -d postgres -t -c "SELECT count(*) FROM \"$t\";" | xargs)
  printf "%-40s %s\n" "$t" "$count"
done

Tables sitting at 0 aren't necessarily a problem — a dev machine that's never had anyone register an OAuth client, enroll a passkey, or configure a pi_worker fleet locally will legitimately have zero rows in those tables on the source too. The useful signal is comparing this list against what you'd expect to be non-trivial (devices, zones, policies, whatever the source machine actually has real data in) — not that every table has rows.

Symptom: org switcher crashes with Cannot read properties of null (reading 'id')

Stack trace points into better-auth-ui's OrganizationSwitcher, in the .map() over the current user's organization list. This means a member row (or session.active_organization_id, or other org-scoped rows) references an organization id that doesn't actually exist in the organization table — a leftover from an earlier, non-atomic restore attempt that partially applied before this playbook's -1 -v ON_ERROR_STOP=1 requirement existed, or was run without it.

Don't reach straight for deleting the dangling rows — first check whether it's actually dangling, or a second real organization whose own row just didn't land while its dependent data did (this is exactly what happened once: a member/policy/zone/device cluster pointed at an org id that turned out to be a second legitimate org, not garbage):

psql -h localhost -p 5432 -U postgres -d postgres -c "
SELECT m.organization_id, count(*) FROM member m
LEFT JOIN organization o ON o.id = m.organization_id
WHERE o.id IS NULL GROUP BY m.organization_id;
"

If that organization genuinely doesn't exist anywhere (not even on the source machine), the row is truly stale and safe to remove. If it turns out to be a real org that's missing its organization row specifically, the correct fix is re-running the full restore properly (with the atomicity guard) — not patching this one table, since the same partial-restore almost certainly left dangling references in others too (policy, zone, device, app_catalog, audit_log are the ones that showed it in practice).

This is a full local-database replacement, not a scoped merge — anything that was only on the destination machine's local database is gone after this runs. Fine for a dev box seeded from the same script everyone else uses; worth a beat of hesitation before running against a local database with something on it you actually care about keeping.

How is this guide?

Last updated on

On this page