Consolidate Alembic migrations before go-live¶
Goal: one migration file that builds the current schema, with no backfills or one-off cleanups in history. Dev, stage and prod are dropped and rebuilt from it on go-live day (2026-09-22). Same mechanism as #2211.
backend/alembic/CUSTOM_DB_OBJECTS.md is the checklist; this collapse rewrote it
(it still listed indexes dropped since #2211 and missed everything added after).
Audit of the 19 migrations since #2211¶
- Schema, already in models (autogenerate re-emits): pod columns, the
carbon_projects/carbon_reportsindexes,ix_factors_res_*, the recalc dedup pair, check constraints,roles_empty_since,runs_jobs, thedata_entry_emissionscolumns and covering index. - Backfills / cleanups (dropped):
277bf6757926guard,95fe938000d4, the1a9837dd65a3backfillUPDATE. - Migration-only index:
ix_classification_translations_label_trgm(956c36805397). Moved intoClassificationTranslation.__table_args__, pinned bytests/unit/models/test_classification_translation_trgm_index.py. - Migration-only check (found while verifying):
ck_data_ingestion_jobs_unit_specific_has_entity_idsat in a first__table_args__that a second one inDataIngestionJoboverrode, so it never reached the metadata. Merged into the single tuple, Postgres-only (the SQLite unit schema never enforced it), pinned bytests/unit/models/test_data_ingestion_job_check_constraint.py. An AST scan found no other class with a duplicate__table_args__. - Reference data: the three label seeds (
3b5609f893f4,fd12a7a0946f,7bff78de3264) are the only source ofclassification_translationsrows. Maintainer decision: keep them as anop.bulk_insertin the new migration, so every fresh DB gets them from the helm migration Job with no manual step.
Steps¶
- Model: declare the trigram index (+ unit test).
git rmthe 20 files; deletetest_2458_orphaned_explore_cleanup(it tested a dropped data migration); repoint comments naming the seed revisions.- Generate against an empty local DB:
DB_URL=<local scratch> make db-revision. - Hand edits only:
CREATE EXTENSION pg_trgmat the top, the labelbulk_insert. - Prove it: a
pg_dump --schema-onlyof the old chain and of the new migration match (modulo the three check constraints,NOT VALIDin the old chain), and a second autogenerate proposes only the knownuq_emission_recalc_active_scopednoise.make test-db-migratepasses.
Result¶
New head: 128d2947054c (2026_09_22_1021-128d2947054c_initial_migration.py).
Schema-only dump of the old chain vs the new migration on local scratch DBs:
identical except the three checks (NOT VALID in the old chain) and the position
of late-added columns in data_entry_emissions and pods. A second autogenerate
proposed only the uq_emission_recalc_active_scoped re-render. 58 label rows
land.
Rollout¶
Every environment was at 9da4a65ef1e3, which no longer exists. Each DB is dropped
and rebuilt with alembic upgrade head from this branch before its migration Job
runs a new image; otherwise the Job fails with "Can't locate revision". Until the
release ships, an Argo re-sync of the old image fails the same way (sync blocked, pods
keep serving). Once prod is built from the new revision, its ID is frozen: any fix is
a new migration on top.