Embodied energy: room_name-keyed derived entries + reconciliation¶
Context¶
Follow-up refactor on top of
878-embodied-energy-csv-ingestion
(unified derived-entry creation behind DERIVED_ENTRY_WORKFLOWS). Two
structural problems remained:
- Derived entries persisted
building_name— a value copied from the parent at creation time. The parentbuildingrows themselves persist onlyroom_nameand resolve everything else from theBuildingRoomreference table, so the derived rows carried a second, denormalized copy that could go stale and that clients could overwrite. - The workflow tracked parent↔derived identity per entry
(
_get_embodied_energy_entry_id, separate create/update/delete branches). Every CRUD path had to know the mutated entry's previous shape, and any missed path (bulk ingest, historically) left orphans or gaps.
Approach¶
Make the derived-entry set a pure function of the parents: one
building_embodied_energy row per parent building room_name
(multiset — duplicate parent rooms get duplicate derived rows), with
{"room_name"} as the entire persisted payload.
- Reference data resolves at compute/read time. The handler's
pre_computelooks upBuildingRoombyroom_nameforbuilding_name+room_surface_square_meter(shared_prefetch_rooms/_resolve_roomhelpers with the rooms handler; slice-cached in bulk). An unresolvable room yields no context, so the formula produces no emission — the module's "skip, don't default" semantic. - CRUD sync is a reconcile, not per-entry bookkeeping.
EmbodiedEnergyWorkflow.post_create/post_update/post_deleteall call one_reconcile: read the module's entries post-commit, count wantedroom_names from the parents, delete surplus derived rows (falsy or over-represented names, highest ids first), create missing ones viaCarbonReportModuleWorkflow. Idempotent; needs no knowledge of what changed.post_deletenow takesdata_entry_type_idinstead of the deleted item's id. - Bulk ingest derives per batch.
_create_derived_entries(batch)runs inside_process_batchon the just-inserted rows instead of a job-end read-back bycreated_by_id(list_by_creator_and_typedeleted). The registry protocol now receives the whole batch and each workflow selects its own source rows. Bulk ingest's delete-then-recreate already clears stale derived rows, so no reconcile is needed there.
Changes¶
workflows/embodied_energy.py— rewrite:_reconcile(Counter-based multiset convergence) replaces_post_create_building/_post_update_building/_get_embodied_energy_entry_id;create_derived_entries_forbecomes a pure mapbuilding rows → {"room_name"} payloadsover the batch.workflows/derived_entry_registry.py— protocol takes a whole ingest batch; workflows filter it themselves.services/data_ingestion/base_csv_provider.py— derived-entry creation moves fromfinalizeto_process_batch.modules/buildings/data_entries.py— DTOs keyed onroom_name;DiscardClientBuildingFieldsMixindrops any client-sentbuilding_name; response exposes resolvedbuilding_name/room_surface_square_meteras optional.modules/buildings/handlers.py—BuildingEmbodiedEnergyModuleHandlergainsprefetch_slice/pre_compute(shared room helpers);group_map/filter_mapreadbuilding_namefrom aBuildingRoomjoin and addroom_name; factor context takesbuilding_namefrom the resolved ctx.repositories/data_entry_emission_repo.py— embodied breakdown joinsBuildingRoom(inner join: entries whoseroom_nameno longer resolves carry no emission and drop out).repositories/data_entry_repo.py—building_embodied_energyincluded in the buildings enrichment branch (resolvedbuilding_namein listings);list_by_creator_and_typeremoved.api/v1/carbon_report_module.py— delete endpoint passes the entry'sdata_entry_type_idtopost_delete.seed/random_generator/seed_data_entries.py— seedsroom_name.
Tests¶
tests/integration/services/data_ingestion/test_buildings_csv_pg.py— derived rows carry{"room_name"}payloads; per-batch creation covered.tests/integration/services/data_ingestion/test_strategy_b_rematch_pg.py— rematch path updated to the batch-level registry protocol.tests/unit/repositories/test_data_entry_emission_repo.py— embodied breakdown resolvesbuilding_namethrough theBuildingRoomjoin.
Delivery¶
Branch fix/878-construction-not-showing, on top of
0e42be999 (bulk-ingest derivation) and bbc8bea21 (registry
unification). No migration: pre-refactor derived rows persisted
building_name without room_name, so they compute no emission under the
new handler and count as surplus (falsy room_name) in the reconcile —
the next parent CRUD mutation or re-ingest of the module deletes them and
recreates room_name-keyed rows from the parents.