Equipment global percentage: aggregate line per type (#2783)¶
Symptom¶
PATCH /carbon-reports/{id}/modules/{module_type_id}/reference-percentage
rewrote every prefilled snapshot line individually
(set_reference_percentage_all, carbon_report_module_service.py:393-429):
one Python loop over all N snapshot entries, each going through
DataEntryEmissionService.upsert_by_data_entry (its own factor resolution +
DELETE + INSERT). For a 950-line equipment module that's ~5,700 sequential
statements and 2.3 minutes on stage. Full trace analysis in #2783.
Decided semantics (#2749)¶
Two readings of "global percentage" were on the table; @agletiec confirmed Option B: setting the global percentage collapses the reference-year snapshot into one aggregated line per equipment type, not editable line-by-line. Users can still add equipment by hand via the input form; those manual lines are unaffected.
Why one line per type, not one line total¶
The equipment UI is three separate tables/fetches
(/modules/equipment/{scientific,it,other}), because data_entry_type_id
(scientific=10, it=11, other=12 in data_entry.py) is the equipment
category axis and drives the report's scientific/it/other split elsewhere
(treemap, CSV). Collapsing across types would lose that. "A single line" is
scoped per type: up to 3 aggregate rows total, not 1 and not 950.
No schema change¶
data_entry_type_id stays as-is. The existing data JSON keys
(source_data_entry_id, percentage_of_reference_year) get one more
combination, no new field, no new DataEntrySourceEnum member, no migration:
| row kind | source_data_entry_id |
percentage_of_reference_year |
|---|---|---|
| manual (unchanged) | null |
null |
| aggregate (new) | null |
set |
| legacy per-line snapshot (phased out below) | set | set |
The presence of an aggregate row is the persisted mode — no separate mode flag. This also fixes the "mode resets to per-line on reload" complaint noted in #2783, for free.
Reading the reference total: no fresh query¶
CarbonReportModule.stats (carbon_report.py:196-217) already carries
by_emission_type (kg per emission_type_id), recomputed by
recompute_stats_many on every write to that module. Equipment's
data_entry_type_id → emission_type_id mapping is 1:1 and enforced in
code (DATA_ENTRY_TO_EMISSION_TYPES in
app/modules/emissions/registry.py:140-168:
scientific→equipment__scientific(80100), it→equipment__it(80200),
other→equipment__other(80300)), and the equipment handler always emits
exactly one DataEntryEmission row per entry (equipment/handlers.py:99-138,
no scope fan-out). So ref_module.stats["by_emission_type"] already is
the per-type reference total — no GROUP BY needed at PATCH time.
Backend change¶
set_reference_percentage_all rewritten:
- Resolve the reference module:
get_calculator_report(unit_id, reference_year)→get_module(...)— existing pattern, used the same way insimulator_plan_service.py:748-757. - Read
ref_module.stats["by_emission_type"], pick out the three equipment leaves viaDATA_ENTRY_TO_EMISSION_TYPES. - For each type present: upsert one aggregate
DataEntryrow + exactly oneDataEntryEmissionrow (kg = ref_total * pct / 100). - Delete any leftover legacy per-line snapshot rows for this module (both keys set) — this is where a module last touched under the old implementation gets cleaned up, the first time this PATCH runs against it post-deploy (see "No backfill" below).
- Existing
recompute_stats_many+recompute_report_stats, unchanged.
Total cost per PATCH: one cheap module lookup, a JSON read, ≤3 upserts, a cleanup delete, existing stats recompute. Flat regardless of entry count.
Switching global → per-line: delete the aggregate row(s), re-run the
existing prefill_module_from_reference snapshot copy to restore per-line
rows at 0% — unchanged mechanism, matches what confirmEquipmentSwitch
already does today.
Prefill writes the aggregate lines itself, at 0%: global is the default mode of every new section, see 2749's follow-up.
Recompute prices aggregate lines (#2749 follow-up). An aggregate line
carries no equipment_class, so the factor path priced it at nothing:
resolve_computations returns [] without a factor, and the percentage
override only runs inside the computation loop. Every recompute (factor
recalc over its (type, year) slice, the plan recalc after a prefill)
deleted its emission row, a silent zero. prepare_create now prices an
aggregate line before the factor path: pct × the reference module's
stats.by_emission_type for its type, the number the PATCH shows. The PATCH
prices through prepare_create too, so there is one formula. Per-line rows
keep their path.
Aggregate lines stay out of the device tables (#2749). An aggregate line
stands for a whole type, not a device: it has no name/equipment_class
(listing it 500'd on EquipmentHandlerResponse), and the global-mode block
already shows its percentage, reference total and planned result.
IS_NOT_PERCENTAGE_AGGREGATE (models/data_entry.py) is the equipment
handler's default_where, so the page, page-id and count queries skip it, and
get_total_count_by_submodule applies it to the table header counts.
POST (manual add) is untouched: CarbonReportModuleWorkflow.create,
source=USER_MANUAL, never sets source_data_entry_id or
percentage_of_reference_year. It sits next to the aggregate row(s) and
adds on top, same as it sits next to per-line rows today.
No backfill¶
Reports currently sitting in global mode with per-line rows are not
migrated by a script. They keep working exactly as they do today — still
per-line rows, still reflecting the last-applied percentage — until someone
next calls reference-percentage on that module, at which point step 4
above collapses them to the aggregate shape. Nothing changes for a report
nobody touches again.
Frontend¶
PlannerYearSection.vue:
equipmentReferenceTotalKg: read from a module GET'sstats.by_emission_typeinstead of the current three?limit=1000snapshot fetches + client-side sum (fetchEquipmentSnapshotRows,equipmentReferenceSum).- Global-mode tables list hand-added equipment only; the aggregate lines
are hidden (see above).
ModuleTable's aggregate-row slider and thepercentageLockedprop (#1981) went with them: global mode no longer has per-line rows to lock. equipmentModeinitializes from row presence (an aggregate row found →global) instead of arefdefault that resets on reload.- Manual add-line form and the switch-confirmation dialog: unchanged.
Tests¶
- Backend: regression test asserting
set_reference_percentage_allissues a fixed, small query count regardless of reference-year entry count (same guard style as #2050's write-path tests). Aggregate upsert produces exactly one row per equipment type present, correctkg, and a repeated PATCH updates those same rows in place (no duplicates). Manual entries survive a PATCH untouched. Switching back to per-line restores snapshot rows at 0%. A module with legacy per-line rows from before this change collapses correctly on its first post-deploy PATCH. - Frontend: global-mode table renders aggregate rows; mode is derived from data and survives a simulated reload.