ARCHIVED: This plan was not delivered. Preserved for context.
Plan: 707-ext — sync_module_factors for research_facilities common (DE type 70)¶
TL;DR¶
Add ResearchFacilitiesCommonFactorUpdateProvider that recomputes kg_co2eq_sum
(single total) for data_entry_type=70 (research_facilities) factors, by summing ALL
DataEntryEmission totals from the corresponding facility's CarbonReport.
Mirrors the animal provider pattern but with a single-field output. Also updates the
ProviderFactory to use 5-tuple keys for computed providers so both 70 and 71
can coexist.
Phase 1 — Factory key update (enables both providers to coexist)¶
Step 1 — Update provider_factory.py to use 5-tuple for computed providers¶
- File:
backend/app/services/data_ingestion/provider_factory.py - Remove the existing 4-tuple entry:
(research_facilities, computed, FACTORS, MODULE_PER_YEAR) - Add new import:
ResearchFacilitiesCommonFactorUpdateProvider - Add two 5-tuple entries to a dedicated
COMPUTED_FACTOR_PROVIDERSdict (or extend PROVIDERS): (research_facilities, mice_and_fish_animal_facilities, computed, FACTORS, MODULE_PER_YEAR)→ResearchFacilitiesAnimalFactorUpdateProvider(research_facilities, research_facilities, computed, FACTORS, MODULE_PER_YEAR)→ResearchFacilitiesCommonFactorUpdateProvider- Update
get_provider_by_keys: whendata_entry_type_idis provided, try 5-tuple(module_type, data_entry_type, ingestion_method, target_type, entity_type)first; fall back to 4-tuple if not found (for CSV/API providers that don't have data_entry_type in key). - Update
PROVIDERS_BY_CLASS_NAMEto include common provider.
Phase 2 — New concrete provider¶
Step 2 — Add ResearchFacilitiesCommonFactorUpdateProvider (new file)¶
- File:
backend/app/services/data_ingestion/computed_providers/research_facilities_common.py - Extends
BaseFactorUpdateProvider compute_factor_values(factor, year, session):- Read
researchfacility_idfromfactor.classification.get("researchfacility_id") - Return
None+ warning log if absent (same pattern as animal provider) UnitRepository(session).get_by_institutional_id(researchfacility_id)→ Unit (raise ValueError if None)CarbonReportRepository(session).get_by_unit_and_year(unit.id, year)→ CarbonReport (raise ValueError if None)DataEntryEmissionRepository(session).get_emission_breakdown(carbon_report.id)→ list of (module_type_id, emission_type_id, kg)- Sum all
kgvalues →total: float - Return
{"kg_co2eq_sum": total} - Only
kg_co2eq_sumis overwritten;use_unitandtotal_useare left untouched (handled by base class merge logic).
Phase 3 — Tests¶
Step 3 — Unit tests for ResearchFacilitiesCommonFactorUpdateProvider (new file)¶
- File:
backend/tests/unit/services/data_ingestion/test_research_facilities_common_factor_update.py - Mirror structure of
test_research_facilities_animal_factor_update.py - Cases:
- Happy path: multiple emission rows from different modules →
kg_co2eq_sum= sum of all - Unit not found →
ValueErrorraised - CarbonReport not found →
ValueErrorraised - Zero emissions → returns
{"kg_co2eq_sum": 0.0} - Missing
researchfacility_idin classification → returnsNone(skipped)
Step 4 — Extend test_provider_factory.py (parallel with Step 3)¶
- File:
backend/tests/unit/services/data_ingestion/test_provider_factory.py - Add assertion: 5-tuple
(research_facilities, research_facilities, computed, FACTORS, MODULE_PER_YEAR)→ResearchFacilitiesCommonFactorUpdateProvider - Update existing assertion for animal provider to use the new 5-tuple key
Relevant files¶
backend/app/services/data_ingestion/provider_factory.py— update registration (5-tuple) + lookupbackend/app/services/data_ingestion/computed_providers/research_facilities_common.py— NEWbackend/app/services/data_ingestion/computed_providers/__init__.py— no change needed (comment-only)backend/app/services/data_ingestion/computed_providers/research_facilities_animal.py— reference onlybackend/app/services/data_ingestion/factor_update_provider.py— reference/no changebackend/app/api/v1/data_sync.py— no change needed (already passes data_entry_type_id in config)backend/app/repositories/data_entry_emission_repo.py— reuse get_emission_breakdown()backend/app/repositories/unit_repo.py— reuse get_by_institutional_id()backend/app/repositories/carbon_report_repo.py— reuse get_by_unit_and_year()backend/tests/unit/services/data_ingestion/test_research_facilities_common_factor_update.py— NEWbackend/tests/unit/services/data_ingestion/test_provider_factory.py— extend
Verification¶
make lint && make typecheckpass on all changed files.- New test file: all 5 cases pass.
- Updated provider factory test: new and updated registrations asserted.
- Manual:
POST /data-sync/factors/research_facilities/research_facilitiesbody{"ingestion_method": 3, "target_type": 1, "year": 2025}→ returns job_id; SSE polling → FINISHED/SUCCESS;Factor.values.kg_co2eq_sumupdated per facility. - Regression:
POST /data-sync/factors/research_facilities/mice_and_fish_animal_facilitiesstill resolves toResearchFacilitiesAnimalFactorUpdateProvider(5-tuple lookup).
Decisions¶
- 5-tuple factory key for computed providers; CSV/API providers remain 4-tuple (fallback).
kg_co2eq_sum= ALL emission types across entire CarbonReport (viaget_emission_breakdown()).use_unitandtotal_useleft untouched — billing/usage denominator comes from CSV ingestion.- No Alembic migration needed (no schema change).
- No changes to
data_sync.pyorfactor_update_provider.py.