ARCHIVED: This plan was not delivered. Preserved for context.
Carbon Report Stats Aggregation Implementation¶
Overview¶
Added a stats column to the CarbonReport model that aggregates emission statistics from all child CarbonReportModule records, similar to the existing module-level stats.
Changes Made¶
1. Database Schema (backend/app/models/carbon_report.py)¶
- Added
statsfield toCarbonReportBasemodel - Type:
Optional[dict]with JSON column - Stores aggregated emission statistics from all child modules
- Format:
{scope1, scope2, scope3, total, by_emission_type, computed_at, entry_count} - Added
last_updatedfield to track when report data changed - Type:
Optional[int](epoch seconds) - Updated automatically when child modules change
2. API Schema (backend/app/schemas/carbon_report.py)¶
- Added
stats: Optional[dict]toCarbonReportReadschema - Added
stats: Optional[dict]toCarbonReportModuleReadschema - Both fields are included in API responses
3. Carbon Report Service (backend/app/services/carbon_report_service.py)¶
- Added
recompute_report_stats(carbon_report_id: int)method - Aggregates stats from all child modules
- Sums scope1, scope2, scope3, total across modules
- Merges
by_emission_typedicts (grouping by emission_type_id) - Sums
entry_countacross all modules - Updates
computed_attimestamp - Persists to
CarbonReport.statscolumn
4. Carbon Report Module Service (backend/app/services/carbon_report_module_service.py)¶
- Modified
recompute_stats()to trigger parent report stats recomputation - After updating module stats, calls
CarbonReportService.recompute_report_stats() - Ensures report-level stats stay synchronized with module stats
- Happens in the same transaction
5. CSV Data Ingestion (backend/app/services/data_ingestion/base_csv_provider.py)¶
- Updated
_recompute_module_stats()method - Now recomputes stats for both modules AND parent carbon reports
- For MODULE_PER_YEAR: recomputes stats for each affected carbon report
- For MODULE_UNIT_SPECIFIC: recomputes stats for the parent carbon report
- Ensures CSV ingestion updates both module and report stats
6. Database Migration¶
- Created Alembic migration:
01b45209983b_add_stats_and_last_updated_to_carbon_reports.py - Adds
statscolumn (JSON, nullable) tocarbon_reportstable - Adds
last_updatedcolumn (Integer, nullable) tocarbon_reportstable - Migration successfully applied to database
Stats Format¶
The stats JSON structure for CarbonReport:
{
"scope1": 1234.56,
"scope2": 789.01,
"scope3": 2345.67,
"total": 4369.24,
"by_emission_type": {
"1": 500.0,
"2": 734.56,
"3": 1000.0,
"...": "..."
},
"computed_at": "2026-03-19T15:19:29.330566+00:00",
"entry_count": 42
}
Benefits¶
- Performance: Pre-computed stats avoid on-the-fly aggregation queries
- Consistency: Report stats automatically update when modules change
- Frontend Optimization: Quick access to report-level totals without complex joins
- Data Integrity: Single source of truth for emission aggregations
Testing Recommendations¶
- Test module stats recomputation triggers report stats update
- Verify CSV ingestion updates both module and report stats
- Check API response includes stats in
CarbonReportRead - Validate stats aggregation:
- Scope totals match sum of module scopes
entry_countmatches sum of module entry_countsby_emission_typeproperly merges across modules- Test with empty modules (no data entries)
- Test with partial data (some modules have data, others don't)
Migration Path¶
- Existing reports will have
stats=nulluntil first recomputation - No backward compatibility maintained for old format
- To backfill historical data:
- Run a script to iterate over all carbon reports
- Call
CarbonReportService.recompute_report_stats(report_id)for each - This will populate stats for all existing reports
Related Files¶
backend/app/models/carbon_report.pybackend/app/schemas/carbon_report.pybackend/app/services/carbon_report_service.pybackend/app/services/carbon_report_module_service.pybackend/app/services/data_ingestion/base_csv_provider.pybackend/alembic/versions/2026_03_19_1519-01b45209983b_add_stats_and_last_updated_to_carbon_.py