Skip to content

Implementation plans

Plans live in docs/src/implementation-plans/. 81 total.

Delivered (60)

Title Issue Last updated Summary
Train CSV station disambiguation — required country_code #1183 2026-05-29 Sources the train location seed from the trainline-eu stations.csv (ISO-2 country per station) and makes country_code REQUIRED in the train CSV trip resolver — no more CH default. Retires the CH-backfill script.
Move 'Incomplete' tag computation to backend #1215 2026-05-22 Backend year-configuration response emits explicit incomplete + reasons per submodule/module. Frontend deletes its client-side derivation. Only factor + reference uploads are mandatory.
BackOffice upload card — API success status + FACTORS-first order #1216 2026-05-28 Data card now turns green when the API ingestion path succeeds (not only CSV). FACTORS-first order across Equipments / Purchases is pinned by a regression test.
1219 Stuck ingestion/recalc jobs + premature pipeline 'success' #1219 2026-05-19 Stage incident: a zombie emission_recalc row tripped uq_emission_recalc_active, the IntegrityError poisoned the runner's job_session, and the job never reached FINISHED — self-propagating the stall. Fixes runner session resilience, adds dedup to the csv/api fan-out, and makes pipeline completion server-authoritative with a 3-phase UI.
Lighthouse: single config + nightly cron #1332 2026-06-08 Merge the duplicate Lighthouse configs into one source of truth and move CI from per-PR to a nightly cron.
1411 — Hide module-status validate button from standard users + consolidate frontend permissions #1411
Implementation Plan: Auto-generate TypeScript types from FastAPI OpenAPI #217 2026-05-28 Stand up a POC that pipes the FastAPI OpenAPI 3.1 schema into a generated openapi.d.ts consumed by the existing ky HTTP client. Recommends openapi-typescript over heavier SDK generators, ships the generator script, makefile target, committed snapshot, and a single migrated interface (User) as proof.
CSV Upload Feature Implementation Summary #220 2026-04-02 This document summarizes the implementation of the CSV upload feature verification and test suite for issue #220 (DB Upload CSV - Improve Data Modifications).
PRD — CSV Upload Feature #220 2026-04-01 The CSV Upload feature allows users to load machine-generated data (emission factors, synthetic data, and other module inputs) into the database from any module table. The backend implementation, UI, permissions,...
Faker seed-data: ~800 data_entry rows, ERD-aligned #222 2026-05-28 Right-size the random generator to ~800 rows, fix payload drift against current Pydantic schemas, wire it into the Makefile.
Implementation Plan: Refactor Pagination & Lazy Loading for Equipment Module #228 2025-12-16 Implement lazy loading with per-submodule pagination and sorting. Load only summary totals on initial page load, then fetch individual submodule data when users expand collapsible sections.
Comprehensive Code Review: Audit Trail Implementation (feat-240) #240 2026-02-16 The implementation has a solid foundation with good design patterns, but has several critical issues that should be addressed before merging to production.
detail mapping #240 2026-02-19 We've implemented pattern 7 as requested, which involves adding a syncstatus column to track the synchronization state of audit records with Elasticsearch. This approach provides reliability by tracking the sync...
Implementation Progress Report #240 2026-02-19 The audit trail infrastructure has been partially implemented with core versioning, database models, and API integration. The foundation is solid but needs external storage integration and UI exposure.
Service Manager — Audit Logs UI & API #240 2026-02-16 This plan covers AUDIT logs only. The system/logs route will expose the auditdocuments table to service managers.
243-backoffice-data-management-factors #243 2026-03-11 (see plan content)
Source Tracking Implementation Summary #243 2026-03-24 Both columns are indexed for query performance.
CSV Data Entry Type Resolution Refactoring #243 2026-03-18 Refactored CSV data ingestion to use module-specific category columns instead of the generic dataentrytype column. This makes the CSV ingestion more explicit and type-safe per module.
Implementation Complete: Delete-Before-Insert Pattern for Factors #243 2026-03-25 Successfully implemented the delete-before-insert pattern for CSV factor uploads to ensure idempotency and prevent duplicate factors.
Plan: Job History Panel + Per-Row Sync Progress #243 2026-03-09 4. Add clearJobHistory() action: jobHistory.value = [].
Filters #246 2026-03-09 (see plan content)
Optimization #246 2026-02-26 Now I have a much better understanding of the existing seed data infrastructure! Looking at the existing seed files in backend/app/seed/, I can see there's already a well-structured approach for generating test data...
Results Page — Single Summary Endpoint #252 2026-02-20 Currently the page hardcodes most values ("37'250", "8.2", "-11.3%"). The existing GET /unit/{unitid}/{year}/totals endpoint is incomplete — it only sums equipment and doesn't return FTE or per-module breakdown. We...
Carbon Report Module — Stats Column Population #252 2026-03-03 The carbonreportmodules table has a stats JSON column that was never populated. Every time the frontend needs emission breakdowns or scope totals for a module, it triggers live aggregation queries across...
Chart Data Endpoints — Implementation Plan #252 2026-02-18 The results page charts (ModuleCarbonFootprintChart and CarbonFootPrintPerPersonChart) previously used hardcoded mock data. A backend endpoint now returns real aggregated emission data so both charts display actual...
Emission Computation — Data Reference #252 2026-03-02 Some formula inputs are not in dataentry.data and must be derived before.
Emission Type Migration — Implementation Plan #252 2026-03-02 We are replacing the old EmissionTypeEnum (int-based, flat) with a new.
Test Plan: 252 Endpoints #252 2026-02-20 Business logic calculations are inline in the getresultssummary and getvalidatedtotals routes. They cannot be unit tested. They must first be extracted into pure functions.
Validated Modules Totals & Results Summary #252 2026-02-20 The existing aggregation endpoints only handle equipment and don't filter by validation status. Headcount has no DataEntryEmission records — FTE is stored in DataEntry.data["fte"] and must be queried separately via...
Lighthouse CI: Frontend Route Coverage #264 2026-04-02 Lighthouse CI previously only audited the login page because all.
310-d Architecture follow-ups (#1062 / #1063 / #1064) #310-d 2026-05-29 Three architectural follow-ups staged from the 310-D landing — passive-monitoring backstop, unified pipeline state store, and DedupConfig generalisation. Each is tracked on a dedicated GitHub issue and lands in the post-merge fix batch.
310-d Frontend — Stale-Stats UX + Pipeline SSE #310-d 2026-05-07 Render the in-flight bulk-pipeline state on carbon-report module cards, subscribing to GET /sync/pipelines/{id}/stream until the chain finishes.
310-d — Bulk Path Pure Async (Path 2 only) #310-d 2026-05-07 Make the bulk pipeline (Path 2) pure async to match its latency profile; Path 1 untouched.
Plan 310-D Follow-up — Strategy B Rematch + Per-Module ITs 2026-05-06 Rematch the FK-link modules (travel, headcount, building embodied energy) plus regression coverage for JSON-link modules including the 1:N building-rooms fan-out.
310-f Ingestion per-row efficiency #310-f 2026-06-16 The CSV ingest parse/validate loop dominated wall-clock (~34s for 9555 purchase rows on dev). The first cut guessed line-level suspects and they were wrong. A row-loop profiler was added, and the trace pinned 33.4s of 34.5s on resolve — a per-row linear scan of ~23k factor keys in lookup_data_entry_type_by_kind (O(rows × factors)). Fixed by memoising the kind→type inference per (kind, subkind); re-measured 34.5s → 1.7s (20×). A one-time index is a documented follow-up, not needed at current data shape.
/me Performance Optimization: Decoupling Role Sync and Frontend Refresh Strategy #334 2026-05-06 Drop /me latency from ~1s to ~8ms by moving role sync trigger from /me to /refresh.
Frontend Error Monitoring via Self-Hosted GlitchTip #344 2026-05-19 Wire @sentry/vue to EPFL's self-hosted GlitchTip with runtime DSN injection under a read-only root filesystem.
Implementation Plan: Unify CSV Upload Ingestion #370 2026-02-04 Unify CSV upload + API sync under one job-creation pipeline that stores file paths in job meta, dispatches providers via the factory, and exposes job status via SSE.
Plan: Travel — Link Traveler Selection to Headcount (Issue #372) #372 2026-03-10 Travel entries currently store travelername as a free-text string and travelerid.
Feature Overview (Why?) #384 2026-04-02 In the calculator a carbon footprint appears in red if crossing a certain threshold, as defined in Data mgt.
Security: authentication & integration hardening #458 2026-05-27 Pin the trust boundaries of the OAuth/JWT auth path with regression tests, fix eleven of twelve findings (F6 logout-denylist deferred to a follow-up — needs server-side store), centralize the JWT→user resolution path, and document the contracts.
Plan: Headcount user_institutional_id — Uniqueness Validation + Institution Label #518 2026-03-10 The headcount module stores a userinstitutionalid (SCIPER at EPFL) per member entry.
Implementation Plan: Headcount Data Validation V2 (Issue #518) #518 2026-03-18 This plan covers five distinct changes to the Headcount module, derived from the updated.
Refactoring: Data Management Upload Components #780 2026-05-06 Deduplicate ~200 lines of upload card UI between ModuleConfig.vue and SubmoduleConfig.vue.
Remove emit_per_factor — always emit one emission per factor #840 2026-05-06 Eliminate the emit_per_factor branch in prepare_create(); always emit one DataEntryEmission per factor.
Root-Level Rollup DataEntryEmission Rows #840 2026-05-06 Add root-level rollup rows on DataEntryEmission to support aggregate views.
Replace distance_km / weight_kg with polymorphic additional_value on data_entry_emission #840 2026-05-06 Replace the two typed columns with a polymorphic additional_value field on data_entry_emission.
Issue 856 — Simulation Explore Flow 2026-05-06
Issue 857 — Back-office: Open year for users (functional, end-to-end) #857 2026-05-12 Consolidate the back-office data-management page year-lifecycle: single-click create+sync, real pipeline observability, hide unstarted years from end-users, and wire the 'Open year for users' button.
Refactor Seed Factors to Use Ingestion Machinery #859 2026-05-06 Replace duplicated factor-seed parsing with a LocalFactorCSVProvider that reuses the ingestion pipeline.
Refactor seed_generic_data_entries to Use Ingestion Machinery #859 2026-05-06 Replace duplicated data-entries seed logic with LocalDataEntryCSVProvider on process_csv_in_batches.
#860 — Dev docs refresh & MkDocs restructure #860 2026-05-06 20-PR refresh of developer documentation across 4 work-streams (A/B/C/F) consolidating, condensing, and structurally reorganizing docs/, with new frontmatter conventions, auto-indexing, ADR backfill, and mermaid-diagrammed architecture.
Backoffice affiliation scope by unit subtree (cf-anchored, intersection clamp) #862 2026-06-03 Resolve backoffice.reporting affiliation scope from a single ACCRED unit cf (any level) into a descendant unit-id set via path_institutional_code, and clamp every backoffice query by intersecting the scope set with the endpoint filter set. Replaces path_name substring matching and list-narrowing.
Workspace shows only level-4 units #930 2026-05-22 Backend /users/units endpoint hardcodes level == 4 so workspace only ever sees actual lab-level units, not EPFL or affiliation-level ancestors.
Tighten unit-scoped permission gates #977 2026-05-22 Close three pre-existing permission soft-spots: 2 carbon_report endpoints + 3 unit_results endpoints get unit gating; remove the coarse modules.* fallback in files.py and data_sync.py.
Emission Pipeline Flow 2026-05-06 Reference for the emission compute pipeline AND for how factor changes propagate to existing emissions (Plan 310-D rematch).
Stop persisting computed fields to DataEntry.data 2026-05-04 Stop persisting computed fields back into DataEntry.data; recompute on read instead. Single-PR refactor delivered during sprint 10; no tracking issue.
Authentication Hardening Implementation Summary 2026-03-20 This document summarizes the implementation of the "Harden Authentication Identity Resolution" PRD to fix a security incident where test users could collide with real user IDs.
Fix: sub_class field disabled state in ModuleForm 2026-03-12 When an equipmentclass (e.g. "3D printer") has an empty subclass array from /api/v1/factors/{id}/class-subclass-map, the subclass field should be hidden/disabled. This worked in ModuleTable (via...
Frontend error reporting — minimal GlitchTip client (drop @sentry/vue) 2026-06-18 Replaces the @sentry/vue SDK with a ~2 KB dependency-free GlitchTip-compatible reporter (src/utils/glitchtip.ts). Captures Vue component, router, global, DOM-event, unhandled-rejection and ky 5xx errors. Fixes the two protocol bugs (missing envelope item header, missing DSN public key) that stopped events ingesting, and documents that the DSN must live in .env.local.

In progress (15)

Title Issue Last updated Summary
310-a — Pod Safety + Atomic Claim #310-a 2026-05-06 Pod-safe atomic claim mechanics for FastAPI background tasks under multi-pod deployment.
310-b — Factor Pipeline + Unit Sync Tracking #310-b 2026-05-06 Bulk-path fixes: factor ingest semantics, recalc enqueue, and unit-sync tracking on Path 2.
310 — Backoffice recalculation upon factor updates #310 2026-05-06 Recalculate downstream emission data when operators update factors via the backoffice.
310-c — DAG + Handler Registry + Observability #310-c 2026-05-06 DAG-driven handler registry with observability columns replacing ad-hoc safety and auto-trigger code.
310 — Background Pipeline Architecture: Overview #310 2026-05-06 Umbrella plan: redesign FastAPI background workflows for multi-pod safety, retry, and observability.
310 — Post-merge fix batch (review findings + #1062/#1063/#1064) #310 2026-05-07 Parallel worker batch landing the 8 High/Critical findings from docs/code-review/310-overall-review.md plus completing the three architecture follow-ups, all targeting feat/310/dev integration branch.
Plan: Manual emission recalculation endpoint with "needs recalculation" status #310 2026-04-10 no new DB table — derived from existing DataIngestionJob rows.
310 — Plan Review #310 2026-05-06 Code-grounded verification of the 310-* plan series against the actual repository state.
Backoffice ACCRED affiliation scoping #459 2026-05-26 Scope backoffice.* permissions by ACCRED-provided affiliation so each backoffice manager sees only their sub-perimeter.
PRD 2 — Results CSV (results.csv) #504 2026-03-31 👉 If byemissiontype is missing → STOP (data issue).
PRD 3 — Detailed CSVs (*_data.csv) #589 2026-03-31 Provide raw, traceable, and schema-specific data for each data entry type.
Update backoffice permissions (page-driven model) #862 2026-06-02 Replace the system.users super-admin gate with one permission per backoffice page (configuration, pipeline_operations, logs, ui_texts), scope only backoffice.reporting by affiliation, and move the #459 affiliation anchor to backoffice.reporting.
Fix ModuleUnitSpecificCSVProvider KeyError + Add CSV Fixture Tests #898 2026-05-06 Resolve KeyError on factor_id_to_factor in ModuleUnitSpecificCSVProvider; add fixture-based tests.
Docs: single source of truth for auth flow 2026-05-28 Strip duplicated and stale auth narratives from backend and frontend overview docs. The architecture/04-auth-flow.md doc is the canonical source; everything else cross-links.
Explicit RoleScope: own / unit / global #862 2026-06-02 Replace the implicit role→scope coupling with a discriminated-union Scope (global/unit/own/affiliation) so the own↔unit boundary is explicit in both the schema and the permission key, fixing unit-level capability leaking to standard users.

Uncategorized (6)

Title Issue Last updated Summary
1234 — Pipeline operations console
1236 — First-class pipelines table
CSV Provider Consolidation Implementation Plan
310-e Async-loop hygiene & pipeline worker hardening #310-e 2026-06-15 Improvement backlog from the 2026-06-15 dev incident: a CSV ingestion job ran in-process on the web event loop, starved /healthz past the 2s liveness timeout, and the pod was restarted mid-job. Captures the shipped mitigation plus the larger refactors it points to — a dedicated pipeline worker, event-loop offloading, and poller/durability sharpening.
PDF Export for Backoffice Reporting — Implementation Plan
Pipeline-debug — living TODO