Backoffice: Copy factors from previous year¶
Removed (2026-07-13). Every copy-from-previous-year affordance was removed. Frontend: the bulk "Copy all factors from {year}" button, the per-submodule "Copy from {year}" buttons,
CopyFactorsDialog.vue, and the whole copy-from-previous-year section of the data-entry dialog (factors, data entries, and reduction objectives alike). Backend:FactorCopyProvider, itsProviderFactoryregistration, thesource_job_idresolution in the dispatch endpoint (_resolve_source_job_to_file_path), and their tests were deleted; only thecopy_previous_year = 4enum member remains so persisted historical jobs still decode.
Problem¶
Factors (factors table) are year-scoped (Factor.year, indexed, unique per
(data_entry_type_id, year, emission_type_id, classification)). The only way
to populate factors for a newly-opened year today is a CSV/API/computed sync
per (module_type_id, data_entry_type_id) via
POST /v1/sync/factors/{module_type_id}/{data_entry_type_id}
(backend/app/api/v1/data_sync.py:998), routed through
ProviderFactory.PROVIDERS/COMPUTED_FACTOR_PROVIDERS
(backend/app/services/data_ingestion/provider_factory.py) keyed on
(IngestionMethod, TargetType) or
(module, data_entry_type, IngestionMethod, TargetType, EntityType).
Most factor tables (emission factor CSVs, unit costs, classifications) do not
change year over year — only a handful of rows get revised. Operators
currently must re-upload the full CSV for every module/data-entry-type
combination for every new year, even when 95%+ of rows are identical to the
prior year. year_configuration creation
(create_year_configuration, backend/app/api/v1/year_configuration.py:608)
already auto-enqueues a unit_sync job but does nothing for factors — every
submodule starts "incomplete" (_enrich_config_with_incomplete_flags) until
manually synced.
getPreviousYearSuccessfulJobs (frontend/src/stores/backofficeDataManagement.ts:572)
already reads the prior year's finished jobs for prerequisite checks, so the
frontend already has the previous-year job list in scope where a "copy"
action would be offered — no new lookup needed there.
No copy/duplicate/clone mechanism exists today (grep for
copy|duplicate|clone across factor_service.py, factor_repo.py,
factors.py, backoffice.py — the only "copy" hit is
FactorRepository._upsert_via_copy, a Postgres COPY-based upsert
implementation detail, unrelated).
Design¶
Reuse the existing ingestion-job pipeline rather than inventing a parallel
synchronous copy path — this keeps SSE progress, latest_factor_job
enrichment, and the pipeline-observability UI (issue #857) working unchanged
for the new action.
Backend
- Add
IngestionMethod.copy_previous_year(int enum,backend/app/models/data_ingestion.py) alongsideapi/csv/manual/computed. - New provider
FactorCopyProvider(backend/app/services/data_ingestion/factor_copy_provider.py, mirrorsfactor_update_provider.py's shape): givenmodule_type_id,data_entry_type_id, targetyear, readssource_yearfromsyncRequest.filters(defaultyear - 1), selects allFactorrows for(data_entry_type_id, source_year)scoped to the module's emission type(s) viaFactorService.list_by_data_entry_type, clones each row withyear=target_yearandid=None, and writes them via the existingFactorRepository.upsert_factors(identity key already excludesid, so re-running is idempotent and safe to retry). - Register
(IngestionMethod.copy_previous_year, TargetType.FACTORS)inProviderFactory.PROVIDERS— no new endpoint needed, the existingPOST /v1/sync/factors/{module_type_id}/{data_entry_type_id}already acceptsingestion_method+filtersinSyncRequestand dispatches throughProviderFactory.create_provider. - Bulk "copy all" for a year: a thin loop endpoint
POST /v1/sync/factors/copy-previous-year/{year}(or a frontend-side loop over configured(module_type_id, data_entry_type_id)pairs calling the per-module endpoint — prefer the frontend loop, it reusesinitiateSync's existing per-submodule pipeline tracking instead of inventing a second bulk-job abstraction). Skip pairs with zero source-year rows (nothing to copy) rather than erroring. - No migration needed —
Factor.yearand its indexes already exist.
Frontend
backofficeDataManagement.ts: addinitiateFactorCopyFromPreviousYear(moduleTypeId, dataEntryTypeId, year), mirrorsinitiateComputedFactorSync(:602) but postsingestion_method: IngestionMethod.COPY_PREVIOUS_YEAR. Gate visibility per-submodule ongetPreviousYearSuccessfulJobs(year - 1, moduleTypeId, TargetType.FACTORS)returning non-empty (nothing to copy from an unsynced prior year).DataManagementPage.vue: add a "Copy from {year-1}" action next to each submodule's existing CSV-upload control, plus one page-level "Copy all factors from {year-1}" bulk button that loops the per-submodule call. Existing job-card/pipeline UI (issue #857) covers progress display with no changes.
Individual-row copy (copying a single factor entry rather than a whole module/data-entry-type table) is out of scope for v1 — the issue's primary ask ("open 2024 ... to have a previous year") is the bulk case; per-row copy is deferred to a follow-up if operators ask for it after using bulk copy.
Steps¶
- Add
IngestionMethod.copy_previous_yeartobackend/app/models/data_ingestion.py - Implement
FactorCopyProvider(source-year lookup, clone-with-new-year,upsert_factors) - Register provider in
ProviderFactory.PROVIDERSfor(copy_previous_year, TargetType.FACTORS) - Unit test:
FactorCopyProvidercopies rows, is idempotent on retry, no-ops when source year is empty - Unit test:
sync_module_factorsendpoint acceptsingestion_method=copy_previous_yearwithfilters.source_yearoverride - Frontend:
initiateFactorCopyFromPreviousYearinbackofficeDataManagement.ts - Frontend: per-submodule "Copy from {year-1}" button (in
UploadCardFactors.vue/SubmoduleItem.vue, consumed fromDataManagementPage.vue's module tree), gated on prior-year success viagetPreviousYearSuccessfulJobs - Frontend: page-level "Copy all factors from {year-1}" bulk action in
DataManagementPage.vuelooping per-submodule calls, skipping pairs with no source-year factors - Manual/logical verification: reasoned through the end-to-end flow (see "Verification" note below) — no live app available in this environment to click through.
Implementation notes (2026-07-09)¶
- The per-submodule "Copy from {year-1}" button lives on the existing
UploadCardFactorscard (next to "Compute Missing Factors"), wired throughSubmoduleItem.vue→useSubmoduleConfig.ts, not directly inDataManagementPage.vue(that file only hosts the page-level bulk button) — the per-submodule UI composition already lived one level down from the page component before this change (mirrors how "Compute Missing Factors" is wired), so the new button follows the same layering rather than moving existing structure. - A confirmation dialog (
CopyFactorsDialog.vue, modeled on the existingComputedFactorDialog.vue) gates both the per-submodule and bulk actions, since copying overwrites already-entered current-year factor rows for the affected data-entry type(s). - Pre-existing, unrelated mechanism found during implementation: a
file-replay "copy from previous year" flow already exists for
individual data-entry-type dialogs (
useDataEntryDialog.ts+config.source_job_id+_resolve_source_job_to_file_pathindata_sync.py), which re-uploads the prior year's processed CSV file through the normal CSV provider. It is orthogonal to this plan's DB-levelFactorCopyProvider: it depends on the source year's processed file still existing in file storage and only replays CSV-sourced factors, whereasFactorCopyProviderclones theFactorrows directly from the database and also covers factors that were never CSV-uploaded (e.g. seeded or computed). Both mechanisms now coexist; no changes were made to the file-replay path.