# Step-13 - Verified synthetic calibration comparison and demonstration ## STEP IDENTIFIER Step-13. Step-12 with Step-12A accepted 2026-10-31. This is the only active implementation assignment; Step-14 remains blocked. Read [STATE.md](STATE.md), [ROADMAP.md](ROADMAP.md), the [accepted execution report](Step-12A-Execution-Report.md) and [WORKER-QUALITY-GATE.md](WORKER-QUALITY-GATE.md). Establish the executable matrix before coding, self-review after first green, save `prompts/Step-13-Report.md` in one plain fenced report and STOP. ## STEP TITLE Compare equivalent real synthetic runs using original configuration and an explicit NEW configuration derived from independently verified rehearsal values. ## OBJECTIVE One user outcome: an auditable before/after synthetic calibration comparison, with real captures/replay and clear separation of proposal, programmed-image verification and observed model behavior. One primary new contract: trusted verified-image admission and exact immutable projection into an equivalent two-run comparison, never mutation of existing configuration/run or implicit programming authority. No hardware, general executor, new protocol/archive schema or runtime manager. Use existing local AI/manual authoring/rehearsal/diagnostic/recording/replay workflows; do not implement parallel versions of those services. ## ARCHITECTURAL CONTEXT Keep all nine projects, one Client and independent Components. Core may own the narrow pure supported-values/configuration projection; Application owns trusted source admission, comparison/metrics/capture/save workflow; UI owns explicit draft/evaluation/report controls and lifetime. Use Diagnostics ports and explicit Simulation provider, not an Application/UI reference to Simulation implementations or privileged access to DynamicsEngine. Rehearsal Apply continues to change only the Simulation-owned supported logical image. Step-13 introduces a separately named explicit projection to a NEW immutable configuration. Original configuration, authoring binding/history, target approvals and existing runs are never overwritten or silently rebased. Replay remains archive playback, not rerunning physics. ## CURRENT IMPLEMENTATION STATE - Accepted baseline: 179 unit + 417 integration = 596 passes, zero failed/ skipped; clean nine-project Release/native evidence from the worker. - `/runbooks` validates original local model content, exact immutable preview/ diff/current-Stale source, and grants no Save/approval/target/model authority. - Authoring supports exact source/head scalar/fixed-axis table proposals. Rehearsal supports actual two-read backup, separately approved single-use CAS, complete readback/reconciliation/Restore and bounded audit. - Rehearsal snapshots contain verified backup/current image/audit results, but public records alone are not unforgeable verified-source admission. - Dynamics config has immutable WarmIdleTargetRpm/EngineTorqueCurve, initial conditions and full five-phase scenario/fingerprint. Existing diagnostics/v2 exposes actual engine/speed/fuel/distance and transmission shift-count signals. - Archive v1 capture/store/save/readback and exact replay are accepted. - No comparison, trusted verified-image projection or complete connected before/after demonstration exists yet. - Local-only Ollama/model is already qualified. No new pull, startup/restart/ config change or approval form for that existing setup is assigned. Use [authoring](../docs/calibration-authoring.md), [rehearsal](../docs/calibration-rehearsal.md), [runbooks](../docs/runbooks-v1.md) and [local generation](../docs/local-runbook-generation.md). ## REQUIRED IMPLEMENTATION ### Frozen contract matrix | ID | Precondition/action | Required result | |---|---|---| | C-01 | Explicit current preview-to-draft action versus invalid/stale/superseded/different source | Exact copied canonical candidate enters the existing pinned authoring draft only; no Save, target call, rebase or automatic action on render | | C-02 | Current owning rehearsal with original verified backup and verified Apply/full actual readback | Trusted copied evidence admits comparison: exact target instance/revision/action/operation/backup/workspace/head/config/definition and complete values; caller-constructed success records are insufficient | | C-03 | Exact supported scalar/table mapping into original config | NEW immutable config changes only WarmIdleTargetRpm and torque cells; axes/all other parameters/initial state/scenario/profile/module/model refs unchanged; explicit lineage and fingerprint | | C-04 | Fresh baseline/derived runs and fixed bounded acquisition | Same explicit real Simulation/v2 provider/scenario/initial state, separate run IDs and actual canonical samples/captures; no replay/oracle/canned telemetry or diagnostics-triggered physics | | C-05 | Decisive warm baseline, real model proposal, explicit Save/backup/approval/Apply/readback | 650->1250 rpm and 2000-rpm torque 360->350 N*m; derived mapping exact, actual last ten warm-idle samples within 1 rpm of each target, stationary; all unrelated config/image values unchanged | | C-06 | Complete equivalent run comparison/report and existing save/replay | Actual idle/speed/fuel/distance/time/shift metrics, exact configuration/proof/run/capture lineage, synthetic limits; explicit bounded saves/readback and real archive replay preserve canonical evidence | | C-07 | Busy, cancellation, close/unmount and source/head/proof supersession during pending work | One admitted owner, responsive Hybrid events, drain outside required gates, no late healthy result/rebase; previous success inspectable with truthful stale/partial status | | C-08 | Missing/invalid proof/config/signals, failure/limits or partially successful pair save | Typed visible rejection before side effects where applicable; incomplete/faulted run is not a successful comparison; exact persisted/unsaved IDs and no overwrite/compensating deletion | | C-09 | Independent verification and supported fault regression matrix | Test-only independent whole-config/image/mapping/numerical assertions and actual producer/consumer faults; no UI/AI access to a private oracle or invented manufacturer checksum/firmware behavior | | C-10 | Real native complete demonstration and preserved behavior | All 596 cases/new focused coverage/nine-project Release build pass, actual local-model-to-verified-image-to-run/capture/replay path and no physical/unassigned mutation | ### A. Explicit preview handoff and trusted verified source Add an explicit current-preview-to-existing-draft action. Application validates actual workspace/head/applicability and expected preview identity at admission; copy the complete canonical candidate. The existing draft pins display units; no string/rounded-display reconstruction. A user must still explicitly Save the Proposed version through existing validation/head checks. No Save/approval/execution on AI validation, navigation, notification or render. Rejected handoff leaves prior draft/head/source/preview unchanged and shows error. Use existing rehearsal commands for backup/preparation/approval/CAS/readback. At an explicit evaluation admission, the actual owning rehearsal must be idle/ Ready with a verified original backup and selected current verified Apply result (including genuinely verified post-reconciliation Apply). Match exact current identity/revision/image, candidate, saved workspace/head/ applicability and original configuration. Reject RecoveryRequired, missing/ divergent/unavailable/stale/closed evidence or forged caller result flags. Use a narrow owning-service admission/helper with copied immutable evidence, not a public constructor accepting arbitrary records and declaring them verified. Bind original backup values to original supported config values and selected saved candidate to actual readback in full, changed AND unchanged values. Preserve target/programming semantics. Any required readonly admission addition does not open/approve/write a target or bypass its owner/generation guards. Captured evidence is historical proof at a stated time, not perpetual target state; supersession must be detected/labeled before current-result publication. ### B. Exact NEW configuration and two equivalent real runs Implement one pure supported projection using existing constructor/validation/ fingerprint logic. Map only: - `engine.warm-idle-target` rpm -> Model.WarmIdleTargetRpm. - Fixed `engine.torque-curve` rpm/N*m axes/cells -> Model.EngineTorqueCurve. Validate exact definition/units/axis/bounds first. Copy every other model field, initial condition, five-phase scenario and identity/reference unchanged. Register/select the NEW config through existing validation/catalogue logic, reusing/extracting helpers rather than JSON string replacement or duplicating an importer. Do not modify either the original config or an existing run. Show original/derived fingerprints and explicit source/proof/value lineage. A no-change supported mapping is a truthful no-change outcome, not fake benefit. One comparison admits immutable original/derived pair and runs two fresh owned diagnostic dynamics sessions sequentially with explicit Simulation/v2 provider. Use real control StepAsync and validated signal replies; no private engine sample list, test oracle, model-generated telemetry or replay as physics. Capture phase before advancing a step where needed for boundary attribution. Fixed limits/cadence for this slice: at most 3900 model steps per run; engine AND transmission signal reads every five completed steps and at completion (avoid duplicate final reads). The decisive 0.1-second warm configuration is therefore sampled every 0.5 virtual seconds. Capture those actual observations through existing recording services, inside unchanged 4096-observation/byte/ archive limits; at most 1562 scheduled signal observations per run. Advance/control phase counts at each step, but do not add unrecorded/private telemetry or per-step polling that exhausts archive byte limits. A run needing step 3901 rejects as incomplete, not Completed or a reason to enlarge existing limits. Retain one current pair/last successful result only, with bounded immutable samples/metrics/captures; no hidden history or unbounded collection. Ordinary manual diagnostic/capture/replay owners are not taken over or cleared. Reuse independent accepted capture/store APIs for the pair; do not silently replace another recording workspace or discard an unsaved first capture when the second begins. Extract a narrow shared helper only if needed. ### C. Comparison, save/readback/replay and truthful lifecycle Compare actual canonical data: last-ten warm-idle RPM and stationary speed, warm phase, acceleration/braking elapsed model time, sampled cruise speed range/count (separate from total cruise model-step count), fuel consumed delta from initial state, distance, total model time and final actual transmission.shift-count. Distinguish fixed-cadence shift sampling from the measured final count; do not infer an exact full shift timeline. Missing/non-Good/nonfinite/wrong-unit/context replies cannot become zero or a healthy complete metric. Failed/limited/cancelled runs retain bounded partial evidence and visible reason, not a complete comparison. Do not promise reduced fuel, better shifts or real-world improvement. For the exact demo, compare 650 rpm baseline with 1250 rpm candidate and torque at 2000 rpm 360->350 N*m. Last ten warm-phase samples on each run must be within 1 rpm of its target and vehicle speed within 0.001 m/s of zero. Fewer than ten valid warm samples is an explicit incomplete/unsupported case, not a shortened passing window. Label sampled metrics with their cadence; do not claim continuous extrema from sparse observations. Report measured driving/fuel/shift differences without requiring an invented sign. Keep all original full-scenario numerical regressions, including baseline 3361 samples/336.1 s and 3000 cruise samples within 45 MPH +/-1 MPH. Report separate proposal/diff, backup/approved mutation/full readback and functional simulation evidence. Include exact source/head/definition/model, target instance/revision/operation/backup, original/derived configs, provider/ run IDs, capture IDs/completion and canonical before/after values/metrics. Provide a safe copyable readonly report; no new persistence/schema/export engine. Save is explicit, reusing archive v1/atomic no-overwrite publication/readback. Save each capture with visible per-ID outcome. If one succeeds and the other fails, preserve the successful file and unsaved capture, identify partial save and retry only missing work; no pair-atomic claim, rollback deletion or fake all-saved state. Open saved results through actual existing replay and raw trace. Units project display only; source/archive/report canonical values stay exact. Use one cancellable owner, generation/source/proof admission and publication checks. Queue only admitted blocking work and release Hybrid command events as accepted in Step-12A. Close/unmount drains owned diagnostic/capture resources without holding required gates, late-result publication or global interference. Invalid competing requests reject before starting/mutating. Retained earlier results remain inspectable, with clear stale/as-of status. ## FILES OR PROJECTS EXPECTED TO CHANGE Narrow Core projection if needed, Application evaluation/trusted admission/ shared-helper reuse, calibration/evaluation UI and DI, tests and related docs/ README. Only `prompts/Step-13-Report.md` under `prompts`. No new project/package/host, archive/runbook/profile format, runtime install/ model pull/configuration change or unrelated recording navigation cleanup. ## INTERFACE AND DEPENDENCY REQUIREMENTS Preserve nine references, independent Components, typed Diagnostics ports, immutable original configs/authoring/runbooks, volatile backup/approval/CAS/ readback/Restore, archive limits/version and replay metadata semantics. No general runbook executor, physical programming, arbitrary frames/offsets, cloud fallback, Git backup, new templates/unit families or privileged oracle. The model supplies untrusted preview data only; neither proof nor authority. No agents, automatic handoff, staging/commit/push or orchestration/shared-gate changes. Step-12A's setup authorization is not recurring runtime management. ## TESTING REQUIREMENTS Map C-01..C-10 before implementation. Verify exact normal/rejected/boundary/ lifecycle/output cases, not only counts: - Actual current preview -> draft -> explicit Save, stale/superseded/wrong source and pinned units. No extra source/target calls on rejected handoff. - Actual owning rehearsal baseline/two-read backup/approval/CAS/full readback -> trusted evaluation source. Forged flags, wrong instance/revision/source/ candidate/backup, recovery/divergence and supersession reject atomically. - Independent whole-config serialized/property comparison proves only the supported warm-idle/torque fields change; originals remain byte/content exact. Nonuniform table/cell changes and invalid/nonfinite/unit/axis/boundaries. - Real two-run v2/recording/store/replay boundary and independent test-only numerical expectations, last-ten idle thresholds, exact baseline scenario and fresh equivalent initial state. Independent tests retain verification of all 3000 cruise model steps; production captures/report explicitly distinguish sampled signal counts/ranges from full model-step counts. No private oracle exposed to application/UI/AI. - Controlled actual owner/transport/save/readback delays and cancel/unmount/ late/superseded/busy failure paths; 3900/3901 limit and unchanged archive caps. Partial saves/readback corruption and wrong archive/config/replay references. - Supported earlier timeout/unsupported service/invalid wire/wrong identity/ version/config/interruption/reset/DTC/recovery/reconciliation/Restore/archive corruption tests remain; name executed evidence. No invented firmware checksum or partial physical-write mechanism merely to fill a fault table. Release test barriers in finally; no sleeps/polling as timing correctness proof. Run focused combined selectors, then sequentially: - `dotnet build .\src\AutoForge.slnx --configuration Release --no-restore` - `dotnet test .\src\AutoForge.slnx --configuration Release --no-build` Preserve all 596 cases/full nine-project/native build. Restore code dependencies only after genuine chosen-command missing dependency; no SDK/workload/model install/update/repair or alternate host. Required actual native demonstration: original warm run/capture; exact local model goal; current preview handoff and separate Proposed Save; actual Simulation target backup/approval/Apply/fresh full readback; explicit trusted projection/ equivalent baseline-derived pair; real measured behavior/report/capture saves/ readback/replay/raw trace; failure/cancel/stale/units and unchanged original configuration/history outside deliberate commands. Existing routes still work. Record exact source/config/run/capture/proof IDs, model original response and measurable outcomes. Controlled responses are not native model proof. No physical target or unrestricted operation; cleanup only exact task-owned native helpers/archives/profiles/PIDs, preserving user data/runtime/model. ## DOCUMENTATION REQUIREMENTS Align README/index/architecture/build/authoring/rehearsal/runbook/evaluation usage with explicit proposal handoff, trusted verified-source admission, NEW-config projection, equivalent fixed-cadence runs, archive bounds, canonical metrics/lineage, partial save, stale/cancel/close and synthetic fidelity limits. Step-12/12A accepted; Step-13 pending manual review; hardware/firmware checksum/ table shift schedules/broader AI evaluation remain future. ## ACCEPTANCE CRITERIA | ID | Criterion | Required evidence | |---|---|---| | AC-01 | C-01..C-03 explicit draft/proof admission and exact immutable projection | Actual owning workflows, independent full-value/config invariants and atomic negative/supersession tests | | AC-02 | C-04..C-06 equivalent real scenarios/captures/measured intent/report/save/replay | Exact idle thresholds/baseline driving gates, actual canonical samples/metrics/lineage and real archive readback/playback | | AC-03 | C-07..C-09 responsive safe owner/failures/limits and independent verification | Controlled real-boundary interleavings/3900 boundary/partial saves/corruption, preserved no-side-effect/isolation and supported fault evidence | | AC-04 | C-10 full 596 regressions/new coverage/nine-project Release and actual native demo pass | Exact worker commands/exits/counts/numerical results, original real model content, proof and captures/replay; no mock-as-native claims | | AC-05 | Matrix/self-review/docs/history/protection/manual gate accurate | Own post-green producer/consumer review, aligned scope/fidelity and `prompts/Step-13-Report.md` | ## OUT-OF-SCOPE WORK Physical writes/read-only hardware onboarding, broad AI/RAG/tools/executors, runtime/model installation/management, firmware layouts/checksums, dynamics fidelity redesign, changed source configs/runs, new archive/report persistence/ Git backups, generic evaluation frameworks, new projects/packages/hosts and unrelated fixes. No Step-14, agents or automatic handoff/commits. ## REPORTING REQUIREMENTS Save `prompts/Step-13-Report.md` wholly inside exactly one plain triple-backtick fence, no language tag/outside text/nested fences. Include identifier/title/status/functions/files/interfaces; C-01..C-10/ AC-01..AC-05 named executed matrix; exact preview/draft/save/proof/projection before/after and unchanged config fields; actual equivalent numerical sample/ metric/run/capture/lineage/save/readback/replay outcomes; owner/negative/limit/ partial failures; commands/exits/warnings/per-project counts; native versus controlled evidence and original model content; docs/self-review/deviations/ limitations/blockers/preservation/cleanup. No secrets/private thinking traces. COMPLETE requires all assigned checks, otherwise INCOMPLETE/BLOCKED with exact unmet criterion. Save and STOP for manual review; do not begin Step-14.