AutoForge/docs/calibration-comparison.md

113 lines
6.8 KiB
Markdown

# Verified synthetic calibration comparison
Step-13 is implemented pending manual review. This is a narrow synthetic
comparison, not manufacturer prediction, safe tuning advice, firmware proof,
physical authority or a general AI/runbook executor.
## Explicit connected workflow
1. Create/select the original warm configuration and authoring Baseline.
Optionally record an actual original warm run through ordinary diagnostics.
2. In `/runbooks`, explicitly validate manual input or original content from
the qualified local model. The decisive goal is warm idle 1250 rpm and
2000-rpm torque 350 N*m; keep every other value unchanged.
3. Return to `/calibration`; explicitly copy the exact current preview into
this existing pinned draft. Copy validates workspace/head/configuration and
keeps canonical doubles and the draft's prior units. Save separately.
4. Select the isolated rehearsal provider, Open, prepare original two-read
backup, prepare exact current Proposed Apply, explicitly confirm/Approve,
Dispatch and inspect independent complete readback.
5. Open comparison controls explicitly, select THIS owning verified Apply and
`autoforge.simulation.dynamics`, then Evaluate. This does not program a target,
overwrite a configuration/run or promote Proposed to Tested/Approved.
6. Inspect/copy the canonical readonly report. Save both captures explicitly.
Only persisted/readback-verified IDs qualify as saved evidence. Open each
exact ID in existing `/telemetry-replay`, inspect prefix/end/raw bytes and
immutable original provenance. Replay is archive playback, not physics.
No action is implied by rendering, navigation, AI validation, unit changes or
notifications. Manual validation remains available without a local model.
## Trusted proof and exact immutable mapping
An actual idle Ready owning rehearsal must have an original verified backup
and current verified Apply (including genuine post-reconciliation verification).
Admission freshly reads the full target and matches exact instance/source/
revision/operation/approval/backup/workspace/head/definition, candidate and every
changed/unchanged value. Original backup must equal original supported config
values. Caller success records cannot construct the internal proof capability.
RecoveryRequired, divergent/unavailable, wrong/closed/stale evidence rejects.
The pure supported mapping changes only Model.WarmIdleTargetRpm and torque
cells. Axis, every other model field, initial conditions, five-phase scenario
and profile/module/model/calibration references remain unchanged. Existing
constructors/validation/fingerprints/catalogue register the NEW immutable config;
original config and authoring binding/history are not rebased. No-change mapping
truthfully retains the same content fingerprint, not a fabricated improvement.
Proof is historical at its readback time, not perpetual external target
attestation. Owning source guards run between scenarios, and another fresh full
readback plus current-head/lifetime checks precede atomic result publication.
## Equivalent acquisition and metrics
Two fresh sequential real Simulation/v2 diagnostic sessions use equivalent
initial state/scenario and distinct run/capture IDs. Physics advances only through
real StepAsync(1). Engine AND transmission signals are read every five completed
steps plus completion, without duplicate final reads. At 0.1 s/step this is
0.5 virtual-second cadence. Maximum is 3900 steps/run and 1562 scheduled
observations; a model needing step 3901 rejects as incomplete. Existing archive
bounds remain 4096 observations, 16 MiB observations, 1 MiB header, 32 MiB archive.
Independent accepted capture APIs leave ordinary unsaved recordings untouched.
Metrics use exact Good finite canonical module/run/config/step/time values.
Missing, non-Good, wrong-unit/context/nonfinite replies cannot become zero.
At least TEN actual warm samples are required; last ten must each be within
1 rpm of their target and 0.001 m/s of stationary. A shorter window is incomplete.
Report includes warm/error/stationary, model/phase step counts, acceleration/
braking time, sampled cruise count/range, fuel/distance deltas from initial state,
total virtual time and final actual transmission.shift-count.
The decisive original/derived pair both completes 3361 steps/336.1 s with phase
counts 100/103/3000/58/100, acceleration 10.3 s and braking 5.8 s. The 3000 cruise
model steps remain independently tested within 45 MPH +/-1 MPH; production
reports 600 sampled cruise signal sets at 20.1168 m/s. Last-ten idle errors are
below 1e-8 rpm with zero stationary speed for 650 and 1250 rpm. Measured fuel is
0.518385971679477 versus 0.5184897031483956 kg; distance 6193.507764002321 versus
6193.389583605566 m; final shift count 4 versus 4. No beneficial sign is promised.
Sampled range is not continuous extrema; final count is not an exact shift
timeline. Request/capture elapsed, diagnostic ticks and virtual model time are
different clocks.
The readonly canonical report identifies proof/time/preparation/backup/mutation/
approval/source/head/definition/model/module/profile, exact torque before/after,
original/derived fingerprints, providers/run/capture IDs, completion and metrics.
Display preferences never alter source/report/archive values.
## Lifetime, partial evidence and save
One admitted comparison/run/save owner rejects competing requests Busy before
starting another. Hybrid callbacks return promptly; admitted work runs off the
UI dispatcher. Cancel/Close/unmount drains owned diagnostics/capture resources
outside required gates, preventing late healthy publication. New evaluation/
Close retires earlier current status; source/head/proof supersession labels it
Stale/as-of without rebase. One previous success and bounded latest partial run/
save evidence remain inspectable, not an unbounded history.
Failed/limited/cancelled physics never becomes a successful pair. A stopped
capture can represent complete coverage of a partial run; the run's separate
error/metrics/completion determine comparison success. This does not expand or
mislabel archive-v1 stop-reason semantics.
Pair save is NOT atomic. Each capture uses existing no-overwrite publication and
exact serialized fresh readback. If one fails, successful files and unsaved
capture remain; UI identifies persisted/verified/error per ID. Retry skips
verified captures and only saves missing work. If a failed publication/cleanup
may have left a file, retry checks the catalogue and verifies it read-only.
Mismatched/corrupt readback remains an error; there is no overwrite, rollback
deletion, hidden new ID or fake all-saved state.
See [authoring](calibration-authoring.md), [rehearsal](calibration-rehearsal.md),
[runbooks](runbooks-v1.md), [recording](telemetry-archive-v1.md),
[replay](telemetry-replay.md) and [build/test evidence](build-and-test.md).