AutoForge/docs/calibration-comparison.md

127 lines
7.7 KiB
Markdown

# Verified synthetic calibration comparison
Step-13 with corrective Step-13A is implemented pending manual review. This is a narrow synthetic
comparison, not manufacturer prediction, safe tuning advice, firmware proof,
physical authority or a general AI/runbook executor.
## Explicit connected workflow
1. Create/select the original warm configuration and authoring Baseline.
Optionally record an actual original warm run through ordinary diagnostics.
2. In `/runbooks`, explicitly validate manual input or original content from
the qualified local model. The decisive goal is warm idle 1250 rpm and
2000-rpm torque 350 N*m; keep every other value unchanged.
3. Return to `/calibration`; explicitly copy the exact current preview into
this existing pinned draft. Copy validates workspace/head/configuration and
keeps canonical doubles and the draft's prior units. Save separately.
4. Select the isolated rehearsal provider, Open, prepare original two-read
backup, prepare exact current Proposed Apply, explicitly confirm/Approve,
Dispatch and inspect independent complete readback.
5. Open comparison controls explicitly, select THIS owning verified Apply and
`autoforge.simulation.dynamics`, then Evaluate. This does not program a target,
overwrite a configuration/run or promote Proposed to Tested/Approved.
6. Inspect/copy the canonical readonly report. Save both captures explicitly.
Only persisted/readback-verified IDs qualify as saved evidence. Open each
exact ID in existing `/telemetry-replay`, inspect prefix/end/raw bytes and
immutable original provenance. Replay is archive playback, not physics.
No action is implied by rendering, navigation, AI validation, unit changes or
notifications. Manual validation remains available without a local model.
## Trusted proof and exact immutable mapping
An actual idle Ready owning rehearsal must have an original verified backup
and current verified Apply (including genuine post-reconciliation verification).
Admission freshly reads the full target and matches exact instance/source/
revision/operation/approval/backup/workspace/head/definition, candidate and every
changed/unchanged value. Original backup must equal original supported config
values. Caller success records cannot construct the internal proof capability.
RecoveryRequired, divergent/unavailable, wrong/closed/stale evidence rejects.
The pure supported mapping changes only Model.WarmIdleTargetRpm and torque
cells. Axis, every other model field, initial conditions, five-phase scenario
and profile/module/model/calibration references remain unchanged. Existing
constructors/validation/fingerprints/catalogue register the NEW immutable config;
original config and authoring binding/history are not rebased. No-change mapping
truthfully retains the same content fingerprint, not a fabricated improvement.
Proof is historical at its readback time, not perpetual external target
attestation. Owning source guards run between scenarios, and another fresh full
readback plus current-head/lifetime checks precede atomic result publication.
## Equivalent acquisition and metrics
Two fresh sequential real Simulation/v2 diagnostic sessions use equivalent
initial state/scenario and distinct run/capture IDs. Physics advances only through
real StepAsync(1). Engine AND transmission signals are read every five completed
steps plus completion, without duplicate final reads. At 0.1 s/step this is
0.5 virtual-second cadence. Maximum is 3900 steps/run and 1562 scheduled
observations; a model needing step 3901 rejects as incomplete. Existing archive
bounds remain 4096 observations, 16 MiB observations, 1 MiB header, 32 MiB archive.
Independent accepted capture APIs leave ordinary unsaved recordings untouched.
Metrics use exact Good finite canonical module/run/config/step/time values.
Missing, non-Good, wrong-unit/context/nonfinite replies cannot become zero.
At least TEN actual warm samples are required; last ten must each be within
1 rpm of their target and 0.001 m/s of stationary. A shorter window is incomplete.
Report includes warm/error/stationary, model/phase step counts, acceleration/
braking time, sampled cruise count/range, fuel/distance deltas from initial state,
total virtual time and final actual transmission.shift-count.
The decisive original/derived pair both completes 3361 steps/336.1 s with phase
counts 100/103/3000/58/100, acceleration 10.3 s and braking 5.8 s. The 3000 cruise
model steps remain independently tested within 45 MPH +/-1 MPH; production
reports 600 sampled cruise signal sets at 20.1168 m/s. Last-ten idle errors are
below 1e-8 rpm with zero stationary speed for 650 and 1250 rpm. Measured fuel is
0.518385971679477 versus 0.5184897031483956 kg; distance 6193.507764002321 versus
6193.389583605566 m; final shift count 4 versus 4. No beneficial sign is promised.
Sampled range is not continuous extrema; final count is not an exact shift
timeline. Request/capture elapsed, diagnostic ticks and virtual model time are
different clocks.
The readonly canonical report identifies proof/time/preparation/backup/mutation/
approval/source/head/definition/model/module/profile, exact torque before/after,
original/derived fingerprints, providers/run/capture IDs, completion and metrics.
Display preferences never alter source/report/archive values.
## Lifetime, partial evidence and save
One admitted comparison/run/save owner rejects competing requests Busy before
starting another. Hybrid callbacks return promptly; admitted work runs off the
UI dispatcher. Cancel/Close/unmount drains owned diagnostics/capture resources
outside required gates, preventing late healthy publication. New evaluation/
Close retires earlier current status; source/head/proof supersession labels it
Stale/as-of without rebase. One previous success and bounded latest partial run/
save evidence remain inspectable, not an unbounded history.
Busy includes the entire completion tail, not merely returned operation work.
The admitted owner and its exact drain remain reserved while cancellation
callbacks/release finish outside the comparison state gate. Only then are
terminal state, owner removal and that completion settled together under the
gate. A competing Evaluate/Save cannot replace the old drain; shared Close/
Dispose includes all admitted cleanup. After a full drain, legitimate new work
can admit only while open; closed sessions require an independent new lifetime.
Cleanup errors fault operation waiters explicitly, preserving primary error/
cancellation and per-capture save evidence; combined failures retain both errors.
Controlled tests use a minimal internal real-release observation seam, not a
public test control or mocked coordinator. Native UI uses the same unmodified
commands with no lifecycle probe subscribed. See the
[Step-13A report](../prompts/Step-13A-Report.md) for pre-fix and corrected evidence.
Failed/limited/cancelled physics never becomes a successful pair. A stopped
capture can represent complete coverage of a partial run; the run's separate
error/metrics/completion determine comparison success. This does not expand or
mislabel archive-v1 stop-reason semantics.
Pair save is NOT atomic. Each capture uses existing no-overwrite publication and
exact serialized fresh readback. If one fails, successful files and unsaved
capture remain; UI identifies persisted/verified/error per ID. Retry skips
verified captures and only saves missing work. If a failed publication/cleanup
may have left a file, retry checks the catalogue and verifies it read-only.
Mismatched/corrupt readback remains an error; there is no overwrite, rollback
deletion, hidden new ID or fake all-saved state.
See [authoring](calibration-authoring.md), [rehearsal](calibration-rehearsal.md),
[runbooks](runbooks-v1.md), [recording](telemetry-archive-v1.md),
[replay](telemetry-replay.md) and [build/test evidence](build-and-test.md).