237 lines
19 KiB
Markdown
237 lines
19 KiB
Markdown
```
|
|
# Step-11 - Deterministic simulation runbook validation and preview
|
|
|
|
## Status
|
|
|
|
COMPLETE. Strict bounded manual schema/source/head/template validation,
|
|
immutable exact preview/staleness, actual local UI, real-source no-side-effect
|
|
tests, post-green self-review, nine-project Release build/full regressions and
|
|
native smoke passed.
|
|
|
|
Step-11 remains pending manual orchestrator review; this report grants no
|
|
acceptance or mutation authority. Corrected Step-10/10A remains accepted.
|
|
Step-12 local inference/knowledge and runbook execution were not started.
|
|
No model/runtime/retrieval/network feature, authoring Save, target command,
|
|
approval or app-level Git persistence was implemented.
|
|
|
|
## Files, functions and interfaces
|
|
|
|
- [RunbookPreviewService.cs](../src/AutoForge.Application/Features/Vehicles/Runbooks/RunbookPreviewService.cs):
|
|
typed RunbookPreviewException/failures, immutable source/result/snapshot
|
|
contracts, strict allowlisted parsing, Validate, CreateInputTemplate and
|
|
fresh Snapshot status. The only constructor dependency is the existing
|
|
CalibrationAuthoringService; validation reads its immutable snapshot and
|
|
publishes under its existing narrow current-head guard.
|
|
- Existing SyntheticEngineCalibration validates complete scalar/table values;
|
|
existing CalibrationValueDifferences supplies the pure exact diff. Neither
|
|
helper, authoring Save, rehearsal, target or source model implementation
|
|
was changed or duplicated.
|
|
- [RunbookPage.razor](../src/AutoForge.UI/Features/Vehicles/Runbooks/RunbookPage.razor):
|
|
/runbooks, manual JSON input, explicit source template/Validate actions,
|
|
typed input errors, previous successful result labels, current/Stale source
|
|
status, canonical candidate/diff and optional display-only units.
|
|
Authoring/preview notifications only schedule rendering; they never rebase
|
|
input, validate, save or execute.
|
|
- [AddAutoForgeUi.cs](../src/AutoForge.UI/Features/Foundation/AddAutoForgeUi.cs),
|
|
[MainLayout.razor](../src/AutoForge.UI/Layout/MainLayout.razor) and
|
|
[Home.razor](../src/AutoForge.UI/Features/Foundation/Home.razor):
|
|
minimal preview registration and shared navigation. Client DI and all
|
|
nine project references remain unchanged.
|
|
|
|
Executable evidence:
|
|
|
|
- [RunbookPreviewTests.cs](../src/tests/AutoForge.IntegrationTests/RunbookPreviewTests.cs)
|
|
- [RunbookPreviewUiTests.cs](../src/tests/AutoForge.IntegrationTests/RunbookPreviewUiTests.cs)
|
|
- [RunbookPreviewIsolationTests.cs](../src/tests/AutoForge.IntegrationTests/RunbookPreviewIsolationTests.cs)
|
|
|
|
The initial frozen schema/source/value/limit/immutability/stale/UI/isolation
|
|
tests were written before production implementation. The post-green review
|
|
added exact exponent/Unicode/escaped-property/count/missing-source vectors.
|
|
|
|
## Frozen C-01..C-10 matrix
|
|
|
|
All rows below are PASSED. Named methods were executed tests, not proposals.
|
|
|
|
| ID | Named evidence and exact result |
|
|
|---|---|
|
|
| C-01 | ExplicitSourceAndNoChanges_PreviewCopiesWholeHeadWithoutImplicitWorkspaceOrSave; MissingActualWorkspace_RejectsStructuredInputWithoutCreatingSourceOrPreview. No source/template/preview is created implicitly. Valid unchanged complete current-head input yields NoChanges, empty diff, exact copied workspace/head/applicability, scalar 650 and six original torque rows. Missing actual workspace rejects SourceUnavailable even for structured otherwise-valid input. Source/history remain the same immutable snapshot. Both Baseline and Proposed current heads are usable. |
|
|
| C-02 | ScalarTableMultiCellReverseAndTinyChanges_AreExactCanonicalDiffNotAuthoringSave. Independent candidate 1250 rpm plus 2000-rpm torque 350 yields exactly two ordered canonical entries: table before 360/after 350/delta -10 N*m/axis 2000 rpm; scalar before 650/after 1250/delta +600 rpm. Multiple-cell ordering, reverse +10/-600 and representable tiny differences are asserted without tolerance. No authoring/config/target is changed by preview. |
|
|
| C-03 | EverySchemaObject_RejectsMissingDuplicateUnknownFieldsAtomically (18 object/mutation vectors); WrongKindsAndIntegerSyntax_RejectWithoutReplacingPriorSuccessfulPreview (24 vectors); MalformedCommentsTrailingDataAndWrongRoot_AreHandledAtomicErrors; EscapedDuplicatePropertiesInvalidUnicodeAndNonJsonNumbers_AreTypedAtomicRejections. Root/step/source/values/scalar/table properties are all required, unique and allowlisted. Wrong kinds, fractional/exponent reference syntax, malformed/comments/trailing input, unsupported version, null input and invalid strings/numbers reject through typed failure/path/reason before replacement. Prior preview and source/head/history remain identical. |
|
|
| C-04 | OperationAllowlistAndSingleStep_RejectExecutionOrOtherTargets (10 vectors); ArbitraryCommandAndAuthorityFields_AreNeverInterpreted (6 vectors); safe rationale rendered test. Only exact simulation/proposeCalibration and one step are allowed. Physical/replay/undefined/case-changed target/operation, empty/multiple steps and SQL/script/frame/offset/approval/execute fields reject. Rationale is safely rendered informational text and never grants command or policy authority. |
|
|
| C-05 | EverySourceComponent_MustMatchActualWorkspaceAndCurrentHead (10 vectors); FieldSpecificCasePolicies_AcceptProfileModuleButNotFingerprintModelDefinitionKeyOrUnit. Every workspace/head/profile revision/config/module/model version/definition reference must match actual current source. Profile/module case variants preserve existing ordinal-insensitive policy; fingerprint/model/definition/key/unit stay exact. No catalogue, role or latest-head fallback occurs. |
|
|
| C-06 | FullTemplateInvalidNumbersKeysUnitsAxisAndShape_RejectAtomically (14 vectors); ExactTemplateBoundsAndNoOp_KeepCanonicalValuesWithoutClipping; SupplementaryStepIdentifierAndWholeCandidateCounts_FollowExactExistingLimits; exact tiny/reverse diff test. Full existing template validates finite scalar/cells, keys/units/fixed axes/counts. Out-of-range/non-finite/wrong-axis/shape/key/unit candidates reject atomically. Inclusive 650/5000 rpm and 1/1000 N*m boundaries remain exact; tiny canonical deltas are not rounded away and no sorting/clipping occurs. |
|
|
| C-07 | SourceChangeAndReplacement_MarkImmutableOldPreviewStaleUntilExplicitCorrectRevalidation; actual rendered stale/draft test. Old-head request rejects; source/head change/replacement after success marks the same immutable captured result Stale. Its before/candidate/diff/identity are not rebased. Explicit correct input/revalidation is required; source changes do not alter draft or invoke validation. Publication reuses the actual source/head guard. |
|
|
| C-08 | Utf8PayloadBoundary_IsExactly131072BytesIncludingMultibyteRationaleAndWhitespace; RationaleAndStepIdentifier_AreExactUtf16CodeUnitLimits; supplementary Unicode/depth tests. Valid original payload exactly 131072 UTF-8 bytes succeeds despite shorter string.Length; one more byte rejects with prior result retained. Rationale 2000 succeeds/2001 rejects; step 128 succeeds/129 rejects. Supplementary Unicode is counted in UTF-16 correctly. Depth 32 parses then receives appropriate schema rejection for the deliberately unknown structure; depth 33 is handled InvalidJson. Invalid UTF-8 encoding/escaped surrogate and non-JSON NaN are handled without fallback. |
|
|
| C-09 | PreviewCollectionsAndRetainedResult_AreDefensiveAndBoundedToOneSuccess; RunbookUi_TemplateValidateRejectedDraftStaleAndUnits_NeverAutoValidateOrRebase. Parsed collections/current-head values/diff are copied/read-only; mutation attempts reject and old preview contents survive a later success. Service retains one last success, invalid attempt preserves its identity, and UI separates old result from rejected/edited input. Units/rerenders/source notifications do not validate or modify canonical input/data/diff. |
|
|
| C-10 | RunbookUi_SafeRationaleUnavailableSourceAndNoExecutionControls_AreRealRenderedBehavior; RunbookIsolation_ActualLiveRecordingReplayAndApprovedRehearsalRemainUntouched; native evidence. Actual route/template/Validate/error/stale/rendering path works without Save/approval/Execute controls. Real live source/recording/replay/approved rehearsal coexist around invocation: transport/model/archive-Save/rehearsal open/read/write deltas=0, authoring changes/history mutation=0, approval remains same/unconsumed, and source export/trace/status/archive bytes/replay snapshot are unchanged. Constructor/source-path review confirms no Save/approval/network/model path. All 442 regressions plus 105 new cases, nine-project Release and native smoke pass. |
|
|
|
|
## Acceptance criteria
|
|
|
|
| ID | Result | Evidence |
|
|
|---|---|---|
|
|
| AC-01 | PASSED | C-03..C-06/C-08 independent strict schema/allowlist/kind/reference/full-template vectors, exact UTF-8/UTF-16/depth bounds and typed atomic rejection assertions. |
|
|
| AC-02 | PASSED | C-01/C-02/C-07/C-09 exact immutable before/candidate/source/head/diff, NoChanges/ChangesProposed, tiny changes, defensive copies, Stale versus current and bounded one-preview retention. |
|
|
| AC-03 | PASSED | C-10 real rendered/native controls, safely encoded rationale, unchanged approved real rehearsal/live/recording/replay/source counters/state, no execution or inference dependency/path. |
|
|
| AC-04 | PASSED | Final 547 passes with all 442 baseline cases preserved; all nine Release projects/native Client, 0 warnings/errors; required native normal preview/rejection/staleness/source/unit smoke passed. |
|
|
| AC-05 | PASSED | Frozen executable matrix, own post-green source-path review, exact schema/usage/docs/README, protection/history/manual gate preserved and this single fenced report saved. |
|
|
|
|
## Final commands and outcomes
|
|
|
|
SDK: dotnet --version -> 10.0.403 under the unchanged global.json policy.
|
|
Existing xUnit v2/VSTest is unchanged. No package/SDK/workload/tool install,
|
|
restore, update or repair occurred.
|
|
|
|
Focused, required build and complete regressions ran sequentially:
|
|
|
|
| Exact command | Exit | Result |
|
|
|---|---:|---|
|
|
| dotnet test .\src\tests\AutoForge.IntegrationTests\AutoForge.IntegrationTests.csproj --configuration Release --no-restore --filter "FullyQualifiedName~RunbookPreviewTests|FullyQualifiedName~RunbookUi_|FullyQualifiedName~RunbookIsolation_" | 0 | 105 new runbook cases passed, zero failed/skipped. |
|
|
| dotnet build .\src\AutoForge.slnx --configuration Release --no-restore | 0 | All nine projects/native Client; 0 warnings, 0 errors. MAUI XAML source-generation output was informational. |
|
|
| dotnet test .\src\AutoForge.slnx --configuration Release --no-build | 0 | 179 unit + 368 integration = 547 passed, zero failed/skipped. |
|
|
| git --no-pager diff --check | 0 | No whitespace errors; existing README/build-guide LF-to-CRLF notices only. |
|
|
| git --no-pager diff --exit-code -- prompts global.json Directory.Build.props Directory.Packages.props NuGet.Config src\AutoForge.Core src\AutoForge.Diagnostics src\AutoForge.Simulation src\AutoForge.Client src\AutoForge.Application\Features\Vehicles\Calibration src\AutoForge.Application\Features\Vehicles\Telemetry | 0 | Existing protected history/dependencies/core/ports/models/Client/authoring/rehearsal/recording/replay implementations unchanged before adding this report. |
|
|
|
|
Per-project Release: AutoForge.Core, AutoForge.Diagnostics,
|
|
AutoForge.Simulation, AutoForge.Application, AutoForge.Components,
|
|
AutoForge.UI, AutoForge.Client, AutoForge.UnitTests and
|
|
AutoForge.IntegrationTests all PASSED. Portable projects remain net10.0;
|
|
Client remains net10.0-windows10.0.26100.0 / win-x64.
|
|
|
|
The baseline is unchanged: 179 unit + 263 integration = 442 cases. This step
|
|
adds 105 integration cases; no baseline safety/source assertion was deleted
|
|
or weakened. Preview limits are 131072 original UTF-8 bytes, depth 32,
|
|
rationale 2000 UTF-16 units, step ID 1..128 trimmed UTF-16 units and one
|
|
allowlisted step. Existing authoring/rehearsal/archive/gauge limits remain.
|
|
|
|
Iteration:
|
|
|
|
- First executable focused run passed 99 cases without compilation/behavioral
|
|
failures. The post-green review added exponent/escaped-property/Unicode/
|
|
full-count cases, yielding 104. A final independent missing-workspace
|
|
validator case raised the final count to 105.
|
|
- The full reviewed suite initially passed 546; final focused/build/full
|
|
checks passed 105/547 after that last test addition. Production source
|
|
was unchanged after the initial implementation/source-path review.
|
|
- Native automation initially matched the explanatory word ChangesProposed
|
|
in a NoChanges status before the actual diff rendered, causing a helper
|
|
null-DOM read. It was corrected to wait for the exact captured indication
|
|
and element presence. This was not a parser/app failure or a successful
|
|
observation. The actual native workflow then completed without an error
|
|
overlay.
|
|
- No pre-fix behavioral reproduction is claimed: this is a new feature, not
|
|
an existing validator bug correction.
|
|
|
|
## Native normal evidence versus headless proof
|
|
|
|
The actual rebuilt Windows Client was operated through task-owned WebView2
|
|
data/local DevTools port 9382 with a temporary built-in Node/WebSocket helper.
|
|
No new application web host, runtime/model or dependency was added.
|
|
|
|
Native observations:
|
|
|
|
1. /runbooks entered with empty input/no preview. Explicit template attempt
|
|
without source gave SourceUnavailable and created no source/default result.
|
|
2. Explicitly created warm source through existing simulation/calibration UI.
|
|
Actual workspace f1b8740bd2414b7aa3e7be78ff35139f, baseline head
|
|
60bc53bc58d04ae38ab208aecf4d8ced. Template copied that source without
|
|
validation; explicit Validate yielded NoChanges/current/empty diff.
|
|
3. Manual canonical edits 1250 rpm and torque 350 at 2000 rpm yielded preview
|
|
493f4f636782496eb50df0e4294e5a9f, ChangesProposed/current, with exactly:
|
|
engine.torque-curve axis 2000 rpm, 360->350 N*m, signed delta -10 N*m;
|
|
engine.warm-idle-target 650->1250 rpm, signed delta +600 rpm.
|
|
4. Rationale "Manual preview only: <b>this is not approval</b>." rendered as
|
|
text. Its HTML used <b> and no b element was created.
|
|
Metric->Imperial changed optional torque display only; JSON draft, preview
|
|
identity and exact canonical diff stayed unchanged.
|
|
5. Malformed JSON, physicalHardware target, wrong expected head and scalar
|
|
600 each yielded the appropriate InvalidJson/OperationNotAllowed/
|
|
StaleSource/InvalidValues error with path/reason, while the previous
|
|
preview ID and diff remained exactly unchanged and explicitly labeled
|
|
Previous successful preview retained.
|
|
6. Returning to authoring confirmed history count 1 and original scalar 650;
|
|
rehearsal remained Inactive/audit 0. Preview created no saved proposal,
|
|
target or approval.
|
|
7. A separate explicit authoring Save changed current head to revision 2 /
|
|
c2cb7ddefe474f44a17754896b3fde9c. On runbooks re-entry the original preview
|
|
was visibly Stale, with its old identity/diff unchanged. Old JSON input
|
|
rejected StaleSource. A new template still did not validate/clear stale
|
|
result automatically; explicit Validate produced NoChanges/current for
|
|
the current Proposed head. History remained exactly 2, not 3.
|
|
8. Original source configuration export stayed exactly identical, fingerprint
|
|
sha256:v1:e8991ae139b54959ec53a4fe64b08dc9bfc03feb533496cf741f2c9f3a2342da.
|
|
No model/session/target execution was opened by this native preview smoke.
|
|
Original Metric/Road preferences were restored and observed; existing
|
|
replay/Home routes worked, with replay Unloaded.
|
|
|
|
Native evidence is normal real-UI/source behavior, not exact-limit or
|
|
concurrency/coexistence proof. Actual active live/recording/replay/approved-
|
|
rehearsal coexistence and zero boundary calls are headless real-service/port
|
|
tests. Normal route cleanup is not attributed to validator invocation.
|
|
No inference, knowledge, physical target or programming was used.
|
|
|
|
Cleanup PASSED: task Client PID 40340 completed after route cleanup. Transient
|
|
WebView PIDs 45756/44632 were checked against the task profile before any
|
|
specific-PID cleanup. No owned WebView/listener remained; port 9382 was closed.
|
|
Exact task-only profile/helper were removed. Native source/proposal/preview
|
|
data were process-memory only; the app-local recordings directory remained
|
|
empty. No unrelated file/archive/default WebView data/preferences were reset.
|
|
|
|
## Own post-green producer/consumer review
|
|
|
|
Reviewed manual JSON -> original strict UTF-8 bound -> JsonDocument depth/
|
|
syntax/object/type/allowlist checks -> detached source/value contracts ->
|
|
actual authoring snapshot/component matching -> full existing template ->
|
|
pure exact diff -> current-head guarded publication -> copied last success ->
|
|
current versus Stale snapshot -> actual input/error/preview UI.
|
|
|
|
Public entry points reviewed: Validate, CreateInputTemplate, Snapshot and
|
|
route template/Validate/subscription/render/disposal paths. Normal, rejected,
|
|
boundary, source-change/replacement, old-preview, draft/error and unit cases
|
|
were walked against C-01..C-10.
|
|
|
|
All schema/source/template checks precede replacement. Publication is under
|
|
the existing exact source/head guard. Snapshot releases the preview gate
|
|
before reading the authoring gate, avoiding inverse lock order. Captured
|
|
values/source/head/diff remain immutable; later source data only determines
|
|
freshness, never rebasing. No method calls Save, Approve, target/model/store/
|
|
network or arbitrary tool code. UI notifications only render, with separate
|
|
rejected/edited draft and previous successful labels. Rationale is plain safe
|
|
text. The retained state is one preview, not a version/history framework.
|
|
|
|
Review strengthened actual integer syntax, escaped duplicate property,
|
|
invalid Unicode/NaN, supplementary-code-unit, duplicate whole-value-array
|
|
and missing-real-workspace assertions. No production defect required a
|
|
follow-up fix. Affected focused and full build/regressions were rerun to
|
|
105/547 passes. Native used the same unchanged production implementation.
|
|
No known unresolved assigned exit-contract defect remains.
|
|
|
|
## Documentation and preservation
|
|
|
|
Added [runbook schema-v1/preview guide](../docs/runbooks-v1.md); updated
|
|
[README.md](../README.md), [documentation index](../docs/README.md),
|
|
[architecture](../docs/architecture.md), [build/test guide](../docs/build-and-test.md)
|
|
and the directly related accepted-status note in
|
|
[rehearsal contracts](../docs/calibration-rehearsal.md).
|
|
They document exact fields/limits/allowlist, source/head/template/canonical
|
|
rules, defensive one-preview/current-Stale behavior, input versus old result,
|
|
manual/no-model/no-save/no-execution/no-authority scope and actual evidence.
|
|
Corrected Step-10/10A is accepted; Step-11 remains pending manual review.
|
|
Future generated Step-12 text must use this same boundary, not a privileged
|
|
path. Inference/knowledge and executor integration remain unimplemented.
|
|
|
|
Worktree began clean on main at bccb797 and HEAD is unchanged. Existing
|
|
prompts/state/roadmap/assignments/shared gate/reports and protected files were
|
|
not edited; this is the only new prompts file. Index remains empty.
|
|
All nine projects/references/dependencies and existing Core/profile/config/
|
|
Simulation/Diagnostics/authoring/rehearsal/recording/replay/physics contracts
|
|
are unchanged. Source/tests remain under src/src/tests; docs under docs.
|
|
The separately disclosed recording navigation issue was not modified.
|
|
|
|
No new project/package/host/model/runtime/schema change to other features,
|
|
permission/version-control/backup framework, app Git persistence, agent,
|
|
automatic handoff, staging, commit, push, system installation or unrelated
|
|
cleanup occurred.
|
|
|
|
STOP.
|
|
```
|