AutoForge/prompts/Step-12-Report.md

245 lines
18 KiB
Markdown

```
# Step-12 - Local AI and applicable knowledge integration
## Status
INCOMPLETE / BLOCKED on required real-model/native generation.
The local-only inference/context/owner/UI implementation and controlled HTTP/
validator/lifetime/isolation tests are complete and pass. All nine Release
projects and 594 tests pass, preserving the 547-case baseline.
This is NOT Step-12 completion or acceptance: required actual installed-model
intent/native-generation evidence cannot be produced from current prerequisites.
Live assigned endpoint http://127.0.0.1:11434 reports:
GET /api/version -> {"version":"0.40.13"}
GET /api/tags -> {"models":[]}
The current documented runtime startup log at
C:\Users\g3\AppData\Local\Ollama\server.log reports:
Ollama cloud disabled: false
The documented .ollama/server.json disable setting was not found.
No eligible installed model/digest/license/resource context or cloud-disabled
runtime evidence exists. No goal or synthetic source context was sent to that
shared runtime. No runtime/model/configuration was installed, pulled, updated,
started/restarted/stopped or changed. Operator action is required separately.
Step-13 was not started.
## Implementation and interfaces
- [LocalGenerationContracts.cs](../src/AutoForge.Application/Features/Vehicles/Runbooks/LocalGenerationContracts.cs):
replaceable ILocalInferencePort, typed failures, model summary/selection,
exact applicable synthetic knowledge and immutable captured context/
bounded generation evidence. The explicit unavailable port preserves
portable/no-model compositions.
- [OllamaLocalInferencePort.cs](../src/AutoForge.Application/Features/Vehicles/Runbooks/OllamaLocalInferencePort.cs):
fixed literal-loopback HTTP with redirects/proxies/cookies/credentials
disabled, catalogue/details selection and pre/post-chat model verification,
finite bounded request/body/depth/deadline handling, critical-field
validation and original assistant content only. No pull/create/delete/push/
cloud/search/tool operation exists.
- [LocalRunbookGenerationSession.cs](../src/AutoForge.Application/Features/Vehicles/Runbooks/LocalRunbookGenerationSession.cs):
fresh page factory; explicit acknowledgement/refresh/selection/context/
Generate/Cancel/Close; one owner; captured goal/model/digest/source/head/
knowledge/prior-preview; late-publication and bounded cleanup/error handling.
Unknown task faults are observed as failure, never completed success.
- [RunbookPreviewService.cs](../src/AutoForge.Application/Features/Vehicles/Runbooks/RunbookPreviewService.cs):
minimal shared ValidateCore/ValidateGenerated path and exact captured-source/
expected-prior-preview/cancellation admission under existing source/head
and preview publication gates. Manual parsing/limits/diff behavior is
unchanged. A pure captured-workspace template overload avoids rebinding.
- [LocalGenerationPanel.razor](../src/AutoForge.UI/Features/Vehicles/Runbooks/LocalGenerationPanel.razor)
within [RunbookPage.razor](../src/AutoForge.UI/Features/Vehicles/Runbooks/RunbookPage.razor):
explicit opt-in, actual catalogue/model selection, goal/context,
Generate/Cancel/Close, separate safe original/unvalidated output,
prerequisite warnings and bounded prior-success provenance.
No request on entry/render and no Save/Approve/Execute/Apply control.
- [AddAutoForgeUi.cs](../src/AutoForge.UI/Features/Foundation/AddAutoForgeUi.cs)
and [MauiProgram.cs](../src/AutoForge.Client/MauiProgram.cs):
unavailable portability fallback and actual safe Ollama HTTP composition.
Existing projects/references/dependencies remain unchanged.
Tests:
[LocalRunbookGenerationTests.cs](../src/tests/AutoForge.IntegrationTests/LocalRunbookGenerationTests.cs),
[LocalInferenceBoundsTests.cs](../src/tests/AutoForge.IntegrationTests/LocalInferenceBoundsTests.cs),
[LocalGenerationReviewTests.cs](../src/tests/AutoForge.IntegrationTests/LocalGenerationReviewTests.cs),
[LocalGenerationUiTests.cs](../src/tests/AutoForge.IntegrationTests/LocalGenerationUiTests.cs),
[LocalGenerationIsolationTests.cs](../src/tests/AutoForge.IntegrationTests/LocalGenerationIsolationTests.cs).
Initial local HTTP/model/source/output/limits/owner contracts were written before
production implementation. Post-green review added real rendered controls/
unmount, exact request bound, invalid critical response/fault observation and
real business-source coexistence coverage.
## Verified public/runtime contracts
Rechecked official documentation before implementation:
- https://docs.ollama.com/api/chat
- https://docs.ollama.com/api/tags
- https://docs.ollama.com/api-reference/show-model-details
- https://docs.ollama.com/faq#how-do-i-disable-ollamas-cloud-features
- Documentation index and documented /api/version contract.
Non-streaming chat uses explicit model/messages, structured format, temperature
0 and num_predict 4096. Only original message.content is accepted; thinking
fields are not displayed or interpreted. Model catalogue exposes exact name/
digest/positive size and remote_model/remote_host; details expose local GGUF/
architecture/completion. Additive metadata is tolerated, duplicate/invalid
critical identity/completion/content/error fields reject.
FAQ documents operator disable_ollama_cloud / OLLAMA_NO_CLOUD=1 plus deliberate
runtime restart and log confirmation. Checkbox acknowledgement is not proof;
current log false and missing models block native success. Runtime PID 7804 /
installed ollama.exe was inspected read-only and never terminated/reconfigured.
## C-01..C-10 mapping
| ID | Result | Named evidence and exact outcome |
|---|---|---|
| C-01 | PASSED | ExplicitLocalSelectionAndContext_UseOneOriginalChatAndAuthoritativeExactPreview; LocalGenerationUi_ExplicitOptInSelectionContextAndGeneration_NeverRunsOnRender. Entry produces zero HTTP calls/model selection/context/generation. Explicit catalogue/select/opt-in is required. Manual validator baseline remains usable without a configured/installed model. |
| C-02 | CONTROL PASSED; REAL PREREQUISITE BLOCKED | RemoteCloudOrAmbiguousModel_NeverReceivesGoalOrSource (remote_model/remote_host/cloud-name/no completion/wrong format); NonLoopbackOrCredentialEndpoint_RejectsBeforeHttp; metadata failures. Only exact eligible selection can chat; ineligible paths ChatCount=0 and requests contain no goal. Real runtime has models:[] and cloud disabled:false, so no native eligible selection/chat occurred. |
| C-03 | PASSED IN CONTROL | GoalAndMatchedKnowledge_RejectBeforeChatAndKeepSourceImmutable; exact context test. Captured actual workspace/head/canonical values/constraints, model name/digest and policy pack autoforge.synthetic.engine-authoring.policy revision 1 with exact model/definition applicability. Missing/bad goal and unmatched pack reject before chat; pack facts are read-only and bounded request enforcement is active. |
| C-04 | CONTROL PASSED; REAL-MODEL BLOCKED | ExplicitLocalSelectionAndContext_UseOneOriginalChatAndAuthoritativeExactPreview uses the real HTTP adapter over a controlled handler: exactly one chat, original response reaches Step-11 validator, scalar 1250/cell 350 yield -10 N*m/+600 rpm diff and all other values unchanged. This response is NOT an LLM result. Required real native decisive-goal response is NOT EXECUTED because prerequisites are absent. |
| C-05 | PASSED IN CONTROL | InvalidOriginalResponse_IsNotRepairedAndPreservesPreviousSuccess (10 vectors); InvalidCriticalCompletionFields_AreHandledWithoutEmptySuccess (4 vectors). Markdown, bad source/unit/key, empty/refusal/tool-call/truncated/wrong-model/duplicate critical output reject without repair, rebinding, defaulting or replacing previous preview. Safe unvalidated diagnostic content is limited and labelled. |
| C-06 | PASSED IN CONTROL | HeldGeneration_CannotOverwriteSupersededSourceModelOrManualPreview (manual preview/source head/rewritten new source/model digest). Held response then source/model/newer manual publication causes typed rejection and identical prior/latest manual preview preserved. Generated-source admission is atomic with existing publication, not a precheck followed by unguarded manual Validate. |
| C-07 | CONTROL PASSED; REAL PENDING-CHAT NATIVE BLOCKED | CancelOrClose_DrainsDelayedHttpAndCannotPublishLatePreview; rendered unmount. Repeated Generate/refresh/selection conflict before a second request. Delayed cancellation/close remains pending until handler drainage; no late preview/Ready state, shared Cancel/Close drain and fresh entry no opt-in/model reuse. Native no-model close/re-entry works, but native real in-flight chat cannot be exercised without a model. |
| C-08 | PASSED IN CONTROL | MetadataHttpAndRedirectFailures_AreVisibleAndDoNotBecomeHealthyEmptyCatalogue; ActualLengthlessResponse_IsBoundedAtOneMiBPlusOneDetectionByte; HttpJsonDepth64_AndAdditiveMetadataHaveExplicitBoundary; ExactRequest131072BytesAndOneMore_AreMeasuredBeforeChatDispatch; Controlled180SecondDeadline_CancelsAndDrainsRatherThanPublishing; UnexpectedPortFault_IsObservedAsFailedOwnerAndCannotBecomeReadySuccess. Actual request 131072 succeeds/131073 rejects before chat; body 1048576 succeeds/1048577 rejects after at most one detection byte. Depth 64 succeeds for additive metadata/65 rejects. Deadline is exactly 180 seconds with controlled time, not sleeps. Errors are typed/visible and old source/preview remain intact. |
| C-09 | PASSED IN CONTROL | Rendered explicit opt-in/context/output and original validator tests. Quoted goal/facts/output never become tools/policy/authority. No hidden loop, output repair, Source Save/approval or validation on render. Unit/manual-path baseline tests and context copying preserve canonical data. Rationale remains safe text. |
| C-10 | CODE/CONTROL/REGRESSIONS PASSED; REAL-MODEL/NATIVE SUCCESS BLOCKED | LocalGenerationIsolation_RealBusinessSourcesRemainUntouchedByControlledModelOutput. Actual live diagnostics/dynamics, saved recording/replay and approved rehearsal retain identical source/head/approval/trace/status/config/archive/replay; transport/model/store/target mutation deltas=0. All 547 baselines plus 47 cases pass, all nine projects build. Real local-model/native exact intent check is absent, so this criterion is not complete. |
## Acceptance criteria
| ID | Result | Evidence/blocker |
|---|---|---|
| AC-01 | PARTIAL / BLOCKED | Local-only HTTP/model/context controls pass; actual installed local model and operator cloud-disabled prerequisites are not available. |
| AC-02 | PARTIAL / BLOCKED | Original controlled output uses authoritative validation unchanged and rejects negatives; real decisive-goal LLM/native exact-diff proof is NOT EXECUTED. |
| AC-03 | PASSED IN CONTROL | One owner, atomic source/model/manual-preview publication, finite request/body/depth/deadline, typed errors, cancellation/drain/unmount and real rendered controls. |
| AC-04 | PARTIAL / BLOCKED | 594 tests/nine-project build/zero business-source effects pass. Required real-model/native goal, model digest/license/resource context and pending-real-chat checks cannot run. |
| AC-05 | PASSED | Executable matrix/self-review/docs/history/protection/manual workflow and honest blocked report; no prerequisites changed or substituted. |
## Final commands/results
SDK remains 10.0.403; existing xUnit v2/VSTest is unchanged.
Final focused/build/full tests ran sequentially:
| Exact command | Exit | Outcome |
|---|---:|---|
| dotnet test .\src\tests\AutoForge.IntegrationTests\AutoForge.IntegrationTests.csproj --configuration Release --no-restore --filter "FullyQualifiedName~LocalRunbookGenerationTests|FullyQualifiedName~LocalInferenceBoundsTests|FullyQualifiedName~LocalGenerationReviewTests|FullyQualifiedName~LocalGenerationUi_|FullyQualifiedName~LocalGenerationIsolation_|FullyQualifiedName~RunbookPreviewTests" | 0 | 149 passed: 47 new cases plus 102 manual parser cases; zero failed/skipped. |
| dotnet build .\src\AutoForge.slnx --configuration Release --no-restore | 0 | All nine projects/native Client; 0 warnings/errors; XAML source-generation notice informational. |
| dotnet test .\src\AutoForge.slnx --configuration Release --no-build | 0 | 179 unit + 415 integration = 594 passed; zero failed/skipped. |
| curl.exe --noproxy '*' --connect-timeout 3 --max-time 10 --max-redirs 0 --silent --show-error http://127.0.0.1:11434/api/tags | 0 | Repeated real prerequisite checks returned {"models":[]}; no context or goal supplied. |
| Same bounded read of documented /api/version | 0 | {"version":"0.40.13"}. |
| git --no-pager diff --check | 0 | No whitespace errors; existing README/build-guide LF-to-CRLF notices only. |
Per-project Release: Core, Diagnostics, Simulation, Application, Components,
UI, Client, UnitTests and IntegrationTests all PASSED. Portable TFMs remain
net10.0, Client net10.0-windows10.0.26100.0 / win-x64.
All 547 baseline cases remain unchanged; 47 integration cases added.
No package/SDK/workload/runtime/model install, restore, update or repair occurred.
Iteration evidence: initial local/parser selection passed 140; expanded UI/
review/isolation selection passed 149. A test-file BL0006 suppression was
initially restored before the second renderer test; the directive scope was
corrected and all checks rerun. Those compile errors are not behavioral
reproductions. Final source/optional-string/context-copy refinements were
followed by the final 149/594 validation. No controlled pass is claimed to
prove an actual LLM generated a response.
## Native/prerequisite evidence
Actual rebuilt Client used task-owned WebView2/local DevTools port 9383.
It entered local generation at Idle, catalogue fresh False, acknowledgement
False and selected model empty. Explicit catalogue refresh called the real
runtime and displayed: Runtime reported zero installed models; generation
unavailable, no substitution/download.
The decisive goal was typed into the app, but Generate rejected ModelRequired
before capturing/sending source/context/chat. Acknowledgement deliberately
remained false because operator cloud-disabled evidence is false.
No source or successful preview was fabricated.
Explicit source creation plus ordinary manual input template/Validate still
produced NoChanges/current. Route re-entry created a fresh generation owner
with no selection/acknowledgement; no automatic request. No-active-request
Cancel/Close and old replay/Home routes worked; original Metric/Road preferences
were preserved/restored. No native unhandled error overlay appeared.
NOT EXECUTED/BLOCKED: eligible actual local model selection/digest/license/
resident-resource context, original real model response to the exact requested
1250 rpm/350 N*m goal, validated exact +600/-10 native intent proof and native
in-flight real-chat cancellation/unmount. Cloud-enabled status is not waived
by a checkbox; empty model catalogue is not replaced by a stub/cloud.
The native "runtime unchanged" scope means no runtime/configuration/model
mutation command was issued; no process-wide attestation is claimed. Runtime
metadata probes and test DevTools transport are not validator/model-context
network calls. Controlled peers/streams/barriers are isolated test evidence.
Cleanup: only task Client PID 1512 was terminated after page cleanup; its
WebView/profile/helper and three exact public-doc temporary outputs were removed.
Port 9383 and owned WebView processes were closed. Shared Ollama PID 7804 /
port 11434 was never stopped/restarted/unloaded. Native recordings directory
stayed empty; user/default WebView data/preferences were not reset.
## Own post-green source-path review
Walked explicit catalogue/selection/acknowledgement -> exact source/pack/goal
capture -> safe loopback metadata rechecks -> bounded single chat -> finished
original content -> post-model verification -> unchanged authoritative parser/
source/head/expected-preview/cancellation publication -> bounded evidence/UI ->
Cancel/Close/unmount. Normal, rejected, malformed, superseded, timeout and
delayed-close paths were reviewed against C-01..C-10.
Review fixes/checks:
- Context and request admission occur under the same owner gate; selected
model/capabilities and canonical source/template are copied, not rebased.
- Atomic generated admission extends the existing parser, not a separate
weaker validator; a newer manual preview and repaired-to-new-source output
cannot overwrite captured expectations.
- All completion tasks are observed before owner release/result publication.
Unexpected port faults become failed typed owners, not finally-generated
success. Catalogue failures revoke freshness without empty-success fallback.
- Critical optional/required string kinds and malformed Unicode use handled
metadata errors. Fixed URI/handler settings and bounded actual reads prevent
redirect/proxy/unbounded chunk buffering.
- Goal/knowledge metadata is validated before chat; no source code/archive/
credentials or arbitrary files enter context. No reasoning trace or model
tool call is used as content/evidence.
- Rendered/unmount, exact request/body/deadline and real business-source zero
call/state assertions were added and executed.
Final controlled/build checks pass. The unresolved assigned limitation is the
real runtime/model/cloud-disabled/native-goal prerequisite, not a waived test.
## Documentation and preservation
Added [local generation guide](../docs/local-runbook-generation.md); updated
[README](../README.md), [index](../docs/README.md),
[architecture](../docs/architecture.md), [build guide](../docs/build-and-test.md)
and [manual runbooks](../docs/runbooks-v1.md). They explicitly describe local
opt-in/eligibility/API/prerequisites, cloud delegation caveat, exact matched
context, unmodified authoritative output, atomic publication/cleanup, controlled
versus real evidence and BLOCKED status.
Worktree began clean on main at 92af07d; HEAD unchanged, index empty.
Existing prompts/state/roadmap/assignment/gate/reports and protected files are
unchanged; this is the only new prompts file. Nine projects/references/central
dependencies, Core/Simulation/Diagnostics/authoring/rehearsal/physics/archives/
replay contracts remain unchanged. Only narrow runbook admission/template
reuse, optional UI and safe Client transport composition changed.
No agent, automatic handoff, staging/commit/push, shared runtime/system change,
cloud/public inference, model pull/install, execution/approval/write or unrelated
recording/Git/persistence work occurred.
Precise blocker: an operator must separately provide an eligible installed local
completion model and verify the running Ollama cloud-disabled configuration,
then manually retry this assignment's real-model/native exact-goal checks.
Step-12 is INCOMPLETE/BLOCKED; Step-13 remains blocked.
STOP.
```