245 lines
18 KiB
Markdown
245 lines
18 KiB
Markdown
```
|
|
# Step-12 - Local AI and applicable knowledge integration
|
|
|
|
## Status
|
|
|
|
INCOMPLETE / BLOCKED on required real-model/native generation.
|
|
|
|
The local-only inference/context/owner/UI implementation and controlled HTTP/
|
|
validator/lifetime/isolation tests are complete and pass. All nine Release
|
|
projects and 594 tests pass, preserving the 547-case baseline.
|
|
This is NOT Step-12 completion or acceptance: required actual installed-model
|
|
intent/native-generation evidence cannot be produced from current prerequisites.
|
|
|
|
Live assigned endpoint http://127.0.0.1:11434 reports:
|
|
|
|
GET /api/version -> {"version":"0.40.13"}
|
|
GET /api/tags -> {"models":[]}
|
|
|
|
The current documented runtime startup log at
|
|
C:\Users\g3\AppData\Local\Ollama\server.log reports:
|
|
Ollama cloud disabled: false
|
|
|
|
The documented .ollama/server.json disable setting was not found.
|
|
No eligible installed model/digest/license/resource context or cloud-disabled
|
|
runtime evidence exists. No goal or synthetic source context was sent to that
|
|
shared runtime. No runtime/model/configuration was installed, pulled, updated,
|
|
started/restarted/stopped or changed. Operator action is required separately.
|
|
Step-13 was not started.
|
|
|
|
## Implementation and interfaces
|
|
|
|
- [LocalGenerationContracts.cs](../src/AutoForge.Application/Features/Vehicles/Runbooks/LocalGenerationContracts.cs):
|
|
replaceable ILocalInferencePort, typed failures, model summary/selection,
|
|
exact applicable synthetic knowledge and immutable captured context/
|
|
bounded generation evidence. The explicit unavailable port preserves
|
|
portable/no-model compositions.
|
|
- [OllamaLocalInferencePort.cs](../src/AutoForge.Application/Features/Vehicles/Runbooks/OllamaLocalInferencePort.cs):
|
|
fixed literal-loopback HTTP with redirects/proxies/cookies/credentials
|
|
disabled, catalogue/details selection and pre/post-chat model verification,
|
|
finite bounded request/body/depth/deadline handling, critical-field
|
|
validation and original assistant content only. No pull/create/delete/push/
|
|
cloud/search/tool operation exists.
|
|
- [LocalRunbookGenerationSession.cs](../src/AutoForge.Application/Features/Vehicles/Runbooks/LocalRunbookGenerationSession.cs):
|
|
fresh page factory; explicit acknowledgement/refresh/selection/context/
|
|
Generate/Cancel/Close; one owner; captured goal/model/digest/source/head/
|
|
knowledge/prior-preview; late-publication and bounded cleanup/error handling.
|
|
Unknown task faults are observed as failure, never completed success.
|
|
- [RunbookPreviewService.cs](../src/AutoForge.Application/Features/Vehicles/Runbooks/RunbookPreviewService.cs):
|
|
minimal shared ValidateCore/ValidateGenerated path and exact captured-source/
|
|
expected-prior-preview/cancellation admission under existing source/head
|
|
and preview publication gates. Manual parsing/limits/diff behavior is
|
|
unchanged. A pure captured-workspace template overload avoids rebinding.
|
|
- [LocalGenerationPanel.razor](../src/AutoForge.UI/Features/Vehicles/Runbooks/LocalGenerationPanel.razor)
|
|
within [RunbookPage.razor](../src/AutoForge.UI/Features/Vehicles/Runbooks/RunbookPage.razor):
|
|
explicit opt-in, actual catalogue/model selection, goal/context,
|
|
Generate/Cancel/Close, separate safe original/unvalidated output,
|
|
prerequisite warnings and bounded prior-success provenance.
|
|
No request on entry/render and no Save/Approve/Execute/Apply control.
|
|
- [AddAutoForgeUi.cs](../src/AutoForge.UI/Features/Foundation/AddAutoForgeUi.cs)
|
|
and [MauiProgram.cs](../src/AutoForge.Client/MauiProgram.cs):
|
|
unavailable portability fallback and actual safe Ollama HTTP composition.
|
|
Existing projects/references/dependencies remain unchanged.
|
|
|
|
Tests:
|
|
[LocalRunbookGenerationTests.cs](../src/tests/AutoForge.IntegrationTests/LocalRunbookGenerationTests.cs),
|
|
[LocalInferenceBoundsTests.cs](../src/tests/AutoForge.IntegrationTests/LocalInferenceBoundsTests.cs),
|
|
[LocalGenerationReviewTests.cs](../src/tests/AutoForge.IntegrationTests/LocalGenerationReviewTests.cs),
|
|
[LocalGenerationUiTests.cs](../src/tests/AutoForge.IntegrationTests/LocalGenerationUiTests.cs),
|
|
[LocalGenerationIsolationTests.cs](../src/tests/AutoForge.IntegrationTests/LocalGenerationIsolationTests.cs).
|
|
|
|
Initial local HTTP/model/source/output/limits/owner contracts were written before
|
|
production implementation. Post-green review added real rendered controls/
|
|
unmount, exact request bound, invalid critical response/fault observation and
|
|
real business-source coexistence coverage.
|
|
|
|
## Verified public/runtime contracts
|
|
|
|
Rechecked official documentation before implementation:
|
|
|
|
- https://docs.ollama.com/api/chat
|
|
- https://docs.ollama.com/api/tags
|
|
- https://docs.ollama.com/api-reference/show-model-details
|
|
- https://docs.ollama.com/faq#how-do-i-disable-ollamas-cloud-features
|
|
- Documentation index and documented /api/version contract.
|
|
|
|
Non-streaming chat uses explicit model/messages, structured format, temperature
|
|
0 and num_predict 4096. Only original message.content is accepted; thinking
|
|
fields are not displayed or interpreted. Model catalogue exposes exact name/
|
|
digest/positive size and remote_model/remote_host; details expose local GGUF/
|
|
architecture/completion. Additive metadata is tolerated, duplicate/invalid
|
|
critical identity/completion/content/error fields reject.
|
|
|
|
FAQ documents operator disable_ollama_cloud / OLLAMA_NO_CLOUD=1 plus deliberate
|
|
runtime restart and log confirmation. Checkbox acknowledgement is not proof;
|
|
current log false and missing models block native success. Runtime PID 7804 /
|
|
installed ollama.exe was inspected read-only and never terminated/reconfigured.
|
|
|
|
## C-01..C-10 mapping
|
|
|
|
| ID | Result | Named evidence and exact outcome |
|
|
|---|---|---|
|
|
| C-01 | PASSED | ExplicitLocalSelectionAndContext_UseOneOriginalChatAndAuthoritativeExactPreview; LocalGenerationUi_ExplicitOptInSelectionContextAndGeneration_NeverRunsOnRender. Entry produces zero HTTP calls/model selection/context/generation. Explicit catalogue/select/opt-in is required. Manual validator baseline remains usable without a configured/installed model. |
|
|
| C-02 | CONTROL PASSED; REAL PREREQUISITE BLOCKED | RemoteCloudOrAmbiguousModel_NeverReceivesGoalOrSource (remote_model/remote_host/cloud-name/no completion/wrong format); NonLoopbackOrCredentialEndpoint_RejectsBeforeHttp; metadata failures. Only exact eligible selection can chat; ineligible paths ChatCount=0 and requests contain no goal. Real runtime has models:[] and cloud disabled:false, so no native eligible selection/chat occurred. |
|
|
| C-03 | PASSED IN CONTROL | GoalAndMatchedKnowledge_RejectBeforeChatAndKeepSourceImmutable; exact context test. Captured actual workspace/head/canonical values/constraints, model name/digest and policy pack autoforge.synthetic.engine-authoring.policy revision 1 with exact model/definition applicability. Missing/bad goal and unmatched pack reject before chat; pack facts are read-only and bounded request enforcement is active. |
|
|
| C-04 | CONTROL PASSED; REAL-MODEL BLOCKED | ExplicitLocalSelectionAndContext_UseOneOriginalChatAndAuthoritativeExactPreview uses the real HTTP adapter over a controlled handler: exactly one chat, original response reaches Step-11 validator, scalar 1250/cell 350 yield -10 N*m/+600 rpm diff and all other values unchanged. This response is NOT an LLM result. Required real native decisive-goal response is NOT EXECUTED because prerequisites are absent. |
|
|
| C-05 | PASSED IN CONTROL | InvalidOriginalResponse_IsNotRepairedAndPreservesPreviousSuccess (10 vectors); InvalidCriticalCompletionFields_AreHandledWithoutEmptySuccess (4 vectors). Markdown, bad source/unit/key, empty/refusal/tool-call/truncated/wrong-model/duplicate critical output reject without repair, rebinding, defaulting or replacing previous preview. Safe unvalidated diagnostic content is limited and labelled. |
|
|
| C-06 | PASSED IN CONTROL | HeldGeneration_CannotOverwriteSupersededSourceModelOrManualPreview (manual preview/source head/rewritten new source/model digest). Held response then source/model/newer manual publication causes typed rejection and identical prior/latest manual preview preserved. Generated-source admission is atomic with existing publication, not a precheck followed by unguarded manual Validate. |
|
|
| C-07 | CONTROL PASSED; REAL PENDING-CHAT NATIVE BLOCKED | CancelOrClose_DrainsDelayedHttpAndCannotPublishLatePreview; rendered unmount. Repeated Generate/refresh/selection conflict before a second request. Delayed cancellation/close remains pending until handler drainage; no late preview/Ready state, shared Cancel/Close drain and fresh entry no opt-in/model reuse. Native no-model close/re-entry works, but native real in-flight chat cannot be exercised without a model. |
|
|
| C-08 | PASSED IN CONTROL | MetadataHttpAndRedirectFailures_AreVisibleAndDoNotBecomeHealthyEmptyCatalogue; ActualLengthlessResponse_IsBoundedAtOneMiBPlusOneDetectionByte; HttpJsonDepth64_AndAdditiveMetadataHaveExplicitBoundary; ExactRequest131072BytesAndOneMore_AreMeasuredBeforeChatDispatch; Controlled180SecondDeadline_CancelsAndDrainsRatherThanPublishing; UnexpectedPortFault_IsObservedAsFailedOwnerAndCannotBecomeReadySuccess. Actual request 131072 succeeds/131073 rejects before chat; body 1048576 succeeds/1048577 rejects after at most one detection byte. Depth 64 succeeds for additive metadata/65 rejects. Deadline is exactly 180 seconds with controlled time, not sleeps. Errors are typed/visible and old source/preview remain intact. |
|
|
| C-09 | PASSED IN CONTROL | Rendered explicit opt-in/context/output and original validator tests. Quoted goal/facts/output never become tools/policy/authority. No hidden loop, output repair, Source Save/approval or validation on render. Unit/manual-path baseline tests and context copying preserve canonical data. Rationale remains safe text. |
|
|
| C-10 | CODE/CONTROL/REGRESSIONS PASSED; REAL-MODEL/NATIVE SUCCESS BLOCKED | LocalGenerationIsolation_RealBusinessSourcesRemainUntouchedByControlledModelOutput. Actual live diagnostics/dynamics, saved recording/replay and approved rehearsal retain identical source/head/approval/trace/status/config/archive/replay; transport/model/store/target mutation deltas=0. All 547 baselines plus 47 cases pass, all nine projects build. Real local-model/native exact intent check is absent, so this criterion is not complete. |
|
|
|
|
## Acceptance criteria
|
|
|
|
| ID | Result | Evidence/blocker |
|
|
|---|---|---|
|
|
| AC-01 | PARTIAL / BLOCKED | Local-only HTTP/model/context controls pass; actual installed local model and operator cloud-disabled prerequisites are not available. |
|
|
| AC-02 | PARTIAL / BLOCKED | Original controlled output uses authoritative validation unchanged and rejects negatives; real decisive-goal LLM/native exact-diff proof is NOT EXECUTED. |
|
|
| AC-03 | PASSED IN CONTROL | One owner, atomic source/model/manual-preview publication, finite request/body/depth/deadline, typed errors, cancellation/drain/unmount and real rendered controls. |
|
|
| AC-04 | PARTIAL / BLOCKED | 594 tests/nine-project build/zero business-source effects pass. Required real-model/native goal, model digest/license/resource context and pending-real-chat checks cannot run. |
|
|
| AC-05 | PASSED | Executable matrix/self-review/docs/history/protection/manual workflow and honest blocked report; no prerequisites changed or substituted. |
|
|
|
|
## Final commands/results
|
|
|
|
SDK remains 10.0.403; existing xUnit v2/VSTest is unchanged.
|
|
Final focused/build/full tests ran sequentially:
|
|
|
|
| Exact command | Exit | Outcome |
|
|
|---|---:|---|
|
|
| dotnet test .\src\tests\AutoForge.IntegrationTests\AutoForge.IntegrationTests.csproj --configuration Release --no-restore --filter "FullyQualifiedName~LocalRunbookGenerationTests|FullyQualifiedName~LocalInferenceBoundsTests|FullyQualifiedName~LocalGenerationReviewTests|FullyQualifiedName~LocalGenerationUi_|FullyQualifiedName~LocalGenerationIsolation_|FullyQualifiedName~RunbookPreviewTests" | 0 | 149 passed: 47 new cases plus 102 manual parser cases; zero failed/skipped. |
|
|
| dotnet build .\src\AutoForge.slnx --configuration Release --no-restore | 0 | All nine projects/native Client; 0 warnings/errors; XAML source-generation notice informational. |
|
|
| dotnet test .\src\AutoForge.slnx --configuration Release --no-build | 0 | 179 unit + 415 integration = 594 passed; zero failed/skipped. |
|
|
| curl.exe --noproxy '*' --connect-timeout 3 --max-time 10 --max-redirs 0 --silent --show-error http://127.0.0.1:11434/api/tags | 0 | Repeated real prerequisite checks returned {"models":[]}; no context or goal supplied. |
|
|
| Same bounded read of documented /api/version | 0 | {"version":"0.40.13"}. |
|
|
| git --no-pager diff --check | 0 | No whitespace errors; existing README/build-guide LF-to-CRLF notices only. |
|
|
|
|
Per-project Release: Core, Diagnostics, Simulation, Application, Components,
|
|
UI, Client, UnitTests and IntegrationTests all PASSED. Portable TFMs remain
|
|
net10.0, Client net10.0-windows10.0.26100.0 / win-x64.
|
|
All 547 baseline cases remain unchanged; 47 integration cases added.
|
|
No package/SDK/workload/runtime/model install, restore, update or repair occurred.
|
|
|
|
Iteration evidence: initial local/parser selection passed 140; expanded UI/
|
|
review/isolation selection passed 149. A test-file BL0006 suppression was
|
|
initially restored before the second renderer test; the directive scope was
|
|
corrected and all checks rerun. Those compile errors are not behavioral
|
|
reproductions. Final source/optional-string/context-copy refinements were
|
|
followed by the final 149/594 validation. No controlled pass is claimed to
|
|
prove an actual LLM generated a response.
|
|
|
|
## Native/prerequisite evidence
|
|
|
|
Actual rebuilt Client used task-owned WebView2/local DevTools port 9383.
|
|
It entered local generation at Idle, catalogue fresh False, acknowledgement
|
|
False and selected model empty. Explicit catalogue refresh called the real
|
|
runtime and displayed: Runtime reported zero installed models; generation
|
|
unavailable, no substitution/download.
|
|
|
|
The decisive goal was typed into the app, but Generate rejected ModelRequired
|
|
before capturing/sending source/context/chat. Acknowledgement deliberately
|
|
remained false because operator cloud-disabled evidence is false.
|
|
No source or successful preview was fabricated.
|
|
|
|
Explicit source creation plus ordinary manual input template/Validate still
|
|
produced NoChanges/current. Route re-entry created a fresh generation owner
|
|
with no selection/acknowledgement; no automatic request. No-active-request
|
|
Cancel/Close and old replay/Home routes worked; original Metric/Road preferences
|
|
were preserved/restored. No native unhandled error overlay appeared.
|
|
|
|
NOT EXECUTED/BLOCKED: eligible actual local model selection/digest/license/
|
|
resident-resource context, original real model response to the exact requested
|
|
1250 rpm/350 N*m goal, validated exact +600/-10 native intent proof and native
|
|
in-flight real-chat cancellation/unmount. Cloud-enabled status is not waived
|
|
by a checkbox; empty model catalogue is not replaced by a stub/cloud.
|
|
|
|
The native "runtime unchanged" scope means no runtime/configuration/model
|
|
mutation command was issued; no process-wide attestation is claimed. Runtime
|
|
metadata probes and test DevTools transport are not validator/model-context
|
|
network calls. Controlled peers/streams/barriers are isolated test evidence.
|
|
|
|
Cleanup: only task Client PID 1512 was terminated after page cleanup; its
|
|
WebView/profile/helper and three exact public-doc temporary outputs were removed.
|
|
Port 9383 and owned WebView processes were closed. Shared Ollama PID 7804 /
|
|
port 11434 was never stopped/restarted/unloaded. Native recordings directory
|
|
stayed empty; user/default WebView data/preferences were not reset.
|
|
|
|
## Own post-green source-path review
|
|
|
|
Walked explicit catalogue/selection/acknowledgement -> exact source/pack/goal
|
|
capture -> safe loopback metadata rechecks -> bounded single chat -> finished
|
|
original content -> post-model verification -> unchanged authoritative parser/
|
|
source/head/expected-preview/cancellation publication -> bounded evidence/UI ->
|
|
Cancel/Close/unmount. Normal, rejected, malformed, superseded, timeout and
|
|
delayed-close paths were reviewed against C-01..C-10.
|
|
|
|
Review fixes/checks:
|
|
|
|
- Context and request admission occur under the same owner gate; selected
|
|
model/capabilities and canonical source/template are copied, not rebased.
|
|
- Atomic generated admission extends the existing parser, not a separate
|
|
weaker validator; a newer manual preview and repaired-to-new-source output
|
|
cannot overwrite captured expectations.
|
|
- All completion tasks are observed before owner release/result publication.
|
|
Unexpected port faults become failed typed owners, not finally-generated
|
|
success. Catalogue failures revoke freshness without empty-success fallback.
|
|
- Critical optional/required string kinds and malformed Unicode use handled
|
|
metadata errors. Fixed URI/handler settings and bounded actual reads prevent
|
|
redirect/proxy/unbounded chunk buffering.
|
|
- Goal/knowledge metadata is validated before chat; no source code/archive/
|
|
credentials or arbitrary files enter context. No reasoning trace or model
|
|
tool call is used as content/evidence.
|
|
- Rendered/unmount, exact request/body/deadline and real business-source zero
|
|
call/state assertions were added and executed.
|
|
|
|
Final controlled/build checks pass. The unresolved assigned limitation is the
|
|
real runtime/model/cloud-disabled/native-goal prerequisite, not a waived test.
|
|
|
|
## Documentation and preservation
|
|
|
|
Added [local generation guide](../docs/local-runbook-generation.md); updated
|
|
[README](../README.md), [index](../docs/README.md),
|
|
[architecture](../docs/architecture.md), [build guide](../docs/build-and-test.md)
|
|
and [manual runbooks](../docs/runbooks-v1.md). They explicitly describe local
|
|
opt-in/eligibility/API/prerequisites, cloud delegation caveat, exact matched
|
|
context, unmodified authoritative output, atomic publication/cleanup, controlled
|
|
versus real evidence and BLOCKED status.
|
|
|
|
Worktree began clean on main at 92af07d; HEAD unchanged, index empty.
|
|
Existing prompts/state/roadmap/assignment/gate/reports and protected files are
|
|
unchanged; this is the only new prompts file. Nine projects/references/central
|
|
dependencies, Core/Simulation/Diagnostics/authoring/rehearsal/physics/archives/
|
|
replay contracts remain unchanged. Only narrow runbook admission/template
|
|
reuse, optional UI and safe Client transport composition changed.
|
|
No agent, automatic handoff, staging/commit/push, shared runtime/system change,
|
|
cloud/public inference, model pull/install, execution/approval/write or unrelated
|
|
recording/Git/persistence work occurred.
|
|
|
|
Precise blocker: an operator must separately provide an eligible installed local
|
|
completion model and verify the running Ollama cloud-disabled configuration,
|
|
then manually retry this assignment's real-model/native exact-goal checks.
|
|
Step-12 is INCOMPLETE/BLOCKED; Step-13 remains blocked.
|
|
|
|
STOP.
|
|
```
|