AutoForge/docs/local-runbook-generation.md

180 lines
10 KiB
Markdown

# Local-only runbook generation
## Current verification status
Step-11 is accepted. Step-12A worker execution on 2026-10-31 qualifies the
required actual local-model/native path; **manual acceptance remains pending**.
The original empty-catalogue and three approval-blocked reports are historical,
preserved unchanged. The selected operator-installed model remains
`huihui_ai/qwen3-abliterated:8b-v2`, digest
`543f9deff86d8347fd96ea1f3e54e73861ea70d52def71d31ccaeaf013de6ba6`,
5,027,793,060 installed bytes, GGUF/qwen3/Q4_K_M, completion capability and
reported Apache License 2.0 metadata.
The corrected owner handoff directly authorized scoped cloud-disable setup and
restart, replacing the unavailable approval-form gate. No concrete active
CLI/runner/API job was observed before interruption. The worker created only
`disable_ollama_cloud:true` in the previously absent per-user server config,
restarted the exact existing instance and verified startup
`2026-10-31T14:39:16.200-04:00`, `Ollama cloud disabled: true`.
Version 0.40.13, model/store and both executable hashes remained unchanged.
No machine-wide environment change, download, alternate backend or cloud call.
The actual native Client selected the exact model and matched source/knowledge,
then validated original unmodified assistant content with exactly +600 rpm and
-10 N*m. Final decisive latency was 20.849 seconds. Real pending conflicts and
Cancel/unmount/drain retained the preview and produced a fresh unacknowledged,
unselected owner. Manual invalid/stale input, display-only units, pinned drafts
and unchanged original source/history/configuration/archive checks passed.
The clean nine-project Release build and 179 unit + 417 integration = 596 tests
pass, preserving all 594 prior cases plus two corrective cases.
See the [execution report](../prompts/Step-12A-Execution-Report.md); previous
[original](../prompts/Step-12-Report.md),
[completion](../prompts/Step-12-Completion-Report.md) and
[Step-12A blocked](../prompts/Step-12A-Report.md) reports remain historical.
## Operator prerequisites, not application configuration
Use an already operator-approved runtime and installed local completion model.
The app never starts/stops shared Ollama, signs in, uses API keys, pulls or
changes a model, or changes server settings.
Loopback alone is not proof that Ollama will not delegate to cloud.
Official contracts verified during implementation:
- [POST /api/chat](https://docs.ollama.com/api/chat): explicit model/messages,
`stream:false`, structured JSON format, finished assistant `message.content`.
- [GET /api/tags](https://docs.ollama.com/api/tags): installed name/digest/size,
optional remote_model/remote_host.
- [POST /api/show](https://docs.ollama.com/api-reference/show-model-details):
completion capabilities and local model metadata.
- [Cloud-disabled FAQ](https://docs.ollama.com/faq#how-do-i-disable-ollamas-cloud-features):
operator sets `disable_ollama_cloud` in their `.ollama/server.json` or
`OLLAMA_NO_CLOUD=1`, restarts their runtime deliberately, and verifies the
documented `Ollama cloud disabled: true` log status.
The UI acknowledgement is a deliberate operator statement, not cryptographic
or API attestation. Do not check it as a substitute for verifying runtime
configuration. Model availability/configuration changes require separate
operator direction. This execution's authority was the corrected owner's scoped
cloud-disable/restart direction, not tool settings or the earlier model download.
Inspect concrete active shared work before interruption; no empty-model list
or unavailable checkbox is a universal idle attestation. Prefer normal
tray Quit/relaunch, preserve the existing model store/settings, and verify a new
current startup log saying `Ollama cloud disabled: true` before acknowledgement
or chat. The app still never manages shared runtime setup automatically.
## Explicit controls and eligibility
Within `/runbooks`, optional generation is separate from ordinary manual JSON.
Entry, source/unit renders and ordinary manual workflows make no inference
request. Explicitly refresh the catalogue, select an exact reported model,
inspect source/context, acknowledge local-only operation, then Generate once.
Missing runtime/model is visible and never replaced by a first-model, stub
or cloud fallback.
Transport is fixed literal HTTP loopback; redirects, proxies, cookies, default
credentials and arbitrary model-provided URLs are disabled/rejected.
Only catalogue/details/chat calls are implemented. Cloud-tagged/remote indicators,
invalid/ambiguous metadata and unsupported local format reject. Eligible
selection requires exact usable name/SHA-256 digest, positive installed size,
local GGUF details/model architecture and completion capability.
Failed refresh retains last-known display without fresh generation authority.
Before each chat, and again after completion, verify that exact catalogue
name/digest/local details. Changed or unavailable metadata rejects output
before preview publication. A response model name alone does not attest digest
or local execution. Additive HTTP metadata is compatible, while duplicate or
invalid critical identity/completion/content/error fields are handled failures.
## Matched source and bounded request
One local `autoforge.synthetic.engine-authoring.policy` revision 1 pack matches
exact `autoforge.synthetic.longitudinal` model 1 and engine-authoring definition
1. Its facts are owned synthetic authoring restrictions, not manufacturer-safe
limits or real-vehicle truth. No nearest/role/RAG/web/file-crawling fallback exists.
Capture exact immutable selected model/digest, actual workspace/head/applicability,
canonical full values, scalar bounds, fixed rpm axis, torque cell bounds/units,
knowledge identity/applicability/facts, explicit goal and prior successful preview
identity. Display these actual facts separately from model rationale/output.
Only explicitly selected bounded synthetic authoring context is included.
Source code, credentials, arbitrary files and raw diagnostic archives are not sent.
Goal is nonblank and at most 2000 UTF-16 code units. Complete UTF-8 request
is at most 131072 bytes. One non-streaming chat uses temperature 0 and
num_predict 4096 with a structured JSON-schema hint for the accepted shape.
This hint and model obedience do not guarantee correctness or determinism;
the unchanged deterministic validator remains authoritative.
Goal/pack/template are quoted data, not a tool or policy instruction channel.
Each actual catalogue/details/chat response is at most 1048576 bytes, checked
while reading even without Content-Length. Parser depth is 64. One-byte-over
or malformed partial JSON never publishes. The complete operation deadline
is 180 seconds, not 180 per unbounded request chain. HTTP/runtime/metadata/
completion/validation/timeout/cancellation failures are distinct and visible.
## Original output and atomic publication
Require the exact requested model, assistant role/string content, done=true and
untruncated stop outcome. Empty/refusal/tool-call/truncated/invalid envelopes
reject. Thinking fields are not content and are not shown as evidence.
Do not strip Markdown, repair JSON, inject source IDs, convert wrong units,
clip values, replace with a template or automatically retry.
Original decoded message.content goes through the same [Step-11 parser,
source/head/template/diff rules](runbooks-v1.md). At publication, the generation
owner/model/cancellation and captured source/head must still match, and the
expected prior successful preview is compared atomically under the validator's
existing publication gate. A newer manual preview cannot be overwritten even
when source head did not change. Source/model changes reject rather than rebase.
Success remains an AI-generated read-only validated preview, not a saved
proposal, tested intent, approval or execution. Exact goal versus candidate/
diff must still be reviewed by a human. Valid in-range JSON alone is not
evidence that the real model followed the decisive requested intent.
Latest request/output and last success are bounded volatile evidence, not a
chat history. Diagnostic unvalidated output is limited to 131072 code units
with an explicit truncation notice; display truncation never repairs content
for validation. Invalid rationale/content is escaped, never executed.
Source staleness continues to apply through the deterministic preview UI.
## Ownership and isolation
One page-owned catalogue/selection/generation owner is active at a time.
Hybrid event callbacks return after admission rather than retaining the IPC
event throughout HTTP/drainage; asynchronous outcomes are observed and explicitly
rendered, including errors. After synchronous source/goal/model capture and owner
admission, replaceable transport work runs off the UI dispatcher, including its
potential synchronous prefix. Cancellation still drains that same owner.
Repeated/competing Generate, refresh or selection rejects before another
request or state change. Cancel/Close share drainage and invalidate publication.
No cleanup await holds a required gate; re-entry creates a fresh owner with
no prior opt-in/selection. Only its HTTP work is drained; shared Ollama or
unrelated models are never stopped/unloaded.
Global units only project display. Nothing saves authoring data, approves/
opens/writes a rehearsal target, controls dynamics or changes archive/replay.
Manual/no-model validation remains available if local inference is unavailable.
## Evidence boundaries
Controlled HttpMessageHandler/streams test actual HTTP serialization/parsing/
limits and original-output admission; their JSON is not a real LLM result.
Controlled barriers test source/model/newer-manual-preview/cancel/unmount races.
Real business-source/recording/replay/approved-rehearsal instances retain exact
state and counted zero unassigned mutations.
The qualified native decisive goal was:
`Propose warm-idle target 1250 rpm and set torque at 2000 rpm to 350 N*m; keep every other value unchanged.`
The exact installed model, current cloud-disabled proof and original
+600 rpm/-10 N*m validated diff satisfied that check in the actual Client.
Controlled counters/barriers remain distinct from native output/pending evidence.
Step-12/12A is accepted. The separate Step-13
[verified comparison](calibration-comparison.md) explicitly consumes the current
preview through the existing draft, separate Save and actual rehearsal proof;
generation still cannot perform any of those actions automatically.
Step-13 awaits manual review; Step-14 remains blocked;
broader research/retrieval/executors/persistence/physical work is unimplemented.