AutoForge/docs/local-runbook-generation.md

142 lines
7.9 KiB
Markdown

# Local-only runbook generation
## Current verification status
Step-11 is accepted. Step-12 code, controlled HTTP/UI tests and regressions
are implemented, but **required real-model/native success is BLOCKED**.
The existing Ollama 0.40.13 at `http://127.0.0.1:11434` reports `models: []`.
Its current documented startup log says `Ollama cloud disabled: false`.
No eligible installed model/digest/license or cloud-disabled execution proof
exists, so no goal/source context was sent to that runtime.
No model/runtime/configuration was installed, pulled, updated or restarted.
See the [Step-12 worker report](../prompts/Step-12-Report.md).
## Operator prerequisites, not application configuration
Use an already operator-approved runtime and installed local completion model.
The app never starts/stops shared Ollama, signs in, uses API keys, pulls or
changes a model, or changes server settings.
Loopback alone is not proof that Ollama will not delegate to cloud.
Official contracts verified during implementation:
- [POST /api/chat](https://docs.ollama.com/api/chat): explicit model/messages,
`stream:false`, structured JSON format, finished assistant `message.content`.
- [GET /api/tags](https://docs.ollama.com/api/tags): installed name/digest/size,
optional remote_model/remote_host.
- [POST /api/show](https://docs.ollama.com/api-reference/show-model-details):
completion capabilities and local model metadata.
- [Cloud-disabled FAQ](https://docs.ollama.com/faq#how-do-i-disable-ollamas-cloud-features):
operator sets `disable_ollama_cloud` in their `.ollama/server.json` or
`OLLAMA_NO_CLOUD=1`, restarts their runtime deliberately, and verifies the
documented `Ollama cloud disabled: true` log status.
The UI acknowledgement is a deliberate operator statement, not cryptographic
or API attestation. Do not check it as a substitute for verifying runtime
configuration. Model availability/configuration changes require separate
operator action; this assignment performs none.
## Explicit controls and eligibility
Within `/runbooks`, optional generation is separate from ordinary manual JSON.
Entry, source/unit renders and ordinary manual workflows make no inference
request. Explicitly refresh the catalogue, select an exact reported model,
inspect source/context, acknowledge local-only operation, then Generate once.
Missing runtime/model is visible and never replaced by a first-model, stub
or cloud fallback.
Transport is fixed literal HTTP loopback; redirects, proxies, cookies, default
credentials and arbitrary model-provided URLs are disabled/rejected.
Only catalogue/details/chat calls are implemented. Cloud-tagged/remote indicators,
invalid/ambiguous metadata and unsupported local format reject. Eligible
selection requires exact usable name/SHA-256 digest, positive installed size,
local GGUF details/model architecture and completion capability.
Failed refresh retains last-known display without fresh generation authority.
Before each chat, and again after completion, verify that exact catalogue
name/digest/local details. Changed or unavailable metadata rejects output
before preview publication. A response model name alone does not attest digest
or local execution. Additive HTTP metadata is compatible, while duplicate or
invalid critical identity/completion/content/error fields are handled failures.
## Matched source and bounded request
One local `autoforge.synthetic.engine-authoring.policy` revision 1 pack matches
exact `autoforge.synthetic.longitudinal` model 1 and engine-authoring definition
1. Its facts are owned synthetic authoring restrictions, not manufacturer-safe
limits or real-vehicle truth. No nearest/role/RAG/web/file-crawling fallback exists.
Capture exact immutable selected model/digest, actual workspace/head/applicability,
canonical full values, scalar bounds, fixed rpm axis, torque cell bounds/units,
knowledge identity/applicability/facts, explicit goal and prior successful preview
identity. Display these actual facts separately from model rationale/output.
Only explicitly selected bounded synthetic authoring context is included.
Source code, credentials, arbitrary files and raw diagnostic archives are not sent.
Goal is nonblank and at most 2000 UTF-16 code units. Complete UTF-8 request
is at most 131072 bytes. One non-streaming chat uses temperature 0 and
num_predict 4096 with a structured JSON-schema hint for the accepted shape.
This hint and model obedience do not guarantee correctness or determinism;
the unchanged deterministic validator remains authoritative.
Goal/pack/template are quoted data, not a tool or policy instruction channel.
Each actual catalogue/details/chat response is at most 1048576 bytes, checked
while reading even without Content-Length. Parser depth is 64. One-byte-over
or malformed partial JSON never publishes. The complete operation deadline
is 180 seconds, not 180 per unbounded request chain. HTTP/runtime/metadata/
completion/validation/timeout/cancellation failures are distinct and visible.
## Original output and atomic publication
Require the exact requested model, assistant role/string content, done=true and
untruncated stop outcome. Empty/refusal/tool-call/truncated/invalid envelopes
reject. Thinking fields are not content and are not shown as evidence.
Do not strip Markdown, repair JSON, inject source IDs, convert wrong units,
clip values, replace with a template or automatically retry.
Original decoded message.content goes through the same [Step-11 parser,
source/head/template/diff rules](runbooks-v1.md). At publication, the generation
owner/model/cancellation and captured source/head must still match, and the
expected prior successful preview is compared atomically under the validator's
existing publication gate. A newer manual preview cannot be overwritten even
when source head did not change. Source/model changes reject rather than rebase.
Success remains an AI-generated read-only validated preview, not a saved
proposal, tested intent, approval or execution. Exact goal versus candidate/
diff must still be reviewed by a human. Valid in-range JSON alone is not
evidence that the real model followed the decisive requested intent.
Latest request/output and last success are bounded volatile evidence, not a
chat history. Diagnostic unvalidated output is limited to 131072 code units
with an explicit truncation notice; display truncation never repairs content
for validation. Invalid rationale/content is escaped, never executed.
Source staleness continues to apply through the deterministic preview UI.
## Ownership and isolation
One page-owned catalogue/selection/generation owner is active at a time.
Repeated/competing Generate, refresh or selection rejects before another
request or state change. Cancel/Close share drainage and invalidate publication.
No cleanup await holds a required gate; re-entry creates a fresh owner with
no prior opt-in/selection. Only its HTTP work is drained; shared Ollama or
unrelated models are never stopped/unloaded.
Global units only project display. Nothing saves authoring data, approves/
opens/writes a rehearsal target, controls dynamics or changes archive/replay.
Manual/no-model validation remains available if local inference is unavailable.
## Evidence boundaries
Controlled HttpMessageHandler/streams test actual HTTP serialization/parsing/
limits and original-output admission; their JSON is not a real LLM result.
Controlled barriers test source/model/newer-manual-preview/cancel/unmount races.
Real business-source/recording/replay/approved-rehearsal instances retain exact
state and counted zero unassigned mutations.
The required native decisive goal remains NOT EXECUTED/BLOCKED:
`Propose warm-idle target 1250 rpm and set torque at 2000 rpm to 350 N*m; keep every other value unchanged.`
Only a real eligible installed local model with operator-verified cloud disabled
and original exact +600 rpm/-10 N*m validated diff can satisfy that check.
No canned test response substitutes for it. Step-13 remains blocked;
broader research/retrieval/executors/persistence/physical work is unimplemented.