AutoForge/docs/telemetry-archive-v1.md

216 lines
7.9 KiB
Markdown

# Telemetry archive schema v1 and recording
## Scope and trust boundary
Recording captures validated diagnostic operations from one explicitly opened
synthetic session. It does not read the bounded 200-entry UI trace, poll a
dashboard, issue commands, step dynamics or decode guessed payloads. Captures
are held in memory until the user stops and explicitly saves them. The saved
file is local and source-sensitive; it is not a trusted statement about a real
vehicle, manufacturer data, current vehicle state or cryptographic authenticity.
Playback, live disk streaming, unlimited retention and crash recovery are not
implemented.
The shared panel is available in both `/diagnostics` (synthetic protocol v1)
and `/simulation` (synthetic protocol v2). Hiding or unmounting a dashboard or
recording panel does not own the capture or cause additional reads. Display-unit
changes affect presentation only; archives preserve source units and source
values.
## Capture lifecycle and clocks
Start attaches to the explicit session's actual-operation observation port.
Earlier trace history is excluded; Start and Stop do not operate the source.
Each observation is published after its validated trace entry is committed,
with that operation's immutable raw trace fields and, only for a successful
`read.signals`, the decoded snapshot from that same response. Exchange ordering
defines capture sequence; model time is not used to sort or group operations.
Stop establishes a cutover by detaching the observer through the session's
serialized observation dispatcher. Callbacks already in dispatch drain before
Stop freezes the result; operations dispatched after detachment are excluded.
Repeated Start on the active owner is idempotent. A second owner for that
session is rejected. Session close/reset stops and detaches the old capture but
retains its source identity and samples for inspection and saving. A new source
does not inherit old samples. Unsaved capture replacement requires explicit
Save or Discard.
The archive keeps three independent time values:
- `captureElapsedTicks`: nonnegative, nondecreasing `TimeSpan` ticks since
Start, from `TimeProvider`.
- `startLogicalTicks` / `endLogicalTicks`: diagnostic protocol logical ticks.
- Per-sample `logicalTimeTicks`, and when present `modelStepIndex` /
`modelTimeSeconds`: the original source/model context. A paused model may
repeat model time while capture elapsed time advances.
## Fixed limits and partial captures
All sizes use compact UTF-8 JSON encoding and binary MiB (1,048,576 bytes).
| Limit | Maximum |
|---|---:|
| Captured operations | 4,096 |
| Sum of encoded observation objects | 16,777,216 bytes |
| Encoded header object | 1,048,576 bytes |
| Complete archive input/output | 33,554,432 bytes |
| JSON parser depth | 128 |
At a count or observation-byte limit, the next operation is not retained; the
valid prefix is frozen with `state: "incomplete"` and
`stopReason: "limitExceeded"`. The diagnostic source operation still returns
its original result. No older record is silently discarded and no unknown
later-loss count is inferred. The UI offers a distinctly labeled **Save partial
capture** action; ordinary Save is rejected for incomplete data. Internal
capture errors use an explicit error state rather than a healthy completion.
Oversized headers and archive inputs fail validation before publishing state.
## JSON schema
The root object has exactly these required camelCase properties:
```json
{
"schemaVersion": 1,
"header": {},
"observations": [],
"completion": {}
}
```
Unknown and duplicate schema-object properties are rejected. Nullable metadata
is explicit `null`; it is not inferred from a current profile, session or
configuration.
`header` contains exactly:
```text
captureId
sessionIdentifier
profileIdentifier
profileRevision
vehicleIdentifier
providerIdentifier
targetKind
protocolName
protocolVersion
dynamicsConfigurationIdentifier
dynamicsRunIdentifier
sourceEndpoints
startedAtUtc
```
`captureId` is a lowercase opaque GUID in `N` format and is the only filename
input. `targetKind` uses the exact camelCase enum token. `protocolName` is
`autoforge.synthetic-diagnostics`; `protocolVersion` is 1 or 2. Static v1 has
null dynamics identifiers. Dynamic v2 carries both the original configuration
and run identifiers. `sourceEndpoints` is an array of `{ "moduleIdentifier",
"address" }` identities. UTC timestamps use the complete round-trip `"O"`
format with an explicit zero offset.
Every `observations` entry contains exactly:
```text
sequence
captureElapsedTicks
moduleIdentifier
address
correlationIdentifier
service
startLogicalTicks
endLogicalTicks
outcome
decodedSummary
error
requestPayloadBase64
responsePayloadBase64
signalSnapshot
```
`sequence` starts at 1 and is contiguous. Raw payloads are canonical base64;
the nullable response is null only when no response bytes were received.
Outcomes are the defined diagnostic outcome tokens. Positive and negative
responses, and recorded protocol errors, require response bytes; timeout,
no-response and cancellation outcomes prohibit them. The raw synthetic request
and any well-formed positive/negative response are checked against the recorded
session, profile, configuration, correlation, endpoint, service, protocol
version and outcome.
`signalSnapshot` is null except for a successful `read.signals`. It contains:
```text
moduleIdentifier
signals
modelStepIndex
modelTimeSeconds
modelConfigurationIdentifier
modelRunIdentifier
```
Each `signals` item contains:
```text
signalIdentifier
moduleIdentifier
value
unit
quality
logicalTimeTicks
origin
modelStepIndex
modelTimeSeconds
modelConfigurationIdentifier
modelRunIdentifier
```
`value` remains a JSON value with its original JSON kind and data; arbitrary
properties inside that value are not interpreted as archive schema. Unit,
quality, origin, diagnostic time and optional model/run context are source
metadata, not display-unit projections.
`completion` contains exactly:
```text
stoppedAtUtc
stopReason
state
observationCount
observationBytes
error
```
`observationCount` and `observationBytes` must match the validated list and its
actual compact encoding. A stopped healthy capture uses `state: "complete"`
and stop reason `"user"` or `"sourceClosed"`. Limit exhaustion is
`"incomplete"` / `"limitExceeded"`. A capture implementation error is
`"error"` / `"observationFailure"` with a visible error message.
## Local persistence and failure handling
The Client composes `FileTelemetryArchiveStore` under
`FileSystem.AppDataDirectory/recordings`. Save is available only after Stop.
The store validates and serializes the whole capture, writes a capture-specific
temporary file, flushes it to disk, closes it, then publishes using a
no-overwrite move. A collision, cancellation, or write failure cannot replace
an existing archive. Temporary-file cleanup failures are surfaced as typed
write failures with an explicit instruction to inspect the named temporary
file; they are not silently hidden.
List and Load enforce the 32 MiB input bound and validate the full schema.
Corrupt matching files, unsupported versions, malformed/truncated JSON,
unknown or duplicate schema properties, bad base64, inconsistent counts,
times, outcomes or source context are visible typed failures. Invalid data does
not replace an existing inspection. Valid incomplete archives remain
incomplete on readback. Readback proves file/schema consistency only; it does
not validate authenticity or source truth.
## Verification
The focused `TelemetryRecordingIntegrationTests` exercise real v1/v2 sessions,
more than 200 operations, all observed outcomes, controlled capture time,
ownership/Stop draining, session reset, exact count/observation/header/archive
boundaries, filesystem round-trip, collision/cancellation/I/O failures,
corrupt-file rejection and rendered controls in both routes. The complete
Release test totals, native-build status and manual-review limitations are
recorded in the [Step-07 worker report](../prompts/Step-07-Report.md).