Cubase-MCP-GAJ3T.MusicGen.R.../docs/architecture.md

1106 lines
71 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Architecture overview
The amended approved [implementation plan](../prompts/plan.md) defines the
authoritative contracts and sequencing. The native/MIDI MCP service is operational
on Windows with real installed-wheel stdio acceptance. Target defaults are the
AME 4.0.1 / AMR 4.1.0 Riffer normal/palm-mute subsets and SD3 3.4.4 SL-DFH
core three-voice subset. SD3 native writing needs an operator-local template.
Every actual saved native/MIDI file is validated before one atomic publication.
Application open/play/save/reload and Cubase acceptance remain unrun.
`agp`/`sdp` are request aliases; actual metadata uses `griff`/`briff`/`sd3p`.
Step-numbered sections retain the original layer acceptance history; the
current [native service report](../prompts/report_9_fix.md),
[Riffer codec](formats/agp.md) and [profiles](articulation-profiles.md#riffer-subset-policy-v1)
supersede historical default/capability statements in earlier step sections.
See the
[Step 10 evidence](../prompts/report_10.md) for separate outcomes.
Current release **0.2.0** includes the accepted Step14 pitch/voicing and
corrected Step15 public hybrid layers below. `__version__` is the sole package
version source; setuptools metadata and MCP initialization derive from it.
The final persisted wheel/sdist, exact original E2 prompt, source/installed
numeric acceptance and new user-output publication are recorded in
[report16](../prompts/report_16.md). Earlier reports/artifacts remain historical.
Restart the MCP after updating, then open a fresh chat for current schemas;
the existing configured venv needs no config edit or config reinstall.
Bounded local deterministic prose is not arbitrary language understanding.
Events `/` and backslash are reserved slides, not implemented: origin/target/
timing codec evidence is still required. Chord slash/Over bass is implemented,
not a slide. Exact explicit pitches are never secretly retuned/revoiced to
the key or to rescue an impossible native fingering. Live playback is unrun.
## Implemented in Step 1
- A `src`-layout Python package, standard setuptools build backend, one literal
public version, and metadata derived from that version.
- An import-only initializer with no service startup, logging, dependency
imports, filesystem writes, or transport initialization.
- pytest unit tests and offline sdist/wheel integration tests, Ruff, and strict
mypy. AnyIO's pytest plugin provides SDK-aligned asynchronous test support.
- Package-data inclusion for six versioned profile JSON resources (three target
defaults and three explicit standard fallbacks).
- An evidence inventory; **no target instrument is certified**.
## Implemented in Step 2
`riff_groove_builder.schemas` is the authoritative Pydantic v2 contract;
`riff_groove_builder.errors` supplies explicit domain exceptions and stable
public codes. Neither imports MCP, starts a server, inspects files, or loads a
profile registry. The initializer remains unchanged and dependency-free.
### Tool boundary and reuse
- `ListInstrumentProfilesRequest` accepts exactly `{}`.
`ProfileListResult` contains three target defaults and three explicit standard fallbacks;
the contract also permits an honestly empty registry in isolated tests.
`ProfileSummary` fixes Eclipse/griff to lead, Ray5/briff to bass, and SD3/sd3p to
drums. Unavailable MIDI or native capabilities require explicit limitations.
IDs are opaque, nonblank strings, not evidence that a mapping exists.
- `GenerateMusicRequest` implements Section 7's parameter names and limits:
nonblank 1–4000-character descriptions; 1–64-character safe basenames;
unique, nonempty selected parts/formats; finite numeric tempo 20–300 BPM
(including integer/fractional BPM); 1–256 integer bars; bounded meter;
key grammar below; per-part profile IDs; seed 0–2147483647 (default 0);
and a strict fallback boolean (default true).
- Optional `cc_events` defaults empty. Each `ScheduledControl` contains exactly
selected `part`, exact `RationalBeat` relative to musical start, strict MIDI
`controller` and `value` integers 0–127. Normalized duplicate points, sub-tick
beats, out-of-duration points and unselected parts are invalid; part association
is rechecked after description inference. The post-compilation
`apply_scheduled_controls` helper preserves original note/switch events and
timing, appends unique sequence keys on the profile channel, sorts controls
before note-ons, and shares the existing 100,000-event budget. Articulation CC
collisions fail with actionable `INVALID_REQUEST`; no implicit resets or
vendor-effect inference. See the [concrete CC example](../README.md#optional-exact-controller-points).
Empty controls return the original compiled plan unchanged. Native preflight
distinguishes supported Riffer CC1 stairs from unsupported controllers and
SD3's precise `sd3_native_controller_not_supported` limitation. Note/channel/
role/release corruption is checked first and is never controller fallback.
- Unspecified `parts`, `formats`, tempo, bars, meter, key, and profiles stay
`None`; explicit null on those optional fields also means unresolved.
`model_fields_set` distinguishes omission from explicit null.
Serialize requests with `model_dump_json(exclude_unset=True)` to preserve
this distinction. `ProfileSelection.selected()` excludes unresolved entries.
Empty profile selections are valid; profile-ID values, when present, are
nonblank. No defaults, text interpretation, or production profiles are chosen.
- Explicit parts constrain profile selection and applicable formats immediately:
agp needs lead/bass, sdp needs drums; actual griff/briff/sd3p suffix requests
require lead/bass/drums respectively. If parts are unresolved, these checks
must run again after text/default resolution. Registry lookup, profile-ID/part
identity, supported vocabulary, text contradictions, and verified articulation
availability necessarily belong to later services.
- `tool_input_schema(name)` and `tool_output_schema(name)` return fresh schema
dictionaries for **exactly** `list_instrument_profiles` and `generate_music`.
Step 3 reuses these and the models rather than duplicating their fields.
Both output schemas have an object root and explicit mutually exclusive
`oneOf` success/error branches. `GENERATION_OUTPUT` and `PROFILE_LIST_OUTPUT`
are reusable typed union adapters for validation and JSON round trips.
All contract objects reject unknown fields. Numeric inputs reject booleans,
numeric strings, nonfinite numbers, and fractional/floating representations of
integer-only fields; meter's integer literals also receive an explicit strict
type check. Lists of enums and declared unique strings reject duplicates.
Empty arrays are rejected where selection/content is required; empty registry,
warnings, assumptions, overrides, supported-articulation lists, and note/event
tracks remain meaningful (including rests or unavailable capabilities).
Basenames use `[A-Za-z0-9][A-Za-z0-9_-]{0,63}` and reject case-insensitive
CON, PRN, AUX, NUL, COM1–COM9, and LPT1–LPT9. Separators, traversal, drive
prefixes, extensions, trailing dots/spaces/newlines, and non-ASCII characters
cannot pass. This is lexical validation, not output-root or link confinement.
Key grammar is one A–G pitch-class letter (case-insensitive), optionally one
ASCII `#` or lowercase `b`, one or more ASCII spaces, then `major` or `minor`
(case-insensitive). Structured key inputs also accept an attached lowercase `m`
minor suffix: `Em`, `F#m`, `Bbm`. Optional outer ASCII spaces are accepted. Examples:
`Bb major` -> `A# major`, `db MINOR` -> `C# minor`, `Cb minor` -> `B minor`.
Normalization uses the twelve sharp-spelled pitch classes and lowercase mode;
it retains pitch-class meaning, not original enharmonic spelling. Tabs,
newlines, Unicode accidentals, double accidentals, bare letters/other modes, and
multiple or contradictory keys are rejected. This is key-string parsing only,
not description interpretation.
### Internal structures and units
`MusicalIntent` requires resolved description, parts, tempo, bars, meter, key,
and seed; rhythm terms/techniques are unique nonblank names, not an implemented
text vocabulary. It computes exact duration as
`bars * numerator * 4 / denominator` quarter-note beats using `Fraction`.
`RationalBeat` serializes as `{"numerator": 1, "denominator": 3}` with strict
nonnegative integer numerator and positive integer denominator. Ratios reduce
exactly (2/6 -> 1/3, 0/n -> 0/1); `.value` returns `fractions.Fraction`.
`PositiveBeat` additionally rejects zero. Floats, decimal approximations, and
string fractions are not timing inputs. Sub-tick logical durations survive
serialization exactly; later quantization must explicitly accept/reject them.
`LogicalNote` is a discriminated union: `PitchedNote(kind="pitched")` has lead
or bass plus numeric pitch, while `DrumNote(kind="drum")` has drums plus a
logical voice name. Both have exact start/duration, normalized finite intensity
0–1 (inclusive, never clamped), and a nonblank articulation name. Numeric
pitches 0–127 only establish MIDI's logical envelope, not an instrument's
playable range. Voice/articulation names are not verified maps.
`Score` retains intent, rational shared start/end, and exactly one track per
selected part in selected-part order. Positive duration equals bars/meter;
every note belongs to its track and fits the shared interval. Empty note lists
allow rests. These contracts do not compose notes or modify random state;
Step 6 supplies the separate composer.
`ScheduledEvent` is a discriminated union of `NoteOnEvent`, `NoteOffEvent`,
and `ControlChangeEvent`, not a bag of optional payloads:
| Quantity | Representation/range |
| --- | --- |
| Absolute event tick, priority, sequence | strict nonnegative integers |
| MIDI channel | 0–15, not 1–16 |
| Note number, controller number/value | 0–127 |
| Note-on velocity (including keyswitches) | 1–127 |
| Note-off velocity | 0–127; default 0 |
| Note role | `musical_note` or `keyswitch` |
| CC role | `articulation_control` |
The stable ordering key is `(tick, priority, sequence)`; keys must be unique
within each part track. Priority is supplied metadata, **not** a scheduler or
a claim that arbitrary numeric priorities satisfy profile-specific causality.
`EventPlan` retains the logical score and one event track per selected part,
positive ticks-per-quarter (default 960), common nonnegative musical start/end,
and a shared overall end allowing control-release tails. Musical tick duration
must exactly match the score; all events fit the overall end, musical events
fit the musical interval, and musical note-ons precede its end. Keyswitches and
controls may occupy pre-roll/tails. Plans reject more than 100,000 events.
Step 4 now implements exact grid checks, sorting, pair/retrigger validation,
channel independence, and event writing. Compiling profile-specific controls,
verified lead times, and articulation state transitions remain Step 5.
### Results, errors, and validation limits
`GenerationResult` contains the planned status, resolved metadata, nonempty
artifacts, and warnings. `ResolvedGeneration` requires exact selected-part
profile coverage, unique profile IDs, and end tick >= start tick. Artifact
size is a positive integer; parts/profile IDs are nonempty and unique. Paths
must be lexically absolute Windows or POSIX local-file paths (without nulls or
line breaks); they are not resolved or checked on disk. Native artifacts
contain exactly one applicable part. Only one combined MIDI can be reported,
and it must cover all selected parts. Artifact profile IDs must match resolved
parts. Duplicate format/part outputs and unselected warning parts are rejected.
`complete` cannot contain substituted parts or native-unavailability warnings.
`midi_fallback` requires a combined MIDI artifact naming substituted parts,
an explicit `NATIVE_FORMAT_UNAVAILABLE` warning for each such part, and no
native artifact for a substituted part. Unavailability warnings must identify
substituted parts. Other public warnings can be part-specific or global.
These validate internally feasible claims, not successful generation: actual
existence/size, requested-format coverage, fallback permission, verified maps,
resolved root/link confinement, atomic publication, and read-back are Step 9's
responsibility. Synthetic in-memory tests do not create or certify artifacts.
`GenerationError {code, message, details}` supports these named exceptions:
`InvalidRequestError`, `UnsupportedDescriptionError`, `ProfileRequiredError`,
`ProfileNotFoundError` (same `PROFILE_REQUIRED` code, with a missing-ID message),
`UnsupportedArticulationError`, `NativeFormatUnavailableError`,
`OutputExistsError`, `OutputIOError`, and `SerializationError`.
Messages are fixed actionable public guidance. `GenerationError.from_domain`
copies only the declared code/message and explicitly provided finite JSON
details; it never reads exception causes, traceback text, or private files.
Callers must supply public-safe details, not raw exception strings/paths.
Step 3 sets MCP `isError` for domain failures; protocol failures remain SDK
errors. These contracts themselves implement no transport behavior.
Published schemas use JSON Schema 2020-12 vocabulary, including `uniqueItems`,
static conditional part/format/profile associations, native artifact shape,
and complete/fallback branch conditions. Standard JSON Schema cannot compare
two arbitrary field values (end >= start, rational arithmetic, profile-ID
correspondence, fallback warning coverage, or cross-track ordering). Those
remain enforced by the authoritative Pydantic validators; schema validation
alone is not sufficient at a service boundary. JSON Schema also treats `4.0`
as an integer mathematically; the typed models intentionally reject it in
integer-only fields. Nonfinite floats are not JSON numbers at all. Downstream
code must always validate with these models, not rely solely on client-side
JSON Schema checks. Do not mutate validated model fields/containers in place;
revalidate changed payloads rather than using unchecked `model_copy(update=...)`.
## Implemented in Step 3
`server.py` uses the installed official MCP **1.30.0 low-level `Server`**,
`@server.list_tools()`, `@server.call_tool(validate_input=False)`, and
`mcp.server.stdio.stdio_server`. The SDK owns framing, JSON-RPC parsing and
dispatch, IDs, notifications, initialization/negotiation, and EOF cancellation.
There is no second protocol layer or custom shutdown method. FastMCP inference
is unnecessary: tool definitions copy the exact reviewed `tool_input_schema`
and `tool_output_schema`, with no wrapper or weakened union.
The SDK's JSON-Schema-only input checker is disabled deliberately. Its error
messages can echo input values, and JSON Schema accepts `4.0` as an integer.
Both handlers instead validate arguments using the authoritative Pydantic
request models, including strict types and cross-field validators. Validation
details contain only model-known field names, numeric list positions,
`<unknown>` placeholders for unknown properties, Pydantic/authored reason codes,
and fixed, authored field corrections (grammar, bounds, shapes and recovery actions).
They never copy input, raw descriptions, validation messages/contexts, exception
representations, or causes. An issue at the whole-request boundary uses
`$request` only when no precise field is available. Part/format/profile and CC
cross-field validators use typed custom reasons; their public issues identify
`formats`, `profiles` or `cc_events`, not a generic root `value_error`.
The service repeats these same safe checks after omitted parts resolve.
Results are constructed as authoritative output models, serialized into explicit
SDK `CallToolResult` objects with structured content and the complete identical
public JSON object in a single text block. Returning
`CallToolResult` bypasses the SDK decorator's automatic output-schema check;
the typed constructors and tests against the published schemas provide that
validation without losing `isError` on the success/error union. Narrow type
ignores are confined to the SDK's untyped decorators; strict project mypy
remains enabled. Schema constraints, `isError`, strict models and SDK lifecycle
are preserved; text/description guidance and structured-key shorthand are Step 11 updates.
Exactly two tools are registered:
- `list_instrument_profiles` returns the real six-profile `ProfileListResult`.
- `generate_music` calls the concrete service described in Step 9, returning
real published Standard MIDI or an explicit typed error. The old development
skeleton is historical in report_3.md and is no longer the server's behavior.
Known `DomainError`s preserve only their declared public code/message/details.
Unexpected handler failures are not silently relabeled as invalid requests:
they log the failure class (without values, traceback, or exception text) on
stderr and return the SDK's ordinary error result with a fixed internal-error
message. Internal output-model failures follow this diagnostic path as well.
Unexpected failures never become success-shaped fallback.
`__main__.py` requires `--output-dir`, validates/resolves an existing directory,
configures stderr logging only on CLI execution, and runs the SDK on asyncio.
The root is retained in the SDK lifespan context and pinned by the service.
Importing the package, server, or entry module starts nothing and prints nothing.
Startup rejects URI/network/device/drive-relative syntax, controls, missing
roots, and regular files. It does not create roots or test writeability.
Step 9 additionally implements root/ancestor reparse checks, volume
classification, identity checks, collisions and atomic publication; trusted ACL
and race/crash limits remain explicit below.
### Observed SDK protocol behavior
Real subprocess tests exercise the installed SDK, not an invented dispatcher:
- Normal initialization sends/accepts the initialized notification. Requested
supported versions are retained; unsupported requested versions negotiate to
an SDK-supported version. Only the tools capability is registered.
- String and integer request IDs are preserved. Valid notifications and unknown
notifications produce no response. Ping continues working after bad input.
- Invalid tool arguments are typed `INVALID_REQUEST` tool results.
Unknown tools are SDK `isError` text results, not declared domain errors.
- An unknown method outside the SDK request union or invalid method parameters
yields `INVALID_PARAMS` (-32602). A recognized but unregistered method
(`resources/list`) reaches SDK dispatch and yields `METHOD_NOT_FOUND` (-32601).
- Invalid JSON, array messages, and messages without a required method are
handled by the SDK as stream-validation exceptions: a generic
`notifications/message` (`Internal Server Error`) is sent, with no invented
parse-error response or ID. The connection remains usable.
- The SDK parses a request-like object with `id: null` as a notification.
An unknown such method is logged/rejected without a response. Tests preserve
this observed behavior rather than asserting a nonexistent parse error.
- The SDK's validation logs can contain raw values. A CLI-only logging filter
replaces its known stream/request/notification-validation and unlisted-tool
diagnostics with fixed safe messages. Parsing, dispatch, and protocol responses
are untouched. Operational diagnostics are not suppressed. Debug logging is
not enabled by this application.
- stdout is exclusively valid UTF-8 newline-delimited JSON-RPC, including the
SDK's generic logging notifications. Human-readable startup errors/diagnostics
remain on stderr. Closing stdin exits zero, including before initialization;
there is no protocol shutdown request.
The tests run the installed editable package with `-I -B` and no `PYTHONPATH`
or `PYTHONHOME`, from a different repository-local working directory. The normal
flow uses the official client, including its bounded stdin-close/termination
cleanup. A small raw subprocess test client validates every stdout line and
enforces bounded waits and exact-process kill fallback before workspace cleanup.
Valid supported requests generate real MIDI; unsupported/native-disabled
requests create no output directory. Built-wheel execution outside the checkout
is additionally covered below. Manual MCP-client/Cubase/plugin checks are UNRUN.
See [startup and Windows client configuration](../README.md#start-the-mcp-stdio-service).
## Implemented in Step 4
`riff_groove_builder.midi` consumes the authoritative `EventPlan`; it neither
composes music nor compiles a `Score` into controls. The reviewed models, tool
schemas and errors are reused by the operational service.
### Exact timing and ordering API
- `beats_to_ticks(Fraction | RationalBeat | int, ppq=960) -> int` requires exact
nonnegative quarter-note beats and a legal positive, non-SMPTE PPQ (1–32767).
Floats, numeric booleans/strings, and sub-tick fractions are rejected.
- `bar_ticks(TimeSignature, bars=1, ppq=960) -> int` uses
`bars * numerator * 4 / denominator` without intermediate floats.
- `quarter_tempo(tempo_bpm) -> int` treats BPM as quarter notes/minute, regardless
of meter denominator. It rounds the exact decimal-rational reciprocal
`60_000_000 / Fraction(str(bpm))` to the nearest integer microseconds/quarter,
ties to even, requiring 1–0xFFFFFF. It never calls a meter-adjusted BPM helper.
The `EventPlan` retains the stricter intent tempo range of 20–300 BPM.
- `validate_event_plan(EventPlan) -> None` revalidates the whole mutable model
graph before checking MIDI-specific invariants. Unsupported payloads, strict
type errors, timeline violations, duplicate ordering keys, and the existing
100,000 scheduled-event limit therefore cannot bypass the writer through
unchecked model copies or list/field mutations. That budget counts scheduled
events, not the bounded conductor/name/EOT metadata.
Logical note offsets relative to `Score.start` and durations must fit the grid.
A nonzero logical score origin is not extra pre-roll: the compiler's intended
conversion is `musical_start_tick + ticks(note.start - score.start)`. The writer
preserves the supplied compiled ticks and does not reconstruct notes from the
score or claim to validate profile-specific compilation.
Absolute events are stable-sorted only by their authoritative
`(tick, priority, sequence)` key, never by pitch or a replacement generic control
priority. Delta times are computed once when constructing Mido tracks.
All deltas are strict integers in 0–0x0FFFFFFF (the four-byte SMF VLQ range).
With no artificial conductor filler events, its final EOT delta spans the
entire plan, so this writer also limits the shared `end_tick` to 0x0FFFFFFF.
It rejects larger plans rather than emitting invalid deltas or inventing timing
events. PPQ 960 represents straight and triplet subdivisions exactly; other
resolutions and fractions are accepted only when exact.
### Notes, channels, and shared tracks
Active notes are keyed by `(channel, note)` within each part. Every musical
note and keyswitch needs a positive-duration matching-role release. Orphan
releases, dangling notes, wrong-role releases, and same-channel/same-pitch
overlapping onsets are errors, including collisions between keyswitch and
musical roles. This conservative policy does not guess voice allocation.
At a same-pitch boundary the supplied ordering must release the old note before
the new onset. Roles may reuse a pitch only after release. Different pitches
on a part's channel are valid polyphony; the same pitch on separate channels
is independent.
Each used channel is reserved to its owning part for the whole plan, including
keyswitch-only and CC-only usage. Different parts cannot share a channel even
at disjoint times. A part may use several otherwise disjoint channels. Empty
parts reserve none. No channel 10/zero-based 9 assumption or program change is
added. Role labels guide validation/diagnostics; they are not extra MIDI metadata.
SMF type 1 has `Conductor` followed by `Lead`, `Bass`, and/or `Drums` in the
resolved selection order, not a fixed instrument order. Conductor name, tempo,
and meter are at zero (standard clocks-per-click 24, notated 32nds/quarter 8).
Part names are at zero. Controls and keyswitches stay in their affected part
track. Exactly one EOT per track is at the common `end_tick`, preserving common
pre-roll, exact musical span, and any control-release tail. Musical onsets
cannot occur before the musical start or at/after the musical end; musical
releases cannot exceed it. Keyswitches/CCs may occupy pre-roll and tails.
### Disk validation and failure boundaries
`write_midi(path: Path, plan: EventPlan) -> Path` constructs and validates Mido
SMF bytes in memory, exclusively opens only the supplied path (`xb`), closes
the saved file, and reads its actual bytes back. `validate_midi_file(path, plan)
-> None` exposes that disk expected-content check without writing or cleanup.
Both enforce header signature/length/type/track count/PPQ and compare every
absolute-tick message payload, track name/order, tempo/meter, channel,
note/CC/velocity, event sequence, and shared EOT. Opening with Mido alone is
not success; readable but changed content, missing/extra events, corruption,
truncation, and trailing data fail. Exact comparison with the already-pair-checked
plan also validates pairing of the saved events.
Failures use existing named `SerializationError`, `OutputExistsError`, and
`OutputIOError` with fixed safe reason codes, never raw exception values or
paths. Expected Mido/encoding and OS failures are caught narrowly; unexpected
programming failures propagate rather than becoming a successful fallback.
Failed writes/read-back remove only the newly exclusively created file,
checking its device/inode identity before unlinking. Pre-existing data and
replacements are not removed. Cleanup failures or ownership changes themselves
raise explicit IO failures; no success is returned if cleanup is denied.
The standalone validator never deletes an input, even if corrupt.
No parent directories are created. Relative and absolute `Path` destinations
are accepted, with no claim of output-root confinement, all-format publication,
fsync/crash durability, or adversarial race-proof filesystem operations.
The Step 9 service supplies staging/root/link checks and publishes the complete
artifact set only after actual saved-file validation.
### Independent evidence and limitations
Tests construct a synthetic three-part Score/EventPlan with a nonzero score
origin, 240-tick common pre-roll, triplets, retriggers, keyswitches, CCs,
120-tick tail, and non-GM drum channel/pitch. Mido reloads real saved production
files; independent MIDIUtil `addNote`/duration and controller APIs build equivalent
reference files in 4/4, 3/4, 6/8, and 7/8, including fractional BPM.
Comparison preserves exact note/control ticks, channels, pitches, onset and
release velocities, and control payloads. Fixtures explicitly match MIDIUtil
1.2.1's release-velocity convention (copies the onset velocity); production
preserves any valid supplied release velocity rather than imposing that rule.
MIDIUtil type 1 inserts a conductor before its three user tracks; meter uses
a denominator exponent in its API. Its tempo truncates instead of rounding
(137.5 BPM: 436363 versus the correctly rounded production 436364).
Tempo is tested independently rather than treated as an equivalent payload.
Reference EOTs occur at each track's local last event; production EOTs share
the full tail. These explicit metadata conventions are excluded from normalized
note/control equality, with separate assertions for conductor/meter/tempo/end
semantics. Duplicate removal, deinterleaving, and origin adjustment are disabled
in the reference to avoid silently changing the fixture's intended events.
Repeated production inputs, including permuted event-list input order, yield
identical bytes with the tested Mido version.
Test workspaces live under `.test-artifacts` and are removed in fixture cleanup.
No licensed or checked-in binary fixtures, production mappings, native codecs,
or Cubase/plugin acceptance are introduced. Focused verification:
```powershell
& '.\.venv\Scripts\python.exe' -m pytest tests\unit\test_midi.py tests\integration\test_midi_files.py -q
```
## Implemented in Step 5: mechanics and Standard MIDI fallback
`profiles.py` defines strict schema-version-1 declarative identity/provenance,
MIDI range/voice/alias/layer/action models, typed registry failures, and
deterministic production selection. `importlib.resources` loads an explicit
manifest from packaged resource initializers; three resources provide standard
pitched lead/bass and GM Level 1 percussion. Inaccessible/malformed/undeclared resources are errors,
not empty-registry success. Test-only data cannot enter production discovery.
`articulations.compile_score` revalidates explicit score/profile graphs and
produces a reviewed EventPlan. It uses exact quarter-note timing, per-part
channel ownership, latched deduplication, per-note momentary pulses/CCs,
reference-counted held controls, declared normal setup/restoration, and
conservative incompatible-state/physical-trigger rejection. One common pre-roll
and shared release/reset/restoration tail retain alignment and logical origin.
Deterministic release/reset/selection/onset priority and sequence preserve
writer causality; the 100,000-event budget and reviewed MIDI validator are
enforced before return.
See [articulation profiles](articulation-profiles.md) for schema/action details,
supported overlap boundaries, interpolation, named errors, API examples, exact
public standard sources and policy, and the remaining vendor evidence gate. Synthetic three-part
tests verify independently expected compiler events and real Mido disk read-back.
Built-wheel installed and ZIP-resource probes run outside the repository and
confirm exactly the three standard-only profiles and normal default selection.
Standard three-part tests also assert independent expected events and real MIDI
read-back. Public summaries add `mapping_kind: instrument | standard_midi`
(default `instrument` for existing contracts). Standard profiles have matching
`standard_midi` provenance, public fact references and separate generator policy;
neither can certify vendor evidence. Channels 0/1/9, normal/sustain without
controls, full legal pitched range, GM percussion numbers and linear positive
velocity policy are explicit. `instrument` identifies routing intent, not
compatibility; public product names and limitations disclose unverified vendors.
Only the explicit summary discriminator/provenance additions changed contracts;
the exact MIDI APIs and generic compiler remain unchanged. Step 9's operational
server lists the three profiles and reuses this compiler.
**Step 6 uses standard fallback**, under the amended plan.
Vendor maps remain unverified; native formats and application acceptance are
separately unverified. Standard fallback does not close those evidence gates.
## Implemented in Step 6: deterministic text and composition
`description.resolve_description(GenerateMusicRequest) -> DescriptionResolution`
revalidates the authoritative request and returns `intent: MusicalIntent`,
`assumptions: list[str]`, `overrides: list[str]` (unique, nonblank records).
The records feed the service's `ResolvedGeneration` metadata, not a
fabricated successful `GenerationResult`. No files, registries, network/model
providers or service transports are used by resolution.
`compose.compose_score(intent, selected_profiles, *, ppq=960) -> Score`
revalidates intent/profile graphs, then composes on one rational timeline.
The registry remains authoritative: callers select using
`registry.select(resolution.intent.parts, request.profiles or ProfileSelection())`.
The composer does not guess instruments from vendor words. Compile that score
with `articulations.compile_score`, then use the existing `midi.write_midi`.
If overriding PPQ for direct use, pass the same PPQ to the compiler.
Format/capability/fallback decisions and post-resolution request-format checks
are service responsibilities; the resolver does not serialize native
formats or infer output filenames/profiles.
### Exact description grammar
Case is folded and all whitespace collapsed before matching whole words.
Clauses may appear in any order. This is a closed grammar, not free-form prose:
| Input | Interpretation |
| --- | --- |
| `120 BPM`, `137.5 bpm`, `+.25 bpm` | Signed decimal numeric quarter-note BPM; final value must be 20–300. No exponent, fraction, textual number or unit conversion. |
| `4 bars`, `1 bar` | Numeric integer-valued count, 1–256. Decimal spelling `4.0 bars` also has integer value; fractional values fail. |
| `in 7/8`, `meter 3/4`, `6 / 8` | Numerator 1–16; denominator 1, 2, 4, 8 or 16. No mixed/changing meter. |
| `E minor`, `Bb major`, `key F# minor`, `Em`, `F#m` | A–G, optional single ASCII `#`/`b`, major or natural minor, including attached minor `m`; canonical sharp-spelled key. Case-insensitive text, including accidentals. |
| `lead`, `lead guitar`, `guitar`, `riff` | Lead part. |
| `bass`, `bass guitar`, `bass riff`, `bass groove`, `bass guitar riff/groove` | Bass part; slash here means either word, not literal `/`. |
| `drum`, `drums`, `drum groove`, `drums groove` | Drum part; groove is optional filler. |
| `lead and bass`, `bass, drums`, `lead + drums`, `bass & drums` | Part list. Separate positive part mentions also combine. Text order resolves to lead, bass, drums order. |
| `only bass and drums`, `lead and bass only` | Only-selection over that entire contiguous list. An additional positive part outside it is contradictory. |
| `no drums`, `without lead and bass`, `exclude bass, drums` | Exclude the entire contiguous list, never include negated words. With only exclusions, subtract from all three and record that baseline assumption. |
| `quarter`, `eighth`, `sixteenth` | Straight slots of 1, 1/2, 1/4 quarter-note beats, respectively. Optional ` note`, ` notes`, `-note`, `-notes`; `straight` is filler. |
| `rests`, `rest`, `no rests`, `no rest` | Omit the final slot of every bar, or omit none. No arbitrary rest schedule. |
| `rock`, `metal` | The two bounded original template policies below; `riff`/`groove` do not invoke a larger genre engine. |
| `slow`, `fast` | Fixed tempo candidates 80/160 BPM. Conflicting numeric/qualitative tempos are errors unless structured tempo overrides them. |
| `normal`, `sustain` | Same global normal technique. Sustain is an alias, not a pedal, CC64 or a different gate. |
| `palm mute`, `palm-mute`, `palm muted`, `palm-muted`, `palm muting` | Canonical `palm_mute`, applied only to selected lead/bass. Mixed-request drums stay normal and assumptions disclose applicability; drum-only palm mute errors. Standard pitched profiles cannot realize it. |
Simple subdivision, template and rest instructions apply globally. Palm mute has
the pitched-only applicability above; no arbitrary per-part technique schedule is
inferred. There are no per-part rhythms, sections, note
spellings, chords, arbitrary durations, accents or dynamics syntax.
Use the structured `parts` list to retain noncanonical part order.
Excluded and included occurrences of the same part are contradictory; an empty
selection is an error. `no drums only` is rejected as ambiguous rather than guessed.
Negation is supported only by the listed part exclusions and `no rest(s)`;
other negated instructions are errors, not silently ignored.
Only these exact filler words are ignored:
`please create produce generate make write a an the some simple music pattern groove
for with and at in of on notes note key tempo meter straight`.
Comma, period, semicolon, colon, exclamation/question marks, `&` and `+` are
separators; other punctuation/unrecognized tokens fail. At least one recognized
instruction or `music`/`pattern`/`groove` is required (`please` alone fails).
Whole-word matching never interprets bassoon as bass or leadership as lead.
Filler is not evidence that arbitrary prose was understood.
Tempo, bars, meter and key default to **120 BPM, 4 bars, 4/4, E minor**.
No explicit part language defaults to **lead, bass, drums**, with a record.
Pattern defaults are **eighth, rock, no_rests, normal**, also with records.
Seed is the authoritative request's seed (default 0), never read from text.
Structured tempo/bars/meter/key/parts take precedence, even over distinct
contradictory or out-of-range scalar text for the same field. Every non-null
structured value records its value and observed text candidates (even if none);
it is not mislabeled as an assumption. Repeated equivalent values, including
enharmonic keys, are not contradictions. Null means unresolved.
Structured overrides never authorize unknown material or changing instructions.
`then`, `to`, `from`, changes/changing, ramp, accelerando, ritardando, rallentando,
speed up and slow down fail explicitly, including with a structured tempo.
Only complete supported beat-range clauses (below) consume `from`/`to` before this guard.
Unresolved conflicts raise `DescriptionConflictError`, an
`UnsupportedDescriptionError` subtype with public `UNSUPPORTED_DESCRIPTION`
and the field to correct. Unknown words, unsupported material and out-of-bound
text raise named `UnsupportedDescriptionError`; invalid/mutated requests retain
`InvalidRequestError`. Unknown instruments (piano, violin, etc.), unsupported
modes/techniques, vendor names, audio, swing/arbitrary triplets, tempo changes,
polyrhythm and song sections are never discarded into a default loop.
### Typed rational phrase schedules (Step 12)
`TimedNote` contains reduced rational `start`/positive `duration`, a closed
`normal`/`palm_mute`/`pinch_harmonic` technique and optional finite intensity.
`PhrasePlan` contains a sole pitched part, low-tonic register policy, exact
duration and an ordered, nonoverlapping, event-budget-bounded note list.
`MusicalIntent.phrase_plan` is optional; a phrase must fit its selected part and
score duration and cannot coexist with global template terms/techniques.
Defensive revalidation rejects mutated invalid plans. No public score tool is added.
The bounded description parser composes two clauses, never a whole-prompt match:
`[chugging] palm mute/muted/muting eighth/8th [notes] from X to Y` and
`triplets on B&A with the A as a pinch harmonic that carries over a whole note
in the second/2nd measure`. Palm/pinch hyphens and spacing around `&` are accepted.
X/Y are inclusive 1–4 labels with optional `&`; B is a beat 1–4. Straight notes
advance by 1/2 quarter beat; triplets by exactly 1/3. The last triplet ends at
quarter beat 8, tying through all of measure 2. Reversed ranges, overlaps,
out-of-bar labels, insufficient length, non-4/4 meter, extra global rhythm or
techniques and other prose fail without dropping instructions. This text grammar
requires lead only and both clauses, not arbitrary melodies/sections/ties.
Longer explicit scores contain the phrase once and then silence.
The original 1-to-3& / beat-4 request yields six PM eighths, two PM triplets and
one PH of duration 13/3, at quarter beats 0,1/2,1,3/2,2,5/2,3,10/3,11/3.
The hold infers two bars when omitted; 120 BPM remains the disclosed default.
`A` is a count, not a pitch. The composer generically applies per-note timing,
technique and intensity, resolving each profile articulation without fallback.
All notes use the tonic nearest MIDI40 within actual range and native tuning;
default AME Em is E2/MIDI40. Seeded intensity uses the unchanged local RNG policy
unless a typed note intensity is present. All PPQ conversion remains exact;
event budgets, compiler ordering, CC handling and atomic file transactions remain
unchanged. Simple patterns do not use the phrase path or change their notes.
### Deterministic composition and exact musical policy
- One common interval starts at 0 and ends at
`bars * numerator * 4 / denominator` quarter-note beats. Every part shares
it, and the existing writer emits a common end-of-track tick even after rests.
BPM is always quarter-note BPM, including 6/8.
- Slots restart at every bar. If the chosen subdivision does not divide the
bar (e.g. quarter in 7/8 or 1/16), only the last slot is shortened to that bar
boundary. This is an explicit policy, not duration drift or rounding.
- Rests omit the last slot, including its drum hits, of every bar. A single-slot
bar with rests is entirely silent but retains its exact duration. No requested
technique is dropped even for silence: profile validation still runs.
- Major intervals are 0,2,4,5,7,9,11; natural minor 0,2,3,5,7,8,10. Lead requires
a full ascending seven-degree scale fitting the actual inclusive profile range.
Bass requires a root/fifth pair (0,7) fitting the range. Choose a root nearest
MIDI 64 for lead and 40 for bass (lower root breaks a distance tie), among
candidates that fit the entire palette. **C4 = 60, E4 = 64, E2 = 40**; profile
note numbers, not vendor octave labels, remain authoritative. No pitch clamping,
chromatic notes, dropped degrees or guessed alternate octave mapping.
- Lead rock degree cycle is (0,2,4,6), metal (0,0,4,1), zero-based; each bar starts
at degree 0. Other slots add a seeded 0/1 degree variation modulo seven.
Bass rock alternates root/fifth; metal uses root,root,fifth,root. These are
original simple policy patterns, not factory/demo content or genre fidelity.
- Drum hats use selected profile alias `closed_hat` on every active slot.
Metrical pulse is `max(slot_length, 4/denominator)`; on pulse-aligned slots,
alternate `kick` and `snare` starting with kick each bar. Metal additionally
adds kick on off-pulse slots. This is the explicit odd/compound-meter rule,
not a claim of idiomatic 6/8 or automatic accent grouping. It avoids losing
snares when a coarse subdivision skips denominator units.
- Resolve all three required drum aliases before allocation. All physical notes
are deferred to the compiler's selected profile lookup; no GM/vendor number
is embedded in composition. Missing aliases fail rather than substituting.
- Pitched gates fill the slot; drum gates are `min(slot_length, 1/4 beat)`.
Adjacent same-pitch notes only touch at boundaries (compiler releases first).
Each slot has seeded intensity from {0.5, 0.625, 0.75}, shared by simultaneous
drum hits. Profiles perform real supported velocity conversion, never clamping.
- Local `Random(seed + part_offset)` uses lead/bass/drums offsets 0/1/2, so
adding/reordering other parts does not perturb a part's notes. Calls never
change global random state. Same intent/seed yields the same score/events/bytes
in the tested Python environment; no cross-runtime RNG certification is claimed.
- Revalidate exact profile coverage, part identity and distinct channels.
Resolve each requested articulation to a canonical profile name, with only
one global canonical technique allowed. `UnsupportedArticulationError`
identifies part/profile/technique, including honest palm-mute failure for
standard maps. No softer-note or guessed-switch approximation exists.
- Before constructing notes, count exact active slots/hits over all bars and
reserve at least two events per note against the existing 100,000-event budget.
The compiler remains authoritative for additional control events/pre-roll/tails.
Grid-check slot offsets, lengths, drum gates and total duration using existing
`beats_to_ticks`; off-grid PPQ fails without rounding.
`CompositionError` (`SERIALIZATION_ERROR`) identifies range, missing-voice,
coverage/channel, unsupported direct-intent terms or budget problems.
Existing exact timing/profile/compiler errors are not hidden by fallback.
The unchanged production registry supplies only Standard MIDI normal/sustain,
channels 0/1/9, legal pitched ranges and declared GM aliases. It supplies no
vendor techniques. One synthetic switch-profile unit test proves real compiler
coupling only; it does not advertise vendor compatibility. No authoritative
shared model/profile/compiler/writer changes were necessary in Step 6.
Focused validation:
```powershell
& '.\.venv\Scripts\python.exe' -m pytest tests\unit\test_description.py tests\unit\test_compose.py tests\integration\test_text_to_midi.py -q
& '.\.venv\Scripts\python.exe' -m pytest -q
& '.\.venv\Scripts\python.exe' -m ruff check .
& '.\.venv\Scripts\python.exe' -m ruff format --check .
& '.\.venv\Scripts\python.exe' -m mypy --strict src tests
```
Actual-disk tests cover lead/bass/drums/all-parts, 3/4, 6/8, 7/8, nondefault
tempo/key/length, independent event/pitch/channel/duration/pairing assertions,
precise expected bass/drum sequences, seed byte determinism and text-driven
rhythm changes. Tests use self-cleaning UUID directories under `.test-artifacts`.
The unchanged Step 3 tool handlers are **not** a text-generation service.
## Implemented in Step 7 fix: real native Riffer codec
`formats.NativeCapability` is a small frozen/slotted internal result with part,
format, available and a fixed safe reason. It is not an exporter framework,
artifact, or caller-supplied authorization. The concrete AGP APIs are:
- `check_agp_capability(part: Part, profile: InstrumentProfile) -> NativeCapability`
- `require_agp_capability(part: Part, profile: InstrumentProfile) -> NativeCapability`
Both reject non-lead/bass requests and defensively revalidate the full profile
through `checked_profile`, preserving the existing target/part/format and
native-metadata constraints. Valid wrong-part profiles raise the existing
`ProfilePartMismatchError`; malformed/mutated profiles raise
`InvalidProfileError`, not native-unavailable.
Checks are still pure/no-IO and keep `format=agp` as a legacy request alias.
The evidenced native subset returns `available=True`, `reason=riffer_xml_2_1`;
standard/synthetic maps return false with `riffer_profile_not_supported`.
Matching metadata alone is insufficient: strict native settings, provenance
and supported target/version are required.
`write_riffer(path, plan, part, profile)` exclusively creates an actual `.griff`
or `.briff` staged file. It validates/recompiles the score/event plan, assigns
physical strings with bounded backtracking, serializes real XML note/CC fields,
and invokes `validate_riffer_file` on actual saved bytes. The separate reader
checks structure, physical validity and requested musical content, not a second
serialization. Native PPQ 480 conversion is exact; unsupported pre-roll/tails,
off-grid events, other CCs/techniques or impossible strings fail explicitly.
Legacy write/validate AGP names alias these APIs, never emit `.agp` files.
At completion of Step 7 the two target profiles were nondefault. The service available-codec
guard remains: native service registration is deliberately not part of this
fix. Standard default MIDI/fallback behavior remains operational. Public
artifacts/summary metadata expose actual suffixes, while requests also accept
`agp`/`sdp`. The subsequent Step 8 fix adds SD3; no application automation is added.
See [native schema/evidence/acceptance](formats/agp.md). Plugin/Cubase acceptance
remains unrun, independently of actual XML generation/read-back success.
## Implemented in Step 8 fix: native SD3 template song codec
`formats.sdp` supplies a bounded balanced-text 3.4.4 parser, read-only
`inspect_sdp_template`, template-aware check/require APIs, exclusive staged
`write_sdp` and independent `validate_sdp_file`. The explicit typed optional
`template: Path | None` is an operator-local input, never a bundled private
project. Missing optional template is `sdp_template_required`; supplied bad,
unreadable, versioned or map-mismatched templates are errors, not fallback.
Standard profiles still report `sdp_profile_not_supported` with no IO.
The nondefault `toontrack.sd3.3.4.4.core.v1` profile covers only locally verified
SL-DFH kickR/36, snareR/38 and hatsTP/42. The codec validates actual pad aliases
and input channel before use. Channel 9, intensity interpolation and neutral
song controls are generator policy, not vendor defaults.
All source active song tracks/clips/references are replaced with one original
note-only track/clip. Packed status/pitch/velocity triples and exact finite
decimal beat times are independently decoded from saved bytes and compared
with a recompiled score/event plan. Unsupported event semantics, pre-roll,
tails, release velocity, precision and corrupt input fail explicitly.
Every non-song byte (kit/mixer/tracker and other operator state) is preserved.
Only the staged output is exclusively written and read back; no publication.
Native tests use newly authored original fixtures and an opt-in read-only local
template integration. SD3 application/Cubase acceptance is UNRUN, not inferred.
At completion of Step 8, service wiring, CLI template configuration and defaults
were untouched; Step 9 fix below now implements all three.
See [SD3 API, limitations and evidence](formats/sdp.md) and
[current verification](../prompts/report_8_fix.md).
<a id="implemented-in-step-9"></a>
## Implemented in Step 9 (updated by native integration fix)
`service.GenerationService(output_root: Path, *, registry: ProfileRegistry | None
= None, sd3_template: Path | None = None)` loads the packaged registry and pins
the existing local root. CLI `--sd3-template` threads a typed operator Path through
`run_stdio`/`create_server`; no tool argument can select a template.
Its two public operations are `list_instrument_profiles() -> ProfileListResult`
and `generate_music(request: GenerateMusicRequest) -> GenerationResult`.
The SDK handlers use this same concrete service, with no alternate composition
or exporter framework. The optional registry supports isolated tests; production
loads exactly six declared resources: three target defaults and three explicitly
selected standard fallbacks. Strict target validation still enforces evidence,
versions, tunings/voices, switches, velocity bands and timing policy.
The service revalidates mutable request graphs, resolves text, rechecks
parts/formats/profile associations after inference, selects/revalidates maps,
composes/compiles once, and preflights actual native capabilities/content before
any output staging. Omitted/null formats request MIDI and relevant native suffixes.
`agp` expands to selected lead `.griff` and bass `.briff`; `sdp` to drums `.sd3p`.
Aliases and actual names deduplicate. Explicit MIDI-only skips native capability
preflight and works with target maps even without a template.
Only known unavailable capability/representation limits permit fallback. Preflight
uses codec content/grid/routing checks without writing. Narrow fixed reason codes
cover common pre-roll/tails, exact native grid/decimal precision, unsupported Riffer
CC and impossible physical-string assignment. Invalid plans/maps, search exhaustion,
template errors, serializer/read-back/IO faults are errors, never fallback.
An explicitly supplied template is validated at construction and checked by source
identity/size/mtime/SHA-256 during listing/generation and before publication.
Even a MIDI-only request never hides an explicitly invalid configured template.
Composition and compilation reuse existing exact-timing APIs and event budget.
Registered writers create `<basename>.mid`, `<basename>_lead.griff`,
`<basename>_bass.briff`, `<basename>_drums.sd3p` as requested. Each exclusively
opens its path and invokes an optional `on_created(fstat)` hook **before writing**:
the service records that actual new file's identity, including on partial failure.
The hook is the only shared writer change; standalone APIs retain their behavior.
The service independently invokes each saved-file validator after the writer's
own read-back, bounds files to positive <=16 MiB, rejects reparse/nonregular/
hardlinked files, and checks identity/size/mtime/SHA-256 stability. It constructs and
revalidates the anticipated result including JSON serialization, then publishes.
A result-model error is `SERIALIZATION_ERROR / invalid_generation_result` before
publication, not a post-publication false failure.
MIDI is emitted only when requested or needed for fallback; it includes all selected
parts. Successful native-only requests have no MIDI. `fallback_for` contains selected native-requested
parts whose capability was unavailable. Each gets `NATIVE_FORMAT_UNAVAILABLE`;
status is `midi_fallback`. Explicit MIDI-only success is `complete`. Every selected
standard-only map gets `STANDARD_MIDI_MAPPING`, never vendor compatibility.
Target subsets get per-part `VERIFIED_SUBSET_LIMITS` warnings.
Missing drum template with allowed fallback preserves lead/bass natives and adds
one combined MIDI with fallback only for drums. Disabled fallback errors before
stage creation. Runtime profile summaries combine real preflight readiness with
version/library/subset/default and application-limit disclosures.
Results contain only actual published absolute paths, actual sizes, part/profile
IDs, resolved musical metadata, assumptions/overrides and public warnings.
Known failures use typed `GenerationError` with MCP `isError`; unexpected bugs
retain the sanitized diagnostic/SDK error path. Profile listing is now real.
### Filesystem guarantees and limitations
- Tool inputs contain only the strict safe basename, not arbitrary paths.
Startup rejects URI/UNC/device/drive-relative/network-drive syntax, `..`, files,
nonexistent roots and all existing root/ancestor symlinks or Windows reparse
points (including junctions). Windows drive type must be fixed/removable/RAM.
The root and every ancestor's device/inode identity are pinned and rechecked.
- One exclusive UUID stage under the root is owned per request (at most three
name attempts). No global scratch or persistent job database is used.
Directory checks verify stage identity, non-reparse status and parent
confinement. Enumerations examine at most requested file count + 1 entries
(at most five); only the exact planned names and claimed file identities are allowed.
No sidecar manifests, renamed MIDI substitutes or extra formats are generated.
All snapshots/hashes are rechecked immediately before commit, including earlier
files after the last writer. Unknown injected content causes an error.
- A same-volume `Path.rename` of the whole completed stage publishes
`<root>\<basename>`. **Windows only:** `os.rename` refuses existing destinations,
including empty directories. Precheck improves diagnostics; the rename itself
supplies no-replace semantics for concurrent same-name requests. No `replace`,
merge, POSIX replacing rename, or delete-destination workaround is used.
Non-Windows generation fails `OUTPUT_IO_ERROR /
no_replace_publication_requires_windows` before staging.
- Existing output/sentinel data are never overwritten. `OUTPUT_EXISTS /
destination_exists` is explicit. Only a verified owned stage is cleaned:
first verify all present entries have recorded exclusive-create identities and
are regular, non-reparse and singly linked; unlink only those owned files, then remove that
specific empty stage. Unexpected content or lost identity refuses cleanup;
nothing follows links or recursively deletes output-root contents.
- Cleanup failure is explicit `OUTPUT_IO_ERROR / stage_cleanup_failed`, carrying
only the original domain code (or `internal`), and can leave the owned stage
for operator inspection. Root/stage replacement similarly refuses unsafe
cleanup. Permissions, disk failures, missing roots and publication failures
remain visible; fallback cannot hide them.
- These portable path/lstat/recheck APIs **do not** provide comprehensive hostile
race safety. A local actor able to replace directories/files between checks
and opens/unlinks/rename can race them. Use an operator-controlled root and
ancestors/volume with trusted ACLs; no handle-relative Windows NT namespace
confinement, cross-process malicious-lock protocol, mount-change defense,
or power-loss/fsync durability is claimed. Directory mode 0700 is advisory on
Windows, not an ACL guarantee. Root identity checks are not indefinite handle
pins and cannot prove against filesystem file-ID reuse.
- Publication success is the commit point; a client disconnect or process crash
afterward can leave a valid published output without a received response.
No post-publication rollback or acknowledgement database is implemented.
The new unit/integration tests exercise these concrete guarantees, including
failure injection, concurrent same-basename publication, actual Windows
junctions, real stdio generation from another working directory and disk MIDI
inspection. Exact executed evidence and persistent example artifacts are in
`prompts/report_9_fix.md`. Native software integration is verified; application
acceptance remains separately unrun.
## Installed software acceptance in Step 10
`tests/integration/test_distribution.py` extends the existing offline build
fixture and reuses the raw stdio client from `test_stdio.py`, without a second
production transport or duplicate build harness. The default build creates an
sdist then builds its wheel. The earlier dependency-free separate-venv import
test remains; the new server test installs that wheel with `pip --no-index
--no-deps --no-cache-dir --target` into an owned test directory.
The `-I -B` bootstrap explicitly prepends this install target and asserts the
actual package, server, service, schemas, profiles, compiler, MIDI, native
boundaries, resource packages and `__main__` entry resolve there (16 locations),
not editable `src`. Distribution metadata must also resolve to the target.
It reuses development-venv runtime dependencies: this is isolated application
installation/execution, not a fresh full dependency-environment certification.
The process working directory is the pre-existing parent **outside** the
checkout; the test does not create files there.
Six packaged JSON files are compared byte-for-byte to wheel archive members,
then discovered/read as installed resources and listed over real stdio.
Initialize, exactly two tools, exactly six profiles, public
input/output schemas, explicit MIDI, default fallback and native-only fallback
are exercised with **explicit standard IDs**. Independent Mido inspection checks type 1 / PPQ 960,
436364 microseconds/quarter at 137.5 BPM, 7/8, Conductor/Lead/Bass/Drums, common
end 6720, exact onsets/gates, legal positive onsets, complete note pairs and
channels 0/1/9. Returned paths/sizes/profile IDs/status/warnings must agree.
All three request-format policies produce the same 324-byte musical content,
not different native encodings. Additional real installed-server calls use an
original minimal SD3 template and target defaults to generate actual full
default/native-alias/actual-name/MIDI-only outputs, exact CC1 stairs/resets and
per-part controller fallback. Missing-template target fallback, disabled fallback
and native collisions are also checked. All saved `.griff`/`.briff`/`.sd3p` and
MIDI files are independently checked against the same requested score.
Native-disabled, unsupported palm mute, Unicode unsupported text and collision
errors are schema-checked and cannot leave new output directories.
The client validates every UTF-8 newline stdout message as MCP JSON-RPC;
deliberate malformed input produces only a generic protocol notification on
stdout and sanitized diagnostics on stderr, with successful ping recovery.
Both initialized and pre-initialize EOF exit zero. Reads/writes/termination are
bounded; cleanup kills only the owned child if EOF times out, reaps it, and
removes the owned distribution/install/output workspace even on failure.
The full suite, Ruff lint/format, strict mypy, `pip check` and final real
sdist/wheel build are separate checks. See [current tested configuration and
outcomes](compatibility.md#tested-software-configuration) and
[report_10.md](../prompts/report_10.md). These are native/MIDI software checks,
not receiver, Eclipse/Ray5/SD3 application or Cubase acceptance.
## Operational service and remaining native boundaries
```text
MCP client
-> SDK stdio lifecycle and two operational tools
-> strict Pydantic request/domain/result models (implemented contracts)
-> deterministic closed-grammar description and coordinated score (implemented)
-> versioned target subset defaults or explicitly selected standard fallback profiles
-> native check/require (Riffer subsets; SD3 subset requires local-template preflight)
-> explicit service fallback policy for known unavailable native capabilities only
-> Mido SMF writer / actual Riffer XML and SD3 template-song codecs
-> read-back validation -> staged, atomic local publication -> typed result
```
The registered tools are `list_instrument_profiles` and `generate_music`.
The SDK owns JSON-RPC lifecycle/negotiation. stdout is reserved for stdio
protocol traffic; diagnostics go to stderr. No HTTP application, GUI, database,
plugin host, audio engine, DAW automation, or model provider is introduced.
Keep intent, logical notes, scheduled MIDI events, and native representation
separate. The planned shared timeline uses rational beat arithmetic before tick
quantization, aligned lead/bass/drum tracks, deterministic local random state,
and profile-specified control timing. Mido is the production MIDI library;
MIDIUtil is an independent test-fixture library, not a second production writer.
Instrument-specific production profiles must identify exact instruments, versions,
source provenance, playable ranges, drum maps, keyswitch/CC semantics and layers.
Standard-only profiles instead identify their standard and explicit generator
policy; they cannot use verified vendor provenance or imply native availability.
Synthetic fixtures test mechanics but cannot become production maps.
Guitar/bass native adapters may share code only if evidence proves shared layout.
The service publishes complete validated native/MIDI artifact sets under an
operator-configured local output root, rejecting unsafe names and collisions.
Unavailable native capabilities may permit explicitly reported MIDI fallback;
unverified vendor mappings and unexpected serialization/IO errors cannot be hidden
by a fallback success. A selected standard profile must remain explicitly
standard-only in metadata/warnings, never represented as a working vendor map.
A preset-only native format may contradict the requested
song-content requirement and needs a decision, not a fabricated implementation.
## Sequencing and evidence
Step 11 fixes a verified text-only feedback failure: the prior profile result
exposed only a count in text, and lead + sd3p with valid `E minor` exposed only
generic guidance while the useful error object lived in structured content.
Both result representations now contain the same complete safe JSON object.
Unit, official-client, raw-stdio and installed-wheel tests decode text and
assert equality with structured content. The text-only recovery scenario lists
real IDs, receives a `formats` correction, receives a separate missing-profile
recovery action, and generates/read-backs actual guitar `.griff` and `.mid`
using a returned ID and structured `Em`. Unrequested CC91 is removed by the
test consumer, not silently stripped by the server. Step 12 adds the bounded
beat-range/triplet/cross-bar-PH phrase above; arbitrary triplet melodies still error.
MCP schema constraints and protocol handling have not been simplified or replaced.
An existing LM Studio setup needs an operator MCP restart and a fresh chat to
refresh cached tool definitions; no live client/configuration is changed here.
Steps 2–4 define typed contracts, an honest stdio skeleton, and a controlled
MIDI writer without inventing instrument data. Step 5 generic profile/compiler
mechanics and standard-fallback production profiles are implemented; vendor
mapping acceptance remains gated and does not block standard-only Step 6.
Standard-only composition, native Riffer writers/validators and the native SD3
template-song codec and native/MIDI service wiring are implemented.
Step 9 fix verifies actual native integration, not a bypass of application acceptance.
Mapping evidence gates production instrument claims;
separate native structure and actual application acceptance gates each native
writer. The [compatibility inventory](compatibility.md) records those gaps.
The existing [AGP native fix](../prompts/step_7_fix.md) and
[SDP native fix](../prompts/step_8_fix.md) distinguish completed codecs from
remaining application work. Independent Step 10
software acceptance does not satisfy the original full native definition of done.
## Step 14: internal explicit pitch and polyphonic attack foundation
Step14 introduced internal atoms/plans, not a public score tool. Step15 below
now exposes those atoms through bounded public hybrid description lanes.
The legacy low-tonic `PhrasePlan` and ordinary pattern paths are unchanged.
`pitch.py` exposes immutable `PitchToken`, `ChordSymbol`, `ExplicitVoicing`,
`NotationToken`, and `ResolvedVoicing` values and these reusable functions:
- `parse_pitch(text)` parses exactly one uppercase A–G, optional single ASCII
`#`/`b`, optional signed octave. C4=60; C-1=0; G9=127. Accidentals cross
octave boundaries correctly (Cb4=59, B#3=60). Unsupported spellings or
explicit pitches outside MIDI 0–127 fail. `.midi` on an octave-free token
fails until register resolution; `.pitch_class` remains available.
- `parse_chord(text)` parses a bare major root, `m`/`min` minor, optional root
octave **after** the quality, optional `(add4)`, and optional `/` or `Over`
bass. Thus Em4 is E4–G4–B4, not a fourth/add4 quality. Intervals are major
`[0,4,7]`, minor `[0,3,7]`, major-add4 `[0,4,5,7]`, and minor-add4
`[0,3,5,7]`. Digits always mean root octave, never seventh/extensions.
Other qualities, extension syntax, or guessed slides fail.
`[C3 E3 G3]` is an explicit whitespace-separated octave-qualified pitch set;
duplicate numeric pitches (including enharmonic duplicates) are invalid.
- `tokenize_notation(text, kind="note" | "chord")` lexes compact runs up to
4000 characters: F#F#, DD, BbBb, CEG, C4E4G4, and concatenated chord atoms.
`text`, `start`, `end`, and exact `leading_space` preserve token boundaries
and spacing width without assigning rhythmic meaning. Note and chord modes
are deliberately separate: C4 is one note in note mode, a major chord root
in chord mode. Unknown/ambiguous tails are errors, not ignored text.
- `resolve_voicing(value, profile)` returns sorted independent MIDI pitches
and disclosed `assumptions`. An absent note/root octave selects the playable
root nearest MIDI40 (lower wins ties) under the chosen profile range/tuning,
**before** constructing the chord. It does not search for another octave
if the complete chord then fails to fit. Explicit pitches are never changed
to match a key. Slash bass is strictly lower than every upper chord tone:
Em4/B2 and Em4OverB2 -> `[47,64,67,71]`; Em4/B and Em4OverB ->
`[59,64,67,71]`, with the nearest-below default reported. Contradictory
explicit bass octaves fail. Identical computed pitches are deduplicated.
`validate_voicing(pitches, profile)` checks every tone against MIDI bounds,
profile range, and evidenced individual-string tuning without dropping,
spreading, transposing, or arbitrarily revoicing.
`schemas.ExplicitAttack` extends exact `TimedNote` timing/technique/intensity
with a nonempty unique set of MIDI `pitches`. `ExplicitAttackPlan` contains
one pitched `part`, positive rational `duration`, ordered `attacks`, and
optional register `assumptions` supplied by the notation resolver. It bounds
the timeline to the existing maximal score length, validates half-open
intervals, forbids independent same-pitch overlaps, permits disjoint compatible
polyphony, and rejects overlapping different profile-wide techniques.
Attack duration means one continuous sustain; gaps mean silence. There are no
rest, continuation, slide, lane-spacing, or prose-interpretation guesses.
Later parsing can represent rests with gaps and continuation by extending the
original duration rather than retriggering. It must propagate assumptions.
`MusicalIntent.explicit_plan` is internal and mutually exclusive with
`phrase_plan`; it requires its sole part, sufficient bars/meter, and no global
rhythm/technique overrides. `compose_score` expands **all** tones as independent
simultaneous `PitchedNote`s at exact starts/durations. One seed draw per attack
provides shared chord intensity unless explicitly supplied, with the existing
bass seed offset. Techniques resolve through the actual selected profile.
The plan counts expanded note-on/off pairs; explicit composition also invokes
the authoritative compiler to enforce the 100,000-event ceiling **including**
articulation controls and to validate grid/control timing. Subsequent explicit
CC points remain subject to the existing combined compiler/control budget.
The unchanged Riffer codec assigns independent tones using bounded physical
string allocation and validates saved pitch/fret/timing/technique data. A
full voicing with no physical string assignment can use only the existing
explicit `no_physical_string_assignment` MIDI-fallback policy; all pitches
survive in MIDI. Out-of-map pitches are composition errors, not fallback.
No profiles, tunings, vendor claims, dependencies, versions, or public request
fields were changed. Integration tests inject an internal resolution solely
to exercise actual service publication; they do not claim public chord prose
or lane parsing. Saved Mido/XML checks independently assert 11 musical tones,
4 MIDI selectors, 30 scheduled events, exact thirds, PM/PH, a non-key F# major
chord, both slash-bass octaves/aliases, and legal occupied physical strings.
## Step 15: public hybrid prose and precise lanes
`hybrid.py` separates lane/hold clauses before the legacy prose sequence guard.
`description.resolve_description` reuses the closed prose resolver for context,
then attaches a strict, profile-independent `HybridPlan` of rational
`HybridSlot`s. Count labels are denominator beats; widths/spaces are lexical only.
`count_grid` is reusable exact Fraction math for complete eighth/sixteenth and
explicit eighth-triplet beats, including 3/4 and non-quarter denominators.
`1&a` is rejected without a disambiguating grid; under sixteenths its `e` is
omitted (positions 0, 1/2, 3/4), while under eighth-triplets it is 0, 1/3, 2/3.
Bare labels remain one denominator beat even with a Grid directive.
After profile selection, service and composer both reuse `hybrid.materialize`.
It calls Step14 tokenization/voicing resolution, never a second voicing engine,
and creates the existing `ExplicitAttackPlan`. `~` extends the previous
adjacent attack duration; rests create real gaps; prose ties extend a final
attack to an exact bar end only if no specified rest/new attack conflicts.
Exact source/installed E2 prose-tie, all-tilde-bar and continuation-clause outputs
are byte-identical. Structured meter changes cannot reinterpret explicit
labels; bars cannot truncate notation or holds.
`o/x/^/-/~`, Notes/Chords empty lanes, compact tokens, explicit chord voicings,
slash/Over bass and ordered Bar blocks or matching `|` spans are public.
Events/Notes brackets group consecutive positions; Chords brackets specify
simultaneous notes. A slide direction in Events returns specific unsupported
origin/destination/timing guidance; slash chord spelling is unrelated.
Unknown prose is rejected, including tempo changes; style does not modify
precise pitches/times. Global techniques and explicit prose pitch claims must
agree with affected events. No implicit CC, transpose, revoice or dropped tone.
The additive default-empty `resolved.interpretation` lists `InterpretedAttack`
records: exact pitches, rational start/duration, part, technique, optional
intensity and held status. Register/bass/default/velocity policy is in
`resolved.assumptions`. Text-only JSON and structuredContent are identical;
errors expose static authored reasons/guidance, never raw user text or paths.
The corrected native fixture uses F#2 major, **D3 major [50,54,57]**, Bb2 major,
then C4/E4/G4: 21 musical tones with legal independent string assignments.
The old explicit D2 major [38,42,45] is deliberately a negative-native /
positive-exact-MIDI test, not a reason to disable the generic parser.
Native validation/tuning/fallback/publication/CC/event budgets are unchanged.
See README's hybrid section for tested public syntax and limits.