17 KiB
17 KiB
Step 9 report — operational Standard MIDI MCP service and explicit native fallback
Date: 2026-10-19
OUTCOME: IMPLEMENTED, PERSISTENT AND VERIFIED ON WINDOWS.
Final complete suite: 1118 passed; 0 failed; 0 skipped.
Final focused service/server/stdio suite: 140 passed; 0 failed; 0 skipped.
AGP/SDP serialization and vendor mapping gates: STILL OPEN / BLOCKED.
Eclips/Ray/SD3/Cubase application acceptance: UNRUN, not passing.
SCOPE AND PERSISTENT CHANGES
Read step_9.md, the amended plan/continuation decision, reports 5_fix/6/7/8,
and the current contracts, profile registry, resolver, composer, articulation
compiler, MIDI writer/read-back, native boundaries and SDK server.
Created:
- src\riff_groove_builder\service.py
- tests\unit\test_service.py
- tests\integration\test_generation_service.py
- prompts\report_9.md
Updated:
- src\riff_groove_builder\server.py and __main__.py
- tests\unit\test_server.py and tests\integration\test_stdio.py
- README.md
- docs\architecture.md, compatibility.md, articulation-profiles.md
- docs\formats\agp.md and sdp.md
Removed registered tools' empty-registry/unfinished-generation behavior.
No shared schemas/errors/profile maps, resolver/composer/compiler/writer,
dependency manifests or prior reports/prompts were changed. No weakened
validation, new suppressions, native codecs, fabricated native outputs,
audio/model/HTTP service, sidecar manifests, commits or delegation.
No later prompt or application-acceptance work was executed.
CONCRETE APIS AND DATA FLOW
service.validate_output_root(value: str) -> Path
Retained as a server-module re-export for existing CLI/tests/callers, with
stronger actual root/ancestor reparse and Windows local-volume checks.
GenerationService(output_root: Path, *, registry: ProfileRegistry | None = None)
Uses the actual packaged three-profile registry by default. The optional
concrete registry is an isolated-test seam, not exporter discovery.
GenerationService.list_instrument_profiles() -> ProfileListResult
Returns three actual, defensively checked maps, sorted by registry ID:
standard.bass.v1, standard.gm-drums.v1, standard.lead.v1.
Each exposes mapping_kind=standard_midi, normal-only articulations,
actual product/standard versions and limitations, midi_available=true,
native_available=false. Intended routing is not vendor compatibility.
GenerationService.generate_music(request: GenerateMusicRequest)
-> GenerationResult, or existing named domain errors.
Revalidates the complete mutable request graph, then:
resolve_description -> post-resolution association validation ->
registry selection/revalidated maps -> real native capability preflight ->
compose_score -> compile_score -> exclusive staged write_midi ->
actual saved-file checks and validate_midi_file -> anticipated result
construction/revalidation/JSON serialization -> directory publication.
No duplicate note-generation logic. Existing exact timing, articulation,
profile provenance, event-pairing and 100,000-event safeguards remain intact.
Only actual published absolute paths and measured sizes are returned.
Result metadata includes parts, profile IDs, BPM/bars/meter/key/seed, common
musical start/end/PPQ, assumptions/structured overrides and explicit warnings.
No raw description, file bytes or private exception text is included.
FORMAT AND WARNING POLICY
- Omitted/null formats request MIDI plus applicable native targets:
AGP for selected lead/bass and SDP for selected drums.
- Explicit ["mid"] means MIDI only, status=complete, fallback_for=[].
Native capability checks are not run for this case.
- Explicit AGP/SDP applies only to the corresponding selected parts.
Applicability/profile associations are rechecked after text inference;
omitted pre-parser parts cannot bypass these checks.
- Current real checks report agp_layout_not_established or
sdp_layout_not_established. With fallback allowed, one combined MIDI
contains ALL selected parts even when "mid" was not requested.
- status=midi_fallback, fallback_for and NATIVE_FORMAT_UNAVAILABLE warnings
cover exactly the selected unavailable native-requested parts.
- Every selected standard-only map always gets STANDARD_MIDI_MAPPING,
including complete MIDI-only success. Example message:
"standard.lead.v1 uses Standard MIDI only; not a verified Eclips/Ray/SD3
mapping or vendor compatibility claim."
- Native lead example:
"agp: agp_layout_not_established; the combined Standard MIDI substitutes
for this native target."
- Native-disabled requests fail before staging. No native path is returned.
A future capability flag without an actual writer fails explicitly;
it cannot produce fake native success.
- Invalid input, unsupported palm mute, corrupt maps, serializer/read-back,
disk/permission/publication failures and unexpected bugs never become
fallback successes.
EXACT NAMED FAILURE BOUNDARIES
Reuses INVALID_REQUEST, UNSUPPORTED_DESCRIPTION, PROFILE_REQUIRED,
UNSUPPORTED_ARTICULATION, NATIVE_FORMAT_UNAVAILABLE, OUTPUT_EXISTS,
OUTPUT_IO_ERROR and SERIALIZATION_ERROR; no new public code invented.
Representative exact details:
- Invalid mutable request: reason=invalid_request_model.
- Inferred invalid associations: reason=resolved_part_association.
- Wrong-part profile: existing PROFILE_REQUIRED / reason=wrong_part.
- Disabled drums native:
{"part":"drums","format":"sdp","reason":"sdp_layout_not_established"}.
- Existing output: OUTPUT_EXISTS / reason=destination_exists.
- Root/ancestor identity or reparse change: OUTPUT_IO_ERROR /
reason=output_root_changed.
- Stage identity/confinement failure: reason=stage_ownership_changed.
- Stage create/name/initial-identity errors:
stage_create_failed, stage_name_collisions, stage_ownership_unestablished.
- Unusable staged file: SERIALIZATION_ERROR / invalid_staged_midi.
- Content changed during read-back: OUTPUT_IO_ERROR / staged_content_changed.
- Result-model failure before rename:
SERIALIZATION_ERROR / invalid_generation_result.
- Rename permission/disk failure: OUTPUT_IO_ERROR / publication_failed.
- Other staged OSError: OUTPUT_IO_ERROR / staging_io_failed.
- Unsafe/failed cleanup: OUTPUT_IO_ERROR / stage_cleanup_failed, with only
original_code=<original domain code or "internal"> when handling failure.
Known domain failures become schema-declared structuredContent with isError.
Successful MCP responses have schema-declared structuredContent and concise
text containing status, parts, warning count and published paths.
Unexpected bugs still propagate through the existing sanitized stderr
diagnostic/SDK error path, not a fabricated typed success or invalid-input blame.
Strict schemas, IDs, initialization, protocol errors, stdout framing and EOF
behavior are preserved.
FILESYSTEM GUARANTEES AND PRECISE LIMITATIONS
- Only the operator chooses the existing root; tools choose only the existing
strict 1-64-character ASCII basename, with reserved Windows names rejected.
- Rejects URI/UNC/device/drive-relative paths, controls, traversal, missing
roots, files, existing root/ancestor symlinks and all Windows reparse points.
GetDriveTypeW rejects mapped network, optical, unknown and no-root volumes;
configured volume must classify fixed/removable/RAM. This is not a write probe.
- Pins device/inode identities for root AND ancestors and rechecks them
throughout staging, cleanup and publication. Checks stage identity,
non-reparse status and resolved-parent confinement.
- One exclusively created UUID stage lives INSIDE the output root. At most
three name attempts. Only <basename>.mid is permitted. Directory inspection
examines at most two entries; unexpected contents refuse publication/cleanup.
- Saved MIDI must be a regular non-reparse file, positive and <=16 MiB.
The service checks actual size, full saved content through the existing
validator, and stable identity/size/mtime. Existing composer/compiler
bounds reject excessive musical events before staging/allocation.
- Constructs/revalidates the anticipated result and JSON representation
before the commit point. No result-model failure publishes a final directory.
- Publishes the whole stage as <root>\<basename> using same-volume Windows
directory rename. The final directory never contains a partial artifact set.
Windows os.rename refuses existing files, populated OR EMPTY directories,
and concurrent same-name destinations. There is no replace/merge/delete
workaround. Exactly one competing request wins; the loser is OUTPUT_EXISTS.
- This publication implementation is explicitly WINDOWS ONLY. Non-Windows
generation fails before staging:
OUTPUT_IO_ERROR / no_replace_publication_requires_windows.
POSIX replacing-rename semantics are never silently accepted.
- Cleanup removes only the specific verified owned stage's expected regular
non-reparse MIDI, then that empty stage directory. It does not recursively
delete root contents, follow junctions, remove other stages or touch sentinels.
Lost ownership, root change, injected content or cleanup permissions can
leave a stage for operator inspection and are explicit cleanup failures.
- Path/lstat/recheck APIs are NOT comprehensive adversarial race protection.
A local actor with replacement rights can race checks versus filesystem
operations. Use trusted operator-controlled ACLs on root/ancestors/volume.
No handle-relative NT namespace confinement, malicious mount/remapping
defense, permanent open identity pin or protection against file-ID reuse
is claimed. mode=0700 is not a Windows ACL guarantee.
- No fsync/power-loss durability or transaction acknowledgement database.
Rename is the publication commit point: a crash/client disconnect afterward
can leave a valid output even if the caller never receives the response.
No destructive post-publication rollback is attempted.
ACTUAL REAL-STDIO AND PERSISTENT DISK EVIDENCE
Launched the actual installed editable module using isolated Python imports:
.venv\Scripts\python.exe -I -B -m riff_groove_builder
--output-dir <repository>\.test-artifacts\step9-verified\outputs
Working directory was the separate owned client-cwd, NOT the repository root.
PID 21540 initialized protocol 2025-11-25, listed exactly two tools and the
three real profiles, generated four files and exercised five known failures.
12 newline-delimited response objects preserved string/integer IDs.
All profile/generation/error structured results passed their declared JSON
Schemas. Closing stdin yielded exit code 0, zero stderr bytes, zero additional
stdout bytes and zero remaining stage directories. The child was reaped.
The empty owned client-cwd was removed after validation.
Persistent outputs are under this absolute root:
D:\_Repos\Code\GAJ3T\MCPs\MusicGen\GAJ3T.MusicGen.Riff-Groove-Builder\
.test-artifacts\step9-verified\outputs
Three-part description:
"lead bass drums 137.5 BPM 2 bars 7/8 Bb minor quarter rests", seed=42.
Resolved: lead/bass/drums; 137.5 BPM; two bars; 7/8; A# minor;
start=0, end=6720, PPQ=960, actual IDs standard.lead.v1/standard.bass.v1/
standard.gm-drums.v1; disclosed omitted rhythm/technique assumptions.
Files relative to that root:
- midi_only\midi_only.mid: 324 bytes; explicit ["mid"]; status=complete;
fallback_for=[]; three STANDARD_MIDI_MAPPING warnings.
- default_all\default_all.mid: 324 bytes; formats omitted;
status=midi_fallback; fallback_for=["lead","bass","drums"]; three standard
mapping plus three native-unavailable warnings.
- native_only\native_only.mid: 324 bytes; formats=["agp","sdp"];
same explicit fallback coverage and six warnings; no native files.
All three actually begin MThd, are SMF type 1 / 960 PPQ, with tracks
Conductor, Lead, Bass, Drums; all four EOTs=6720.
Note-on counts by track: 0,6,6,12. SHA256 for EACH:
7d61b8d4a8c42a19091126ce7301b159744bbae4e78456b0cd1894f6269aea4b
Identical bytes demonstrate unchanged musical content across requested-format
policies, not three independent native encodings.
- bass_only\bass_only.mid: 237 bytes; "bass only 72.5 BPM 3 bars 6/8
Bb minor eighth", seed=0, explicit mid; complete, one mapping warning.
Type 1 / PPQ=960; Conductor/Bass; both EOTs=8640; note-on counts 0,18.
SHA256:
9dbd7e537c462a4d1a041d954cb91cd586c23c7b85ca5f820d33eda86724daf9
The same real process also returned exact expected typed failures for:
bars=1.0 -> INVALID_REQUEST;
inferred drums plus agp -> INVALID_REQUEST;
native-disabled drums -> NATIVE_FORMAT_UNAVAILABLE;
lead palm mute -> UNSUPPORTED_ARTICULATION;
existing midi_only basename -> OUTPUT_EXISTS.
No bad/disabled/unsupported directories, AGP/SDP files or sidecars exist.
Each published directory contains only its actual correctly named .mid.
An independent follow-up disk inspection verified all sizes, hashes, headers,
track names, common endings and note counts above.
TESTS, STATIC CHECKS AND EXACT FINAL RESULTS
Environment: existing .venv, CPython 3.13.13, Windows AMD64.
No other runtime/OS/receiver/plugin setup is certified.
No dependencies were installed/restored or manifests changed.
Existing distribution tests still perform their isolated offline wheel installs.
Repository-local subprocess scratch setup:
$scratch = New-Item -ItemType Directory -Force '.test-artifacts\scratch'
$env:TEMP = $scratch.FullName
$env:TMP = $scratch.FullName
$env:TMPDIR = $scratch.FullName
$env:PIP_NO_INDEX = '1'
Final focused:
& '.\.venv\Scripts\python.exe' -m pytest tests\unit\test_service.py tests\unit\test_server.py tests\integration\test_generation_service.py tests\integration\test_stdio.py -q --tb=short
PASS: 140 passed in 13.89s; 0 failed, 0 skipped.
Final complete current suite:
& '.\.venv\Scripts\python.exe' -m pytest -q --tb=short
PASS: 1118 passed in 34.95s; 0 failed, 0 skipped.
Includes strict contracts/errors, all standard/vendor-test mechanics, parser,
composer, compiler, MIDI/MIDIUtil read-back, both native-unavailable boundaries,
real stdio protocol/generation, offline sdist/wheel build/install and installed/
ZIP-wheel packaged-resource regressions from different working directories.
& '.\.venv\Scripts\python.exe' -m ruff check .
PASS: All checks passed!
& '.\.venv\Scripts\python.exe' -m ruff format --check .
PASS: 67 files already formatted (final rerun after persisting this report).
& '.\.venv\Scripts\python.exe' -m mypy --strict src tests
PASS: Success: no issues found in 38 source files.
& '.\.venv\Scripts\python.exe' -m pip check
PASS: No broken requirements found.
git --no-pager diff --check
PASS: no tracked-diff whitespace errors. Most existing implementation files
remain untracked; Ruff checked actual authored Python files independently.
All seven authored/updated Python files also have no editor-reported problems.
Editor runTests reported no discovered tests; this is not a pass or pytest skip.
The explicit venv pytest commands above are the actual execution evidence.
TEST COVERAGE AND ISSUES RESOLVED
Tests cover default/explicit/native-only formats; exact warning/status/part/
profile/fallback coverage; MIDI-only with fallback disabled; post-inference
associations; unsupported palm mute; mutated/unchecked request graphs and
corrupt maps; future flags cannot invent codecs; event-budget rejection;
partial writes, actual corruption, read-back IO, result-model failures,
unexpected bugs, changing content, root/ancestor identity replacement,
bounded stage retries, create/publication/cleanup failures and ownership.
Existing file/populated/EMPTY directory identity and sentinel preservation
are checked, as are competing same-basename requests with one winner.
Genuine Windows junction tests ALL RAN AND PASSED (five parameter cases):
root, ancestor, pinned root replacement, destination and staged-directory
replacement. These are actual mklink /J reparse points, not only mocked flags.
They do not certify comprehensive hostile race safety. Every created junction
was explicitly removed; outside sentinels survived. No environment skips.
Local-volume classification and reparse flags also have focused unit coverage.
Initial focused run: six assertions incorrectly expected lowercase MIDI track
names. Existing writer contract is Conductor/Lead/Bass/Drums; corrected the new
tests, not the writer or its checks. Initial Ruff findings were unused import
and line wrapping; fixed/formatted authored files with no rule relaxation.
Final suite, lint, format, strict mypy and pip check all pass.
CLEANUP AND SEPARATE READINESS
Owned UUID service/server/stdio/junction/MIDI/distribution test workspaces
cleaned up. Only existing scratch and the deliberately retained step9-verified
MIDI evidence directory remain under .test-artifacts. No generated evidence
script, child server, stage directory or background helper remains.
Existing caches/venv/dist and unrelated worktree content were preserved.
Software Standard MIDI MCP generation: VERIFIED, ready for review/use on the
tested Windows configuration. No implementation blocker remains for Step 9.
Verified vendor Eclips/Ray/SD3 mapping acceptance: STILL OPEN.
Installed product/build/library/preset/map, actual control/range/layer behavior
and evidence are not established by these standard maps.
Real AGP and SDP layouts/serializers: BLOCKED separately. Existing step_7_fix.md
and step_8_fix.md remain untouched and unresolved. No genuine native fixtures,
native serialization/parser round trips or native success were manufactured.
Actual Eclips/Ray/SD3 and Cubase import/routing/load/save/reload acceptance:
UNRUN, not test passes or skips. The amended plan permits this independent
standard-MIDI completion, not full native/application/release acceptance.
Ready for review of this step, not a claim that those separate gates are passed.
Stopped after verified implementation/evidence/report; no later prompt executed.