Voice-Model-Training/AceStepXL-Training
gaj3t 3b8fd86f0d Update AceStepXL-Training/README.md 2026-11-12 03:56:21 +00:00
..
src/voice_studio Copying More Repo to Demo 2026-10-26 05:16:59 -04:00
.gitignore Copying More Repo to Demo 2026-10-26 05:16:59 -04:00
README.md Update AceStepXL-Training/README.md 2026-11-12 03:56:21 +00:00
Start-VoiceStudio.ps1 Copying More Repo to Demo 2026-10-26 05:16:59 -04:00
requirements.txt Copying More Repo to Demo 2026-10-26 05:16:59 -04:00

README.md

Voice Studio

A local, browser-based voice-character training workbench for ACE-Step 1.5 XL. Maintain any number of characters, each with separate recordings, annotations, dataset versions, jobs, logs, and exports.

Delivery status

Source implementation only. At the user's request, the application has not been launched or tested, dependencies have not been installed, and no models have been trained. No services or containers have been started.

The default paths reflect the existing local installation inspected during development.

Start when ready

From this project directory in PowerShell:

.\Start-VoiceStudio.ps1

Open http://127.0.0.1:9874. The launcher does not install packages or open another ACE-Step instance. It uses:

  • ACE-Step installation: D:\ai\ACE-Step-1.5
  • Python: D:\ai\ACE-Step-1.5\.venv\Scripts\python.exe
  • Character library: ..\Voice Characters
  • Generated data: .\workspace

To supply another interpreter or installation:

.\Start-VoiceStudio.ps1 -AceStepRoot "D:\Tools\ACE-Step-1.5" `
  -Python "D:\Tools\ACE-Step-1.5\.venv\Scripts\python.exe" -Port 9874

The browser server requires Python 3.10+ and the small set of packages in requirements.txt. It does not install the ACE-Step runtime. If those packages are missing, install them deliberately into the chosen interpreter:

& "D:\ai\ACE-Step-1.5\.venv\Scripts\python.exe" -m pip install -r .\requirements.txt

Review changes to a shared environment before installing. ACE-Step itself must have its own CUDA-enabled Torch, training dependencies, and downloaded model assets. Training requires an NVIDIA CUDA GPU; memory requirements depend on the XL variant, clip duration, and upstream implementation.

No Docker, Postman, cloud services, paid APIs, or MCP server is required. This is a local web application, not an MCP integration. The browser server binds only to 127.0.0.1; do not expose it through a public proxy.

Multiple voice characters

Every immediate subfolder of the character library is a separate character. The existing GothMetalMaleV2 folder is discovered, not hard-coded.

Use New character to create another folder and independent workspace, then Add recordings to copy WAV, FLAC, or OGG files into it. Uploads receive unique filenames; existing files are never overwritten. Alternatively, add a character subfolder and audio files directly to the library before opening or refreshing the studio.

Display names, generation triggers, default captions, and language are editable. Use a distinct trigger per voice. Identity changes apply to future datasets and do not silently rewrite old annotations.

Environment overrides:

Variable Meaning
ACESTEP_ROOT Default ACE-Step installation
VOICE_STUDIO_SOURCE Default recording library
VOICE_STUDIO_WORKSPACE Generated datasets, settings, job history, and exports

Saved settings override the default installation and source paths. The output workspace is selected before launch, not moved through the UI.

Dataset preparation and review

  1. Select or create a character.
  2. Add isolated vocals that you own or are authorized to use.
  3. Set the clip duration and silence threshold, then prepare a dataset.
  4. Listen to the resulting clips. Correct captions and enter accurate lyrics, syllables, or descriptions of the actual non-lexical vocalisation.
  5. Mark each clip reviewed and save.

Preparation creates a new version every time:

workspace\
  <character-id>\
    character.json
    datasets\
      <version>\
        ace_audio\       48 kHz stereo PCM-16 WAV
        svc_audio\       40 kHz mono PCM-16 WAV
        dataset.json     Native ACE-Step samples/metadata
        segments.json    Original paths and segment timestamps
        tensors\         Generated only by explicit preprocessing
        ready.json       Written only after complete preprocessing
    jobs\
      <job-id>\
        job.json
        output.log
        recipe.json      Training jobs only
        adapter\         Training outputs, if produced by the backend

Silence detection uses 20 ms energy windows, brief boundary padding, and a minimum retained segment length of 1.5 seconds. Long spans are cut near a low-energy boundary where possible; audition cuts because this is not a phoneme-aware segmenter. Quiet clips below the configured threshold are excluded and reported. Very short isolated sounds may be omitted.

Audio is not denoised, separated, pitch-corrected, or transcribed automatically. Resampling peaks are limited only when needed to prevent output clipping. For SVC, a phase-cancelling stereo downmix falls back to the strongest channel and produces a warning. Original recordings are never modified.

Unknown lyrics remain blank. Vocal data is explicitly not marked instrumental. ACE-Step preprocessing is blocked until every clip has vocal text, a caption, and a listening-review confirmation. This does not guarantee that an annotation is correct; it is a human review gate.

Path A: new songs with ACE-Step XL

  1. Open Train a voice.
  2. Select the exact installed XL variant: XL-SFT, XL-Turbo, or XL-Base.
  3. Confirm permission and suitable vocal data.
  4. Run Preprocess clips. The studio invokes upstream's two-pass preprocess_audio_files in a separate process using the ACE-Step interpreter.
  5. Run Start training after preprocessing completes.

The integration targets the local ACE-Step source interfaces inspected at commit 14c0211:

  • acestep.training_v2.preprocess.preprocess_audio_files
  • train.py --plain --yes fixed

Commands use the literal checkpoint directory name plus an explicit --base-model xl_*. This avoids the inspected CLI path validator's missing XL aliases while retaining the correct XL family for training configuration. No upstream files are modified.

Starting settings: batch size 1, BF16, rank 16, learning rate 0.0001, gradient accumulation 4, gradient checkpointing, encoder offload, AdamW, and zero DataLoader workers. These are starting settings, not a validated 24 GB memory guarantee. Default 12-second clips reduce activation memory. Close other GPU workloads before training. Lower clip length/rank if needed.

Every preprocessing attempt gets a separate tensor directory. Completion requires all expected samples to succeed. A dataset hash and XL variant are recorded; changes invalidate readiness and require preprocessing again.

The console displays actual subprocess output, not estimated loss curves or invented progress. Nonzero exits and explicit backend [FAIL] reports are failures. Training completion additionally requires final .safetensors weights and their adapter configuration. That check does not certify voice quality: audition checkpoints with prompts and melodies not used in training.

Load the resulting adapter directory, retaining adapter_config.json and its weights together, through the matching ACE-Step XL model's adapter loader. Include the trigger recorded in that training job's recipe.json in the generation caption; renaming a character does not rewrite an old model's trigger. In ComfyUI, the installed nodes must explicitly support the XL variant and the saved adapter format; compatibility is not assumed and weights are not silently converted.

Syllable runs can provide timbre and pitch coverage, but do not establish good lyric pronunciation or full-song generalisation. A voice-only set can also bias outputs toward a cappella performances. Add authorized clean sung phrases with varied pitches, vowels, consonants, dynamics, and accurately annotated lyrics; evaluate on held-out performances before relying on the character.

Path B: singing voice conversion

For preserving an existing singer's melody, timing, and performance while changing the voice, use a separate RVC model, for example through Applio.

  1. Prepare and listen to the character's vocal clips.
  2. Open Models & exports and build an SVC dataset export.
  3. Download and extract the ZIP. It contains the mono 40 kHz WAVs and local training handoff instructions, not a trained conversion model.
  4. In a separately installed Applio environment, select a unique model name and the extracted character folder; use 40 kHz, preprocess, extract pitch/features, train, and build its retrieval index following the current Applio training guide.
  5. Use its .pth and .index for conversion of isolated source vocals, then mix the converted vocals back with the instrumental.

Applio installation, downloads, training, and conversion are not automated by this studio. That boundary avoids guessing an unselected backend version or mixing conflicting Torch dependencies. SVC training generally does not need lyrics, so the export is available without ACE-Step lyric annotation. It still requires human listening and authorized audio.

An ACE-Step LoRA is not an RVC/so-vits-svc model. Renaming an adapter does not convert it. ACE-Step audio conditioning/cover workflows may be useful, but are not a guaranteed drop-in replacement for performance-preserving SVC.

Job and file behaviour

  • One dataset/GPU/export job runs at a time. Mutations are blocked during jobs.
  • History, logs, and artifacts persist per character. Interrupted jobs are marked failed on the next studio startup, never reported as completed.
  • Stopping the studio requests termination of its active backend process. There is no browser cancellation/resume UI. A subsequent training request starts a new run; native CLI resume remains an upstream advanced workflow.
  • Preparation and uploads write only new files. Earlier datasets and partial failed runs are retained for inspection; no automatic deletion is performed.
  • Model weights and audio remain local. The application contains no telemetry, remote fonts, external JavaScript, or audio-upload services.

Upstream references