10 KiB
Voice Studio
A local, browser-based voice-character training workbench for ACE-Step 1.5 XL. Maintain any number of characters, each with separate recordings, annotations, dataset versions, jobs, logs, and exports.
Delivery status
Source implementation only. At the user's request, the application has not been launched or tested, dependencies have not been installed, and no models have been trained. No services or containers have been started.
The default paths reflect the existing local installation inspected during development. Header inspection found six stereo, 48 kHz PCM-16 WAV recordings in the initial character folder, totaling approximately 17 minutes. The audio has not been listened to or assessed for vocal quality or training permission.
Start when ready
From this project directory in PowerShell:
.\Start-VoiceStudio.ps1
Open http://127.0.0.1:9874. The launcher does not install packages or open another ACE-Step instance. It uses:
- ACE-Step installation:
D:\ai\ACE-Step-1.5 - Python:
D:\ai\ACE-Step-1.5\.venv\Scripts\python.exe - Character library:
..\Voice Characters - Generated data:
.\workspace
To supply another interpreter or installation:
.\Start-VoiceStudio.ps1 -AceStepRoot "D:\Tools\ACE-Step-1.5" `
-Python "D:\Tools\ACE-Step-1.5\.venv\Scripts\python.exe" -Port 9874
The browser server requires Python 3.10+ and the small set of packages in
requirements.txt. It does not install the ACE-Step runtime. If those
packages are missing, install them deliberately into the chosen interpreter:
& "D:\ai\ACE-Step-1.5\.venv\Scripts\python.exe" -m pip install -r .\requirements.txt
Review changes to a shared environment before installing. ACE-Step itself must have its own CUDA-enabled Torch, training dependencies, and downloaded model assets. Training requires an NVIDIA CUDA GPU; memory requirements depend on the XL variant, clip duration, and upstream implementation.
No Docker, Postman, cloud services, paid APIs, or MCP server is required.
This is a local web application, not an MCP integration. The browser server
binds only to 127.0.0.1; do not expose it through a public proxy.
Multiple voice characters
Every immediate subfolder of the character library is a separate character.
The existing GothMetalMaleV2 folder is discovered, not hard-coded.
Use New character to create another folder and independent workspace, then Add recordings to copy WAV, FLAC, or OGG files into it. Uploads receive unique filenames; existing files are never overwritten. Alternatively, add a character subfolder and audio files directly to the library before opening or refreshing the studio.
Display names, generation triggers, default captions, and language are editable. Use a distinct trigger per voice. Identity changes apply to future datasets and do not silently rewrite old annotations.
Environment overrides:
| Variable | Meaning |
|---|---|
ACESTEP_ROOT |
Default ACE-Step installation |
VOICE_STUDIO_SOURCE |
Default recording library |
VOICE_STUDIO_WORKSPACE |
Generated datasets, settings, job history, and exports |
Saved settings override the default installation and source paths. The output workspace is selected before launch, not moved through the UI.
Dataset preparation and review
- Select or create a character.
- Add isolated vocals that you own or are authorized to use.
- Set the clip duration and silence threshold, then prepare a dataset.
- Listen to the resulting clips. Correct captions and enter accurate lyrics, syllables, or descriptions of the actual non-lexical vocalisation.
- Mark each clip reviewed and save.
Preparation creates a new version every time:
workspace\
<character-id>\
character.json
datasets\
<version>\
ace_audio\ 48 kHz stereo PCM-16 WAV
svc_audio\ 40 kHz mono PCM-16 WAV
dataset.json Native ACE-Step samples/metadata
segments.json Original paths and segment timestamps
tensors\ Generated only by explicit preprocessing
ready.json Written only after complete preprocessing
jobs\
<job-id>\
job.json
output.log
recipe.json Training jobs only
adapter\ Training outputs, if produced by the backend
Silence detection uses 20 ms energy windows, brief boundary padding, and a minimum retained segment length of 1.5 seconds. Long spans are cut near a low-energy boundary where possible; audition cuts because this is not a phoneme-aware segmenter. Quiet clips below the configured threshold are excluded and reported. Very short isolated sounds may be omitted.
Audio is not denoised, separated, pitch-corrected, or transcribed automatically. Resampling peaks are limited only when needed to prevent output clipping. For SVC, a phase-cancelling stereo downmix falls back to the strongest channel and produces a warning. Original recordings are never modified.
Unknown lyrics remain blank. Vocal data is explicitly not marked instrumental. ACE-Step preprocessing is blocked until every clip has vocal text, a caption, and a listening-review confirmation. This does not guarantee that an annotation is correct; it is a human review gate.
Path A: new songs with ACE-Step XL
- Open Train a voice.
- Select the exact installed XL variant: XL-SFT, XL-Turbo, or XL-Base.
- Confirm permission and suitable vocal data.
- Run Preprocess clips. The studio invokes upstream's two-pass
preprocess_audio_filesin a separate process using the ACE-Step interpreter. - Run Start training after preprocessing completes.
The integration targets the local ACE-Step source interfaces inspected at
commit 14c0211:
acestep.training_v2.preprocess.preprocess_audio_filestrain.py --plain --yes fixed
Commands use the literal checkpoint directory name plus an explicit
--base-model xl_*. This avoids the inspected CLI path validator's missing
XL aliases while retaining the correct XL family for training configuration.
No upstream files are modified.
Starting settings: batch size 1, BF16, rank 16, learning rate 0.0001, gradient accumulation 4, gradient checkpointing, encoder offload, AdamW, and zero DataLoader workers. These are starting settings, not a validated 24 GB memory guarantee. Default 12-second clips reduce activation memory. Close other GPU workloads before training. Lower clip length/rank if needed.
Every preprocessing attempt gets a separate tensor directory. Completion requires all expected samples to succeed. A dataset hash and XL variant are recorded; changes invalidate readiness and require preprocessing again.
The console displays actual subprocess output, not estimated loss curves or
invented progress. Nonzero exits and explicit backend [FAIL] reports are
failures. Training completion additionally requires final .safetensors
weights and their adapter configuration. That check does not certify voice
quality: audition checkpoints with prompts and melodies not used in training.
Load the resulting adapter directory, retaining adapter_config.json and
its weights together, through the matching ACE-Step XL model's adapter loader.
Include the trigger recorded in that training job's recipe.json in the
generation caption; renaming a character does not rewrite an old model's
trigger. In ComfyUI, the
installed nodes must explicitly support the XL variant and the saved adapter
format; compatibility is not assumed and weights are not silently converted.
Syllable runs can provide timbre and pitch coverage, but do not establish good lyric pronunciation or full-song generalisation. A voice-only set can also bias outputs toward a cappella performances. Add authorized clean sung phrases with varied pitches, vowels, consonants, dynamics, and accurately annotated lyrics; evaluate on held-out performances before relying on the character.
Path B: singing voice conversion
For preserving an existing singer's melody, timing, and performance while changing the voice, use a separate RVC model, for example through Applio.
- Prepare and listen to the character's vocal clips.
- Open Models & exports and build an SVC dataset export.
- Download and extract the ZIP. It contains the mono 40 kHz WAVs and local training handoff instructions, not a trained conversion model.
- In a separately installed Applio environment, select a unique model name and the extracted character folder; use 40 kHz, preprocess, extract pitch/features, train, and build its retrieval index following the current Applio training guide.
- Use its
.pthand.indexfor conversion of isolated source vocals, then mix the converted vocals back with the instrumental.
Applio installation, downloads, training, and conversion are not automated by this studio. That boundary avoids guessing an unselected backend version or mixing conflicting Torch dependencies. SVC training generally does not need lyrics, so the export is available without ACE-Step lyric annotation. It still requires human listening and authorized audio.
An ACE-Step LoRA is not an RVC/so-vits-svc model. Renaming an adapter does not convert it. ACE-Step audio conditioning/cover workflows may be useful, but are not a guaranteed drop-in replacement for performance-preserving SVC.
Job and file behaviour
- One dataset/GPU/export job runs at a time. Mutations are blocked during jobs.
- History, logs, and artifacts persist per character. Interrupted jobs are marked failed on the next studio startup, never reported as completed.
- Stopping the studio requests termination of its active backend process. There is no browser cancellation/resume UI. A subsequent training request starts a new run; native CLI resume remains an upstream advanced workflow.
- Preparation and uploads write only new files. Earlier datasets and partial failed runs are retained for inspection; no automatic deletion is performed.
- Model weights and audio remain local. The application contains no telemetry, remote fonts, external JavaScript, or audio-upload services.