Using AudioDojo on Chutes

AudioDojo is a community chute by vonkaiser that runs 12 audio models behind 14 REST paths on one deployment: text-to-speech, voice cloning, voice design, three ASR engines, and six audio-processing utilities. Every model stays loaded in GPU memory, so switching endpoints has no cold-start delay, and one API key on one host covers a complete audio pipeline.

Overview

Rather than one model per chute, AudioDojo co-locates a curated audio stack. The TTS side pairs the tiny, fast Kokoro-82M with the January-2026 Qwen3-TTS 12Hz 1.7B family, which brings zero-shot voice cloning, preset speakers with emotion instructions, and voice design from text descriptions. The STT side offers three engines at different accuracy/language/latency trade-offs (Whisper large-v3-turbo, NVIDIA Canary-Qwen 2.5B, NVIDIA Parakeet TDT 0.6B v3). The processing side wraps standard open-source components: Silero VAD, SpeechBrain's MetricGAN+ and ECAPA-TDNN, SepFormer separation, Whisper-based language detection, and speaker diarization. The /speak path is a deliberate drop-in alias for the original standalone chutes/kokoro chute.

Model specifications

Every bundled model, its endpoint, and its verified facts:

Quick start

All endpoints share the host https://vonkaiser-audiodojo.chutes.ai, flat JSON bodies, and Bearer auth. TTS returns WAV bytes; STT and analysis return JSON.

# TTS (Kokoro): returns audio/wav
curl -X POST "https://vonkaiser-audiodojo.chutes.ai/tts/kokoro" \
  -H "Authorization: Bearer $CHUTES_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text": "Hello from AudioDojo.", "voice": "af_heart", "speed": 1}' \
  --output output.wav

# STT (Whisper): returns JSON with transcript and word timestamps
curl -X POST "https://vonkaiser-audiodojo.chutes.ai/stt/whisper" \
  -H "Authorization: Bearer $CHUTES_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"audio_b64": "<base64-encoded audio>", "return_timestamps": true}'

import base64
import os
import requests

HOST = "https://vonkaiser-audiodojo.chutes.ai"
HEADERS = {"Authorization": f"Bearer {os.environ['CHUTES_API_KEY']}"}

# Voice cloning: 3-6s reference clip -> new speech in that voice
ref_b64 = base64.b64encode(open("reference.wav", "rb").read()).decode()
resp = requests.post(f"{HOST}/tts/qwen3clone", headers=HEADERS, json={
    "text": "This is my cloned voice speaking new words.",
    "ref_audio_b64": ref_b64,
    "language": "English",     # default "English"
    "trim_sample": False,      # default False: reject refs over 6s
    "high_fidelity": False,    # default False: embedding-only mode, no bleed-through
})
open("cloned.wav", "wb").write(resp.content)

# Speaker verification: JSON back
a = base64.b64encode(open("clip_a.wav", "rb").read()).decode()
b = base64.b64encode(open("clip_b.wav", "rb").read()).decode()
print(requests.post(f"{HOST}/audio/verifyspeaker", headers=HEADERS,
                    json={"audio_a_b64": a, "audio_b_b64": b}).json())

import { writeFile } from "node:fs/promises";

const HOST = "https://vonkaiser-audiodojo.chutes.ai";

// Voice design: describe the voice in plain English
const resp = await fetch(`${HOST}/tts/qwen3design`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.CHUTES_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    text: "Welcome to the show.",
    instruct: "A warm, confident male narrator with a slight British accent",
    language: "English",
  }),
});
await writeFile("designed.wav", Buffer.from(await resp.arrayBuffer()));

Parameters and tuning

Key request fields per endpoint group (defaults from the chute's llms.txt):

• Kokoro TTS (/speak, /tts/kokoro): text (required), voice (default af_heart, 54 options), speed (default 1, range 0.1-3.0).
• Voice cloning (/tts/qwen3clone): ref_audio_b64 (required, max 6s), ref_text (optional transcript; Whisper auto-transcribes if empty), trim_sample (default false: over-6s refs are rejected), auto_denoise (default false: runs MetricGAN+ on noisy references), high_fidelity (default false: speaker-embedding mode; true switches to ICL mode for closer prosody at the risk of reference bleed-through).
• Custom speaker (/tts/qwen3custom): speaker (Vivian, Serena, Uncle Fu, Dylan, Eric, Ryan, Aiden, Ono Anna, Sohee; defaults to Ryan), instruct (free-text delivery style such as "whispering").
• Voice design (/tts/qwen3design): instruct (required voice description) plus text and language.
• STT: audio_b64 (required everywhere); Whisper adds language (BCP-47, auto-detected if omitted) and return_timestamps (default true); Canary adds max_new_tokens (default 256); Parakeet adds timestamps (default true).
• Processing: all six take audio_b64 (verification takes audio_a_b64 and audio_b_b64).

What it's best at

The chute shines when a workflow touches several audio tasks: record, denoise, transcribe, verify the speaker, then synthesize a reply, all against one host with no cold starts. Voice cloning from a few seconds of reference audio and voice design from a text description are capabilities the standalone Kokoro chute does not have. For ASR, you can route per clip: Whisper for anything multilingual, Canary for maximum-accuracy English, Parakeet for fast European-language transcription with timestamps.

Less ideal: single-model workloads at large scale (a dedicated chute is simpler to reason about), music or singing synthesis (these are speech models), and ASR in languages outside Whisper's coverage.

How Chutes serves this model

All 14 paths live on https://vonkaiser-audiodojo.chutes.ai with flat JSON request bodies and Authorization: Bearer $CHUTES_API_KEY. Output content types follow the task: audio/wav for the five TTS paths plus denoise and separate, application/json for STT, VAD, diarization, speaker verification, and language detection. Audio inputs are raw base64 strings in common formats; large playground uploads use temporary blob storage automatically, but API clients send base64 directly. Per the chute readme, all 12 models are held in GPU memory simultaneously, eliminating cold starts between calls, and /speak preserves the exact contract of the retired standalone Kokoro deployment.

Related resources: the model page, the machine-readable llms.txt, and the callable OpenAPI spec.

FAQ

Which model serves which endpoint?

TTS: /speak and /tts/kokoro use Kokoro-82M; /tts/qwen3clone, /tts/qwen3custom, and /tts/qwen3design use Qwen3-TTS 1.7B (Base, CustomVoice, and VoiceDesign checkpoints respectively). STT: /stt/whisper is Whisper large-v3-turbo, /stt/canary is NVIDIA Canary-Qwen 2.5B, /stt/parakeet is NVIDIA Parakeet TDT 0.6B v3. Processing: /audio/vad is Silero VAD, /audio/denoise is MetricGAN+, /audio/separate is SepFormer, /audio/verifyspeaker is ECAPA-TDNN, /audio/detectlanguage is Whisper, and /audio/diarize is a diarization pipeline (model unnamed in the chute readme).

How does voice cloning work and what reference audio do I need?

POST to /tts/qwen3clone with ref_audio_b64 (a base64 reference clip of up to 6 seconds), the text to speak, and optionally ref_text (its transcript; Whisper auto-transcribes if omitted). Set trim_sample true to auto-trim longer clips, auto_denoise true to clean noisy references with MetricGAN+, and high_fidelity true for ICL mode, which mimics prosody more closely but can bleed the reference's last syllable into the output.

Which STT endpoint should I pick?

Whisper large-v3-turbo for multilingual audio (99 languages) with optional word timestamps. Canary-Qwen 2.5B for the highest-accuracy English transcription, especially complex or domain-specific audio; it is English-only. Parakeet TDT 0.6B v3 for fast transcription of 25 European languages with word and segment timestamps and automatic punctuation.

What formats do requests and responses use?

All requests are flat JSON. Audio inputs are base64 strings in fields like audio_b64 and ref_audio_b64, accepting common audio formats. TTS endpoints and denoise/separate return raw audio/wav bytes; STT, VAD, diarization, speaker verification, and language detection return JSON.

Is /speak compatible with the old standalone Kokoro chute?

Yes, by design. It keeps the same request and response contract (text, voice, speed in, audio/wav at 24 kHz out) and runs the identical Kokoro-82M pipeline as /tts/kokoro. Migrating from chutes-kokoro means changing only the base URL to https://vonkaiser-audiodojo.chutes.ai.

Can I use the outputs commercially?

Licenses vary per bundled model: Kokoro, Qwen3-TTS, MetricGAN+, and ECAPA-TDNN are Apache-2.0; Whisper large-v3-turbo is MIT; NVIDIA's Canary-Qwen and Parakeet are CC-BY-4.0 (commercial use permitted with attribution, per their model cards). Check the license of the specific model behind the endpoint you use.

Do models cold-start when I switch endpoints?

No. The chute readme states all 12 models are loaded in GPU memory at all times, so any endpoint responds without a model-load delay. This is the main operational advantage over running 12 separate single-model deployments.

What languages does Qwen3-TTS support?

Ten, per the Qwen3-TTS model cards: Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, and Italian, plus several dialect voice profiles. Pass the target language name in the endpoint's language field (default English).

Sources

• AudioDojo on Chutes
• Chute llms.txt (endpoint reference)
• hexgrad/Kokoro-82M
• Qwen/Qwen3-TTS-12Hz-1.7B-Base
• Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice
• Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign
• openai/whisper-large-v3-turbo
• nvidia/canary-qwen-2.5b
• nvidia/parakeet-tdt-0.6b-v3
• onnx-community/silero-vad
• speechbrain/metricgan-plus-voicebank
• speechbrain/spkrec-ecapa-voxceleb