Using Qwen3.5-397B-A17B on Chutes

Qwen3.5-397B-A17B is the flagship open-weight model of the Qwen3.5 generation: a unified vision-language Mixture-of-Experts model with 397B total parameters, only 17B active per token, and a 262K native context, released under Apache-2.0. On Chutes it runs as the official FP8 build inside a Trusted Execution Environment, behind the OpenAI-compatible gateway.

Overview

Qwen3.5 integrates multimodal learning, architectural efficiency, and large-scale reinforcement learning into one foundation model. Early-fusion training on multimodal tokens gives it cross-generational parity with the text-only Qwen3 line while outperforming Qwen3-VL models on reasoning, coding, agents, and visual understanding. RL was scaled across million-agent environments with progressively complex task distributions, and language coverage extends to 201 languages and dialects. It is the open counterpart of the hosted Qwen3.5-Plus service.

The architecture is a hybrid: the 60-layer language model interleaves Gated DeltaNet linear-attention blocks with periodic gated-attention blocks in a 3:1 pattern (15 repetitions of three DeltaNet-MoE blocks followed by one gated-attention-MoE block). Each MoE layer holds 512 experts, activating 10 routed plus 1 shared per token. Full-attention layers use 32 query heads against just 2 KV heads. This layout keeps inference fast and memory-light at long context: 262,144 tokens natively, extensible up to 1,010,000.

Model specifications

Quick start

Base URL https://llm.chutes.ai/v1, model Qwen/Qwen3.5-397B-A17B-TEE, Bearer auth with your Chutes API key.

curl -X POST "https://llm.chutes.ai/v1/chat/completions" \
  -H "Authorization: Bearer $CHUTES_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3.5-397B-A17B-TEE",
    "messages": [{"role": "user", "content": "Explain the trade-offs of linear attention."}],
    "stream": true,
    "max_tokens": 1024,
    "temperature": 0.7
  }'

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://llm.chutes.ai/v1",
    api_key=os.environ["CHUTES_API_KEY"],
)

response = client.chat.completions.create(
    model="Qwen/Qwen3.5-397B-A17B-TEE",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What does this diagram show?"},
            {"type": "image_url", "image_url": {"url": "data:image/png;base64,<BASE64>"}},
        ],
    }],
    max_tokens=2048,
    temperature=0.6,
)
print(response.choices[0].message.content)

const res = await fetch("https://llm.chutes.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.CHUTES_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "Qwen/Qwen3.5-397B-A17B-TEE",
    messages: [{ role: "user", content: "Translate this changelog into Japanese and Swahili." }],
    stream: false,
    max_tokens: 1024,
    temperature: 0.7,
  }),
});
const data = await res.json();
console.log(data.choices[0].message.content);

Parameters and tuning

Chute request fields (from the live llms.txt) and the model card's recommended sampling:

The card's other best practices: keep thinking content out of multi-turn history (only resend final outputs), and use explicit output-format instructions when benchmarking.

What it's best at

• Multimodal reasoning. One model handles documents, screenshots, diagrams, and video alongside text, with visual understanding the card reports as ahead of the Qwen3-VL line.
• Agentic workflows. RL scaled across million-agent environments targets exactly the robustness tool-use loops need.
• Multilingual products. 201 languages and dialects with regional nuance.
• Long-context work. 262K native window, extensible to about 1M, with linear attention keeping long-context inference affordable.
• High-throughput serving. 17B active parameters per token means far lower per-request compute than dense models of comparable quality.

It does not accept audio input, produce speech, or generate images (it consumes images but only emits text). And since this chute serves the FP8 build, outputs are not bit-identical to the BF16 checkpoint.

How Chutes serves this model

This chute serves the official FP8 release (Qwen/Qwen3.5-397B-A17B-FP8: e4m3 weights, dynamic activation scaling) inside a TEE, meaning inference runs on attested confidential-compute hardware and prompts and outputs are processed within the enclave. It runs on the vLLM template with the OpenAI-compatible surface: POST /v1/chat/completions, POST /v1/completions, and GET /v1/models, all with streaming and flat JSON request bodies. See the model page, llms.txt, and openapi.json.

FAQ

What does 397B-A17B mean?

397B total parameters with 17B activated per token. It is a sparse Mixture-of-Experts model: each MoE layer holds 512 experts but routes each token through only 10 routed plus 1 shared expert, so you get large-model quality at a fraction of the per-token compute.

Can it process images?

Yes. Qwen3.5 is a unified vision-language model with a vision encoder trained by early fusion, and its card reports results on visual reasoning and video benchmarks. Send OpenAI-style multimodal content parts (image_url) in your chat messages.

What context window does it support?

262,144 tokens natively, extensible up to 1,010,000 per the model card. The chute's default max_tokens is 1,024; the card recommends 32,768-token outputs for most queries and up to 81,920 for hard math and coding.

What sampling settings should I use?

For thinking mode: temperature 0.6, top_p 0.95, top_k 20, min_p 0 (also the generation_config.json defaults). For non-thinking mode: temperature 0.7, top_p 0.8, top_k 20. presence_penalty between 0 and 2 reduces endless repetition, at some risk of language mixing.

Can I use it commercially?

Yes. The model is released under the Apache-2.0 license, which permits commercial use, modification, and redistribution.

What does the TEE suffix mean?

TEE stands for Trusted Execution Environment. Inference runs inside attested confidential-compute hardware, so your prompts and the model's outputs are processed within a hardware-isolated enclave rather than on an open host.

How do I call it from the OpenAI SDK?

Set base_url to https://llm.chutes.ai/v1 with your Chutes API key and model to Qwen/Qwen3.5-397B-A17B-TEE. Chat completions, raw completions, and streaming work through the standard SDK methods.

Should I keep thinking content in multi-turn history?

No. The model card's best practices say historical turns should contain only the final output, not the thinking content. The official chat template handles this automatically; if you manage history yourself, strip reasoning blocks before resending.

Sources

• Qwen/Qwen3.5-397B-A17B on Hugging Face
• Qwen/Qwen3.5-397B-A17B-FP8 on Hugging Face
• Qwen3.5-397B-A17B-TEE model page on Chutes
• Live llms.txt for this chute