Using Qwen3-32B on Chutes

Qwen3-32B is the largest dense model in the Qwen3 family: a 32.8B-parameter hybrid-reasoning LLM that switches between explicit thinking and fast direct answers. On Chutes it runs inside a Trusted Execution Environment (TEE) in FP8 with DFLASH speculative decoding, behind a standard OpenAI-compatible API.

Overview

Released by the Qwen team in April 2025 under Apache-2.0, Qwen3-32B is a dense causal transformer (Qwen3ForCausalLM) with 64 layers, grouped-query attention (64 query heads, 8 KV heads, head dim 128), a 151,936-token vocabulary, and RoPE theta of 1,000,000. Its headline feature is seamless mode switching: thinking mode (on by default) emits reasoning inside <think> tags before the final answer, while non-thinking mode responds directly. Modes are controlled with the enable_thinking chat-template switch or per turn with /think and /no_think in the prompt. The model card reports improved reasoning over QwQ and Qwen2.5-Instruct, strong agent and tool-calling behavior, and support for over 100 languages.

Model specifications

Quick start

The chute is OpenAI-compatible. Base URL https://llm.chutes.ai/v1, model name Qwen/Qwen3-32B-TEE, Bearer auth with your Chutes API key.

curl -X POST "https://llm.chutes.ai/v1/chat/completions" \
  -H "Authorization: Bearer $CHUTES_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-32B-TEE",
    "messages": [{"role": "user", "content": "Explain GQA in two sentences."}],
    "stream": true,
    "max_tokens": 1024,
    "temperature": 0.6
  }'

import os
from openai import OpenAI

client = OpenAI(base_url="https://llm.chutes.ai/v1", api_key=os.environ["CHUTES_API_KEY"])

resp = client.chat.completions.create(
    model="Qwen/Qwen3-32B-TEE",
    messages=[{"role": "user", "content": "Explain GQA in two sentences."}],
    max_tokens=1024,
    temperature=0.6,
)
print(resp.choices[0].message.content)

const res = await fetch("https://llm.chutes.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.CHUTES_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "Qwen/Qwen3-32B-TEE",
    messages: [{ role: "user", content: "Explain GQA in two sentences." }],
    max_tokens: 1024,
    temperature: 0.6,
  }),
});
const data = await res.json();
console.log(data.choices[0].message.content);

Parameters and tuning

Chute request fields and defaults (from the live llms.txt): max_tokens 1024, temperature 0.7, seed 42, stream supported on both /v1/chat/completions and /v1/completions. The completions endpoint additionally accepts vLLM-style knobs such as top_p, top_k, min_p, presence_penalty, frequency_penalty, and repetition_penalty.

The Qwen team's recommendations differ by mode. Thinking mode (the default): temperature 0.6, top_p 0.95, top_k 20, min_p 0 — this matches generation_config.json — and never greedy decoding, which can trigger repetition loops. Non-thinking mode: temperature 0.7, top_p 0.8, top_k 20. Because thinking output counts against max_tokens, raise the limit well above 1024 for hard reasoning problems, or append /no_think to the prompt when you want short direct answers.

What it's best at

Use Qwen3-32B where its hybrid reasoning pays off: math, code, and logic problems in thinking mode; agentic tool-calling loops (the card recommends Qwen-Agent, which wraps Qwen3's tool-call templates and parsers); and multilingual chat or instruction following across 100+ languages. The per-turn /think and /no_think switches make it a good fit for products that mix quick replies with occasional deep reasoning in the same conversation.

It is not the right choice for prompts beyond roughly 40K tokens as served (longer contexts require YaRN rescaling upstream), for any image or audio input (text only), or for hard-latency paths that leave thinking mode enabled, since reasoning tokens lengthen every response.

How Chutes serves this model

This chute runs the model inside a Trusted Execution Environment: attested confidential-compute hardware that keeps prompts and outputs protected while they are processed. The weights are served in FP8 precision and generation is accelerated with DFLASH speculative decoding; both are serving-level optimizations (speculative decoding preserves the target model's output distribution, FP8 can introduce minor numerical differences versus BF16). The endpoint lives on the shared OpenAI-compatible gateway with streaming enabled, and usage is billed per token through your Chutes account. See the model page, the machine-readable llms.txt, and the callable OpenAPI spec.

FAQ

What context window does Qwen3-32B support on Chutes?

The upstream model natively handles 32,768 tokens and its config sets max_position_embeddings to 40,960. The model card documents up to 131,072 tokens only when YaRN RoPE scaling is enabled by the serving stack, so plan around the native window unless you have verified longer prompts against the live endpoint.

Does Qwen3-32B think before answering?

Yes, by default. Qwen3 is a hybrid-reasoning model: thinking mode emits reasoning wrapped in <think> tags before the final answer. You can disable it with the enable_thinking template switch or per turn with /no_think in the prompt, which trades reasoning depth for speed and fewer output tokens.

What sampling settings should I use?

The Qwen team recommends temperature 0.6, top_p 0.95, top_k 20, min_p 0 for thinking mode (these match generation_config.json) and temperature 0.7, top_p 0.8, top_k 20 for non-thinking mode. They explicitly warn against greedy decoding in thinking mode because it can cause repetition loops.

Can I use it with the OpenAI SDK?

Yes. Point the SDK at base_url https://llm.chutes.ai/v1 with your Chutes API key and set model to Qwen/Qwen3-32B-TEE. Standard chat.completions calls, streaming included, work unchanged.

What does the TEE suffix mean?

The model runs inside a Trusted Execution Environment: confidential-compute hardware that keeps prompts and outputs encrypted in memory and attestable. It is a deployment property of this chute, not a different model; the weights are Qwen/Qwen3-32B served in FP8.

Does FP8 or speculative decoding change the model's outputs?

FP8 is a weight/activation precision used for serving efficiency and DFLASH speculative decoding accelerates generation; the model card facts (architecture, training) are unchanged. Speculative decoding is lossless with respect to the target model's distribution, while FP8 quantization can introduce small numerical differences versus BF16.

Is Qwen3-32B free for commercial use?

Yes. The upstream repository is licensed Apache-2.0, which permits commercial use, modification, and redistribution with attribution. Your usage of the Chutes endpoint is billed per Chutes pricing, which is separate from the model license.

Does it support function calling and tools?

Yes. The model card highlights agent and tool-calling capability and recommends the Qwen-Agent framework, which bundles Qwen3's tool-call templates and parsers. Through the OpenAI-compatible endpoint you can also drive tool use with standard tools/tool_choice request fields where supported.

Sources

• Qwen/Qwen3-32B on Hugging Face
• config.json and generation_config.json
• Qwen3-32B-TEE model page on Chutes
• Live llms.txt for this chute