Using Gemma 4 31B Turbo on Chutes

Gemma 4 31B is the largest dense model in Google DeepMind's Gemma 4 family: a 30.7B multimodal reasoner with a 256K context window, configurable thinking, and native function calling. The turbo chute on Chutes serves NVIDIA's NVFP4 quantization with DFlash speculative decoding inside a Trusted Execution Environment (TEE), behind an OpenAI-compatible API.

Overview

Released in March 2026 under the Gemma 4 Apache-2.0 license, Gemma 4 31B (Gemma4ForConditionalGeneration) takes text and image input and generates text, supporting more than 140 languages. Its 60 layers interleave local sliding-window attention (1024-token window) with full global attention, the final layer always global; global layers use unified Keys and Values plus Proportional RoPE (p-RoPE) to control memory at long context. Attention is grouped-query with 32 query heads and 16 KV heads at head dim 256 over a 262K vocabulary, with a ~550M-parameter vision encoder handling variable image aspect ratios and resolutions. Card benchmarks: MMLU Pro 85.2, AIME 2026 (no tools) 89.2, LiveCodeBench v6 80.0, GPQA Diamond 84.3, Codeforces ELO 2150, MMMU Pro 76.9, MRCR v2 8-needle 128k 66.4.

Model specifications

Quick start

Base URL https://llm.chutes.ai/v1, model name google/gemma-4-31B-turbo-TEE, Bearer auth with your Chutes API key.

curl -X POST "https://llm.chutes.ai/v1/chat/completions" \
  -H "Authorization: Bearer $CHUTES_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemma-4-31B-turbo-TEE",
    "messages": [{"role": "user", "content": "Summarize the tradeoffs of sliding-window attention."}],
    "stream": true,
    "max_tokens": 1024,
    "temperature": 1.0
  }'

import os
from openai import OpenAI

client = OpenAI(base_url="https://llm.chutes.ai/v1", api_key=os.environ["CHUTES_API_KEY"])

resp = client.chat.completions.create(
    model="google/gemma-4-31B-turbo-TEE",
    messages=[{"role": "user", "content": "Summarize the tradeoffs of sliding-window attention."}],
    max_tokens=1024,
    temperature=1.0,
)
print(resp.choices[0].message.content)

const res = await fetch("https://llm.chutes.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.CHUTES_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "google/gemma-4-31B-turbo-TEE",
    messages: [{ role: "user", content: "Summarize the tradeoffs of sliding-window attention." }],
    max_tokens: 1024,
    temperature: 1.0,
  }),
});
const data = await res.json();
console.log(data.choices[0].message.content);

Parameters and tuning

Chute request fields and defaults (from the live llms.txt): max_tokens 1024, temperature 0.7, seed 42, streaming on both /v1/chat/completions and /v1/completions; the completions endpoint also accepts vLLM-style fields such as top_p, top_k, min_p, and the penalty knobs.

Google's recommended sampling for Gemma 4 is temperature 1.0, top_p 0.95, top_k 64, exactly what ships in generation_config.json. Note the chute default temperature is 0.7, so pass 1.0 explicitly for card-recommended behavior. Thinking is controlled by the <|think|> token: include it at the start of the system prompt to enable step-by-step reasoning, omit it to disable. Two card rules matter for agents: previous turns' thinking content must not be replayed into history, and Gemma 4 natively supports the system role, unlike Gemma 3.

What it's best at

Gemma 4 31B is a strong general-purpose reasoner and coder for its size: near-frontier scores on MMLU Pro (85.2), GPQA Diamond (84.3), and LiveCodeBench v6 (80.0), with a 2150 Codeforces ELO. Native function calling and system-prompt support make it a solid base for autonomous agents, and its vision encoder handles document and image understanding (0.131 average edit distance on OmniDocBench 1.5, 85.6 MATH-Vision). The 256K context plus hybrid attention makes long-document work practical on modest hardware budgets, and multilingual coverage spans 140+ languages.

It does not accept audio (only the E2B/E4B family members do), and this chute's NVFP4 weights are not bit-identical to the BF16 originals, so avoid it where exact upstream parity is required. For very fast, cheap inference at lower quality, the MoE sibling Gemma 4 26B A4B activates only 3.8B parameters per token.

How Chutes serves this model

The turbo suffix is the serving story: this chute runs nvidia/Gemma-4-31B-IT-NVFP4, NVIDIA's ModelOpt NVFP4 quantization of the instruction-tuned weights with an FP8 KV cache, and accelerates generation with DFlash speculative decoding. Speculative decoding preserves the served model's output distribution; NVFP4 introduces small numerical differences versus BF16 in exchange for materially higher throughput. Everything executes inside a Trusted Execution Environment, so prompts and outputs are processed on attested confidential-compute hardware, and usage is billed per token. See the model page, the machine-readable llms.txt, and the callable OpenAPI spec.

FAQ

What does turbo mean in this model's name?

It is a serving configuration, not a different Google model. The chute runs NVIDIA's NVFP4 quantization of Gemma 4 31B IT (nvidia/Gemma-4-31B-IT-NVFP4, built with TensorRT Model Optimizer) plus DFlash speculative decoding, trading a small amount of numerical precision for substantially faster, cheaper inference.

How do I enable thinking mode?

Gemma 4 enables reasoning when the <|think|> control token appears at the start of the system prompt; remove it to disable thinking. When enabled, the model emits its reasoning before the final answer. Do not feed previous turns' thinking content back into the conversation history.

What context window does it support?

256K tokens per the model card; config.json sets max_position_embeddings to 262,144. The hybrid local/global attention design with p-RoPE is what keeps memory usage manageable at that length.

Can it process images?

Yes. Gemma 4 31B accepts text and image input with variable aspect ratio and resolution support, and scores 76.9 on MMMU Pro per the card. It does not accept audio; only the smaller E2B and E4B family members have audio encoders.

What sampling settings does Google recommend?

Temperature 1.0, top_p 0.95, top_k 64, which match the shipped generation_config.json. The chute's own default temperature is 0.7, so pass temperature explicitly if you want the card-recommended behavior.

How do I call it from the OpenAI SDK?

Set base_url to https://llm.chutes.ai/v1 with your Chutes API key and model to google/gemma-4-31B-turbo-TEE. Chat completions and streaming work as with any OpenAI-compatible endpoint.

Is Gemma 4 licensed for commercial use?

Yes. Google released Gemma 4 under an Apache 2.0 license (see the Gemma 4 license page linked from the model card), and NVIDIA's NVFP4 quantization page states the model is ready for commercial and non-commercial use. Chutes billing is separate from the license.

What does the TEE suffix mean?

The chute runs inside a Trusted Execution Environment: attested confidential-compute hardware that keeps prompts and outputs protected while being processed. It is a deployment guarantee on Chutes and does not change the model.

Sources

• google/gemma-4-31B-it on Hugging Face
• nvidia/Gemma-4-31B-IT-NVFP4 (served quantization)
• config.json and generation_config.json
• Gemma 4 31B Turbo TEE model page on Chutes
• Live llms.txt for this chute