Using Mistral-Nemo-Instruct-2407 on Chutes

Mistral-Nemo-Instruct-2407 is a 12B instruction-tuned model trained jointly by Mistral AI and NVIDIA, with a 128k context window and the efficient Tekken tokenizer. On Chutes it runs inside a Trusted Execution Environment (TEE) behind an OpenAI-compatible API, served from the unsloth mirror of the Apache-2.0 weights.

Overview

Released in July 2024 under Apache-2.0, Mistral Nemo was positioned as a drop-in replacement for Mistral 7B: same standard architecture family, substantially more capability. It is a dense 40-layer transformer (MistralForCausalLM) with hidden size 5120, grouped-query attention (32 query heads, 8 KV heads, head dim 128), RoPE theta 1,000,000, and no sliding window. It uses the Tekken tokenizer with a ~128k vocabulary (2^17), built for efficient compression of multilingual text and code, and was trained with a 128k context window on data containing a large proportion of multilingual and code content. This chute serves unsloth/Mistral-Nemo-Instruct-2407, a mirror that is architecturally identical to the Mistral AI original: same config, same 12.25B safetensors parameter total, same license.

Model specifications

Quick start

Base URL https://llm.chutes.ai/v1, model name unsloth/Mistral-Nemo-Instruct-2407-TEE, Bearer auth with your Chutes API key. Use a low temperature; the card recommends 0.3.

curl -X POST "https://llm.chutes.ai/v1/chat/completions" \
  -H "Authorization: Bearer $CHUTES_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "unsloth/Mistral-Nemo-Instruct-2407-TEE",
    "messages": [{"role": "user", "content": "Draft a polite follow-up email in French."}],
    "stream": true,
    "max_tokens": 1024,
    "temperature": 0.3
  }'

import os
from openai import OpenAI

client = OpenAI(base_url="https://llm.chutes.ai/v1", api_key=os.environ["CHUTES_API_KEY"])

resp = client.chat.completions.create(
    model="unsloth/Mistral-Nemo-Instruct-2407-TEE",
    messages=[{"role": "user", "content": "Draft a polite follow-up email in French."}],
    max_tokens=1024,
    temperature=0.3,
)
print(resp.choices[0].message.content)

const res = await fetch("https://llm.chutes.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.CHUTES_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "unsloth/Mistral-Nemo-Instruct-2407-TEE",
    messages: [{ role: "user", content: "Draft a polite follow-up email in French." }],
    max_tokens: 1024,
    temperature: 0.3,
  }),
});
const data = await res.json();
console.log(data.choices[0].message.content);

Parameters and tuning

Chute request fields and defaults (from the live llms.txt): max_tokens 1024, temperature 0.7, seed 42, streaming on /v1/chat/completions and /v1/completions; the completions endpoint additionally accepts vLLM fields such as top_p, top_k, min_p, presence_penalty, frequency_penalty, and repetition_penalty.

The single most important tuning fact comes straight from the Mistral card: unlike previous Mistral models, Nemo requires smaller temperatures, with 0.3 recommended and official examples using up to 0.35. Since the chute defaults to 0.7, pass temperature explicitly in every request. There is no thinking mode or reasoning budget to manage; responses are direct, so the 1024-token default max_tokens is workable for chat and worth raising for long-form generation.

What it's best at

Mistral Nemo is a compact workhorse: general chat, instruction following, summarization, and drafting in a 12B footprint that keeps per-token costs low. Multilingual work is a genuine strength; it was trained on a large proportion of multilingual data, the Tekken tokenizer compresses non-English text efficiently, and the card publishes multilingual MMLU results across French, German, Spanish, Italian, Portuguese, Russian, Chinese, and Japanese. The card also demonstrates native function calling via the Mistral tool-call format, and its heavy code training data makes it serviceable for lightweight code tasks. Teams already on Mistral 7B can treat it as a drop-in upgrade.

Set expectations accordingly: it is a 2024-era instruct model with no thinking mode, so it will not match modern reasoning models on hard math or multi-step logic; it is text-only; and newer long-context models score better near the full 128k window.

How Chutes serves this model

This chute runs the model inside a Trusted Execution Environment: attested confidential-compute hardware that keeps prompts and outputs protected while they are processed, a serving-level guarantee that does not alter the model. Inference is exposed on the shared OpenAI-compatible gateway with streaming and billed per token through your Chutes account. The served weights are the unsloth mirror of the original Mistral AI release. See the model page, the machine-readable llms.txt, and the callable OpenAPI spec.

FAQ

What context window does Mistral Nemo support?

It was trained with a 128k context window and config.json sets max_position_embeddings to 131,072. Both prompt and completion share that budget.

What temperature should I use?

0.3. The Mistral card states explicitly that, unlike previous Mistral models, Nemo requires smaller temperatures, and its own examples use 0.3 to 0.35. The chute default is 0.7, so pass temperature explicitly for card-recommended behavior.

Is the unsloth repo different from the Mistral AI original?

No functional difference: unsloth/Mistral-Nemo-Instruct-2407 carries the same MistralForCausalLM architecture, the same 12.25B safetensors parameter count, and the same Apache-2.0 license as mistralai/Mistral-Nemo-Instruct-2407. The unsloth mirror exists mainly to pair with their fine-tuning notebooks.

Does it support function calling?

Yes. The model card demonstrates tool use through the Mistral function-calling format, defining tools with JSON schemas and parsing structured tool-call output. Via the OpenAI-compatible endpoint you can send standard tools fields where supported.

How do I call it from the OpenAI SDK?

Use base_url https://llm.chutes.ai/v1 with your Chutes API key and set model to unsloth/Mistral-Nemo-Instruct-2407-TEE. Chat completions with streaming work unchanged.

What does the TEE suffix mean?

The chute runs inside a Trusted Execution Environment: attested confidential-compute hardware that keeps prompts and outputs protected in memory during inference. It is a serving property of this chute, not a change to the model.

Can I use it commercially?

Yes. Mistral-Nemo-Instruct-2407 is released under Apache-2.0, which permits commercial use, modification, and redistribution with attribution. Chutes endpoint usage is billed separately through your account.

Is it multilingual?

Yes. It was trained on a large proportion of multilingual and code data, uses the Tekken tokenizer designed for efficient multilingual compression, and the card publishes multilingual MMLU scores for French, German, Spanish, Italian, Portuguese, Russian, Chinese, and Japanese.

Sources

• unsloth/Mistral-Nemo-Instruct-2407 on Hugging Face
• mistralai/Mistral-Nemo-Instruct-2407 (upstream)
• config.json
• Mistral-Nemo-Instruct-2407-TEE model page on Chutes
• Live llms.txt for this chute