I let Hermes write its own Chutes config

I handed Hermes the Chutes Agent Toolkit and asked it to set up a Chutes provider. Less than a minute later, it had written a config, selected the model I’d specified, and tried a test call. That last part failed because I’d let my account balance slip below zero.

I’ve been using Hermes, the open-source agent from Nous Research, for a while. Its memory across sessions is a big part of why I keep reaching for it. For this run, I wanted it to handle the provider setup from the toolkit’s instructions, with one restriction: write a proposal I could review, not a live change to my working config.

Why Chutes

The setup looked straightforward. Chutes has an OpenAI-compatible API, so Hermes can connect through one base URL and one API key. The toolkit includes a Hermes skill, a doctor script for checking the setup, and four example configurations. I didn’t need to write the YAML myself.

I’m also interested in where the money goes during an agent session. A task can involve reading files, calling tools, summarizing results, and working through another round of decisions. I don’t want every turn billed at my primary model’s rate when a less expensive model can handle some of that work.

Hermes gives you two ways to split it up: smart routing for simpler turns, and delegation for background subtasks. You can keep your preferred model for the harder work and send selected tasks to Chutes. I haven’t benchmarked savings for this setup, so check the catalog’s per-token prices against your own workload.

Then there’s the context itself. Once an agent starts reading a repo or an inbox, a model request can contain a lot more than the sentence you typed. It can include file contents, tools, and the steps the agent has already taken. Where that lands matters to me.

Chutes runs its served models inside hardware-attested trusted execution environments, or TEEs, with end-to-end encryption. The stack is designed so the operator cannot read what it processes. Its TEE stack, sek8s, and the encryption layer are open source, and there is a first-hand write-up of how it was built. The toolkit also includes an attestation script, which I ran against the GLM-5.2 chute.

Handing the setup to Hermes

You’ll need Hermes installed, a Chutes API key, the toolkit, and a funded Chutes account. Don’t skip that last one, as I apparently did.

git clone https://github.com/chutesai/chutes-agent-toolkit.git
cd chutes-agent-toolkit
echo 'CHUTES_API_KEY=cpk_...' >> ~/.hermes/.env

Replace the example key with your own locally. Keep it in Hermes’s environment file, not in the YAML config or the prompt.

I started Hermes from the toolkit directory. It was still using its previously configured model; Chutes wasn’t powering the setup. Here’s the exact prompt:

You are being handed the Chutes Agent Toolkit, checked out in the current directory. Set yourself up to run on Chutes, step by step. Read `other-agents/hermes/README.md` and the `chutes-ai` skill. Run the doctor script with `--emit-config` and report what it found. Never print the API key value, not even part of it. From the live catalog at llm.chutes.ai/v1/models, pick `zai-org/GLM-5.2-TEE` as the default and say why. Write the provider block you would add to `~/.hermes/config.yaml` to a file called `proposed-config.yaml`. Do not modify the real config. Then test the provider with one minimal chat completion and report the response.

The proposal file was deliberate. I wanted to review what Hermes planned to change before letting it touch the configuration I actually use.

What happened

These were the tool calls, in order, with timings removed:

read      README.md
read      SKILL.md
$         python3 scripts/hermes_chutes_doctor.py --emit-config
exec      from hermes_tools import terminal
write     proposed-config.yaml
$         python3 - <<'PY' ... POST https://llm.chutes.ai/v1/chat/completions ... PY

And this was Hermes’s wrap-up. The wording is unchanged apart from spelling out the model count. The catalog details are from the September 14 run, not a promise about what you’ll see when you try it:

Read the README and the chutes-ai skill; both say to use llm.chutes.ai/v1/models as the source of truth and CHUTES_API_KEY with Bearer auth.
Doctor found the Hermes CLI, CHUTES_API_KEY present in the environment, fourteen live models, all of them confidential_compute=true, and the auth check skipped because --check-auth was not requested.
Selected zai-org/GLM-5.2-TEE as the default because the live catalog reports a 1M context length and confidential_compute=true.
Wrote the proposed provider block to proposed-config.yaml only; I did not modify ~/.hermes/config.yaml or ~/.hermes/.env.
Minimal chat completion test failed with HTTP quota exceeded, so response_text and model were not returned.

The doctor displayed a redacted key and where it found it, rather than the full value. Hermes wrote the proposal without modifying my working config.

Then the completion request hit the quota error. My account balance was below zero, so there was no model response to show. Hermes reported the failure instead of typing “ready” and calling it a day. I’ll take that over a fake pass.

The config it wrote

This is proposed-config.yaml, unedited:

# Proposed ~/.hermes/config.yaml additions for Chutes.
# Store CHUTES_API_KEY in ~/.hermes/.env; do not put the key in this file.
providers:
  chutes:
    name: Chutes
    base_url: https://llm.chutes.ai/v1
    key_env: CHUTES_API_KEY
    transport: chat_completions
    default_model: zai-org/GLM-5.2-TEE
    discover_models: true
    models:
      "default": {}
      "default:latency": {}
      "default:throughput": {}
      "zai-org/GLM-5.2-TEE": {}

model:
  provider: custom:chutes
  default: zai-org/GLM-5.2-TEE

I asked for GLM-5.2 explicitly. In that catalog snapshot, it was listed with a 1M context window and confidential_compute=true. The doctor reads that field rather than assuming a model is confidential because its name ends in -TEE.

The proposal also keeps the routing aliases. default:latency is the latency-oriented option for interactive work; default:throughput favors longer generations and background jobs. They let you express a performance preference without pinning every request to a particular model.

After reviewing the proposal, merge the relevant settings into ~/.hermes/config.yaml. Then start a new session or run hermes model and select the saved Chutes provider.

Keeping your primary model

You don’t have to move the whole session to Chutes. The toolkit includes a smart-routing example that keeps your primary model in place and sends simpler work to the Chutes provider.

Here’s chutes-cheap-routing.yaml, unedited:

# ~/.hermes/config.yaml
# Keep your primary model/provider elsewhere, and let Hermes route cheap/simple work to Chutes.

providers:
  chutes:
    name: Chutes
    base_url: https://llm.chutes.ai/v1
    key_env: CHUTES_API_KEY
    transport: chat_completions
    default_model: default:latency
    discover_models: true
    models:
      default: {}
      "default:latency": {}
      "default:throughput": {}

smart_model_routing:
  enabled: true
  cheap_model:
    provider: custom:chutes
    model: default:latency

For delegated tasks, chutes-delegation.yaml uses this block alongside the Chutes provider configuration:

delegation:
  provider: custom:chutes
  model: default:throughput
  reasoning_effort: medium

The cost decision still comes down to the model and its rates. The latency and throughput aliases describe performance preferences, not a guarantee of the lowest price. To choose by price and tool support, use the toolkit’s picker script:

python3 scripts/pick_model.py --task cheap --need tools

It reads the live catalog and prints the lowest-priced models that support tool calling, with their per-token prices. Use those results to choose a specific model instead of an alias when cost is the priority.

Testing the connection

The recorded run stopped at the quota error. With a funded account and CHUTES_API_KEY available in your shell, this is the minimal completion request to retry:

curl -s https://llm.chutes.ai/v1/chat/completions \
  -H "Authorization: Bearer $CHUTES_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"zai-org/GLM-5.2-TEE","messages":[{"role":"user","content":"Reply with the single word: ready"}],"max_tokens":8}'

A written config isn’t a successful connection test. Check the actual response before treating the provider as ready.

What the enclave check returned

Each model in the catalog has a chute_id. Use that ID with the attestation script in the toolkit’s TEE skill, replacing the placeholder below:

python3 plugins/chutes-ai/skills/chutes-tee/scripts/attest_chute.py --chute-id <chute_id from /v1/models>

I ran it against the GLM-5.2 chute the same afternoon. It requested an evidence envelope with a random nonce, parsed the instance’s Intel TDX quote, and parsed the accompanying NVIDIA GPU attestation records. The result was explicit:

verdict:                  shape-valid
cryptographic validation: not run. Install Intel DCAP and run it against the saved envelope.

shape-valid means the evidence passed the script’s structural checks. The output explicitly says cryptographic validation was not run, so this was not a verified attestation.

The script points to Intel DCAP for validating the quote’s signature chain. Run it with --save to keep the evidence envelope for that follow-up. The open-source sek8s repository is also available to inspect the implementation, but reading the code and validating attestation evidence are separate steps.

Try the setup

The Chutes Agent Toolkit is at github.com/chutesai/chutes-agent-toolkit. You can get a key and fund your account at chutes.ai, work through the quickstart if you are new to the API, then give Hermes the prompt above.

Start with the proposal file and read what it writes. Make sure the completion test passes before switching over. If your run goes differently from mine, I’d like to hear about it.