Hermes Agent: Model Configuration
Last updated: 2026-08-31
Model configuration is like choosing an engine — different tasks need different engines. Simple questions use the fuel-efficient one, complex reasoning uses the powerful one. Hermes switches flexibly.
💡 Tip: Hermes supports model routing strategies that automatically select the best model based on task complexity, balancing quality and cost.
📋 Prerequisites: Lesson 3 — Configuration Files
1. What You Will Learn
| # | Content |
|---|---|
| ❶ | Supported model providers |
| ❷ | models.yaml configuration explained |
| ❸ | Local model integration (Ollama/vLLM) |
| ❹ | Model routing strategies |
| ❺ | Cost optimization techniques |
2. Story
(1) Pain Point: GPT-4 Too Expensive, Small Models Too Weak
Bob uses GPT-4 for everything, spending 200+ USD/month on API costs. But 80% of tasks would work fine with GPT-4o-mini. Alice uses model routing — simple tasks automatically use small models, complex reasoning uses large models — and costs drop to 40 USD.
(2) Solution: Smart Model Routing
YAML
routing:
simple_tasks: "gpt-4o-mini"
medium_tasks: "gpt-4o"
complex_tasks: "gpt-4o"
code_execution: "claude-3.5-sonnet"
fallback: "llama3.2"
3. Supported Model Providers
| Provider | Model Examples | Configuration |
|---|---|---|
| OpenAI | gpt-4o, gpt-4o-mini, o1 | OPENAI_API_KEY |
| Anthropic | claude-3.5-sonnet, claude-3-haiku | ANTHROPIC_API_KEY |
| gemini-2.0-flash, gemini-pro | GOOGLE_API_KEY |
|
| Mistral | mistral-large, mistral-small | MISTRAL_API_KEY |
| Ollama (local) | llama3.2, qwen2.5, deepseek | Local service |
| vLLM (local) | Any HuggingFace model | Local service |
| OpenRouter | 200+ models via unified API | OPENROUTER_API_KEY |
4. models.yaml Configuration
YAML
# ~/.hermes/models.yaml
providers:
openai:
api_key: "${OPENAI_API_KEY}"
base_url: "https://api.openai.com/v1"
rate_limit: 100
anthropic:
api_key: "${ANTHROPIC_API_KEY}"
ollama:
base_url: "http://localhost:11434"
timeout: 120
openrouter:
api_key: "${OPENROUTER_API_KEY}"
base_url: "https://openrouter.ai/api/v1"
models:
gpt-4o:
provider: openai
context_window: 128000
input_cost: 2.50
output_cost: 10.00
capabilities:
- chat
- vision
- function_calling
- json_mode
gpt-4o-mini:
provider: openai
context_window: 128000
input_cost: 0.15
output_cost: 0.60
capabilities:
- chat
- function_calling
llama3.2:
provider: ollama
context_window: 128000
input_cost: 0
output_cost: 0
capabilities:
- chat
- function_calling
routing:
strategy: "capability-based"
rules:
- condition: "complexity < 0.3"
model: "gpt-4o-mini"
- condition: "complexity >= 0.3 and complexity < 0.7"
model: "gpt-4o"
- condition: "task_type == 'code'"
model: "claude-3.5-sonnet"
- condition: "offline == true"
model: "llama3.2"
5. Local Model Integration
(1) Ollama
YAML
providers:
ollama:
base_url: "http://localhost:11434"
models:
llama3.2:
provider: ollama
context_window: 128000
qwen2.5-coder:
provider: ollama
context_window: 128000
BASH
ollama pull llama3.2
hermes model list --provider ollama
(2) vLLM
YAML
providers:
vllm:
base_url: "http://localhost:8000"
api_format: "openai"
models:
deepseek-v3:
provider: vllm
model_id: "deepseek-ai/DeepSeek-V3"
context_window: 128000
6. Model Routing Strategies
(1) Capability Routing (Recommended)
PYTHON
def select_model(task):
if task.requires_vision:
return "gpt-4o"
if task.complexity < 0.3:
return "gpt-4o-mini"
if task.type == "code":
return "claude-3.5-sonnet"
return config.default_model
(2) Cost Routing
YAML
routing:
strategy: "cost-based"
quality_threshold: 0.8
max_cost_per_request: 0.05
(3) Manual Specification
BASH
hermes chat --model gpt-4o
hermes chat --model claude-3.5-sonnet --mode code
hermes chat --model llama3.2 --offline
7. Cost Optimization
(1) Cost Monitoring
BASH
hermes cost report --month
# ┌──────────────┬──────────┬───────────┬─────────┐
# │ Model │ Requests │ Tokens In │ Cost │
# ├──────────────┼──────────┼───────────┼─────────┤
# │ gpt-4o-mini │ 1,200 │ 2.4M │ $2.16 │
# │ gpt-4o │ 80 │ 0.8M │ $12.00 │
# │ Total │ 1,280 │ 3.2M │ $14.16 │
# └──────────────┴──────────┴───────────┴─────────┘
(2) Optimization Strategies
| Strategy | Savings | Implementation |
|---|---|---|
| Model routing | 60-80% | Small models for simple tasks |
| Response caching | 10-20% | No repeat calls for same questions |
| Local model hybrid | 50-90% | Ollama for offline tasks |
| Token compression | 15-30% | Trim System Prompts |
| Batch requests | 5-10% | Merge small requests |
❓ FAQ
Q How many models are supported?
A 200+ models via OpenRouter for all mainstream models, plus unlimited Ollama local models.
Q Can I use multiple API Keys simultaneously?
A Yes. Configure multiple providers in models.yaml, each with its own API Key. Routing decides which to use.
Q Is model routing accurate?
A Capability routing based on task type tags and complexity scoring achieves ~90% accuracy. Manually adjust routing rules to optimize.
Q Are local models good enough?
A 8B models suit simple Q&A; 70B+ models approach GPT-3.5 level. Code and reasoning tasks still recommend cloud large models.
Q How to estimate monthly costs?
A
hermes cost estimate --daily-requests 50 --model gpt-4o for usage-based estimates.📖 Summary
- 200+ models from 6 providers + local models
- models.yaml defines model attributes, costs, capabilities
- Model routing: capability-based (recommended), cost-based, manual
- Local models via Ollama/vLLM for zero-cost inference
- Cost optimization: routing + caching + local hybrid saves 60-80%
📝 Exercises
- Basic (⭐): Configure two model providers (e.g., OpenAI + Ollama), verify both work.
- Intermediate (⭐⭐): Set up model routing: simple tasks use gpt-4o-mini, complex tasks use gpt-4o.
- Advanced (⭐⭐⭐): Design a cost-optimal hybrid plan with a 20 USD monthly budget, calculate how many requests it supports.
