Ollama: Phase 2 Comprehensive Exercises
Phase 2 comprehensive exercises are the leap from "can use" to "proficient" — Modelfile customization, GPU speedup, API migration all covered.
💡 Tip: These comprehensive exercises aim to consolidate Phase 2 core skills (Modelfile customization, Python SDK integration, GPU acceleration, OpenAI compatible API, Multimodal Models). We recommend completing exercises 1→2→3→4 in order, as each exercise builds on the knowledge and output of the previous ones. After completing all exercises, you'll have a complete SupportBot V2 architecture plan, laying a solid foundation for Phase 3's Docker Deployment and RAG integration.
📋 Prerequisites: You need to master the following first
- Lesson 1: Local AI Concepts and Environment Overview
- Lesson 2: Ollama Installation and Environment Configuration
- Lesson 3: CLI Basic Interaction
- Lesson 4: Model Management
- Lesson 5: REST API Fundamentals
- Lesson 6: Phase 1 Comprehensive Exercises
- Lesson 7: Python SDK Integration
- Lesson 8: Modelfile Custom Models
- Lesson 9: GPU and CUDA Configuration
- Lesson 10: Multimodal Models
- Lesson 11: OpenAI Compatible API
1. What You Will Learn
- Modelfile + Python SDK combined practice
- CPU vs GPU Inference speed benchmarking
- Complete OpenAI application migration to Ollama
- Multimodal image description microservice implementation
- SupportBot V2 architecture planning
2. A Real Story from a SaaS Founder
ℹ️ Info: The output of Phase 2 comprehensive exercises is a SupportBot V2 architecture plan — this is not just practice, but also input for subsequent Phase 3 (Docker/Quantization/multi-Model orchestration) and Phase 4 (security/monitoring/production Deployment). Invest time in good planning now, and you'll reap the benefits later.
⚠️ Warning: After migrating to the OpenAI compatible API, you must do regression testing — local Model behavior differs from GPT-4, with possible output format deviations, varying System Prompt adherence, and multi-turn dialogue context understanding differences.
(1) The Pain Point: Prototype Features Are Too Limited
Alice's SupportBot V1 can only do simple text conversation, lacking role customization, GPU acceleration, image analysis, and API compatibility. She needs to integrate all capabilities learned in Phase 2 to build a more powerful V2 version.
(2) The Solution: Phase 2 Full Capability Integration
PYTHON
# SupportBot V2 leverages all Phase 2 skills:
# - Custom Modelfile for role specialization
# - GPU acceleration for speed
# - OpenAI-compatible API for easy integration
# - Multimodal for image analysis
3. Exercise 1: Modelfile + Python SDK Combined Practice
💡 Tip: Modelfile + Python SDK is the best combination — Modelfile solidifies roles and parameters, while the SDK handles business logic and conversation management. Don't hardcode System Prompts in code; let the Modelfile serve as the "configuration file," and let the code just call it.
(1) Create a Custom Model and Call It with the SDK
flowchart TD
A[Write Modelfile] --> B[ollama create]
B --> C[Python SDK Import]
C --> D[Build Application]
| Step | Action | Key Point |
|---|---|---|
| 1 | Write SupportBot Modelfile | SYSTEM + PARAMETER + MESSAGE |
| 2 | ollama create |
Build custom Model |
| 3 | Python call ollama.chat(model='supportbot') |
No need to pass System Prompt |
| 4 | Compare output with/without Modelfile | Role consistency improvement |
▶ Example 1: Modelfile + SDK Complete Workflow
TEXT
# supportbot.Modelfile
FROM qwen2.5:7b
SYSTEM You are SupportBot for GlobalShop e-commerce. Be polite, concise (2-3 sentences). For order queries ask for order number. If unsure, say 'Let me connect you with a human agent.'
PARAMETER temperature 0.4
PARAMETER num_ctx 4096
PARAMETER num_predict 256
MESSAGE user What is your return policy?
MESSAGE assistant Our return policy allows returns within 30 days in original condition. Free return shipping included.
PYTHON
import ollama
# Create model (run once)
# ollama.create(model='supportbot', from_='./supportbot.Modelfile')
# Use the custom model - no system prompt needed!
def test_supportbot():
questions = [
"I want to return order #12345",
"Do you ship to Germany?",
"What's the price of the iPhone 15?"
]
for q in questions:
response = ollama.chat(
model='supportbot',
messages=[{'role': 'user', 'content': q}],
stream=False
)
print(f"Q: {q}")
print(f"A: {response['message']['content']}\n")
test_supportbot()
4. Exercise 2: CPU vs GPU Benchmarking
⚠️ Note: When benchmarking, be sure to make 2-3 warmup calls first (discard results) before starting formal timing. The first request includes cold-start Model loading time (5-30 seconds), which will significantly inflate the average. Also, don't run other GPU-intensive tasks during testing to ensure a fair comparison.
(1) Benchmark Dimensions
| Metric | CPU Expected | GPU Expected | Difference |
|---|---|---|---|
| First-Token Latency (TTFT) | 2-5s | 0.2-0.5s | 5-10x |
| Generation Speed (tok/s) | 5-10 | 30-60 | 5-10x |
| Total Latency (200 tokens) | 20-40s | 3-7s | 5-10x |
| Memory Usage | RAM | VRAM | Different types |
▶ Example 2: CPU vs GPU Comparison Script
PYTHON
import ollama
import time
from statistics import mean
def benchmark(model: str, prompt: str, runs: int = 5) -> dict:
"""Benchmark model inference speed."""
latencies = []
token_speeds = []
# Warm up (first call includes model loading)
ollama.chat(model=model, messages=[{'role': 'user', 'content': 'warm up'}],
stream=False)
for _ in range(runs):
start = time.time()
response = ollama.chat(
model=model,
messages=[{'role': 'user', 'content': prompt}],
stream=False
)
elapsed = time.time() - start
# Estimate tokens (rough: ~4 chars per token for English)
content = response['message']['content']
token_count = len(content) // 4
latencies.append(elapsed)
token_speeds.append(token_count / elapsed if elapsed > 0 else 0)
return {
"model": model,
"avg_latency_s": round(mean(latencies), 2),
"avg_tokens_per_s": round(mean(token_speeds), 1),
"runs": runs
}
# Run benchmark
print("=== CPU vs GPU Benchmark ===")
result = benchmark("qwen2.5", "Explain machine learning in 100 words")
print(f"Model: {result['model']}")
print(f"Avg latency: {result['avg_latency_s']}s")
print(f"Avg speed: {result['avg_tokens_per_s']} tokens/s")
# Compare different models
for model in ["llama3.2:3b", "qwen2.5", "mistral"]:
result = benchmark(model, "Explain AI in 50 words", runs=3)
print(f"{model}: {result['avg_tokens_per_s']} tok/s, {result['avg_latency_s']}s")
Output:
TEXT
=== CPU vs GPU Benchmark ===
5. Exercise 3: OpenAI Application Migration
(1) Migration Workflow
flowchart TD
A[Original OpenAI App] --> B{Identify API calls}
B --> C[Change base_url to Ollama]
C --> D[Change model name]
D --> E[Test each feature]
E --> F{All features work?}
F -->|Yes| G[Migration Complete]
F -->|No| H[Fix incompatibilities]
H --> E
| Check Step | Action | Verification |
|---|---|---|
| 1 | Change base_url | curl localhost:11434/v1/models returns Model list |
| 2 | Change Model name | qwen2.5 responds normally |
| 3 | Test streaming output | stream=True works properly |
| 4 | Test JSON mode | response_format works properly |
| 5 | Test Embeddings | nomic-embed-text works properly |
| 6 | Test Function Calling | If unsupported, switch to JSON mode alternative |
▶ Example 3: Complete Migration Practice
PYTHON
from openai import OpenAI
import json
# Before migration
# client = OpenAI(api_key="sk-xxx")
# model = "gpt-4"
# After migration - only these 3 lines changed
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
model = "qwen2.5"
embed_model = "nomic-embed-text"
# Test 1: Chat
print("=== Test 1: Chat ===")
response = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": "Hello, who are you?"}],
temperature=0.3
)
print(response.choices[0].message.content)
# Test 2: Streaming
print("\n=== Test 2: Streaming ===")
stream = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": "Count from 1 to 5"}],
stream=True
)
for chunk in stream:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="")
print()
# Test 3: JSON mode
print("\n=== Test 3: JSON Mode ===")
response = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": "List 3 colors as JSON array"}],
response_format={"type": "json_object"},
temperature=0.1
)
print(response.choices[0].message.content)
# Test 4: Embeddings
print("\n=== Test 4: Embeddings ===")
response = client.embeddings.create(model=embed_model, input="Hello world")
print(f"Embedding dimension: {len(response.data[0].embedding)}")
print("\n=== All tests passed! ===")
Output:
TEXT
=== Test 1: Chat ===
\n=== Test 2: Streaming ===
\n=== Test 3: JSON Mode ===
\n=== Test 4: Embeddings ===
\n=== All tests passed! ===
6. Multimodal Challenge: Image Description Microservice
(1) Microservice Architecture
flowchart TD
A[HTTP Request<br/>image + prompt] --> B[FastAPI Endpoint]
B --> C[Base64 Encode Image]
C --> D[Ollama llava Model]
D --> E[Image Description]
E --> F[JSON Response]
▶ Example 4: FastAPI Image Description Microservice
PYTHON
from fastapi import FastAPI, UploadFile, File, Form
from fastapi.responses import JSONResponse
import ollama
import base64
app = FastAPI(title="Image Description Service")
@app.post("/describe")
async def describe_image(
file: UploadFile = File(...),
prompt: str = Form("Describe this image in detail")
):
"""Describe an uploaded image using llava."""
image_data = base64.b64encode(await file.read()).decode("utf-8")
response = ollama.chat(
model="llava",
messages=[{
"role": "user",
"content": prompt,
"images": [image_data]
}],
stream=False,
options={"temperature": 0.3}
)
return JSONResponse({
"description": response["message"]["content"],
"model": "llava",
"filename": file.filename
})
@app.post("/extract-text")
async def extract_text(file: UploadFile = File(...)):
"""OCR-like text extraction from image."""
image_data = base64.b64encode(await file.read()).decode("utf-8")
response = ollama.chat(
model="llava",
messages=[{
"role": "user",
"content": "Extract all visible text from this image verbatim.",
"images": [image_data]
}],
stream=False,
options={"temperature": 0.1}
)
return JSONResponse({
"text": response["message"]["content"],
"model": "llava"
})
# Run: uvicorn service:app --host 0.0.0.0 --port 8000
Output:
TEXT
# Function defined successfully
▶ Example 5: SupportBot V2 Architecture Plan
PYTHON
# SupportBot V2 Architecture Blueprint
SUPPORTBOT_V2_SPEC = {
"name": "SupportBot V2",
"components": {
"custom_model": {
"tool": "Modelfile",
"purpose": "Role-specialized customer service model",
"config": "qwen2.5:7b + custom SYSTEM + PARAMETER"
},
"api_gateway": {
"tool": "OpenAI compatible API",
"purpose": "Easy integration with existing tools",
"config": "/v1/chat/completions endpoint"
},
"image_service": {
"tool": "llava multimodal",
"purpose": "Damage report analysis, product photo description",
"config": "FastAPI + llava + Base64 encoding"
},
"embedding_service": {
"tool": "nomic-embed-text",
"purpose": "Product knowledge base (RAG ready for Phase 3)",
"config": "/v1/embeddings endpoint"
},
"gpu_acceleration": {
"tool": "CUDA / Metal",
"purpose": "Fast inference for real-time responses",
"config": "Auto-detected by Ollama"
}
},
"cost_comparison": {
"before": "GPT-4 API: ~2,000 USD/month",
"after": "Local Ollama: ~100 USD/month (hardware + electricity)",
"saving": "1,900 USD/month (95% reduction)"
}
}
print("SupportBot V2 Architecture:")
for name, spec in SUPPORTBOT_V2_SPEC["components"].items():
print(f" {name}: {spec['tool']} - {spec['purpose']}")
Output:
TEXT
SupportBot V2 Architecture:
7. Comprehensive Example: Phase 2 Full Skills Verification
PYTHON
# ============================================
# Comprehensive: Phase 2 full skills verification
# Modelfile + SDK + GPU + OpenAI compat + Multimodal
# ============================================
import ollama
from openai import OpenAI
import base64
import time
import json
from pathlib import Path
def phase2_verify():
results = {}
# Test 1: Custom Modelfile model
print("=== Test 1: Custom Modelfile Model ===")
try:
resp = ollama.chat(
model="supportbot",
messages=[{"role": "user", "content": "I want a refund for order #12345"}],
stream=False
)
results["modelfile"] = "PASS"
print(f" Response: {resp['message']['content'][:80]}...")
except Exception as e:
results["modelfile"] = f"FAIL: {e}"
# Test 2: GPU acceleration
print("\n=== Test 2: GPU Acceleration ===")
start = time.time()
resp = ollama.chat(
model="qwen2.5",
messages=[{"role": "user", "content": "Hello"}],
stream=False
)
elapsed = time.time() - start
results["gpu_speed"] = f"{elapsed:.2f}s"
print(f" Response time: {elapsed:.2f}s ({'GPU' if elapsed < 3 else 'CPU'})")
# Test 3: OpenAI compatible API
print("\n=== Test 3: OpenAI Compatible API ===")
try:
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
resp = client.chat.completions.create(
model="qwen2.5",
messages=[{"role": "user", "content": "Say OK"}],
temperature=0.1
)
results["openai_compat"] = "PASS"
print(f" Response: {resp.choices[0].message.content}")
except Exception as e:
results["openai_compat"] = f"FAIL: {e}"
# Test 4: Embeddings
print("\n=== Test 4: Embeddings ===")
try:
resp = client.embeddings.create(model="nomic-embed-text", input="test")
dim = len(resp.data[0].embedding)
results["embeddings"] = f"PASS ({dim}d)"
print(f" Embedding dimension: {dim}")
except Exception as e:
results["embeddings"] = f"FAIL: {e}"
# Test 5: Streaming
print("\n=== Test 5: Streaming ===")
try:
stream = client.chat.completions.create(
model="qwen2.5",
messages=[{"role": "user", "content": "Count 1 to 3"}],
stream=True
)
chunks = 0
for chunk in stream:
if chunk.choices[0].delta.content:
chunks += 1
results["streaming"] = f"PASS ({chunks} chunks)"
print(f" Received {chunks} chunks")
except Exception as e:
results["streaming"] = f"FAIL: {e}"
# Summary
print("\n" + "=" * 40)
print("Phase 2 Verification Results:")
for test, status in results.items():
print(f" {test}: {status}")
passed = sum(1 for v in results.values() if "PASS" in str(v))
print(f"\n Total: {passed}/{len(results)} passed")
if passed >= 4:
print(" ✅ Ready for Phase 3!")
else:
print(" ⚠️ Review failed tests.")
if __name__ == "__main__":
phase2_verify()
❓ FAQ
Q My Modelfile-created Model isn't found by the Python SDK, what should I do?
A Ensure
ollama create succeeded (no errors). Use ollama list to confirm the Model exists. The Python SDK's model name must match exactly what was specified in ollama create.Q Benchmark results vary widely, what should I do?
A The first call includes Model loading time (cold start). Do 2 warmup calls before measuring. Don't run other programs during testing. Take the average of 5+ runs.
Q Some features break after OpenAI migration, what should I do?
A Troubleshoot one by one — test basic Chat first, then Streaming, then JSON mode. Verify each feature independently to pinpoint the specific incompatibility.
Q The llava Model hasn't been pulled yet, what should I do?
A Exercise 4 requires
ollama pull llava first. If disk space is insufficient, you can skip Exercise 4 — Phase 3's RAG doesn't require Multimodal.Q What's the core difference between SupportBot V2 and V1?
A V1 is a Python script-level wrapper; V2 integrates Modelfile role customization, OpenAI compatible API gateway, and Multimodal image analysis — it's a microservice architecture.
Q What prerequisite knowledge is needed for Phase 3?
A LangChain basics (Prompt Template, Chain, Agent), RAG concepts (document loading, Vector retrieval), and Docker fundamentals. Lessons 13-18 will cover these one by one.
📖 Summary
- Modelfile + Python SDK combination enables zero-configuration use of custom Models in code
- Benchmarking requires warmup + multiple averages; CPU vs GPU difference is 5-10x
- OpenAI migration requires changing only base_url/api_key/model — three places
- Multimodal microservice uses FastAPI + Base64 encoding + llava
- SupportBot V2 integrates Modelfile + OpenAI compatible + Multimodal + Embedding
- Phase 2 full skills verification: pass 5 tests to proceed to Phase 3
📝 Exercises
- Basic (Difficulty ⭐): Create a SupportBot Modelfile, call it with the Python SDK and test 3 different questions, recording role consistency.
- Intermediate (Difficulty ⭐⭐): Run CPU vs GPU benchmarks, compare speed across 3 different Models, and output a performance report table.
- Advanced (Difficulty ⭐⭐⭐): Complete SupportBot V2's full implementation — Modelfile role customization + OpenAI compatible API + Multimodal image analysis (optional), and write a migration document.
