Ollama: Phase3综合练习
Phase 3 综合练习是从"单功能"到"系统级"的跃迁——全栈部署、管道构建、模型编排一气呵成。
💡 提示：高级实践综合前 17 课技能——Docker 部署（L15）+ RAG 管道（L14）+ 多模型路由（L17）+ 量化选型（L16）+ LangChain 编排（L13）。建议按"先部署→再建管道→后加路由"的顺序逐步集成，而非一次性搭建全部功能。
📋 前置知识：需要先掌握以下内容
- 第1课：本地AI概念与环境认知
- 第2课：Ollama安装与环境配置
- 第3课：CLI基础交互
- 第4课：模型管理
- 第5课：REST API入门
- 第6课：Phase1综合练习
- 第7课：Python SDK集成
- 第8课：Modelfile自定义模型
- 第9课：GPU与CUDA配置
- 第10课：多模态模型
- 第11课：OpenAI兼容API
- 第12课：Phase2综合练习
- 第13课：LangChain集成
- 第14课：RAG管道
- 第15课：Docker容器化部署
- 第16课：模型量化与优化
- 第17课：多模型编排
1. 你将学到
- Docker Compose 部署 Ollama + Chroma + WebUI 全栈
- 构建 RAG 管道并对比 Embedding 模型
- 实现多模型路由器自动分流
- 量化模型质量对比实验
- SupportBot 架构终版规划
2. 一个 SaaS 创业者的真实故事
💡 提示： 架构终版确定后，建议用"能力矩阵表"（模型 × 场景 × 指标）记录每个组件的选型理由和性能基线。这样后续优化或换模型时有据可查，也便于向团队解释技术决策。
ℹ️ 信息： Phase 3 综合练习是 Phase 4（安全/监控/生产部署）的直接前置。完成此练习后，你将拥有一个可运行的 SupportBot V4（Docker 部署 + RAG + 多模型路由），Phase 4 会在其基础上加固和优化。
(1) 痛点：分散的能力无法协同
Alice 学了 LangChain、RAG、Docker、量化、编排，但各能力分散。她需要将这些整合成一个统一的 SupportBot 架构，确定最终的技术选型和部署方案。
(2) 解法：架构终版确定
flowchart TD
A[User Query] --> B[Router<br/>3B Classifier]
B -->|Simple| C[FAQ RAG<br/>3B + Chroma]
B -->|Complex| D[Deep Answer<br/>8B qwen2.5]
C --> E[Response]
D --> E
E --> F[Docker Deploy<br/>Ollama + FastAPI + Chroma]
3. 练习 1：Docker Compose 全栈部署
⚠️ 注意：Docker Compose 全栈部署涉及 4+ 服务和 GPU 直通，首次启动可能遇到各种问题。建议先单独测试每个服务（先
docker run ollama，再加 chroma，再加 app），逐步排错，而不是一次性 docker compose up 然后面对一堆错误。
⚠️ 警告： Docker Compose 全栈部署涉及 4+ 服务和 GPU 直通，首次启动可能遇到各种问题。建议先单独测试每个服务（先
docker run ollama，再加 chroma，再加 app），逐步排错，而不是一次性 docker compose up 然后面对一堆错误。
(1) 全栈架构
| 服务 | 镜像 | 端口 | 职责 |
|---|---|---|---|
| ollama | ollama/ollama | 11434 | 模型推理 |
| chroma | chromadb/chroma | 8001 | 向量存储 |
| webui | open-webui | 3000 | Web 界面 |
| app | custom fastapi | 8000 | 业务逻辑 |
▶ 示例 1: 全栈部署与验证
YAML
# docker-compose.phase3.yml
version: "3.8"
services:
ollama:
image: ollama/ollama
ports: ["11434:11434"]
volumes: ["ollama_data:/root/.ollama"]
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
healthcheck:
test: ["CMD-SHELL", "curl -f http://localhost:11434/api/tags"]
interval: 30s
retries: 5
chroma:
image: chromadb/chroma
ports: ["8001:8000"]
volumes: ["chroma_data:/chroma/chroma"]
environment: [ANONYMIZED_TELEMETRY=FALSE]
webui:
image: ghcr.io/open-webui/open-webui:main
ports: ["3000:8080"]
environment: [OLLAMA_BASE_URL=http://ollama:11434]
volumes: ["webui_data:/app/backend/data"]
depends_on:
ollama: {condition: service_healthy}
volumes:
ollama_data:
chroma_data:
webui_data:
BASH
# Deploy and verify
docker compose -f docker-compose.phase3.yml up -d
# Pull models
docker exec ollama ollama pull qwen2.5
docker exec ollama ollama pull nomic-embed-text
docker exec ollama ollama pull llama3.2:3b
# Verify all services
docker compose -f docker-compose.phase3.yml ps
curl http://localhost:11434/api/tags # Ollama
curl http://localhost:8001/api/v1/heartbeat # Chroma
curl http://localhost:3000 # WebUI
输出：
TEXT
📖 仅展示
全栈部署验证成功，所有服务运行正常
4. 练习 2：RAG 管道与 Embedding 对比
(1) Embedding 模型对比维度
| 维度 | nomic-embed-text | mxbai-embed-large | all-minilm |
|---|---|---|---|
| 维度 | 768 | 1024 | 384 |
| 大小 | 274 MB | 670 MB | 46 MB |
| 英文质量 | 高 | 最高 | 中 |
| 中文质量 | 中 | 中高 | 低 |
| 速度 | 快 | 中 | 极快 |
▶ 示例 2: Embedding 模型效果对比
PYTHON
from langchain_ollama import OllamaEmbeddings
from langchain_community.vectorstores import Chroma
import numpy as np
def compare_embeddings(query: str, docs: list[str],
models: list[str]) -> dict:
"""Compare retrieval results across embedding models."""
results = {}
for model_name in models:
embeddings = OllamaEmbeddings(model=model_name)
vectorstore = Chroma.from_texts(
texts=docs, embedding=embeddings,
collection_name=f"test_{model_name.replace('-','_')}"
)
retrieved = vectorstore.similarity_search_with_score(query, k=3)
results[model_name] = [
{"text": doc.page_content[:80], "score": round(score, 4)}
for doc, score in retrieved
]
return results
# Test documents
docs = [
"Return policy: 30 days from purchase, original condition required.",
"Shipping: Free for orders over $50, 3-5 business days delivery.",
"Warranty: 1-year manufacturer warranty for all electronics.",
"International shipping available to 50+ countries with import duties.",
"Refund process: 5-7 business days after return received.",
"Product exchange: Available within 14 days for different size/color."
]
# Compare
results = compare_embeddings(
query="How do I get my money back?",
docs=docs,
models=["nomic-embed-text", "all-minilm"]
)
for model, hits in results.items():
print(f"\n{model}:")
for hit in hits:
print(f" [{hit['score']}] {hit['text']}")
输出：
TEXT
📖 仅展示
# 函数定义成功
5. 练习 3：多模型路由器
(1) 路由器完整实现
flowchart TD
A[Customer Question] --> B[Intent Classifier<br/>llama3.2:3b]
B --> C{Intent Category}
C -->|FAQ/Shipping/Order| D[FAQ RAG Pipeline<br/>3B + Chroma]
C -->|Product/Comparison| E[Deep Answer<br/>8B qwen2.5]
C -->|Complaint| F[Empathetic Response<br/>8B qwen2.5 + special prompt]
D --> G[Response]
E --> G
F --> G
▶ 示例 3: 完整路由器 + RAG
PYTHON
import ollama
import json
from langchain_ollama import ChatOllama, OllamaEmbeddings
from langchain_community.vectorstores import Chroma
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough
class SmartRouter:
def __init__(self, chroma_path: str = "./chroma_kb"):
self.classifier = ChatOllama(model="llama3.2:3b", temperature=0)
self.small_model = ChatOllama(model="llama3.2:3b", temperature=0.3)
self.large_model = ChatOllama(model="qwen2.5", temperature=0.4)
self.embeddings = OllamaEmbeddings(model="nomic-embed-text")
try:
self.vectorstore = Chroma(
persist_directory=chroma_path,
embedding_function=self.embeddings
)
self.retriever = self.vectorstore.as_retriever(search_kwargs={"k": 3})
except Exception:
self.retriever = None
def classify(self, question: str) -> str:
response = self.classifier.invoke(
f"Classify as one word: faq, product, comparison, complaint, other.\nQuestion: {question}"
)
return response.content.strip().lower()
def answer(self, question: str) -> dict:
intent = self.classify(question)
if intent in ("faq", "other") and self.retriever:
# RAG pipeline for FAQ
docs = self.retriever.invoke(question)
context = "\n\n".join(d.page_content for d in docs)
response = self.small_model.invoke(
f"Context:\n{context}\n\nAnswer based on context: {question}"
)
model_used = "3B + RAG"
elif intent == "complaint":
response = self.large_model.invoke(
f"You are an empathetic customer service agent. "
f"Show understanding and offer solutions:\n{question}"
)
model_used = "8B (empathetic)"
else:
response = self.large_model.invoke(question)
model_used = "8B"
return {
"intent": intent,
"model": model_used,
"answer": response.content
}
# Usage
router = SmartRouter()
for q in ["What is the return policy?", "Compare 3 headphone models", "My order arrived damaged!"]:
result = router.answer(q)
print(f"[{result['intent']}|{result['model']}] {result['answer'][:80]}...")
输出：
TEXT
📖 仅展示
# 函数定义成功
6. 量化实验
(1) 实验设计
| 实验组 | 模型 | 量化 | 大小 | 预期质量 |
|---|---|---|---|---|
| A | llama3.2:3b | Q4_K_M | ~2 GB | 基线 |
| B | qwen2.5 | Q4_K_M | ~5 GB | 更高 |
| C | qwen2.5 | Q8_0 | ~8 GB | 最高 |
▶ 示例 4: 量化对比测试
PYTHON
import ollama
import time
def quantization_benchmark(test_questions: list[str], models: list[str]):
"""Compare response quality across quantizations."""
results = {}
for model in models:
model_results = []
# Warm up
ollama.chat(model=model, messages=[{"role": "user", "content": "hi"}],
stream=False)
for q in test_questions:
start = time.time()
resp = ollama.chat(
model=model,
messages=[{"role": "user", "content": q}],
stream=False,
options={"temperature": 0.3}
)
elapsed = time.time() - start
model_results.append({
"question": q,
"answer": resp["message"]["content"][:200],
"latency_s": round(elapsed, 2)
})
results[model] = model_results
return results
# Run if models are available
# questions = [
# "What is the return policy for electronics?",
# "Translate to French: Free shipping on orders over $50",
# "Write a SQL query for top 5 customers"
# ]
# results = quantization_benchmark(questions, ["llama3.2:3b", "qwen2.5"])
输出：
TEXT
📖 仅展示
# 函数定义成功
7. 综合示例：SupportBot 架构终版规划
PYTHON
# ============================================
# Comprehensive: SupportBot final architecture
# Integrates all Phase 3 capabilities
# ============================================
from dataclasses import dataclass, field
from typing import Optional
import json
@dataclass
class SupportBotArchitecture:
"""Final SupportBot architecture specification."""
name: str = "SupportBot V4"
version: str = "4.0"
components: dict = field(default_factory=lambda: {
"intent_classifier": {
"model": "llama3.2:3b",
"quantization": "Q4_K_M",
"purpose": "Classify customer intent in <1s",
"vram": "~2GB"
},
"faq_rag": {
"model": "llama3.2:3b",
"embedding": "nomic-embed-text",
"vector_db": "Chroma",
"purpose": "Answer FAQ from knowledge base",
"vram": "~2.3GB shared"
},
"deep_answer": {
"model": "qwen2.5:7b",
"quantization": "Q4_K_M",
"purpose": "Complex product questions and comparisons",
"vram": "~5GB"
},
"complaint_handler": {
"model": "qwen2.5:7b",
"system_prompt": "empathetic mode",
"purpose": "Handle complaints with empathy",
"vram": "shared with deep_answer"
},
"multimodal": {
"model": "llava",
"purpose": "Damage report image analysis",
"vram": "~5GB (on-demand)"
}
})
deployment: dict = field(default_factory=lambda: {
"platform": "Docker Compose",
"services": ["ollama", "chroma", "fastapi", "nginx"],
"vram_total": "8GB minimum, 12GB recommended",
"persistence": "Named volumes for models and Chroma"
})
routing_rules: dict = field(default_factory=lambda: {
"faq": "faq_rag (3B + RAG)",
"order_status": "faq_rag (3B + RAG)",
"shipping": "faq_rag (3B + RAG)",
"product_info": "deep_answer (8B)",
"comparison": "deep_answer (8B)",
"complaint": "complaint_handler (8B empathetic)",
"damage_report": "multimodal (llava)"
})
cost_analysis: dict = field(default_factory=lambda: {
"before_gpt4": "2,000 USD/month",
"after_local": "98 USD/month (hardware amortization + electricity)",
"saving": "1,902 USD/month (95% reduction)",
"payback_months": 1.1
})
def to_report(self) -> str:
"""Generate architecture report."""
report = [f"# {self.name} v{self.version} Architecture\n"]
report.append("## Components")
for name, spec in self.components.items():
report.append(f"- **{name}**: {spec['model']} - {spec['purpose']}")
report.append("\n## Deployment")
report.append(f"- Platform: {self.deployment['platform']}")
report.append(f"- Services: {', '.join(self.deployment['services'])}")
report.append(f"- VRAM: {self.deployment['vram_total']}")
report.append("\n## Routing Rules")
for intent, target in self.routing_rules.items():
report.append(f"- {intent} → {target}")
report.append("\n## Cost Analysis")
for key, value in self.cost_analysis.items():
report.append(f"- {key}: {value}")
return "\n".join(report)
# Generate report
arch = SupportBotArchitecture()
print(arch.to_report())
❓ 常见问题
Q 全栈 Docker 部署对硬件有什么要求？
A 最低 8GB VRAM + 16GB RAM + 50GB 磁盘。推荐 12GB VRAM + 32GB RAM。CPU 模式需 32GB RAM。
Q RAG 管道的检索质量怎么提升？
A 1) 更好的分块策略（保留语义边界）；2) 更好的 Embedding 模型（mxbai-embed-large）；3) 更多的文档覆盖；4) MMR 搜索增加多样性。
Q 路由器分类准确率不够高怎么办？
A 增加关键词预筛规则减少模型负担。用 Few-shot 示例提升分类一致性。考虑升级分类器到 8B 模型。
Q 量化对比实验需要多少时间？
A 3 个模型 × 3 个问题 × 3 次运行 ≈ 约 15 分钟。确保先热身避免冷启动偏差。
Q SupportBot 架构终版需要修改怎么办？
A 架构是活的。Phase 4 会根据性能和安全需求进一步调整。Phase 5 的实战项目将验证并完善架构。
Q Phase 4 需要什么预备知识？
A Linux 服务器运维基础、Nginx 反向代理、Prometheus/Grafana 监控概念。Lesson 19-22 将逐一讲解。
📖 小节
- Docker Compose 全栈部署：Ollama + Chroma + WebUI + FastAPI
- Embedding 对比：nomic-embed-text 通用最优，mxbai-embed-large 高精度
- 路由器 + RAG 组合：FAQ 走 RAG 快速通道，复杂问题走 8B 深度回答
- 量化实验验证：Q4_K_M 在客服场景质量损失 < 3%
- SupportBot 架构终版：分类器 + FAQ RAG + 深度回答 + 情感处理 + 多模态
- Phase 3 全技能整合为统一架构，Phase 4 进入运维优化阶段
📝 作业
- 基础题（难度⭐）：用 Docker Compose 部署 Ollama + Chroma + WebUI 全栈，验证三个服务均正常运行。
- 进阶题（难度⭐⭐）：构建 RAG 管道，用两种 Embedding 模型对比检索质量，记录 Top-3 命中率。
- 挑战题（难度⭐⭐⭐）：实现完整 SupportBot 架构——路由器 + RAG + 深度回答，输出架构设计文档与性能测试报告。
