Ollama: LangChain Integration
LangChain is the orchestration maestro of AI applications — models are actors, Chains are scripts, and Agents are directors.
💡 Tip: The core of LangChain is the Agent/Chain/Tool three-layer architecture — Chain is a fixed pipeline (Prompt→LLM→Parser), Tool is a callable external capability (query a database, search documents), and Agent is an autonomous decision-maker that dynamically chooses which Tools to invoke. 90% of scenarios are well-served by Chains; use Agents only when dynamic decision-making is needed.
📋 Prerequisites: You should first master the following
- Lesson 5: REST API Basics
- Lesson 7: Python SDK Integration
1. What You Will Learn
- LangChain + Ollama connection configuration
- Prompt Template + Chain for chained prompt engineering
- Output Parser for structured output
- Tool-calling Agent
- Alice's SaaS metrics analysis Agent
2. A Real Story from a Startup CTO
(1) The Pain Point: Manually Analyzing 50 Metric Reports Every Month
Alice's SaaS company tracks 50 business metrics, requiring manual trend analysis and report writing each month. One analyst spends 3 days completing the work, and important anomaly signals are often missed.
(2) The Solution: LangChain Agent for Automated Analysis
Using LangChain + Ollama to build an AI Agent that automatically reads metric data, identifies anomalies, and generates analysis reports:
PYTHON
from langchain_ollama import ChatOllama
llm = ChatOllama(model="qwen2.5", temperature=0.2)
result = llm.invoke("Analyze the 20% drop in MRR last month")
3. LangChain + Ollama Connection
💡 Tip: The
base_url parameter of ChatOllama defaults to http://localhost:11434. If Ollama is deployed on another host or in a Docker container, you need to explicitly set base_url="http://ollama:11434" or the corresponding address.
(1) Installation and Initialization
BASH
pip install langchain langchain-ollama langchain-community
(2) Two Core Components
| Component | Class | Purpose |
|---|---|---|
| Chat Model | ChatOllama | Conversation generation |
| Embeddings | OllamaEmbeddings | Text vectorization |
flowchart TD
A[LangChain App] --> B[ChatOllama<br/>qwen2.5 / llama3.2]
A --> C[OllamaEmbeddings<br/>nomic-embed-text]
B --> D[Ollama Server<br/>localhost:11434]
C --> D
| ChatOllama Parameter | Type | Default | Description |
|---|---|---|---|
| model | str | — | Model name |
| temperature | float | 0.8 | Randomness |
| base_url | str | http://localhost:11434 | Server address |
| num_ctx | int | 2048 | Context window |
▶ Example 1: Basic ChatOllama Invocation
PYTHON
from langchain_ollama import ChatOllama
from langchain_core.messages import HumanMessage, SystemMessage
# Initialize ChatOllama
llm = ChatOllama(model="qwen2.5", temperature=0.3)
# Simple invocation
response = llm.invoke("What is RAG?")
print(response.content)
# With system message
response = llm.invoke([
SystemMessage(content="You are a data analyst. Be concise."),
HumanMessage(content="What causes MRR to drop?")
])
print(response.content)
Output:
TEXT
# Execution successful
4. Prompt Template + Chain
(1) Chained Prompt Engineering
Prompt Templates make prompts parameterizable, reusable, and composable:
flowchart LR
A[Input Variables] --> B[PromptTemplate]
B --> C[ChatOllama]
C --> D[Output Parser]
D --> E[Structured Result]
| Chain Type | Description | Use Case |
|---|---|---|
| LLMChain | Prompt → LLM → Output | Single-step reasoning |
| SequentialChain | Multiple Chains in series | Multi-step analysis |
| RouterChain | Route based on input | Branch processing |
▶ Example 2: Prompt Template + Chain
PYTHON
from langchain_ollama import ChatOllama
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
llm = ChatOllama(model="qwen2.5", temperature=0.3)
# Define prompt template
prompt = ChatPromptTemplate.from_messages([
("system", "You are a {role}. Be concise and professional."),
("human", "{input}")
])
# Build chain: prompt -> llm -> parser
chain = prompt | llm | StrOutputParser()
# Invoke with variables
result = chain.invoke({
"role": "customer service agent for e-commerce",
"input": "Customer wants to cancel order #12345"
})
print(result)
Output:
TEXT
# Execution successful
5. Output Parser for Structured Output
⚠️ Warning: Small models (under 3B parameters) generate unstable JSON structures, causing
PydanticOutputParser parsing failures. Use an 8B+ model with format="json", and add try/except retry logic in your code.
(1) Common Output Parsers
| Parser | Purpose | Output Format |
|---|---|---|
| StrOutputParser | Plain text | string |
| JsonOutputParser | JSON object | dict |
| PydanticOutputParser | Pydantic model | typed object |
| CommaSeparatedListOutputParser | Comma-separated list | list[str] |
(2) Pydantic Structured Output Comparison
| Approach | Advantages | Disadvantages |
|---|---|---|
| format=json | Simple | No type validation |
| JsonOutputParser | Has schema hints | Parsing may fail |
| PydanticOutputParser | Full type validation | Depends on model following schema |
▶ Example 3: Pydantic Structured Output
PYTHON
from langchain_ollama import ChatOllama
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import PydanticOutputParser
from pydantic import BaseModel, Field
# Define output schema
class CustomerIntent(BaseModel):
intent: str = Field(description="Customer intent category")
confidence: float = Field(description="Confidence score 0-1")
order_number: str | None = Field(description="Order number if mentioned")
# Create parser
parser = PydanticOutputParser(pydantic_object=CustomerIntent)
# Build chain with format instructions
llm = ChatOllama(model="qwen2.5", temperature=0.1, format="json")
prompt = ChatPromptTemplate.from_messages([
("system", "Classify the customer intent.\n{format_instructions}"),
("human", "{query}")
])
chain = prompt | llm | parser
# Invoke
result = chain.invoke({
"query": "I want to track my order #88765",
"format_instructions": parser.get_format_instructions()
})
print(f"Intent: {result.intent}")
print(f"Confidence: {result.confidence}")
print(f"Order: {result.order_number}")
Output:
TEXT
# Execution successful
6. Tool-calling Agent
⚠️ Note: Agents may produce unstable output — local model reasoning loops can result in "infinite loops" (repeatedly calling the same tool) or "hallucinated tool calls" (calling non-existent tools). Always set
max_iterations=3-5 to limit loop count, and use temperature=0 for higher determinism. 3B models have limited Agent capabilities; 8B+ is recommended.
💡 Tip: Ollama local models' Function Calling capability is not as stable as GPT-4. It is recommended to use ReAct Agent +
@tool decorator pattern instead of native Function Calling for better compatibility, and temperature=0 can improve tool-calling determinism.
(1) Agent Architecture
flowchart TD
A[User Input] --> B[Agent]
B --> C{Need Tool?}
C -->|Yes| D[Call Tool]
D --> E[Tool Result]
E --> B
C -->|No| F[Final Answer]
| Agent Type | Description | Use Case |
|---|---|---|
| ReAct Agent | Reasoning + action loop | Complex tasks requiring tool calls |
| Structured Chat | Structured tool calling | Multi-tool, multi-parameter |
| OpenAI Functions | Function calling mode | Partially supported by Ollama |
(2) Tool Definition Patterns
| Definition Method | Description | Example |
|---|---|---|
| @tool decorator | Simple tool | @tool def func(...) |
| StructuredTool | Tool with description | StructuredTool.from_function(...) |
| Pydantic + BaseTool | Strongly typed tool | class MyTool(BaseTool) |
▶ Example 4: ReAct Agent in Practice
PYTHON
from langchain_ollama import ChatOllama
from langchain_core.tools import tool
from langchain.agents import AgentExecutor, create_react_agent
from langchain_core.prompts import PromptTemplate
# Define tools
@tool
def get_order_status(order_id: str) -> str:
"""Get the status of an order by its ID."""
# Simulated database lookup
orders = {
"12345": "Shipped, arriving in 2 days",
"88765": "Processing, shipping tomorrow",
"99999": "Delivered on 2024-01-15"
}
return orders.get(order_id, f"Order {order_id} not found")
@tool
def check_product_stock(product_name: str) -> str:
"""Check if a product is in stock."""
stock = {
"wireless headphones": "In stock: 45 units",
"laptop stand": "Out of stock, restock in 3 days",
"usb cable": "In stock: 200 units"
}
return stock.get(product_name.lower(), f"Product '{product_name}' not found")
# Build agent
llm = ChatOllama(model="qwen2.5", temperature=0)
tools = [get_order_status, check_product_stock]
prompt = PromptTemplate.from_template(
"""Answer the following question. You have access to these tools:
{tools}
Use this format:
Question: the input question
Thought: what to do
Action: tool name (one of [{tool_names}])
Action Input: tool input
Observation: tool output
... (repeat Thought/Action/Action Input/Observation)
Thought: I know the final answer
Final Answer: the final answer
Question: {input}
{agent_scratchpad}"""
)
agent = create_react_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
# Run agent
result = agent_executor.invoke({
"input": "What is the status of order 88765?"
})
print(result["output"])
Output:
TEXT
# Function defined successfully
7. Comprehensive Example: Alice's SaaS Metrics Analysis Agent
ℹ️ Info: LangChain's
AgentExecutor defaults to max_iterations=15, but local models reason more slowly. Setting it to 3-5 prevents infinite loops. Also set verbose=True to observe the Agent's reasoning process.
PYTHON
# ============================================
# Comprehensive: SaaS metrics analysis agent
# Uses LangChain + Ollama for automated analysis
# ============================================
from langchain_ollama import ChatOllama
from langchain_core.tools import tool
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain.agents import AgentExecutor, create_react_agent
from langchain_core.prompts import PromptTemplate
import json
# Simulated metrics database
METRICS_DB = {
"mrr": {"current": 85000, "previous": 106250, "change": -20},
"churn_rate": {"current": 8.5, "previous": 5.2, "change": 63},
"arr": {"current": 1020000, "previous": 1275000, "change": -20},
"active_users": {"current": 12500, "previous": 11800, "change": 6},
"cac": {"current": 250, "previous": 180, "change": 39},
"ltv": {"current": 2400, "previous": 3200, "change": -25}
}
@tool
def get_metric(metric_name: str) -> str:
"""Get a SaaS metric value. Available: mrr, churn_rate, arr, active_users, cac, ltv."""
data = METRICS_DB.get(metric_name.lower())
if data:
return json.dumps(data)
return f"Metric '{metric_name}' not found. Available: {list(METRICS_DB.keys())}"
@tool
def compare_metrics(metric1: str, metric2: str) -> str:
"""Compare two metrics and identify correlations."""
m1 = METRICS_DB.get(metric1.lower())
m2 = METRICS_DB.get(metric2.lower())
if not m1 or not m2:
return "One or both metrics not found."
return json.dumps({
"metric1": {"name": metric1, **m1},
"metric2": {"name": metric2, **m2},
"correlation_note": "Both metrics show negative trends"
})
# Build analysis agent
llm = ChatOllama(model="qwen2.5", temperature=0.2)
tools = [get_metric, compare_metrics]
prompt = PromptTemplate.from_template(
"""You are a SaaS metrics analyst. Analyze data and provide insights.
Available tools: {tools}
Format:
Question: the input
Thought: what to do
Action: tool name from [{tool_names}]
Action Input: tool input
Observation: tool output
... (repeat as needed)
Thought: I have enough information
Final Answer: detailed analysis with actionable recommendations
Question: {input}
{agent_scratchpad}"""
)
agent = create_react_agent(llm, tools, prompt)
executor = AgentExecutor(agent=agent, tools=tools, verbose=True, max_iterations=5)
# Run analysis
if __name__ == "__main__":
result = executor.invoke({
"input": "MRR dropped 20% last month. Analyze the key metrics and identify likely causes with recommendations."
})
print("\n=== Analysis Report ===")
print(result["output"])
❓ FAQ
Q What is the difference between langchain-ollama and langchain-community?
A
langchain-ollama is the official Ollama integration package, containing ChatOllama and OllamaEmbeddings. langchain-community also includes legacy integrations, but the official package is recommended.Q What should I do if the Agent's tool calls are unstable?
A Lower temperature to 0, ensure tool descriptions are clear, and use format=json to force structured output. Small models (3B) have limited Agent capabilities; 8B+ is recommended.
Q How do I choose between LangChain Chains and Agents?
A Use Chains for fixed workflows (Prompt → LLM → Parser), and Agents for dynamic decision-making (autonomously selecting tools and steps). Chains are sufficient for 90% of scenarios.
Q What should I do if PydanticOutputParser parsing fails?
A Add
format="json" to force JSON output. Add explicit format instructions in the prompt. Fall back to JsonOutputParser on failure.Q Do Ollama models support Function Calling?
A Partially. qwen2.5 and llama3.1 have some Function Calling capability, but it is not as stable as GPT-4. ReAct Agent + @tool decorator is recommended as an alternative.
Q How can I debug an Agent's reasoning process?
A Set
verbose=True to view the Agent's Thought/Action/Observation loop. LangSmith (tracelangchain.com) can visualize the full execution chain.📖 Summary
- ChatOllama is the core class for connecting LangChain to Ollama, supporting conversation and streaming output
- Prompt Templates make prompts parameterizable; Chains make workflows composable
- Output Parsers convert LLM text into structured data (JSON/Pydantic/List)
- ReAct Agents implement a reasoning + action loop that can call custom tools
- Alice used a LangChain Agent to automate SaaS metrics analysis, reducing 3 days of work to 3 minutes
- Agents require 8B+ models for stability; 3B models lack sufficient reasoning capability
📝 Exercises
- Basic (⭐): Build a Chain with ChatOllama + PromptTemplate to implement customer service intent classification (output in JSON format).
- Intermediate (⭐⭐): Create a ReAct Agent with 2 tools (query order + check stock), and test tool-calling with 3 different questions.
- Advanced (⭐⭐⭐): Build a SaaS metrics analysis Agent for Alice — include at least 4 tools (get metric, compare metrics, generate chart description, send alert), outputting a structured analysis report.