Ollama: LangChain Integration

LangChain is the orchestration maestro of AI applications — models are actors, Chains are scripts, and Agents are directors.

💡 Tip: The core of LangChain is the Agent/Chain/Tool three-layer architecture — Chain is a fixed pipeline (Prompt→LLM→Parser), Tool is a callable external capability (query a database, search documents), and Agent is an autonomous decision-maker that dynamically chooses which Tools to invoke. 90% of scenarios are well-served by Chains; use Agents only when dynamic decision-making is needed.

📋 Prerequisites: You should first master the following

1. What You Will Learn


2. A Real Story from a Startup CTO

(1) The Pain Point: Manually Analyzing 50 Metric Reports Every Month

Alice's SaaS company tracks 50 business metrics, requiring manual trend analysis and report writing each month. One analyst spends 3 days completing the work, and important anomaly signals are often missed.

(2) The Solution: LangChain Agent for Automated Analysis

Using LangChain + Ollama to build an AI Agent that automatically reads metric data, identifies anomalies, and generates analysis reports:

PYTHON
from langchain_ollama import ChatOllama

llm = ChatOllama(model="qwen2.5", temperature=0.2)
result = llm.invoke("Analyze the 20% drop in MRR last month")

3. LangChain + Ollama Connection

💡 Tip: The base_url parameter of ChatOllama defaults to http://localhost:11434. If Ollama is deployed on another host or in a Docker container, you need to explicitly set base_url="http://ollama:11434" or the corresponding address.

(1) Installation and Initialization

BASH
pip install langchain langchain-ollama langchain-community

(2) Two Core Components

Component Class Purpose
Chat Model ChatOllama Conversation generation
Embeddings OllamaEmbeddings Text vectorization
100%
flowchart TD
    A[LangChain App] --> B[ChatOllama<br/>qwen2.5 / llama3.2]
    A --> C[OllamaEmbeddings<br/>nomic-embed-text]
    B --> D[Ollama Server<br/>localhost:11434]
    C --> D
ChatOllama Parameter Type Default Description
model str Model name
temperature float 0.8 Randomness
base_url str http://localhost:11434 Server address
num_ctx int 2048 Context window

▶ Example 1: Basic ChatOllama Invocation

PYTHON
from langchain_ollama import ChatOllama
from langchain_core.messages import HumanMessage, SystemMessage

# Initialize ChatOllama
llm = ChatOllama(model="qwen2.5", temperature=0.3)

# Simple invocation
response = llm.invoke("What is RAG?")
print(response.content)

# With system message
response = llm.invoke([
    SystemMessage(content="You are a data analyst. Be concise."),
    HumanMessage(content="What causes MRR to drop?")
])
print(response.content)

Output:

TEXT
# Execution successful

4. Prompt Template + Chain

(1) Chained Prompt Engineering

Prompt Templates make prompts parameterizable, reusable, and composable:

100%
flowchart LR
    A[Input Variables] --> B[PromptTemplate]
    B --> C[ChatOllama]
    C --> D[Output Parser]
    D --> E[Structured Result]
Chain Type Description Use Case
LLMChain Prompt → LLM → Output Single-step reasoning
SequentialChain Multiple Chains in series Multi-step analysis
RouterChain Route based on input Branch processing

▶ Example 2: Prompt Template + Chain

PYTHON
from langchain_ollama import ChatOllama
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

llm = ChatOllama(model="qwen2.5", temperature=0.3)

# Define prompt template
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a {role}. Be concise and professional."),
    ("human", "{input}")
])

# Build chain: prompt -> llm -> parser
chain = prompt | llm | StrOutputParser()

# Invoke with variables
result = chain.invoke({
    "role": "customer service agent for e-commerce",
    "input": "Customer wants to cancel order #12345"
})
print(result)

Output:

TEXT
# Execution successful

5. Output Parser for Structured Output

⚠️ Warning: Small models (under 3B parameters) generate unstable JSON structures, causing PydanticOutputParser parsing failures. Use an 8B+ model with format="json", and add try/except retry logic in your code.

(1) Common Output Parsers

Parser Purpose Output Format
StrOutputParser Plain text string
JsonOutputParser JSON object dict
PydanticOutputParser Pydantic model typed object
CommaSeparatedListOutputParser Comma-separated list list[str]

(2) Pydantic Structured Output Comparison

Approach Advantages Disadvantages
format=json Simple No type validation
JsonOutputParser Has schema hints Parsing may fail
PydanticOutputParser Full type validation Depends on model following schema

▶ Example 3: Pydantic Structured Output

PYTHON
from langchain_ollama import ChatOllama
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import PydanticOutputParser
from pydantic import BaseModel, Field

# Define output schema
class CustomerIntent(BaseModel):
    intent: str = Field(description="Customer intent category")
    confidence: float = Field(description="Confidence score 0-1")
    order_number: str | None = Field(description="Order number if mentioned")

# Create parser
parser = PydanticOutputParser(pydantic_object=CustomerIntent)

# Build chain with format instructions
llm = ChatOllama(model="qwen2.5", temperature=0.1, format="json")
prompt = ChatPromptTemplate.from_messages([
    ("system", "Classify the customer intent.\n{format_instructions}"),
    ("human", "{query}")
])

chain = prompt | llm | parser

# Invoke
result = chain.invoke({
    "query": "I want to track my order #88765",
    "format_instructions": parser.get_format_instructions()
})
print(f"Intent: {result.intent}")
print(f"Confidence: {result.confidence}")
print(f"Order: {result.order_number}")

Output:

TEXT
# Execution successful

6. Tool-calling Agent

⚠️ Note: Agents may produce unstable output — local model reasoning loops can result in "infinite loops" (repeatedly calling the same tool) or "hallucinated tool calls" (calling non-existent tools). Always set max_iterations=3-5 to limit loop count, and use temperature=0 for higher determinism. 3B models have limited Agent capabilities; 8B+ is recommended.

💡 Tip: Ollama local models' Function Calling capability is not as stable as GPT-4. It is recommended to use ReAct Agent + @tool decorator pattern instead of native Function Calling for better compatibility, and temperature=0 can improve tool-calling determinism.

(1) Agent Architecture

100%
flowchart TD
    A[User Input] --> B[Agent]
    B --> C{Need Tool?}
    C -->|Yes| D[Call Tool]
    D --> E[Tool Result]
    E --> B
    C -->|No| F[Final Answer]
Agent Type Description Use Case
ReAct Agent Reasoning + action loop Complex tasks requiring tool calls
Structured Chat Structured tool calling Multi-tool, multi-parameter
OpenAI Functions Function calling mode Partially supported by Ollama

(2) Tool Definition Patterns

Definition Method Description Example
@tool decorator Simple tool @tool def func(...)
StructuredTool Tool with description StructuredTool.from_function(...)
Pydantic + BaseTool Strongly typed tool class MyTool(BaseTool)

▶ Example 4: ReAct Agent in Practice

PYTHON
from langchain_ollama import ChatOllama
from langchain_core.tools import tool
from langchain.agents import AgentExecutor, create_react_agent
from langchain_core.prompts import PromptTemplate

# Define tools
@tool
def get_order_status(order_id: str) -> str:
    """Get the status of an order by its ID."""
    # Simulated database lookup
    orders = {
        "12345": "Shipped, arriving in 2 days",
        "88765": "Processing, shipping tomorrow",
        "99999": "Delivered on 2024-01-15"
    }
    return orders.get(order_id, f"Order {order_id} not found")

@tool
def check_product_stock(product_name: str) -> str:
    """Check if a product is in stock."""
    stock = {
        "wireless headphones": "In stock: 45 units",
        "laptop stand": "Out of stock, restock in 3 days",
        "usb cable": "In stock: 200 units"
    }
    return stock.get(product_name.lower(), f"Product '{product_name}' not found")

# Build agent
llm = ChatOllama(model="qwen2.5", temperature=0)
tools = [get_order_status, check_product_stock]

prompt = PromptTemplate.from_template(
    """Answer the following question. You have access to these tools:
{tools}

Use this format:
Question: the input question
Thought: what to do
Action: tool name (one of [{tool_names}])
Action Input: tool input
Observation: tool output
... (repeat Thought/Action/Action Input/Observation)
Thought: I know the final answer
Final Answer: the final answer

Question: {input}
{agent_scratchpad}"""
)

agent = create_react_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

# Run agent
result = agent_executor.invoke({
    "input": "What is the status of order 88765?"
})
print(result["output"])

Output:

TEXT
# Function defined successfully

7. Comprehensive Example: Alice's SaaS Metrics Analysis Agent

ℹ️ Info: LangChain's AgentExecutor defaults to max_iterations=15, but local models reason more slowly. Setting it to 3-5 prevents infinite loops. Also set verbose=True to observe the Agent's reasoning process.

PYTHON
# ============================================
# Comprehensive: SaaS metrics analysis agent
# Uses LangChain + Ollama for automated analysis
# ============================================

from langchain_ollama import ChatOllama
from langchain_core.tools import tool
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain.agents import AgentExecutor, create_react_agent
from langchain_core.prompts import PromptTemplate
import json

# Simulated metrics database
METRICS_DB = {
    "mrr": {"current": 85000, "previous": 106250, "change": -20},
    "churn_rate": {"current": 8.5, "previous": 5.2, "change": 63},
    "arr": {"current": 1020000, "previous": 1275000, "change": -20},
    "active_users": {"current": 12500, "previous": 11800, "change": 6},
    "cac": {"current": 250, "previous": 180, "change": 39},
    "ltv": {"current": 2400, "previous": 3200, "change": -25}
}

@tool
def get_metric(metric_name: str) -> str:
    """Get a SaaS metric value. Available: mrr, churn_rate, arr, active_users, cac, ltv."""
    data = METRICS_DB.get(metric_name.lower())
    if data:
        return json.dumps(data)
    return f"Metric '{metric_name}' not found. Available: {list(METRICS_DB.keys())}"

@tool
def compare_metrics(metric1: str, metric2: str) -> str:
    """Compare two metrics and identify correlations."""
    m1 = METRICS_DB.get(metric1.lower())
    m2 = METRICS_DB.get(metric2.lower())
    if not m1 or not m2:
        return "One or both metrics not found."
    return json.dumps({
        "metric1": {"name": metric1, **m1},
        "metric2": {"name": metric2, **m2},
        "correlation_note": "Both metrics show negative trends"
    })

# Build analysis agent
llm = ChatOllama(model="qwen2.5", temperature=0.2)
tools = [get_metric, compare_metrics]

prompt = PromptTemplate.from_template(
    """You are a SaaS metrics analyst. Analyze data and provide insights.
Available tools: {tools}

Format:
Question: the input
Thought: what to do
Action: tool name from [{tool_names}]
Action Input: tool input
Observation: tool output
... (repeat as needed)
Thought: I have enough information
Final Answer: detailed analysis with actionable recommendations

Question: {input}
{agent_scratchpad}"""
)

agent = create_react_agent(llm, tools, prompt)
executor = AgentExecutor(agent=agent, tools=tools, verbose=True, max_iterations=5)

# Run analysis
if __name__ == "__main__":
    result = executor.invoke({
        "input": "MRR dropped 20% last month. Analyze the key metrics and identify likely causes with recommendations."
    })
    print("\n=== Analysis Report ===")
    print(result["output"])

❓ FAQ

Q What is the difference between langchain-ollama and langchain-community?
A langchain-ollama is the official Ollama integration package, containing ChatOllama and OllamaEmbeddings. langchain-community also includes legacy integrations, but the official package is recommended.
Q What should I do if the Agent's tool calls are unstable?
A Lower temperature to 0, ensure tool descriptions are clear, and use format=json to force structured output. Small models (3B) have limited Agent capabilities; 8B+ is recommended.
Q How do I choose between LangChain Chains and Agents?
A Use Chains for fixed workflows (Prompt → LLM → Parser), and Agents for dynamic decision-making (autonomously selecting tools and steps). Chains are sufficient for 90% of scenarios.
Q What should I do if PydanticOutputParser parsing fails?
A Add format="json" to force JSON output. Add explicit format instructions in the prompt. Fall back to JsonOutputParser on failure.
Q Do Ollama models support Function Calling?
A Partially. qwen2.5 and llama3.1 have some Function Calling capability, but it is not as stable as GPT-4. ReAct Agent + @tool decorator is recommended as an alternative.
Q How can I debug an Agent's reasoning process?
A Set verbose=True to view the Agent's Thought/Action/Observation loop. LangSmith (tracelangchain.com) can visualize the full execution chain.

📖 Summary


📝 Exercises

  1. Basic (⭐): Build a Chain with ChatOllama + PromptTemplate to implement customer service intent classification (output in JSON format).
  2. Intermediate (⭐⭐): Create a ReAct Agent with 2 tools (query order + check stock), and test tool-calling with 3 different questions.
  3. Advanced (⭐⭐⭐): Build a SaaS metrics analysis Agent for Alice — include at least 4 tools (get metric, compare metrics, generate chart description, send alert), outputting a structured analysis report.
Web-Tutorial.com

Web-Tutorial Tech Team

A team of developers maintaining programming tutorials. Each tutorial is written and reviewed by developers with expertise in that field. We work to keep our content accurate and reliable — if you spot an issue, please let us know.

100%

🙏 帮我们做得更好

我们是刚上线的编程教程站,几个人的小团队,精力有限。页面虽经检查,难免还有疏漏——链接失效、排版错乱、内容有误、语言生硬……

如果您发现了,麻烦告诉我们,我们会在收到反馈后第一时间进行修复,再次感谢您的光临 🙏