DeepSeek Harness: LLM Adapter

Q: Which methods must a custom adapter implement?

Must implement complete , stream , getModels , getModelCapabilities . Missing any method causes a TypeScript compile error.

Q: What if streaming output is interrupted?

The StreamChunk protocol includes an error type. Consumers should handle error chunks, log them, and decide whether to retry.

Q: How to securely store API keys in adapters?

Use environment variables or .env files; don't hardcode in configuration. Schemastery supports .hidden() to mark sensitive fields.

Last updated: 2026-08-31

DSH's "model-agnostic" approach isn't just a slogan — it's an architectural design. LLM adapters, as Providers in the Cordis capability system, make switching models as easy as swapping batteries. This lesson dives into LLM adapter registration, implementation, and multi-model management.

💡 Tip: All LLM adapters implement the same Definition — LLMCapability. Agents and tools only depend on this interface, not whether the underlying model is DeepSeek or GPT-4o.

📋 Prerequisites: Completed 22-capability.md, understand capability three roles

1. What You'll Learn

ctx.llm adapter registration
OpenAI-compatible endpoint adapter
Custom model routing
StreamChunk protocol
Model capability declarations
Multi-model load balancing

LLM Adapter

2. ctx.llm Adapter Registration

(1) LLM Capability Interface

DSH's LLM adapter implements the LLMCapability interface:

TYPESCRIPT

interface LLMCapability {
  complete(request: LLMRequest): Promise<LLMResponse>
  stream(request: LLMRequest): AsyncIterable<StreamChunk>
  getModels(): ModelInfo[]
  getModelCapabilities(model: string): ModelCapabilities
}

(2) ▶ Example 2

TYPESCRIPT

import { Service, Context } from '@deepseek-ai/cordis'

export default class MyLLMAdapter extends Service {
  constructor(ctx: Context) {
    super(ctx, 'llm')
  }
}

super(ctx, 'llm') registers the adapter as the llm service.

(3) Built-in Adapters

DSH comes with two LLM adapters:

Adapter	Service Name	Supported Models
DeepSeek	`llm`	deepseek-chat, deepseek-reasoner
OpenAI Compatible	`llm`	Any OpenAI-compatible API

(4) ▶ Example 4

YAML

# cordis.yml
plugins:
  llm:
    $replace: ./adapters/my-custom-llm
    config:
      apiKey: sk-xxx
      endpoint: https://api.example.com/v1

3. OpenAI-Compatible Endpoint Adapter

(1) Protocol Overview

The OpenAI Chat Completions API is the de facto industry standard. DSH's built-in adapter is compatible with this protocol:

TYPESCRIPT

interface OpenAIRequest {
  model: string
  messages: { role: string; content: string }[]
  temperature?: number
  max_tokens?: number
  stream?: boolean
  tools?: ToolDefinition[]
}

(2) ▶ Example 2

YAML

# cordis.yml
plugins:
  llm:
    config:
      provider: openai-compatible
      apiKey: sk-xxx
      endpoint: https://api.openai.com/v1
      models:
        - id: gpt-4o
          capabilities: [chat, tool-use, vision]
        - id: gpt-4o-mini
          capabilities: [chat, tool-use]

(3) Custom Endpoints

Any OpenAI-compatible API can be plugged in:

YAML

# Connect to local Ollama
plugins:
  llm:
    config:
      provider: openai-compatible
      endpoint: http://localhost:11434/v1
      apiKey: ollama
      models:
        - id: llama3
          capabilities: [chat]

# Connect to Azure OpenAI
plugins:
  llm:
    config:
      provider: openai-compatible
      endpoint: https://my-resource.openai.azure.com/openai/deployments/my-deployment
      apiKey: xxx
      headers:
        api-key: xxx

(4) Writing a Custom Adapter

When the built-in adapter doesn't meet your needs, write a custom one:

TYPESCRIPT

import { Service, Context } from '@deepseek-ai/cordis'
import { LLMCapability, LLMRequest, LLMResponse, StreamChunk } from '@deepseek-ai/dsh'

export const Config = Schema.object({
  apiKey: Schema.string().required().description('API key'),
  endpoint: Schema.string().required().description('API endpoint'),
  defaultModel: Schema.string().default('custom-model').description('Default model ID')
})

export default class CustomLLMAdapter extends Service implements LLMCapability {
  static inject = []

  private apiKey: string
  private endpoint: string
  private defaultModel: string

  constructor(ctx: Context) {
    super(ctx, 'llm')
    this.apiKey = ctx.config.apiKey
    this.endpoint = ctx.config.endpoint
    this.defaultModel = ctx.config.defaultModel
  }

  async complete(request: LLMRequest): Promise<LLMResponse> {
    const response = await fetch(`${this.endpoint}/chat/completions`, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'Authorization': `Bearer ${this.apiKey}`
      },
      body: JSON.stringify({
        model: request.model || this.defaultModel,
        messages: request.messages,
        temperature: request.temperature,
        max_tokens: request.maxTokens,
        stream: false
      })
    })

    const data = await response.json()
    return {
      content: data.choices[0].message.content,
      model: data.model,
      usage: data.usage
    }
  }

  async *stream(request: LLMRequest): AsyncIterable<StreamChunk> {
    const response = await fetch(`${this.endpoint}/chat/completions`, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'Authorization': `Bearer ${this.apiKey}`
      },
      body: JSON.stringify({
        model: request.model || this.defaultModel,
        messages: request.messages,
        temperature: request.temperature,
        max_tokens: request.maxTokens,
        stream: true
      })
    })

    const reader = response.body!.getReader()
    const decoder = new TextDecoder()
    let buffer = ''

    while (true) {
      const { done, value } = await reader.read()
      if (done) break

      buffer += decoder.decode(value, { stream: true })
      const lines = buffer.split('\n')
      buffer = lines.pop() || ''

      for (const line of lines) {
        if (line.startsWith('data: ')) {
          const data = line.slice(6)
          if (data === '[DONE]') return
          try {
            const parsed = JSON.parse(data)
            const delta = parsed.choices[0].delta
            if (delta.content) {
              yield { type: 'text', content: delta.content }
            }
          } catch {}
        }
      }
    }
  }

  getModels() {
    return [{ id: this.defaultModel, capabilities: ['chat', 'tool-use'] }]
  }

  getModelCapabilities(model: string) {
    return { chat: true, toolUse: true, vision: false }
  }
}

4. Custom Model Routing

(1) Routing Concept

Model routing selects different LLM adapters based on request characteristics:

graph TB
    REQ[LLM Request] --> ROUTER{Model Router}
    ROUTER -->|code task| CODE[DeepSeek Coder]
    ROUTER -->|reasoning| REASON[DeepSeek Reasoner]
    ROUTER -->|simple chat| CHAT[GPT-4o-mini]

(2) Configuring Routing

YAML

plugins:
  llm:
    config:
      routing:
        default: deepseek-chat
        rules:
          - match:
              mode: ptc
            model: deepseek-reasoner
          - match:
              mode: creative
            model: deepseek-chat
          - match:
              tools: [shell]
            model: deepseek-coder

(3) Custom Routing Logic

TYPESCRIPT

export default class RoutingLLMAdapter extends Service {
  private adapters: Map<string, LLMCapability> = new Map()

  constructor(ctx: Context) {
    super(ctx, 'llm')
  }

  async complete(request: LLMRequest): Promise<LLMResponse> {
    const model = this._selectModel(request)
    const adapter = this._getAdapter(model)
    return adapter.complete({ ...request, model })
  }

  private _selectModel(request: LLMRequest): string {
    if (request.tools?.some(t => t.name === 'shell')) {
      return 'deepseek-coder'
    }
    if (request.metadata?.mode === 'ptc') {
      return 'deepseek-reasoner'
    }
    return 'deepseek-chat'
  }

  private _getAdapter(model: string): LLMCapability {
    const prefix = model.split('-')[0]
    return this.adapters.get(prefix) || this.adapters.get('default')!
  }
}

5. StreamChunk Protocol

(1) Protocol Definition

StreamChunk is DSH's unified streaming output protocol:

TYPESCRIPT

type StreamChunk =
  | { type: 'text'; content: string }
  | { type: 'tool_call'; id: string; name: string; arguments: string }
  | { type: 'tool_result'; id: string; result: any }
  | { type: 'error'; error: Error }
  | { type: 'done'; reason: 'stop' | 'tool_use' | 'length' }

(2) Chunk Types Explained

Type	Description	Source
`text`	Text content fragment	LLM streaming output
`tool_call`	Tool call request	LLM decides to call a tool
`tool_result`	Tool execution result	After tool execution
`error`	Error information	Errors at any stage
`done`	Stream end	LLM output finished

(3) Consuming StreamChunks

TYPESCRIPT

for await (const chunk of ctx.llm.stream(request)) {
  switch (chunk.type) {
    case 'text':
      process.stdout.write(chunk.content)
      break
    case 'tool_call':
      console.log(`\nTool call: ${chunk.name}`)
      break
    case 'tool_result':
      console.log(`Tool result:`, chunk.result)
      break
    case 'error':
      console.error(`Error:`, chunk.error)
      break
    case 'done':
      console.log(`\nDone (${chunk.reason})`)
      break
  }
}

6. Model Capability Declarations

(1) ModelCapabilities Interface

TYPESCRIPT

interface ModelCapabilities {
  chat: boolean
  toolUse: boolean
  vision: boolean
  maxTokens: number
  supportedModes: string[]
}

(2) Declaring Model Capabilities

TYPESCRIPT

getModels(): ModelInfo[] {
  return [
    {
      id: 'deepseek-chat',
      capabilities: {
        chat: true,
        toolUse: true,
        vision: false,
        maxTokens: 65536,
        supportedModes: ['standard', 'ptc', 'minimal', 'creative']
      }
    },
    {
      id: 'deepseek-reasoner',
      capabilities: {
        chat: true,
        toolUse: true,
        vision: false,
        maxTokens: 65536,
        supportedModes: ['ptc']
      }
    }
  ]
}

(3) Purpose of Capability Declarations

The framework uses capability declarations to decide:

Whether to allow tool use for this model (toolUse)
Whether to allow sending images (vision)
Maximum token limit (maxTokens)
Supported run modes (supportedModes)

7. Multi-Model Load Balancing

(1) Configuration

YAML

plugins:
  llm:
    config:
      loadBalancing:
        strategy: round-robin
        endpoints:
          - model: deepseek-chat
            endpoint: https://api.deepseek.com/v1
            apiKey: sk-key1
            weight: 3
          - model: gpt-4o
            endpoint: https://api.openai.com/v1
            apiKey: sk-key2
            weight: 1

(2) Load Balancing Strategies

Strategy	Description
`round-robin`	Round-robin
`random`	Random selection
`weighted`	Weighted distribution
`least-latency`	Select lowest latency

(3) Custom Load Balancing

TYPESCRIPT

export default class LoadBalancedLLM extends Service {
  private endpoints: Endpoint[]
  private currentIndex = 0

  async complete(request: LLMRequest): Promise<LLMResponse> {
    const endpoint = this._nextEndpoint()
    return endpoint.complete(request)
  }

  private _nextEndpoint(): Endpoint {
    const endpoint = this.endpoints[this.currentIndex]
    this.currentIndex = (this.currentIndex + 1) % this.endpoints.length
    return endpoint
  }
}

❓ FAQ

Q Can multiple LLM adapters be registered simultaneously?

A Not by default — later registrations override earlier ones. Use realm isolation or a routing adapter for multi-model management.

Q Which methods must a custom adapter implement?

A Must implement complete, stream, getModels, getModelCapabilities. Missing any method causes a TypeScript compile error.

Q How to test a custom adapter?

A Use an InMemoryProvider pattern — don't send real HTTP requests, return preset responses. Replace the llm service in unit tests.

Q What if streaming output is interrupted?

A The StreamChunk protocol includes an error type. Consumers should handle error chunks, log them, and decide whether to retry.

Q What's the difference between model routing and load balancing?

A Routing selects a model based on request characteristics (code→coder), while load balancing distributes requests across multiple endpoints of the same model.

Q How to securely store API keys in adapters?

A Use environment variables or .env files; don't hardcode in configuration. Schemastery supports .hidden() to mark sensitive fields.

📖 Summary

LLM adapters are Providers in the Cordis capability system, implementing the LLMCapability interface
Built-in OpenAI-compatible adapter supports any compatible API (DeepSeek, GPT-4o, Ollama, etc.)
Custom adapters implement four methods: complete/stream/getModels/getModelCapabilities
Model routing selects different models based on request characteristics; load balancing distributes across multiple endpoints
StreamChunk protocol unifies streaming output: text/tool_call/tool_result/error/done
Model capability declarations tell the framework what a model can do; the framework restricts features accordingly

📝 Exercises

1. ⭐ Basic： Configure DSH to use an OpenAI-compatible endpoint for GPT-4o, set API Key and model ID, and have the Agent complete a conversation successfully.

2. ⭐⭐ Intermediate： Write a custom LLM adapter that forwards requests to a local Ollama service (http://localhost:11434/v1). Implement complete and stream methods, test streaming output.

3. ⭐⭐⭐ Challenge： Implement a routing adapter that selects models based on Agent run mode — PTC mode uses reasoner, other modes use chat. Create two sessions using different modes and verify they use different models.

Web-Tutorial Tech Team

A team of developers maintaining programming tutorials. Each tutorial is written and reviewed by developers with expertise in that field. We work to keep our content accurate and reliable — if you spot an issue, please let us know.

About Us

DeepSeek Harness: LLM Adapter

1. What You'll Learn

2. ctx.llm Adapter Registration

(1) LLM Capability Interface

(2) ▶ Example 2

(3) Built-in Adapters

(4) ▶ Example 4

3. OpenAI-Compatible Endpoint Adapter

(1) Protocol Overview

(2) ▶ Example 2

(3) Custom Endpoints

(4) Writing a Custom Adapter

4. Custom Model Routing

(1) Routing Concept

(2) Configuring Routing

(3) Custom Routing Logic

5. StreamChunk Protocol

(1) Protocol Definition

(2) Chunk Types Explained

(3) Consuming StreamChunks

6. Model Capability Declarations

(1) ModelCapabilities Interface

(2) Declaring Model Capabilities

(3) Purpose of Capability Declarations

7. Multi-Model Load Balancing

(1) Configuration

(2) Load Balancing Strategies

(3) Custom Load Balancing

❓ FAQ

📖 Summary

📝 Exercises

🙏 帮我们做得更好