English 中文 العربية 日本語 Português

Go: Go HTTP Clients and Middleware

Last updated: 2026-08-26

HTTP clients and وسيط are the two cornerstones of microservice communication—one manages connections, and the other handles cross-cutting concerns.

When your service needs to call five other downstream services, each call requires logging, timeout handling, retries, and circuit breaking—should you write this code five times or abstract it once? In this lesson, you'll master advanced usage of the Go HTTP عميل and وسيط design patterns.

1. You will learn

2. A True Story of a Backend Engineer

(1) Pain Point: Five downstream services, each requiring logging, timeout handling, and retries

Charlie is in charge of the company's API gateway service, which needs to call five downstream services:

"Every downstream call must: print a طلب log (to facilitate troubleshooting), set a timeout (to prevent deadlocks), and record the duration (for monitoring and alerts). I ended up writing the same code five times—one دالة per service, each containing the same repetitive 'sandwich' of logging, timeout, and the call itself."

His code at the time:

GO 
// Bad code: every downstream service repeats the same logic
func callUserService(w http.ResponseWriter, r *http.Request) {
    log.Printf("[%s] Requesting user service: %s", r.Method, r.URL.Path)
    start := time.Now()
    resp, err := http.Get("http://user-service/api/users")
    log.Printf("[%s] User service elapsed: %v", r.Method, time.Since(start))
    // ... process response
}

func callOrderService(w http.ResponseWriter, r *http.Request) {
    log.Printf("[%s] Requesting order service: %s", r.Method, r.URL.Path)
    start := time.Now()
    resp, err := http.Get("http://order-service/api/orders")
    log.Printf("[%s] Order service elapsed: %v", r.Method, time.Since(start))
    // ... same pattern again!
}
// Every new service repeats!

(2) Go Solution: Middleware Pattern + Custom Client

GO 
// Middleware: wraps http.RoundTripper
type LoggingMiddleware struct {
    next http.RoundTripper
}

func (m *LoggingMiddleware) RoundTrip(req *http.Request) (*http.Response, خطأ) {
    start := time.Now()
    log.Printf("[%s] %s %s", req.Method, req.URL.Host, req.URL.Path)
    resp, err := m.next.RoundTrip(req)
    log.Printf("[%s] %s elapsed: %v", req.Method, req.URL.Path, time.Since(start))
    return resp, err
}

// Unified Client: shared by all downstream services
عميل := &http.Client{
    Timeout: 5 * time.Second,
    Transport: &LoggingMiddleware{
        next: http.DefaultTransport,
    },
}

// All downstream calls automatically get logging + timeout
resp1, _ := عميل.Get("http://user-service/api/users")  // Auto logging
resp2, _ := عميل.Get("http://order-service/api/orders") // Auto logging

(3) Benefits: Before and After Middleware Implementation

Dimension Duplicate Code Middleware Pattern
Code Volume 30 lines per downstream, 5 downstreams = 150 lines 15 lines of middleware, for a total of 30 lines
Add downstream Copy and paste 30 lines Call client.Get directly
Edit timeout Change 5 places Change 1 client.Timeout

3. HTTP Client Basics

▶ Example: GET / POST / Custom Requests

GO 
package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "log"
    "net/http"
)

func main() {
    // 1. GET request
    resp, err := http.Get("https://api.example.com/users")
    if err != nil {
        log.Fatal(err)
    }
    defer resp.Body.Close()

    body, _ := io.ReadAll(resp.Body)
    fmt.Printf("GET status: %d\n", resp.StatusCode)
    fmt.Printf("Response: %s\n", body)

    // 2. POST JSON request
    data := map[string]string{"name": "Alice"}
    jsonData, _ := json.Marshal(data)

    resp, err = http.Post(
        "https://api.example.com/users",
        "application/json",
        bytes.NewReader(jsonData),
    )
    if err != nil {
        log.Fatal(err)
    }
    defer resp.Body.Close()

    // 3. Custom request (set Headers)
    req, _ := http.NewRequest("DELETE", "https://api.example.com/users/1", nil)
    req.Header.Set("Authorization", "Bearer token-123")
    req.Header.Set("X-Request-ID", "req-456")

    resp, err = http.DefaultClient.Do(req)
    if err != nil {
        log.Fatal(err)
    }
    defer resp.Body.Close()

    fmt.Printf("DELETE status: %d\n", resp.StatusCode)
}
▶ Try it Yourself

(2) http.Get vs http.Client

GO 
// Method 1: Use http.Get directly (default Client)
resp, _ := http.Get(url)
// Problem: No default timeout, may hang forever!

// Method 2: Custom Client (recommended)
client := &http.Client{
    Timeout: 10 * time.Second,
}
resp, _ := client.Get(url)
// Safe: 10-second timeout auto-cancels
Method Timeout Connection Pool Recommendation
http.Get(url) No timeout Default (limited reuse) ❌ For quick testing only
http.DefaultClient No timeout Default ⚠️ Use with caution in production
&http.Client{Timeout: 10s} Global timeout Default ✅ Recommended
&http.Client{Timeout, Transport} Global timeout Custom connection pool ✅ Production-ready

4. http.Client Timeouts and Transport

▶ Example: Configuring Timeouts and Connection Pools

GO 
package main

import (
    "fmt"
    "net"
    "net/http"
    "time"
)

func main() {
    // Custom Transport
    transport := &http.Transport{
        // Connection pool
        MaxIdleConns:        100,              // Max idle connections
        MaxIdleConnsPerHost: 10,               // Max idle connections per host
        IdleConnTimeout:     90 * time.Second, // Idle connection timeout

        // TLS
        TLSHandshakeTimeout: 10 * time.Second,

        // Dial
        DialContext: (&net.Dialer{
            Timeout:   30 * time.Second, // Connection timeout
            KeepAlive: 30 * time.Second, // Keep-Alive interval
        }).DialContext,
    }

    // Custom Client
    عميل := &http.Client{
        Timeout:   30 * time.Second, // Total طلب timeout (includes all stages)
        Transport: transport,
    }

    // Use عميل to send طلب
    resp, err := عميل.Get("https://api.example.com/users")
    if err != nil {
        fmt.Printf("Request failed: %v\n", err)
        return
    }
    defer resp.Body.Close()
    fmt.Printf("Status: %d\n", resp.StatusCode)
}
▶ Try it Yourself

▶ Example: Tiered Timeout Control

GO 
package main

import (
    "fmt"
    "net"
    "net/http"
    "time"
)

func main() {
    // Timeout tiers:
    // Transport.DialContext     → Connection timeout (10s)
    // Transport.TLSHandshake    → TLS handshake timeout (5s)
    // Transport.ResponseHeader  → Response header timeout (10s)
    // Client.Timeout            → Total timeout (30s, covers Dial + TLS + send + read)

    transport := &http.Transport{
        DialContext: (&net.Dialer{
            Timeout: 10 * time.Second,
        }).DialContext,
        TLSHandshakeTimeout:   5 * time.Second,
        ResponseHeaderTimeout: 10 * time.Second,
        ExpectContinueTimeout: 1 * time.Second,
    }

    client := &http.Client{
        Timeout:   30 * time.Second,
        Transport: transport,
    }

    start := time.Now()
    resp, err := client.Get("https://httpbin.org/delay/5")
    if err != nil {
        fmt.Printf("Error: %v (elapsed: %v)\n", err, time.Since(start))
        return
    }
    defer resp.Body.Close()
    fmt.Printf("Success: %d (elapsed: %v)\n", resp.StatusCode, time.Since(start))
}
▶ Try it Yourself

(3) Transport Configuration Parameters

Parameter Default Recommended Description
MaxIdleConns 100 100–200 Global maximum number of idle connections
MaxIdleConnsPerHost 2 10–50 Maximum number of idle connections per host (the default of 2 is too low!)
IdleConnTimeout 90s 30–90s Idle connection timeout
TLSHandshakeTimeout 10s 5–10s TLS handshake timeout
ResponseHeaderTimeout 0 (Never) 10–30s Response header timeout
DialContext.Timeout None 10–30s TCP connection timeout
💡 Tip: The default value for MaxIdleConnsPerHost in http.Transport is only 2, which is severely insufficient for scenarios involving high-concurrency calls to the same service. Be sure to increase this value based on the level of concurrency (e.g., 50–100).

5. Middleware Pattern

▶ Example: The Onion Model of Middleware

GO 📖 Display only 
package main

import (
    "context"
    "fmt"
    "log"
    "net/http"
    "time"
)

// Middleware type: receives Handler, returns Handler
type Middleware func(http.Handler) http.Handler

// Middleware chain: assembles all middleware into one
func Chain(handler http.Handler, middlewares ...Middleware) http.Handler {
    for i := len(middlewares) - 1; i >= 0; i-- {
        handler = middlewares[i](handler)
    }
    return handler
}

// ---------- Concrete middleware ----------

// 1. Logging middleware
func LoggingMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        start := time.Now()
        log.Printf("[%s] %s %s", r.Method, r.URL.Path, r.RemoteAddr)
        next.ServeHTTP(w, r)
        log.Printf("[%s] %s elapsed: %v", r.Method, r.URL.Path, time.Since(start))
    })
}

// 2. Recovery middleware (prevents panic from crashing the service)
func RecoveryMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        defer func() {
            if err := recover(); err != nil {
                log.Printf("[PANIC] %v", err)
                http.Error(w, "Internal Server Error", http.StatusInternalServerError)
            }
        }()
        next.ServeHTTP(w, r)
    })
}

// 3. CORS middleware
func CORSMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        w.Header().Set("Access-Control-Allow-Origin", "*")
        w.Header().Set("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE, OPTIONS")
        w.Header().Set("Access-Control-Allow-Headers", "Content-Type, Authorization")

        if r.Method == http.MethodOptions {
            w.WriteHeader(http.StatusNoContent)
            return
        }
        next.ServeHTTP(w, r)
    })
}

// 4. Timeout middleware
func TimeoutMiddleware(timeout time.Duration) Middleware {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            ctx, cancel := context.WithTimeout(r.Context(), timeout)
            defer cancel()
            next.ServeHTTP(w, r.WithContext(ctx))
        })
    }
}

// ---------- Usage ----------

func helloHandler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Hello, %s!", r.URL.Query().Get("name"))
}

func main() {
    mux := http.NewServeMux()
    mux.HandleFunc("GET /hello", helloHandler)

    // Chain all middleware (onion model: Recovery → Logging → CORS → Handler)
    handler := Chain(mux,
        RecoveryMiddleware,
        LoggingMiddleware,
        CORSMiddleware,
        TimeoutMiddleware(5*time.Second),
    )

    log.Print(http.ListenAndServe(":8080", handler))
}
69 logic lines (exceeds 40-line limit, display only)
100% 
graph LR
    Req[Request] --> R[Recovery]
    R --> L[Logging]
    L --> C[CORS]
    C --> T[Timeout]
    T --> H[Handler]
    H -->|Response| T
    T --> C
    C --> L
    L --> R
    R --> Resp[Response]

(2) The Responsibilities of the Four Core Middleware Components

Middleware Responsibilities Order
Recovery Catch panic, return 500 Outermost
Logging Logging Request and Response Times Layer 2
CORS Handling Cross-Origin Request Headers Layer 3
Timeout Request Timeout Control Innermost Layer (Close to Handler)

6. httputil.ReverseProxy Reverse Proxy

▶ Example: Reverse Proxy

GO 
package main

import (
    "log"
    "net/http"
    "net/http/httputil"
    "net/url"
)

func main() {
    // Target service address
    target, _ := url.Parse("http://localhost:8081")

    // Create reverse proxy
    proxy := httputil.NewSingleHostReverseProxy(target)

    // Custom Director (modify طلب headers)
    proxy.Director = func(req *http.Request) {
        req.URL.Scheme = target.Scheme
        req.URL.Host = target.Host
        req.URL.Path = target.Path + req.URL.Path
        req.Header.Set("X-Forwarded-Host", req.Header.Get("Host"))
        req.Header.Set("X-Real-IP", req.RemoteAddr)
    }

    // Custom خطأ handling
    proxy.ErrorHandler = func(w http.ResponseWriter, r *http.Request, err خطأ) {
        log.Printf("Proxy خطأ: %v", err)
        http.Error(w, "Bad Gateway", http.StatusBadGateway)
    }

    mux := http.NewServeMux()
    mux.HandleFunc("/api/", proxy.ServeHTTP)

    log.Print("API gateway started on :8080")
    log.Fatal(http.ListenAndServe(":8080", mux))
}
▶ Try it Yourself
💡 Tip: httputil.ReverseProxy automatically handles most proxy details: Host header forwarding, X-Forwarded-For, response header passing, and connection pool reuse. All you need to do is provide a Director function to modify the request.

7. Complete Example: API Gateway + Middleware Pipeline

GO 
// api_gateway.go
package main

import (
    "context"
    "encoding/json"
    "log"
    "net/http"
    "net/http/httputil"
    "net/url"
    "os"
    "os/signal"
    "strings"
    "time"
)

// ---------- Configuration ----------

type Route struct {
    Path   string
    Target *url.URL
}

type GatewayConfig struct {
    Port        string
    Routes      []Route
    Middlewares []Middleware
}

// ---------- Middleware ----------

type Middleware func(http.Handler) http.Handler

func Chain(handler http.Handler, middlewares ...Middleware) http.Handler {
    for i := len(middlewares) - 1; i >= 0; i-- {
        handler = middlewares[i](handler)
    }
    return handler
}

// Logging
func Logging(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        start := time.Now()
        log.Printf("[%s] %s", r.Method, r.URL.Path)
        next.ServeHTTP(w, r)
        log.Printf("[%s] %s → %v", r.Method, r.URL.Path, time.Since(start))
    })
}

// Recovery
func Recovery(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        defer func() {
            if err := recover(); err != nil {
                log.Printf("[PANIC] %v", err)
                http.Error(w, `{"error":"internal error"}`, http.StatusInternalServerError)
            }
        }()
        next.ServeHTTP(w, r)
    })
}

// CORS
func CORS(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        w.Header().Set("Access-Control-Allow-Origin", "*")
        w.Header().Set("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE, OPTIONS")
        w.Header().Set("Access-Control-Allow-Headers", "Content-Type, Authorization")
        if r.Method == http.MethodOptions {
            w.WriteHeader(http.StatusNoContent)
            return
        }
        next.ServeHTTP(w, r)
    })
}

// Rate limiting (simplified token bucket)
type RateLimiter struct {
    tokens chan struct{}
}

func NewRateLimiter(rate int) *RateLimiter {
    rl := &RateLimiter{tokens: make(chan struct{}, rate)}
    for i := 0; i < rate; i++ {
        rl.tokens <- struct{}{}
    }
    // Refill every second
    go func() {
        ticker := time.NewTicker(time.Second)
        defer ticker.Stop()
        for range ticker.C {
            select {
            case rl.tokens <- struct{}{}:
            default:
            }
        }
    }()
    return rl
}

func (rl *RateLimiter) Middleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        select {
        case <-rl.tokens:
            next.ServeHTTP(w, r)
        default:
            http.Error(w, `{"error":"rate limit exceeded"}`, http.StatusTooManyRequests)
        }
    })
}

// ---------- Gateway ----------

type Gateway struct {
    config GatewayConfig
    proxy  *httputil.ReverseProxy
}

func NewGateway(config GatewayConfig) *Gateway {
    proxy := &httputil.ReverseProxy{
        Director: func(req *http.Request) {
            // Match route based on request path
            for _, route := range config.Routes {
                if strings.HasPrefix(req.URL.Path, route.Path) {
                    req.URL.Scheme = route.Target.Scheme
                    req.URL.Host = route.Target.Host
                    req.URL.Path = strings.TrimPrefix(req.URL.Path, route.Path)
                    req.Header.Set("X-Forwarded-Host", req.Header.Get("Host"))
                    return
                }
            }
        },
        ErrorHandler: func(w http.ResponseWriter, r *http.Request, err error) {
            log.Printf("Proxy error: %v", err)
            http.Error(w, `{"error":"bad gateway"}`, http.StatusBadGateway)
        },
    }

    return &Gateway{config: config, proxy: proxy}
}

func (g *Gateway) Start() error {
    routesJSON, _ := json.MarshalIndent(g.config.Routes, "", "  ")
    log.Printf("Loading routes:\n%s\n", routesJSON)

    // Route distribution
    mux := http.NewServeMux()
    for _, route := range g.config.Routes {
        pattern := route.Path
        if !strings.HasSuffix(pattern, "/") {
            pattern += "/"
        }
        mux.Handle(pattern, g.proxy)
    }

    // Health check
    mux.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) {
        json.NewEncoder(w).Encode(map[string]string{"status": "ok"})
    })

    // Middleware chain
    handler := Chain(mux, g.config.Middlewares...)

    server := &http.Server{
        Addr:    ":" + g.config.Port,
        Handler: handler,
    }

    // Graceful shutdown
    go func() {
        sigCh := make(chan os.Signal, 1)
        signal.Notify(sigCh, os.Interrupt)
        <-sigCh
        log.Println("Shutting down server...")
        ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
        defer cancel()
        server.Shutdown(ctx)
    }()

    log.Printf("API gateway started on :%s", g.config.Port)
    return server.ListenAndServe()
}

func main() {
    userService, _ := url.Parse("http://localhost:8081")
    orderService, _ := url.Parse("http://localhost:8082")

    config := GatewayConfig{
        Port: "8080",
        Routes: []Route{
            {Path: "/api/users", Target: userService},
            {Path: "/api/orders", Target: orderService},
        },
    }

    config.Middlewares = []Middleware{
        Recovery,
        Logging,
        CORS,
    }

    gateway := NewGateway(config)
    if err := gateway.Start(); err != nil && err != http.ErrServerClosed {
        log.Fatal(err)
    }
}
🔥 Common Mistake: httputil.ReverseProxy modifies the X-Forwarded-For header by default. If you don't want this behavior, remove the header in Director. Additionally, ReverseProxy does not automatically handle WebSocket upgrades—the ServeHTTP طريقة of httputil.ReverseProxy does not support WebSockets, so you must handle the upgrade طلب separately.

❓ FAQ

Q How do I choose between http.Get and http.Client?
A Use http.Get for quick testing. In a production environment, always use a custom http.Client—set the Timeout to prevent requests from hanging, and configure Transport to control connection pooling and timeout details. http.Get uses http.DefaultClient, which has no default timeout.
Q How do I configure the Transport connection pool?
A The key parameters are MaxIdleConns and MaxIdleConnsPerHost. By default, MaxIdleConnsPerHost is set to 2, which is severely insufficient for high-concurrency services. It is recommended to set it to 50–100 (depending on concurrency). Set IdleConnTimeout to 30–90 seconds to keep connections active.
Q What is the execution order of وسيط?
A The onion model—requests enter from the outermost layer and travel through each layer to the Handler; responses travel back from the Handler through each layer to the outermost layer. The registration order determines the execution order: Chain(h, A, B, C) → Requests pass through A → B → C → Handler; responses pass through C → B → A.
Q What scenarios is httputil.ReverseProxy suitable for?
A API gateways, reverse proxies, and service routing. It automatically handles the Host header, the X-Forwarded-For header, and connection pool reuse. You only need to implement the Director دالة to modify the طلب's Scheme, Host, and Path. It is suitable as a traffic entry point for microservice architectures.
Q How do I implement طلب retries?
A Implement a custom RoundTripper at the Transport layer, or wrap the Do طريقة at the Client layer. Note the idempotency of retries—only safe methods such as GET, HEAD, and OPTIONS can be automatically retried. For POST/PUT requests, you must verify that the body can be re-read before retrying.
Q Is http.Client خيط-safe?
A Yes. http.Client can be used concurrently (multiple calls to Get or Do do not require additional locks). However, the Transport fields should not be modified after the first use. Best practice: Create one Client for each downstream service cluster; do not create a new Client for every طلب.
Q How do I set a header for a طلب?
A After creating a طلب using http.NewRequest, set the header as follows: req.Header.Set("Key", "Value"). Do not use req.Header.Add("Key", "Value")—Add appends rather than overwrites. Common headers such as Auth, TraceID, and Content-Type are all passed in the طلب headers.

📖 Summary

📝 Exercises

  1. Basic Exercise (Difficulty: ⭐): Write a custom http.Client with a 5-second timeout and MaxIdleConnsPerHost=20. Use this Client to send a GET طلب to https://httpbin.org/delay/3 and verify that the timeout works as expected.

  2. Advanced Problem (Difficulty ⭐⭐): Implement a طلب retry وسيط (RoundTripper layer). Requirements: (1) Automatically retry GET requests up to 3 times; (2) Set the retry intervals to 100 ms, 200 ms, and 400 ms, respectively (exponential backoff); (3) Retry only 5xx responses; do not retry 4xx responses; (4) Log each retry attempt.

  3. Challenge (Difficulty: ⭐⭐⭐): Implement a reverse proxy with load balancing. Requirements: (1) Support registration of multiple واجهة خلفية instances (e.g., 3 user-service instances); (2) Distribute requests using the Round-Robin algorithm; (3) Perform passive health checks—remove an مثيل after 3 consecutive failures and re-add it once it recovers; (4) Log the واجهة خلفية address to which each طلب is routed; (5) Implement using httputil.ReverseProxy.

Web-Tutorial.com

Web-Tutorial Tech Team

A team of developers maintaining programming tutorials. Each tutorial is written and reviewed by developers with expertise in that field. We work to keep our content accurate and reliable — if you spot an issue, please let us know.

100%

🙏 帮我们做得更好

我们是刚上线的编程教程站，几个人的小团队，精力有限。页面虽经检查，难免还有疏漏——链接失效、排版错乱、内容有误、语言生硬……

如果您发现了，麻烦告诉我们，我们会在收到反馈后第一时间进行修复，再次感谢您的光临 🙏