JSON Processing

Lesson 21: JSON Processing

Life Analogy

Imagine you are a translator. JSON is the most widely used "universal language" in the world. When a Go program needs to communicate with other systems (frontend, API, database), you need to:

Encode (Marshal): "Translate" Go structs into JSON format to send out
Decode (Unmarshal): "Translate" received JSON back into Go structs for use

Just as a translator needs to understand the rules and idioms of both languages, Go's encoding/json package is your powerful tool for handling JSON.

Core Concepts

Concept	Description
Serialization (Marshal)	Convert Go data structures to JSON byte slices
Deserialization (Unmarshal)	Parse JSON byte slices into Go data structures
Struct Tag	Metadata that controls JSON field names and behavior
Streaming	Use Decoder/Encoder for large data or network streams
Custom Serialization	Implement Marshaler/Unmarshaler interfaces for custom conversion logic

Basic Syntax and Usage

1. Import the Package

import "encoding/json"

2. Serialization: Struct → JSON

// Define struct
type User struct {
    Name  string
    Age   int
    Email string
}

user := User{Name: "Alice", Age: 28, Email: "alice@example.com"}

// Serialize
data, err := json.Marshal(user)
if err != nil {
    log.Fatal(err)
}
fmt.Println(string(data))
// Output: {"Name":"Alice","Age":28,"Email":"alice@example.com"}

💡 Tip: json.Marshal returns []byte, which needs string() conversion to print readable JSON.

3. Deserialization: JSON → Struct

jsonStr := `{"Name":"Bob","Age":32,"Email":"bob@example.com"}`

var user User
err := json.Unmarshal([]byte(jsonStr), &user)
if err != nil {
    log.Fatal(err)
}
fmt.Printf("Name: %s, Age: %d\n", user.Name, user.Age)
// Output: Name: Bob, Age: 32

💡 Tip: The second parameter of Unmarshal must be a pointer, otherwise changes won't take effect.

4. Struct Tags

type Product struct {
    ID    int     `json:"id"`           // Specify JSON field name
    Name  string  `json:"name"`         // Lowercase naming is more JSON-friendly
    Price float64 `json:"price"`
    Desc  string  `json:"description,omitempty"` // Omit when empty
    internal string `json:"-"`          // Completely ignore this field
}

💡 Tip:

omitempty: When the field is a zero value, it will be omitted from JSON output

-: This field will never appear in JSON

Tag names take priority over field names

5. Common Type Mappings

Go Type	JSON Type
`string`	string
`int`, `float64`	number
`bool`	boolean
`nil`	null
`[]T`	array
`map[string]T`	object
`struct`	object

Examples

Example: Basic JSON Serialization and Deserialization (Difficulty ⭐)

package main

import (
    "encoding/json"
    "fmt"
    "log"
)

// Book struct
type Book struct {
    Title    string   `json:"title"`
    Author   string   `json:"author"`
    Pages    int      `json:"pages"`
    Tags     []string `json:"tags"`
    InStock  bool     `json:"in_stock"`
}

func main() {
    // === Serialization ===
    book := Book{
        Title:   "Go in Action",
        Author:  "John Smith",
        Pages:   350,
        Tags:    []string{"Programming", "Go", "Backend"},
        InStock: true,
    }

    // Pretty print (with indentation)
    jsonData, err := json.MarshalIndent(book, "", "  ")
    if err != nil {
        log.Fatal("Serialization failed:", err)
    }
    fmt.Println("=== Serialization Result ===")
    fmt.Println(string(jsonData))

    // === Deserialization ===
    jsonStr := `{
        "title": "Mastering Go",
        "author": "Jane Doe",
        "pages": 480,
        "tags": ["Go", "Advanced", "Concurrency"],
        "in_stock": false
    }`

    var newBook Book
    err = json.Unmarshal([]byte(jsonStr), &newBook)
    if err != nil {
        log.Fatal("Deserialization failed:", err)
    }
    fmt.Println("\n=== Deserialization Result ===")
    fmt.Printf("Title: %s\n", newBook.Title)
    fmt.Printf("Author: %s\n", newBook.Author)
    fmt.Printf("Tags: %v\n", newBook.Tags)
    fmt.Printf("In Stock: %v\n", newBook.InStock)
}

▶ Try it Yourself

Output:

TEXT

=== Serialization Result ===
{
  "title": "Go in Action",
  "author": "John Smith",
  "pages": 350,
  "tags": [
    "Programming",
    "Go",
    "Backend"
  ],
  "in_stock": true
}

=== Deserialization Result ===
Title: Mastering Go
Author: Jane Doe
Tags: [Go Advanced Concurrency]
In Stock: false

Example: Nested JSON and Map Handling (Difficulty ⭐⭐)

package main

import (
    "encoding/json"
    "fmt"
    "log"
)

// Address struct
type Address struct {
    City    string `json:"city"`
    Street  string `json:"street"`
    ZipCode string `json:"zip_code"`
}

// Contact information
type Contact struct {
    Phone string `json:"phone"`
    Email string `json:"email"`
}

// Employee struct (with nesting)
type Employee struct {
    Name      string            `json:"name"`
    Age       int               `json:"age"`
    Address   Address           `json:"address"`     // Nested struct
    Contact   Contact           `json:"contact"`     // Nested struct
    Skills    []string          `json:"skills"`      // Slice
    Metadata  map[string]string `json:"metadata"`    // Dynamic fields
}

func main() {
    // Construct nested data
    emp := Employee{
        Name: "Alice",
        Age:  35,
        Address: Address{
            City:    "Beijing",
            Street:  "88 Jianguo Road, Chaoyang District",
            ZipCode: "100022",
        },
        Contact: Contact{
            Phone: "13800138000",
            Email: "alice@example.com",
        },
        Skills: []string{"Go", "Python", "Docker"},
        Metadata: map[string]string{
            "department": "Engineering",
            "level":      "P7",
            "joined":     "2020-03-15",
        },
    }

    // Serialize
    data, err := json.MarshalIndent(emp, "", "  ")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println("=== Nested JSON Serialization ===")
    fmt.Println(string(data))

    // Handle dynamic JSON (using map)
    dynamicJSON := `{
        "event": "user_login",
        "timestamp": 1700000000,
        "data": {
            "user_id": 12345,
            "ip": "192.168.1.100",
            "browser": "Chrome"
        },
        "tags": ["web", "auth"]
    }`

    var result map[string]interface{}
    err = json.Unmarshal([]byte(dynamicJSON), &result)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println("\n=== Dynamic JSON Parsing ===")
    fmt.Printf("Event: %s\n", result["event"])
    fmt.Printf("Timestamp: %.0f\n", result["timestamp"])

    // Access nested map
    if data, ok := result["data"].(map[string]interface{}); ok {
        fmt.Printf("User ID: %.0f\n", data["user_id"])
        fmt.Printf("IP Address: %s\n", data["ip"])
    }

    // Access array
    if tags, ok := result["tags"].([]interface{}); ok {
        fmt.Print("Tags: ")
        for _, tag := range tags {
            fmt.Printf("%s ", tag)
        }
        fmt.Println()
    }
}

▶ Try it Yourself

Output:

TEXT

=== Nested JSON Serialization ===
{
  "name": "Alice",
  "age": 35,
  "address": {
    "city": "Beijing",
    "street": "88 Jianguo Road, Chaoyang District",
    "zip_code": "100022"
  },
  "contact": {
    "phone": "13800138000",
    "email": "alice@example.com"
  },
  "skills": [
    "Go",
    "Python",
    "Docker"
  ],
  "metadata": {
    "department": "Engineering",
    "joined": "2020-03-15",
    "level": "P7"
  }
}

=== Dynamic JSON Parsing ===
Event: user_login
Timestamp: 1700000000
User ID: 12345
IP Address: 192.168.1.100
Tags: web auth

Example: Custom Serialization and Streaming (Difficulty ⭐⭐⭐)

package main

import (
    "encoding/json"
    "fmt"
    "log"
    "strings"
    "time"
)

// CustomTime custom time type
type CustomTime struct {
    time.Time
}

// Implement json.Marshaler interface
func (ct CustomTime) MarshalJSON() ([]byte, error) {
    // Output format: 2006-01-02 15:04:05
    formatted := ct.Format("2006-01-02 15:04:05")
    return json.Marshal(formatted)
}

// Implement json.Unmarshaler interface
func (ct *CustomTime) UnmarshalJSON(data []byte) error {
    var s string
    if err := json.Unmarshal(data, &s); err != nil {
        return err
    }
    // Support multiple format parsing
    formats := []string{
        "2006-01-02 15:04:05",
        "2006-01-02T15:04:05",
        "2006/01/02",
    }
    for _, format := range formats {
        t, err := time.Parse(format, s)
        if err == nil {
            ct.Time = t
            return nil
        }
    }
    return fmt.Errorf("unable to parse time: %s", s)
}

// Status custom enum type
type Status int

const (
    StatusActive   Status = iota // 0
    StatusInactive               // 1
    StatusBanned                 // 2
)

// Status to string mapping
var statusNames = map[Status]string{
    StatusActive:   "active",
    StatusInactive: "inactive",
    StatusBanned:   "banned",
}

// String to status mapping
var statusValues = map[string]Status{
    "active":   StatusActive,
    "inactive": StatusInactive,
    "banned":   StatusBanned,
}

// MarshalJSON custom serialization
func (s Status) MarshalJSON() ([]byte, error) {
    name, ok := statusNames[s]
    if !ok {
        return json.Marshal("unknown")
    }
    return json.Marshal(name)
}

// UnmarshalJSON custom deserialization
func (s *Status) UnmarshalJSON(data []byte) error {
    var name string
    if err := json.Unmarshal(data, &name); err != nil {
        return err
    }
    val, ok := statusValues[name]
    if !ok {
        return fmt.Errorf("unknown status: %s", name)
    }
    *s = val
    return nil
}

// EventLog event log entry
type EventLog struct {
    Event     string     `json:"event"`
    Timestamp CustomTime `json:"timestamp"`
    Status    Status     `json:"status"`
    Details   string     `json:"details,omitempty"`
}

func main() {
    // === Custom serialization demo ===
    logEntry := EventLog{
        Event:     "user_register",
        Timestamp: CustomTime{time.Date(2024, 1, 15, 14, 30, 0, 0, time.Local)},
        Status:    StatusActive,
        Details:   "New user registered successfully",
    }

    data, _ := json.MarshalIndent(logEntry, "", "  ")
    fmt.Println("=== Custom Serialization ===")
    fmt.Println(string(data))

    // === Custom deserialization demo ===
    jsonStr := `{
        "event": "user_login",
        "timestamp": "2024/01/15",
        "status": "inactive"
    }`

    var entry EventLog
    err := json.Unmarshal([]byte(jsonStr), &entry)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("\nParse result: Event=%s, Time=%s, Status=%d\n",
        entry.Event,
        entry.Timestamp.Format("2006-01-02 15:04:05"),
        entry.Status,
    )

    // === Streaming demo ===
    fmt.Println("\n=== Streaming Decoder ===")
    // Simulate JSON stream received from network
    jsonStream := `[
        {"name": "Alice", "score": 95},
        {"name": "Bob", "score": 87},
        {"name": "Charlie", "score": 92}
    ]`

    decoder := json.NewDecoder(strings.NewReader(jsonStream))

    // Read start token
    token, err := decoder.Token()
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("Start token: %v\n", token)

    // Read array elements one by one
    type Student struct {
        Name  string `json:"name"`
        Score int    `json:"score"`
    }

    var students []Student
    for decoder.More() {
        var s Student
        if err := decoder.Decode(&s); err != nil {
            log.Fatal(err)
        }
        students = append(students, s)
    }

    for _, s := range students {
        fmt.Printf("Student: %s, Score: %d\n", s.Name, s.Score)
    }

    // === Streaming Encoder demo ===
    fmt.Println("\n=== Streaming Encoder ===")
    var buf strings.Builder
    encoder := json.NewEncoder(&buf)
    encoder.SetIndent("", "  ")

    // Encode individual objects
    for _, s := range students {
        if err := encoder.Encode(s); err != nil {
            log.Fatal(err)
        }
    }
    fmt.Println(buf.String())
}

▶ Try it Yourself

Output:

TEXT

=== Custom Serialization ===
{
  "event": "user_register",
  "timestamp": "2024-01-15 14:30:00",
  "status": "active",
  "details": "New user registered successfully"
}

Parse result: Event=user_login, Time=2024-01-15 00:00:00, Status=1

=== Streaming Decoder ===
Start token: [
Student: Alice, Score: 95
Student: Bob, Score: 87
Student: Charlie, Score: 92

=== Streaming Encoder ===
{
  "name": "Alice",
  "score": 95
}
{
  "name": "Bob",
  "score": 87
}
{
  "name": "Charlie",
  "score": 92
}

Application Scenarios

Scenario 1: API Response Wrapper

package main

import (
    "encoding/json"
    "fmt"
    "log"
    "net/http"
)

// APIResponse unified API response structure
type APIResponse struct {
    Code    int         `json:"code"`
    Message string      `json:"message"`
    Data    interface{} `json:"data,omitempty"`
    Error   string      `json:"error,omitempty"`
}

// SuccessResponse success response
func SuccessResponse(w http.ResponseWriter, data interface{}) {
    resp := APIResponse{
        Code:    200,
        Message: "success",
        Data:    data,
    }
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    json.NewEncoder(w).Encode(resp)
}

// ErrorResponse error response
func ErrorResponse(w http.ResponseWriter, statusCode int, errMsg string) {
    resp := APIResponse{
        Code:    statusCode,
        Message: "error",
        Error:   errMsg,
    }
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(statusCode)
    json.NewEncoder(w).Encode(resp)
}

// UserHandler handles user requests
func UserHandler(w http.ResponseWriter, r *http.Request) {
    // Simulated user data
    users := []map[string]interface{}{
        {"id": 1, "name": "Alice", "role": "admin"},
        {"id": 2, "name": "Bob", "role": "user"},
        {"id": 3, "name": "Charlie", "role": "user"},
    }

    SuccessResponse(w, users)
}

func main() {
    // Simulated API response
    fmt.Println("=== Simulated API Response ===")

    // Success response
    successResp := APIResponse{
        Code:    200,
        Message: "success",
        Data: map[string]interface{}{
            "id":   1,
            "name": "Alice",
        },
    }
    data, _ := json.MarshalIndent(successResp, "", "  ")
    fmt.Println("Success response:")
    fmt.Println(string(data))

    // Error response
    errorResp := APIResponse{
        Code:    404,
        Message: "error",
        Error:   "User not found",
    }
    data, _ = json.MarshalIndent(errorResp, "", "  ")
    fmt.Println("\nError response:")
    fmt.Println(string(data))

    _ = log.Fatal // Avoid unused warning
}

Output:

TEXT

=== Simulated API Response ===
Success response:
{
  "code": 200,
  "message": "success",
  "data": {
    "id": 1,
    "name": "Alice"
  }
}

Error response:
{
  "code": 404,
  "message": "error",
  "error": "User not found"
}

Scenario 2: Configuration File Reading and Validation

package main

import (
    "encoding/json"
    "fmt"
    "log"
    "os"
)

// DatabaseConfig database configuration
type DatabaseConfig struct {
    Host     string `json:"host"`
    Port     int    `json:"port"`
    Username string `json:"username"`
    Password string `json:"password"`
    DBName   string `json:"dbname"`
}

// ServerConfig server configuration
type ServerConfig struct {
    Host         string   `json:"host"`
    Port         int      `json:"port"`
    ReadTimeout  int      `json:"read_timeout"`
    WriteTimeout int      `json:"write_timeout"`
    AllowOrigins []string `json:"allow_origins"`
}

// AppConfig application configuration
type AppConfig struct {
    AppName  string         `json:"app_name"`
    Debug    bool           `json:"debug"`
    Server   ServerConfig   `json:"server"`
    Database DatabaseConfig `json:"database"`
}

// Validate validates the configuration
func (c *AppConfig) Validate() error {
    if c.AppName == "" {
        return fmt.Errorf("app_name cannot be empty")
    }
    if c.Server.Port <= 0 || c.Server.Port > 65535 {
        return fmt.Errorf("server.port must be between 1-65535")
    }
    if c.Database.Host == "" {
        return fmt.Errorf("database.host cannot be empty")
    }
    return nil
}

func main() {
    // Simulated configuration file content
    configJSON := `{
        "app_name": "GoWebApp",
        "debug": true,
        "server": {
            "host": "0.0.0.0",
            "port": 8080,
            "read_timeout": 30,
            "write_timeout": 30,
            "allow_origins": ["http://localhost:3000", "https://example.com"]
        },
        "database": {
            "host": "localhost",
            "port": 3306,
            "username": "root",
            "password": "secret123",
            "dbname": "myapp"
        }
    }`

    // Parse configuration
    var config AppConfig
    err := json.Unmarshal([]byte(configJSON), &config)
    if err != nil {
        log.Fatalf("Failed to parse config: %v", err)
    }

    // Validate configuration
    if err := config.Validate(); err != nil {
        log.Fatalf("Config validation failed: %v", err)
    }

    // Print configuration info
    fmt.Printf("App Name: %s\n", config.AppName)
    fmt.Printf("Debug Mode: %v\n", config.Debug)
    fmt.Printf("Server Address: %s:%d\n", config.Server.Host, config.Server.Port)
    fmt.Printf("Database Connection: %s:%d/%s\n",
        config.Database.Host,
        config.Database.Port,
        config.Database.DBName,
    )
    fmt.Printf("Allowed Origins: %v\n", config.Server.AllowOrigins)

    // Write example (save modified config)
    config.Debug = false
    config.Server.Port = 9090

    output, err := json.MarshalIndent(config, "", "  ")
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println("\n=== Modified Configuration ===")
    fmt.Println(string(output))

    // In a real project, you would write to file:
    // os.WriteFile("config.json", output, 0644)
    _ = os.WriteFile // Avoid unused warning
}

Output:

TEXT

App Name: GoWebApp
Debug Mode: true
Server Address: 0.0.0.0:8080
Database Connection: localhost:3306/myapp
Allowed Origins: [http://localhost:3000 https://example.com]

=== Modified Configuration ===
{
  "app_name": "GoWebApp",
  "debug": false,
  "server": {
    "host": "0.0.0.0",
    "port": 9090,
    "read_timeout": 30,
    "write_timeout": 30,
    "allow_origins": [
      "http://localhost:3000",
      "https://example.com"
    ]
  },
  "database": {
    "host": "localhost",
    "port": 3306,
    "username": "root",
    "password": "secret123",
    "dbname": "myapp"
  }
}

❓ FAQ

Q1: Why are JSON field names uppercase?

Reason: Go only exports fields with an uppercase first letter, and json.Marshal uses the field name as the JSON key by default.

Solution: Use struct tags to specify lowercase names:

type User struct {
    Name  string `json:"name"`   // "name" in JSON
    Age   int    `json:"age"`    // "age" in JSON
    Email string `json:"email"`  // "email" in JSON
}

Q2: How to ignore empty value fields?

Use the omitempty tag:

type Request struct {
    Name  string `json:"name"`
    Email string `json:"email,omitempty"` // Omit when empty string
    Age   int    `json:"age,omitempty"`   // Omit when 0
    Items []string `json:"items,omitempty"` // Omit when nil or empty slice
}

// Test
req := Request{Name: "Alice"}
data, _ := json.Marshal(req)
fmt.Println(string(data))
// Output: {"name":"Alice"} — email, age, items are all omitted

Q3: How to handle number precision issues in JSON?

Go's json.Unmarshal parses JSON numbers as float64 by default, which loses precision for large integers:

// Problem example
var result map[string]interface{}
json.Unmarshal([]byte(`{"id": 12345678901234567}`), &result)
fmt.Printf("%.0f\n", result["id"]) // Output: 12345678901234568 (precision lost!)

// Solution: use json.Number
decoder := json.NewDecoder(strings.NewReader(`{"id": 12345678901234567}`))
decoder.UseNumber()
decoder.Decode(&result)

id, _ := result["id"].(json.Number).Int64()
fmt.Println(id) // Output: 12345678901234567 (correct)

Q4: How to handle JSON with unknown structure?

Use map[string]interface{} or json.RawMessage:

// Method 1: Use map
var data map[string]interface{}
json.Unmarshal(jsonBytes, &data)

// Method 2: Use json.RawMessage for deferred parsing
type Message struct {
    Type    string          `json:"type"`
    Payload json.RawMessage `json:"payload"` // Deferred parsing
}

// Determine how to parse Payload based on Type field
switch msg.Type {
case "user":
    var user User
    json.Unmarshal(msg.Payload, &user)
case "order":
    var order Order
    json.Unmarshal(msg.Payload, &order)
}

📖 Summary

This lesson covered the core content of Go JSON processing:

Basic Operations: json.Marshal for serialization and json.Unmarshal for deserialization
Struct Tags: Use json:"name" to control field names, omitempty to omit empty values, - to ignore fields
Nested Handling: Struct nesting, map[string]interface{} for dynamic JSON
Custom Serialization: Implement Marshaler/Unmarshaler interfaces
Streaming: json.Decoder and json.Encoder for stream data
Practical Applications: API response wrappers, configuration file management

💡 Key Points:

Always check error handling

Pass pointers for deserialization

Use struct tags to maintain JSON naming conventions

Use streaming for large data volumes

📝 Exercises

Exercise 1: Basic Practice

Write a program that defines a Student struct (with name, age, and a list of grades), and implements:

Create 3 student objects
Serialize to a JSON array with pretty printing
Deserialize back to structs and print the information

Exercise 2: Intermediate Practice

Implement a simple JSON configuration manager:

Define an application configuration struct (containing server, database, logging, etc.)
Implement a LoadConfig(filename) function to read config from file
Implement a SaveConfig(filename, config) function to save config to file
Implement configuration validation

Exercise 3: Advanced Practice

Implement a JSON-RPC message handler:

Define request and response structures
Use json.RawMessage for deferred parsing
Dispatch to different handler functions based on the request method name
Support batch request processing

// Hint: JSON-RPC request format
type RPCRequest struct {
    JSONRPC string          `json:"jsonrpc"`
    Method  string          `json:"method"`
    Params  json.RawMessage `json:"params"`
    ID      interface{}     `json:"id"`
}

type RPCResponse struct {
    JSONRPC string      `json:"jsonrpc"`
    Result  interface{} `json:"result,omitempty"`
    Error   *RPCError   `json:"error,omitempty"`
    ID      interface{} `json:"id"`
}

Next Lesson

After completing this lesson, please continue to Lesson 22: HTTP Services, where we will learn how to use Go to build HTTP servers and clients.

Web-Tutorial Tech Team

A team of developers maintaining programming tutorials. Each tutorial is written and reviewed by developers with expertise in that field. We work to keep our content accurate and reliable — if you spot an issue, please let us know.

About Us

JSON Processing

Lesson 21: JSON Processing

Life Analogy

Core Concepts

Basic Syntax and Usage

1. Import the Package

2. Serialization: Struct → JSON

3. Deserialization: JSON → Struct

4. Struct Tags

5. Common Type Mappings

Examples

Example: Basic JSON Serialization and Deserialization (Difficulty ⭐)

Example: Nested JSON and Map Handling (Difficulty ⭐⭐)

Example: Custom Serialization and Streaming (Difficulty ⭐⭐⭐)

Application Scenarios

Scenario 1: API Response Wrapper

Scenario 2: Configuration File Reading and Validation

❓ FAQ

Q1: Why are JSON field names uppercase?

Q2: How to ignore empty value fields?

Q3: How to handle number precision issues in JSON?

Q4: How to handle JSON with unknown structure?

📖 Summary

📝 Exercises

Exercise 1: Basic Practice

Exercise 2: Intermediate Practice

Exercise 3: Advanced Practice

Next Lesson

🙏 帮我们做得更好