Go: Go Database Operations
Go's
قاعدة بيانات/sqlpackage provides a unified interface for SQL قاعدة بيانات operations—you can switch between MySQL, SQLite, and PostgreSQL simply by changing the driver.
When you need to work with databases securely—to prevent SQL injection, manage transactions, and configure connection pools—the قاعدة بيانات/sql standard library provides a complete and secure set of tools.
1. You will learn
قاعدة بيانات/sql: Registering drivers and opening connections- Three query methods:
db.Query,db.QueryRow, anddb.Exec db.Prepare: Prevent SQL injection using precompilation- Transaction:
Begin/Commit/Rollback - Connection pool configuration:
SetMaxOpenConns/SetMaxIdleConns - Using SQLite and MySQL Drivers
2. A True Story of a Backend Engineer
(1) Pain Point: String concatenation in SQL led to a قاعدة بيانات deletion via a hacker's injection attack
Bob is a واجهة خلفية engineer at an e-commerce platform. He needs to implement a product search API:
"I used
fmt.Sprintfto construct an SQL query for products:SELECT * FROM products WHERE name LIKE '%" + search + "%'. On the second day after launch, someone entered' OR 1=1; DROP TABLE products;--into the search box. My product table was gone. My boss asked, 'Where did the 100,000 products in the قاعدة بيانات go?'"
// Bad code: string concatenation SQL, fatal vulnerability
func searchProducts(w http.ResponseWriter, r *http.Request) {
search := r.URL.Query().Get("q")
// If search = "' OR 1=1; DROP TABLE products;--"
// Final SQL: SELECT * FROM products WHERE name LIKE '%' OR 1=1; DROP TABLE products;--%'
query := fmt.Sprintf("SELECT * FROM products WHERE name LIKE '%%%s%%'", search)
rows, err := db.Query(query) // Nightmare begins
}
(2) Go Solution: Precompiled PreparedStatement
// Good code: precompiled, parameters automatically escaped
func searchProducts(w http.ResponseWriter, r *http.Request) {
search := r.URL.Query().Get("q")
// ? is a placeholder; the قاعدة بيانات driver automatically escapes parameters
// The search value is always treated as a سلسلة, never as part of SQL syntax
stmt, err := db.Prepare("SELECT * FROM products WHERE name LIKE ?")
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
defer stmt.Close()
rows, err := stmt.Query("%" + search + "%")
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
defer rows.Close()
// Process results...
}
(3) Performance: String Concatenation vs. Precompiled SQL
| Dimension | String Concatenation | PreparedStatement |
|---|---|---|
| SQL Injection | ❌ High Risk | ✅ Automatic Escaping |
| Performance | SQL Parsing per Execution | ✅ Parsed only once; faster on subsequent executions |
| Readability | Confusing | ✅ Clear |
| Parameter Type | Convert All to Strings | ✅ Preserve Type |
3. Database Connection
▶ Example: Connecting to SQLite
⚙️ Prerequisite: Run
go get github.com/mattn/go-sqlite3⚠️ Note: go-sqlite3 requires CGO; Windows needs gcc (MinGW-w64), macOS/Linux includes it by default
package main
import (
"database/sql"
"fmt"
"log"
_ "github.com/mattn/go-sqlite3" // Import driver (_ means only init is executed, not directly used)
)
func main() {
// Open database (actual connection is established on first query)
db, err := sql.Open("sqlite3", "./test.db")
if err != nil {
log.Fatal(err)
}
defer db.Close()
// Test if connection is reachable
err = db.Ping()
if err != nil {
log.Fatal("Cannot connect to database:", err)
}
fmt.Println("Database connected successfully")
}
▶ Example: Connecting to MySQL
⚙️ Prerequisite: Run
go get github.com/go-sql-driver/mysql
package main
import (
"database/sql"
"fmt"
"log"
"time"
_ "github.com/go-sql-driver/mysql"
)
func main() {
// DSN format: user:password@tcp(host:port)/dbname?params
dsn := "root:password@tcp(127.0.0.1:3306)/shop?charset=utf8mb4&parseTime=true"
db, err := sql.Open("mysql", dsn)
if err != nil {
log.Fatal(err)
}
defer db.Close()
// Connection pool configuration
db.SetMaxOpenConns(25) // Maximum open connections
db.SetMaxIdleConns(10) // Maximum idle connections
db.SetConnMaxLifetime(5 * time.Minute) // Maximum connection lifetime
db.SetConnMaxIdleTime(2 * time.Minute) // Maximum idle connection lifetime
if err = db.Ping(); err != nil {
log.Fatal("Cannot connect to database:", err)
}
fmt.Println("MySQL connected successfully")
}
(3) Key Methods in database/sql
| Method | Purpose | Return |
|---|---|---|
sql.Open(driver, dsn) |
Opens the database (lazy loading) | *DB, error |
db.Ping() |
Tests whether the connection is actually reachable | error |
db.Close() |
Close the database | error |
db.Exec(sql, args...) |
Execute INSERT/UPDATE/DELETE | Result, error |
db.Query(sql, args...) |
Executes a SELECT statement and returns multiple rows | *Rows, error |
db.QueryRow(sql, args...) |
Executes a SELECT statement and returns a single row | *Row |
db.Prepare(sql) |
Precompile SQL | *Stmt, error |
sql.Open does not actually create a connection—it merely validates the DSN format. The actual connection is established during the first Ping or Query / Exec. Therefore, the fact that sql.Open returns nil error does not mean the database is accessible.
4. CRUD Operations
▶ Example: Creating a Table and Inserting Data
package main
import (
"قاعدة بيانات/sql"
"fmt"
"log"
_ "github.com/mattn/go-sqlite3"
)
func main() {
db, err := sql.Open("sqlite3", "./shop.db")
if err != nil {
log.Fatal(err)
}
defer db.Close()
// Create table
createSQL := `
CREATE TABLE IF NOT EXISTS products (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL,
price REAL NOT NULL,
stock INTEGER NOT NULL DEFAULT 0,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
)`
_, err = db.Exec(createSQL)
if err != nil {
log.Fatal(err)
}
fmt.Println("Table created successfully")
// Insert data (Exec returns Result)
result, err := db.Exec(
"INSERT INTO products (name, price, stock) VALUES (?, ?, ?)",
"Laptop", 999.99, 10,
)
if err != nil {
log.Fatal(err)
}
id, _ := result.LastInsertId()
affected, _ := result.RowsAffected()
fmt.Printf("Inserted: id=%d, rows affected=%d\n", id, affected)
}
▶ Example: Querying Data
package main
import (
"database/sql"
"fmt"
"log"
_ "github.com/mattn/go-sqlite3"
)
type Product struct {
ID int
Name string
Price float64
Stock int
}
func main() {
db, err := sql.Open("sqlite3", "./shop.db")
if err != nil {
log.Fatal(err)
}
defer db.Close()
// QueryRow: query a single row
var p Product
err = db.QueryRow("SELECT id, name, price, stock FROM products WHERE id = ?", 1).
Scan(&p.ID, &p.Name, &p.Price, &p.Stock)
if err == sql.ErrNoRows {
fmt.Println("Product not found")
} else if err != nil {
log.Fatal(err)
} else {
fmt.Printf("Product: %+v\n", p)
}
// Query: query multiple rows
rows, err := db.Query("SELECT id, name, price, stock FROM products WHERE price < ?", 500)
if err != nil {
log.Fatal(err)
}
defer rows.Close()
for rows.Next() {
var pr Product
err := rows.Scan(&pr.ID, &pr.Name, &pr.Price, &pr.Stock)
if err != nil {
log.Fatal(err)
}
fmt.Printf(" Product: %+v\n", pr)
}
// Check if iteration encountered an error
if err = rows.Err(); err != nil {
log.Fatal(err)
}
}
(3) Comparison of CRUD Methods
| Operation | Method | Return Value | Use Case |
|---|---|---|---|
| Create | db.Exec(INSERT...) |
Result (LastInsertId + RowsAffected) |
Insert/Update/Delete |
| Read Single Row | db.QueryRow(SELECT...).Scan() |
*Row + Auto-Close |
Single-Row Query |
| Read Multiple Rows | db.Query(SELECT...); rows.Scan() |
*Rows (requires iteration + closing) |
Multi-row query |
| Update | db.Exec(UPDATE...) |
Result (RowsAffected) |
Update data |
| Delete | db.Exec(DELETE...) |
Result (RowsAffected) |
Delete data |
rows.Close() must be called—even if you've already iterated through all rows in for rows.Next(). Failure to close will result in a connection leak (the connection won't be released back to the pool). Use defer rows.Close() to ensure it's released.
5. PreparedStatement
▶ Example: Precompiled Batch Insert
package main
import (
"database/sql"
"fmt"
"log"
_ "github.com/mattn/go-sqlite3"
)
type Product struct {
Name string
Price float64
Stock int
}
func main() {
db, err := sql.Open("sqlite3", "./shop.db")
if err != nil {
log.Fatal(err)
}
defer db.Close()
// Precompile SQL statement
stmt, err := db.Prepare("INSERT INTO products (name, price, stock) VALUES (?, ?, ?)")
if err != nil {
log.Fatal(err)
}
defer stmt.Close()
// Batch insert (SQL compiled only once)
products := []Product{
{"Mouse", 29.99, 100},
{"Keyboard", 79.99, 50},
{"Monitor", 299.99, 20},
{"USB-C Hub", 49.99, 200},
}
for _, p := range products {
result, err := stmt.Exec(p.Name, p.Price, p.Stock)
if err != nil {
log.Printf("Insert failed %s: %v", p.Name, err)
continue
}
id, _ := result.LastInsertId()
fmt.Printf("Inserted successfully: %s (id=%d)\n", p.Name, id)
}
}
(2) Precompiled vs. Concatenated SQL
| Comparison | Precompilation Prepare |
SQL Concatenation |
|---|---|---|
| SQL Injection | ✅ Automatic Parameter Escaping | ❌ High Risk |
| Performance (multiple executions) | ✅ Parse only once | ❌ Parse each time |
| Code Readability | Clear (parameters use ? as placeholders) | Confusing (nested quotes) |
| Type Safety | ✅ Preserve Parameter Types | ❌ Convert All to Strings |
| Applicable Scenarios | All user input | Static SQL only (table names/column names, not user input) |
6. Transactions
▶ Example: Transaction Transfer
package main
import (
"قاعدة بيانات/sql"
"fmt"
"log"
_ "github.com/mattn/go-sqlite3"
)
func transferFunds(db *sql.DB, fromID, toID int, amount float64) خطأ {
// Begin transaction
tx, err := db.Begin()
if err != nil {
return fmt.Errorf("failed to begin transaction: %w", err)
}
// Rollback on transaction failure
defer tx.Rollback() // If Commit succeeds, Rollback is a no-op
// 1. Deduct from fromID
result, err := tx.Exec(
"UPDATE accounts SET balance = balance - ? WHERE id = ? AND balance >= ?",
amount, fromID, amount,
)
if err != nil {
return fmt.Errorf("deduction failed: %w", err)
}
affected, _ := result.RowsAffected()
if affected == 0 {
return fmt.Errorf("insufficient balance or account does not exist")
}
// 2. Add to toID
result, err = tx.Exec(
"UPDATE accounts SET balance = balance + ? WHERE id = ?",
amount, toID,
)
if err != nil {
return fmt.Errorf("credit failed: %w", err)
}
affected, _ = result.RowsAffected()
if affected == 0 {
return fmt.Errorf("recipient account does not exist")
}
// Commit transaction
return tx.Commit()
}
func main() {
db, err := sql.Open("sqlite3", "./bank.db")
if err != nil {
log.Fatal(err)
}
defer db.Close()
// Initialize accounts
db.Exec("CREATE TABLE IF NOT EXISTS accounts (id INTEGER PRIMARY KEY, balance REAL)")
db.Exec("INSERT OR IGNORE INTO accounts VALUES (1, 1000.00)")
db.Exec("INSERT OR IGNORE INTO accounts VALUES (2, 500.00)")
// Transfer 200 from account 1 to 2
err = transferFunds(db, 1, 2, 200)
if err != nil {
log.Printf("Transfer failed: %v\n", err)
} else {
fmt.Println("Transfer successful!")
}
}
sequenceDiagram
participant App as Application
participant DB as Database
App->>DB: BEGIN
DB-->>App: OK
App->>DB: UPDATE SET balance = balance - ? WHERE id = 1
DB-->>App: 1 row affected
App->>DB: UPDATE SET balance = balance + ? WHERE id = 2
DB-->>App: 1 row affected
App->>DB: COMMIT
DB-->>App: OK (persisted)
Note over App,DB: If error midway → ROLLBACK<br/>All changes reverted
defer tx.Rollback() in a transaction is a safe pattern—if Commit succeeds, the Rollback call is safe (a no-op). If Commit fails, defer automatically rolls back the transaction. Never omit the Rollback!
7. Connection Pool Configuration
package main
import (
"database/sql"
"fmt"
"time"
_ "github.com/go-sql-driver/mysql"
)
func configurePool(db *sql.DB) {
// Maximum open connections (new requests queue when this limit is reached)
db.SetMaxOpenConns(25)
// Maximum idle connections (kept open but unused)
db.SetMaxIdleConns(10)
// Maximum connection lifetime (prevents long-running connections from being dropped by the database)
db.SetConnMaxLifetime(5 * time.Minute)
// Maximum idle connection lifetime
db.SetConnMaxIdleTime(2 * time.Minute)
}
func main() {
db, _ := sql.Open("mysql", "user:pass@/dbname")
configurePool(db)
fmt.Println("Connection pool configured")
}
(1) Connection Pool Parameters
| Parameter | Default Value | Recommended Value | Description |
|---|---|---|---|
SetMaxOpenConns |
0 (unlimited) | 25–100 | Maximum number of concurrent connections |
SetMaxIdleConns |
2 | 10–25 | Maximum number of idle connections (≤ MaxOpenConns) |
SetConnMaxLifetime |
0 (never expires) | 5–30 min | Maximum connection lifetime |
SetConnMaxIdleTime |
0 (never expires) | 2–5 min | Idle connection timeout |
MaxIdleConns cannot be greater than MaxOpenConns—the database/SQL will automatically adjust this. Also, do not set MaxOpenConns=0 (unlimited)—under high concurrency, this will create a large number of connections and overload the database. Always set a reasonable upper limit.
8. Complete Example: E-commerce Inventory Management
⚙️ Prerequisite: Run
go get github.com/mattn/go-sqlite3
// inventory.go
package main
import (
"قاعدة بيانات/sql"
"fmt"
"log"
"sync"
"time"
_ "github.com/mattn/go-sqlite3"
)
// ---------- Store ----------
type InventoryStore struct {
db *sql.DB
mu sync.RWMutex
}
func NewInventoryStore(dbPath سلسلة) (*InventoryStore, خطأ) {
db, err := sql.Open("sqlite3", dbPath)
if err != nil {
return nil, fmt.Errorf("failed to open قاعدة بيانات: %w", err)
}
// Connection pool configuration
db.SetMaxOpenConns(10)
db.SetMaxIdleConns(5)
db.SetConnMaxLifetime(5 * time.Minute)
store := &InventoryStore{db: db}
if err := store.initSchema(); err != nil {
return nil, fmt.Errorf("failed to initialize schema: %w", err)
}
return store, nil
}
func (s *InventoryStore) initSchema() خطأ {
schema := `
CREATE TABLE IF NOT EXISTS products (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL,
price REAL NOT NULL,
stock INTEGER NOT NULL DEFAULT 0,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS orders (
id INTEGER PRIMARY KEY AUTOINCREMENT,
product_id INTEGER NOT NULL,
quantity INTEGER NOT NULL,
total REAL NOT NULL,
حالة TEXT NOT NULL DEFAULT 'pending',
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (product_id) REFERENCES products(id)
);`
_, err := s.db.Exec(schema)
return err
}
// ---------- Product Operations ----------
func (s *InventoryStore) AddProduct(name سلسلة, price float64, stock int) (int64, خطأ) {
result, err := s.db.Exec(
"INSERT INTO products (name, price, stock) VALUES (?, ?, ?)",
name, price, stock,
)
if err != nil {
return 0, err
}
return result.LastInsertId()
}
func (s *InventoryStore) GetProduct(id int) (product, خطأ) {
var p product
err := s.db.QueryRow(
"SELECT id, name, price, stock, created_at FROM products WHERE id = ?", id,
).Scan(&p.ID, &p.Name, &p.Price, &p.Stock, &p.CreatedAt)
if err == sql.ErrNoRows {
return p, fmt.Errorf("product not found")
}
return p, err
}
type product struct {
ID int
Name سلسلة
Price float64
Stock int
CreatedAt time.Time
}
// ---------- Order Operations (transaction) ----------
type OrderRequest struct {
ProductID int
Quantity int
}
func (s *InventoryStore) PlaceOrder(req OrderRequest) (int64, خطأ) {
s.mu.Lock() // Prevent overselling: only one order deduction at a time
defer s.mu.Unlock()
tx, err := s.db.Begin()
if err != nil {
return 0, fmt.Errorf("failed to begin transaction: %w", err)
}
defer tx.Rollback() // Safe rollback
// 1. Query product and lock row (⚠️ SQLite does not support FOR UPDATE; use MySQL/PostgreSQL for true row locking)
var price float64
var stock int
err = tx.QueryRow(
"SELECT price, stock FROM products WHERE id = ? FOR UPDATE",
req.ProductID,
).Scan(&price, &stock)
if err == sql.ErrNoRows {
return 0, fmt.Errorf("product not found")
}
if err != nil {
return 0, err
}
// 2. Check stock
if stock < req.Quantity {
return 0, fmt.Errorf("insufficient stock: need %d, have %d", req.Quantity, stock)
}
// 3. Deduct stock
result, err := tx.Exec(
"UPDATE products SET stock = stock - ? WHERE id = ? AND stock >= ?",
req.Quantity, req.ProductID, req.Quantity,
)
if err != nil {
return 0, err
}
affected, _ := result.RowsAffected()
if affected == 0 {
return 0, fmt.Errorf("concurrent stock shortage")
}
// 4. Create order
total := price * float64(req.Quantity)
result, err = tx.Exec(
"INSERT INTO orders (product_id, quantity, total) VALUES (?, ?, ?)",
req.ProductID, req.Quantity, total,
)
if err != nil {
return 0, err
}
orderID, _ := result.LastInsertId()
// 5. Commit
if err := tx.Commit(); err != nil {
return 0, fmt.Errorf("failed to إيداع transaction: %w", err)
}
return orderID, nil
}
// ---------- Report ----------
func (s *InventoryStore) LowStockReport(threshold int) ([]product, خطأ) {
rows, err := s.db.Query(
"SELECT id, name, price, stock, created_at FROM products WHERE stock < ? ORDER BY stock ASC",
threshold,
)
if err != nil {
return nil, err
}
defer rows.Close()
var products []product
for rows.Next() {
var p product
if err := rows.Scan(&p.ID, &p.Name, &p.Price, &p.Stock, &p.CreatedAt); err != nil {
return nil, err
}
products = append(products, p)
}
return products, rows.Err()
}
// ---------- Main ----------
func main() {
store, err := NewInventoryStore("./inventory.db")
if err != nil {
log.Fatal(err)
}
// Add products
laptopID, _ := store.AddProduct("Laptop", 999.99, 10)
mouseID, _ := store.AddProduct("Mouse", 29.99, 100)
fmt.Printf("Products added: laptop=%d, mouse=%d\n", laptopID, mouseID)
// Place order (transaction-safe)
orderID, err := store.PlaceOrder(OrderRequest{ProductID: int(laptopID), Quantity: 2})
if err != nil {
log.Printf("Order failed: %v\n", err)
} else {
fmt.Printf("Order placed successfully: order=%d\n", orderID)
}
// Low stock report
lowStock, _ := store.LowStockReport(20)
fmt.Printf("Low stock products (%d items):\n", len(lowStock))
for _, p := range lowStock {
fmt.Printf(" %s: stock=%d\n", p.Name, p.Stock)
}
}
FOR UPDATE row locking—the clause is silently ignored. The code above only achieves true row locking in MySQL/PostgreSQL; in SQLite, safety is ensured via the mu.Lock() mutex. For cross-database compatibility, use conditional updates (UPDATE ... WHERE stock >= ?) instead of row locks.
❓ FAQ
database/sql + a query builder (such as squirrel) for large projects.' OR 1=1 is just a string and will not become a WHERE clause.tx.Rollback() to roll back all changes in the event of an error. defer tx.Rollback() + tx.Commit() is the standard pattern—if Commit succeeds, Rollback is a no-op; if Commit fails, the transaction is automatically rolled back. This ensures atomicity.MaxOpenConns to 2–3 times the concurrent connection count, and MaxIdleConns to MaxOpenConns/2. Set SetConnMaxLifetime to 5 minutes to prevent connections from being terminated by the database middleware.database/sql interface. Differences: (1) Placeholders: SQLite uses ?, MySQL uses ?, and PostgreSQL uses $1; (2) Differences in SQL syntax (such as the syntax for auto-incrementing primary keys); (3) SQLite's concurrency performance is not as good as MySQL's.rows.Close() after iterating through rows.Next()?rows still holds the database connection. rows.Close() releases the connection back to the pool. Use defer rows.Close() to register this immediately after rows is created, ensuring that resources are released in any scenario.db.SetMaxOpenConns and db.SetMaxIdleConns at the database/sql layer to ensure the connection pool does not become a bottleneck; (2) Use the database's built-in slow query log; (3) Use EXPLAIN ANALYZE to analyze the query plan; (4) Consider adding SQL logging middleware to record the execution time of all queries.📖 Summary
sql.Openopens a connection (lazy loading) +db.Pingverifies connectivity- The three types of queries:
db.Exec,db.Query, anddb.QueryRow Prepare: Prevent SQL injection through precompilation- Transaction:
Begin→Commit/Rollback defer tx.Rollback()Safe Mode- Connection pool:
SetMaxOpenConns/SetMaxIdleConns/SetConnMaxLifetime - SQLite uses
?, MySQL uses?, and PostgreSQL uses$1
📝 Exercises
-
Basic Problem (Difficulty ⭐): Create an SQLite database and create a
taskstable (id, title, done, created_at). Implement three functions: InsertTask, ListTasks (sorted by date), and MarkDone (update by id). -
Advanced Problem (Difficulty ⭐⭐): Implement the database layer for a blog system. Requirements: (1)
poststable (id,title,content,author_id,created_at); (2) Support paginated queries (LIMIT/OFFSET); (3) Use prepared statements for inserts to prevent injection attacks; (4) Transactions: Update the author's post count when a post is published; (5) Use-raceto verify concurrency safety. -
Challenge (Difficulty: ⭐⭐⭐): Implement concurrency-safe operations for an inventory management system. Requirements: (1) 100 goroutines place orders simultaneously (each deducting stock from a different item); (2) Use database/sql transactions + application-layer mutexes to prevent overselling; (3) Precompile all SQL statements; (4) Configure the connection pool appropriately; (5) Count the number of successful and failed orders and calculate the final inventory.