Regular Expressions
A regular expression (regex) is a special language for describing text patterns. One line of regex can do what dozens of lines of manual code would take to match and extract text. It's everywhere in data cleaning, log analysis, and form validation.
1. What Is a Regular Expression
A regular expression uses special symbols to describe "what kind of string I'm looking for."
PYTHON
import re
# Simplest regex: directly match a string
pattern = r"hello"
text = "hello world"
result = re.search(pattern, text)
if result:
print("Found!") # Found!
print(result.group()) # hello
print(result.start()) # 0 (start position)
print(result.end()) # 5 (end position)
⚠️ Always prefix regex strings with
r (raw string). r"\n" is backslash + letter n, not a newline. Regex uses backslashes extensively; without r, you'd have to write \\n, which is painful.
2. Common Methods
PYTHON
import re
text = "My email is alice@example.com, also bob@test.com"
# search() — find the first match
result = re.search(r"\w+@\w+\.\w+", text)
print(result.group()) # alice@example.com
# findall() — find all matches
emails = re.findall(r"\w+@\w+\.\w+", text)
print(emails) # ['alice@example.com', 'bob@test.com']
# match() — match from the beginning
print(re.match(r"My", text)) # Match (at start)
print(re.match(r"email", text)) # None (not at start)
# sub() — replace
masked = re.sub(r"\w+@", "***@", text)
print(masked) # My email is *@example.com, also *@test.com
3. Metacharacters Quick Reference
| Metacharacter | Meaning | Example | Matches |
|---|---|---|---|
. |
Any single char (except newline) | h.t |
hat, hot, hit |
\d |
Digit | \d{3} |
123, 456 |
\w |
Letter/digit/underscore | \w+ |
hello, abc123 |
\s |
Whitespace | \s |
space, tab, newline |
* |
0 or more of previous | ab*c |
ac, abc, abbc |
+ |
1 or more of previous | ab+c |
abc, abbc (not ac) |
? |
0 or 1 of previous | colou?r |
color, colour |
{n} |
Exactly n times | \d{4} |
2026, 1990 |
{n,m} |
n to m times | \d{2,4} |
23, 456, 2026 |
^ |
Start of string | ^Hello |
Hello... |
$ |
End of string | end$ |
...end |
[] |
Character set | [aeiou] |
Any vowel |
| ` | ` | OR | `cat |
Example: Phone Number Validation (Difficulty ⭐⭐)
PYTHON
import re
def is_valid_phone(phone):
"""Validate a Chinese mobile phone number (11 digits, starts with 1)"""
pattern = r"^1[3-9]\d{9}$"
return bool(re.match(pattern, phone))
phones = ["13800138000", "12345678901", "010-12345678", "1380013800a"]
for p in phones:
print(f"{p}: {'✅' if is_valid_phone(p) else '❌'}")
# Extract all phone numbers from text
text = "Contact: 13800138000, backup: 13912345678, landline: 010-12345678"
phones = re.findall(r"1[3-9]\d{9}", text)
print(f"Found phones: {phones}")
4. Group Capture
Use parentheses () to split matched content into multiple parts:
PYTHON
import re
# Extract username and domain from an email
email = "alice@example.com"
pattern = r"(\w+)@(\w+\.\w+)"
result = re.search(pattern, email)
if result:
print(f"Full match: {result.group(0)}") # alice@example.com
print(f"Username: {result.group(1)}") # alice
print(f"Domain: {result.group(2)}") # example.com
# Extract info from a log
log = "2026-06-23 15:30:45 ERROR Database connection timeout"
pattern = r"(\d{4}-\d{2}-\d{2}) (\d{2}:\d{2}:\d{2}) (\w+) (.+)"
result = re.search(pattern, log)
print(f"Date: {result.group(1)}") # 2026-06-23
print(f"Time: {result.group(2)}") # 15:30:45
print(f"Level: {result.group(3)}") # ERROR
print(f"Message: {result.group(4)}") # Database connection timeout
Example: URL Parser (Difficulty ⭐⭐⭐)
PYTHON
import re
def parse_url(url):
"""Parse a URL to extract protocol, domain, path, and query parameters"""
pattern = r"(https?)://([^/]+)(/[^?]*)?(?:\?([^#]*))?"
result = re.search(pattern, url)
if not result:
return None
return {
"protocol": result.group(1),
"domain": result.group(2),
"path": result.group(3) or "/",
"query": result.group(4) or ""
}
urls = [
"https://www.example.com/path/page.html?id=123&name=test",
"http://localhost:8080/api/users",
"https://google.com"
]
for url in urls:
info = parse_url(url)
if info:
print(f"\nURL: {url}")
for key, value in info.items():
print(f" {key}: {value}")
5. Greedy vs Lazy Matching
By default, * and + are greedy — they match as much as possible. Adding ? makes them lazy — they match as little as possible:
PYTHON
import re
text = "<h1>Title</h1><p>Paragraph</p>"
# Greedy mode — matches as much as possible
greedy = re.search(r"<.+>", text)
print(greedy.group()) # <h1>Title</h1><p>Paragraph</p>
# Lazy mode — matches as little as possible (add ?)
lazy = re.search(r"<.+?>", text)
print(lazy.group()) # <h1>
💡
.*? (lazy) vs .* (greedy): When working with structured content like HTML or JSON, lazy mode is more common. Greedy mode often "oversteps" to match content it shouldn't. If unsure, start with lazy.
Common Use Cases
- Form validation: Validate email format, phone numbers, ID numbers.
- Data extraction: Extract specific patterns from web pages, logs, and text.
- Text replacement: Batch replace content matching a pattern (e.g., masking phone numbers).
- Syntax highlighting: Editors use regex to identify keywords and strings.
- Log analysis: Filter and process specific patterns from massive log files.
❓ FAQ
Q What does the
r prefix in regex mean?A
r indicates a raw string. In a normal string, \\n is a newline; in a raw string, \\n is just backslash + letter n. Regex uses backslashes extensively (like \d, \w); without r, you'd have to write \\\\d, which is very painful.
Q: How performant are regular expressions? A: Fast enough for most cases. But poorly written patterns (like nested quantifiers (.*)*) can cause "catastrophic backtracking" — taking minutes for simple text. Solutions: avoid nested quantifiers, use lazy mode, and use re.compile() for frequently used patterns.
Q: Complex regex is hard to read. Any tips? A: ① Add comments — re.VERBOSE mode allows spaces and comments in regex. ② Break it into multiple simple patterns. ③ Use online tools like regex101.com for debugging. Regex is the classic "fun to write, painful to read" — don't try to solve everything with one regex.📖 Summary
re.search()finds the first match;findall()finds all;sub()replaces- Metacharacters:
\ddigit,\wword char,\swhitespace,.any char - Quantifiers:
*any,+at least 1,?0 or 1,{n}exactly n - Groups
()extract parts;group(1)gets the first group - Greedy matches as much as possible; add
?for lazy (as little as possible) - Always prefix regex with
rto avoid backslash hell
📝 Exercises
-
Basic (Difficulty ⭐): Write a function
is_valid_email(email)that uses regex to validate an email address (contains@, non-empty on both sides, has a domain suffix). -
Intermediate (Difficulty ⭐⭐): Write a function
extract_numbers(text)that extracts all numbers (including integers and decimals) from a string and returns their sum. For example,"Price is 19.99, shipping 5, coupon -10"→ returns 14.99. -
Challenge (Difficulty ⭐⭐⭐): Write a "simple template engine." Given the template
"Hello, {name}! Your order {order_id} has been shipped, arriving in {days} days."and a dict{"name": "Alice", "order_id": "2024001", "days": 3}, use regex to replace all{variable}placeholders with actual values. Hint: Usere.sub()with a callback function.