Pandas: Data Types

Q: What's the difference between object and string?

object is an array of Python object pointers — each element is a separate Python string object with significant memory overhead. string (StringDtype) is a dedicated string type introduced in Pandas 1.0+ that uses pd.NA for missing values and offers more consistent operation semantics. For new projects, prefer string over object.

Q: What are the benefits of Nullable types?

They solve the long-standing problem where "an integer column with missing values gets forced into float." Int64 can hold both integers and pd.NA while preserving integer semantics. For example, a customer age column with missing values stays an integer type (Int8) instead of becoming float64.

Q: When should I use category?

When a column's unique values are far fewer than its row count (rule of thumb: unique values < 50% of rows). Typical cases: finite-category columns like city, gender, status, or tier. category replaces repeated strings with integer codes plus a lookup table, cutting memory by 90%+. It's not suitable for high-cardinality columns (like user IDs).

Q: What's the difference between convert_dtypes and astype?

astype requires you to manually specify the target type (e.g. astype('int64') ), while convert_dtypes automatically infers the best Nullable type (object→string, int64→Int64, float64→Float64). astype is more precise; convert_dtypes is more convenient.

Q: How do I check memory usage?

df.memory_usage(deep=True) returns the byte count for each column. deep=True computes the actual string memory for object columns (not just pointer sizes). df.info(memory_usage='deep') also shows total memory.

Q: Is downcast safe?

Yes — Pandas' pd.to_numeric(downcast=...) automatically selects the smallest type that can hold the data range. For example, integers from 0-100 will pick uint8 (0-255) with no overflow. But if you later add data outside that range, it may be silently truncated or upcast. It's best to run downcast as the final step after data cleaning is complete.

Q: Why is memory_usage so large for string columns?

Each element in an object/string column is a separate Python object with 50+ bytes of object-header overhead. 10,000 rows of strings can take up 600 KB, while the same number of int64 rows takes only 80 KB. Solution: use category for low-cardinality data, and consider chunked processing for high-cardinality data.

Last updated: 2026-08-26

The data type (dtype) determines how Pandas stores and operates on your data. Pick the right dtype and your code can run 10x faster while using 80% less memory; pick the wrong one and you'll face lost precision, operation errors, and memory blowups. This section walks through the Pandas dtype system, helping you understand the characteristics and conversion methods of each type, along with the most practical memory optimization techniques.

⚠️ Note: The code below must be run in a local Python environment.

1. What You'll Learn

❶ The differences between the Pandas dtype system and NumPy dtypes
❷ Choosing between object, StringDtype, and category
❸ Type conversion with astype / convert_dtypes / infer_objects
❹ Nullable types (Int64 / Float64 / boolean)
❺ Memory optimization strategies (category / downcast)

2. Bob's Memory Blowup Crisis

(1) The Pain Point: A 500 MB Customer Table

Bob loaded a customer dataset into a DataFrame and checked the memory usage — 500 MB! 100 thousand rows shouldn't take up that much space:

PYTHON

import pandas as pd

# Load customer data (simulated)
df = pd.DataFrame({
    'customer_id': range(100000),
    'name': ['Customer_' + str(i) for i in range(100000)],
    'city': ['New York', 'London', 'Tokyo', 'Paris', 'Sydney'] * 20000,
    'age': [25 + i % 50 for i in range(100000)],
    'loyalty_points': [100 + i * 10 for i in range(100000)]
})

print(f"Memory before optimization: {df.memory_usage(deep=True).sum() / 1024 / 1024:.1f} MB")
# Typical output: ~18 MB (this example is small)
# Real 100K-row dataset with many string columns can reach 500+ MB

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

The culprit: the object type. By default, Pandas stores string columns as object (Python object pointers), where every element is a full Python string object — a huge memory overhead.

(2) The Solution: 10x Compression with the category Type

▶ Example

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

: category Memory Optimization (Difficulty ⭐⭐)

PYTHON

import pandas as pd

df = pd.DataFrame({
    'customer_id': range(100000),
    'name': ['Customer_' + str(i) for i in range(100000)],
    'city': ['New York', 'London', 'Tokyo', 'Paris', 'Sydney'] * 20000,
    'age': [25 + i % 50 for i in range(100000)],
    'loyalty_points': [100 + i * 10 for i in range(100000)]
})

# Check memory before optimization
mem_before = df.memory_usage(deep=True).sum() / 1024 / 1024

# Optimize: city has only 5 unique values → category
df['city'] = df['city'].astype('category')

# Optimize: age range 25-74 → int8 is enough (max 127)
df['age'] = df['age'].astype('int8')

# Optimize: loyalty_points range → int32 is enough
df['loyalty_points'] = df['loyalty_points'].astype('int32')

mem_after = df.memory_usage(deep=True).sum() / 1024 / 1024
print(f"Before: {mem_before:.1f} MB")
print(f"After:  {mem_after:.1f} MB")
print(f"Saved:  {(1 - mem_after/mem_before)*100:.0f}%")
# Before: ~18 MB
# After:  ~5 MB
# Saved:  ~72%

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

(3) The Payoff: 3 Key Optimization Techniques

Technique	Effect	When to Use
object→category	90%+ memory reduction	Low-cardinality strings (unique values < 50% of row count)
int64→int8/int16/int32	~75% memory reduction	Small numeric ranges
float64→float32	50% memory reduction	Low precision requirements

3. The Pandas dtype System

(1) The Full Type Landscape

graph TB
    DT["Pandas dtype"] --> NUM["Numeric"]
    DT --> STR["String-like"]
    DT --> DT_TYPE["Datetime"]
    DT --> BOOL["Boolean"]
    DT --> CAT["Categorical"]
    DT --> NULL["Nullable"]

    NUM --> I["int8/16/32/64"]
    NUM --> UI["uint8/16/32/64"]
    NUM --> F["float32/float64"]

    STR --> O["object (legacy)"]
    STR --> SD["StringDtype (new)"]

    DT_TYPE --> DT64["datetime64[ns]"]
    DT_TYPE --> TD64["timedelta64[ns]"]

    BOOL --> PB["bool (NumPy)"]
    BOOL --> NB["boolean (Nullable)"]

    NULL --> NI["Int8/16/32/64"]
    NULL --> NF["Float32/64"]
    NULL --> NBO["boolean"]
    NULL --> NS["string"]

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

(2) Pandas dtype vs NumPy dtype

Feature	NumPy dtype	Pandas dtype
Numeric types	int8~64 / float32~64	Same as NumPy + Nullable (Int64/Float64)
Strings	object	object / StringDtype / string
Boolean	bool	bool / boolean (Nullable)
Missing values	np.nan (float only)	NaN / NaT / pd.NA (full coverage)
Categorical	None	category
Time	datetime64	datetime64[ns] / timedelta64[ns]
Extension	None	ExtensionDtype (custom)

📌 Key point: Pandas' Nullable types (capitalized: Int64 / Float64 / boolean / string) are a feature introduced in 1.0. They use pd.NA to represent missing values uniformly, solving the long-standing problem that "integer columns can't hold NaN."

4. object vs StringDtype vs category

(1) Comparing the Three

Feature	object	StringDtype	category
Underlying storage	Python object pointers	Dedicated string storage	Integer codes + lookup table
Missing values	None / np.nan	pd.NA	pd.NA
Memory	Highest	Medium	Lowest (for low cardinality)
String methods	.str accessor	.str accessor	.str accessor
Comparisons	Can be inconsistent	Consistent	Consistent
Sorting	Lexicographic	Lexicographic	Custom order supported
Best for	Legacy code compatibility	Recommended for new projects	Low-cardinality columns

▶ Example

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

: Memory Comparison of the Three Types (Difficulty ⭐⭐)

PYTHON

import pandas as pd
import numpy as np

# 1000 rows, 5 unique cities
cities = ['New York', 'London', 'Tokyo', 'Paris', 'Sydney']
data = [cities[i % 5] for i in range(1000)]

s_object = pd.Series(data, dtype='object')
s_string = pd.Series(data, dtype='string')
s_category = pd.Series(data, dtype='category')

print(f"object:   {s_object.memory_usage(deep=True) / 1024:.1f} KB")
print(f"string:   {s_string.memory_usage(deep=True) / 1024:.1f} KB")
print(f"category: {s_category.memory_usage(deep=True) / 1024:.1f} KB")
# object:   ~62 KB
# string:   ~55 KB
# category: ~5 KB  ← 12x smaller!

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

🔥 Common pitfall: When the number of unique values approaches the row count (high cardinality), category actually uses more memory than object — because it has to maintain a full mapping table. Rule of thumb: category pays off when unique values are under 50% of the row count.

5. Type Conversion Methods

(1) astype: Explicit Conversion

▶ Example

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

: astype Type Conversion (Difficulty ⭐)

PYTHON

import pandas as pd

df = pd.DataFrame({
    'price_str': ['9.99', '19.99', '29.99'],
    'quantity_int': [1, 3, 5],
    'flag_str': ['True', 'False', 'True']
})

# String to float
df['price'] = df['price_str'].astype(float)

# Int to string (for ID columns)
df['qty_str'] = df['quantity_int'].astype(str)

# String to boolean (caution: 'True'/'False' as strings → bool)
df['flag'] = df['flag_str'].map({'True': True, 'False': False})

print(df.dtypes)
# price_str       object
# quantity_int     int64
# flag_str        object
# price          float64
# qty_str         object
# flag              bool

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

(2) convert_dtypes: Automatically Infer the Best Type

▶ Example

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

: convert_dtypes Automatic Conversion (Difficulty ⭐⭐)

PYTHON

import pandas as pd

df = pd.DataFrame({
    'name': ['Alice', 'Bob', 'Charlie'],
    'age': [28, 34, 25],
    'score': [85.5, 92.0, 78.3],
    'active': [True, True, False]
})

# Convert to best nullable dtypes
df_converted = df.convert_dtypes()
print(df_converted.dtypes)
# name      string    ← object → StringDtype
# age        Int64    ← int64 → nullable Int64
# score    Float64    ← float64 → nullable Float64
# active   boolean    ← bool → nullable boolean

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

(3) infer_objects: Infer the Best Type for object Columns

▶ Example

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

: infer_objects Inference (Difficulty ⭐)

PYTHON

import pandas as pd

df = pd.DataFrame({
    'a': [1, 2, 3],       # stored as object
    'b': ['x', 'y', 'z']  # stays as object
}, dtype='object')

print(df.dtypes)
# a    object
# b    object

df_inferred = df.infer_objects()
print(df_inferred.dtypes)
# a     int64   ← inferred from content
# b    object   ← still object (strings stay)

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

(4) Conversion Method Comparison

Method	Purpose	Automatic/Manual	Nullable
astype()	Force-convert to a specified type	Manual	Not automatic
convert_dtypes()	Infer the best Nullable type	Automatic	Yes
infer_objects()	Infer the type of object columns	Automatic	Not automatic

6. Nullable Types: Solving the Integer Missing-Value Problem

(1) Why Nullable Types Are Needed

NumPy's int64 can't store NaN — when an integer column has missing values, Pandas is forced to upcast the whole column to float64, losing the original type.

▶ Example

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

: Nullable vs Traditional Types (Difficulty ⭐⭐)

PYTHON

import pandas as pd
import numpy as np

# Traditional: int column with NaN → forced to float
s_trad = pd.Series([1, 2, np.nan, 4])
print(s_trad)         # 1.0, 2.0, NaN, 4.0 — floats!
print(s_trad.dtype)   # float64

# Nullable: Int64 keeps integer type with <NA>
s_null = pd.Series([1, 2, pd.NA, 4], dtype='Int64')
print(s_null)         # 1, 2, <NA>, 4 — integers!
print(s_null.dtype)   # Int64

# Nullable boolean
b_null = pd.Series([True, False, pd.NA], dtype='boolean')
print(b_null)

# Nullable string
str_null = pd.Series(['Alice', pd.NA, 'Charlie'], dtype='string')
print(str_null)

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

(2) Nullable Types at a Glance

Nullable Type	Traditional Equivalent	Missing-Value Marker	When to Use
Int8/16/32/64	int8/16/32/64	pd.NA	Integer columns with missing values
Float32/64	float32/64	pd.NA	Float columns with missing values
boolean	bool	pd.NA	Boolean columns with missing values
string	object	pd.NA	String columns (replaces object)

💡 Tip: Nullable types use pd.NA to represent missing values uniformly, replacing the messy mix of np.nan / None / NaT. During operations, pd.NA follows the "propagation rule" — any operation involving pd.NA returns pd.NA.

7. Memory Optimization in Practice

(1) Checking Memory Usage

▶ Example

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

: memory_usage Analysis (Difficulty ⭐)

PYTHON

import pandas as pd

df = pd.DataFrame({
    'id': range(10000),
    'city': ['New York', 'London', 'Tokyo', 'Paris', 'Sydney'] * 2000,
    'score': [85.5 + i * 0.1 for i in range(10000)],
    'status': ['Active', 'Inactive', 'Pending'] * 3333 + ['Active']
})

# Per-column memory usage (deep=True for object columns)
print(df.memory_usage(deep=True))
# Index           80
# id            80000    ← int64 for 10K rows
# city         630000    ← object (huge!)
# score        80000    ← float64
# status       460000    ← object (huge!)

# Total
total = df.memory_usage(deep=True).sum() / 1024 / 1024
print(f"Total: {total:.1f} MB")

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

(2) Optimization Strategies

▶ Example

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

: End-to-End Memory Optimization (Difficulty ⭐⭐⭐)

PYTHON

import pandas as pd

df = pd.DataFrame({
    'id': range(10000),
    'city': ['New York', 'London', 'Tokyo', 'Paris', 'Sydney'] * 2000,
    'score': [85.5 + i * 0.1 for i in range(10000)],
    'status': ['Active', 'Inactive', 'Pending'] * 3333 + ['Active']
})

mem_before = df.memory_usage(deep=True).sum() / 1024 / 1024

# 1. String columns with low cardinality → category
df['city'] = df['city'].astype('category')
df['status'] = df['status'].astype('category')

# 2. Numeric downcast
df['id'] = pd.to_numeric(df['id'], downcast='integer')   # int64 → int16
df['score'] = pd.to_numeric(df['score'], downcast='float')  # float64 → float32

mem_after = df.memory_usage(deep=True).sum() / 1024 / 1024

print(f"Before: {mem_before:.1f} MB")
print(f"After:  {mem_after:.1f} MB")
print(f"Saved:  {(1 - mem_after/mem_before)*100:.0f}%")
print(f"\nOptimized dtypes:")
print(df.dtypes)

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

(3) Downcast Reference Table

Original Type	downcast='integer'	downcast='float'	Savings
int64	int8/int16/int32 (auto-selects smallest)	—	50%~75%
float64	—	float32	50%
uint64	uint8/uint16/uint32	—	50%~75%

8. Complete Example: Optimizing a Large Customer Table

▶ Example

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

: Full Customer Table Optimization Pipeline (Difficulty ⭐⭐⭐)

PYTHON

import pandas as pd
import numpy as np

# ============================================
# Comprehensive example: Customer table
# full optimization pipeline
# ============================================

# 1. Create realistic customer data
n = 50000
df = pd.DataFrame({
    'customer_id': range(n),
    'name': [f'Customer_{i:05d}' for i in range(n)],
    'city': np.random.choice(['New York', 'London', 'Tokyo', 'Paris',
                               'Sydney', 'Berlin', 'Toronto', 'Seoul'], n),
    'age': np.random.randint(18, 80, n),
    'membership': np.random.choice(['Basic', 'Silver', 'Gold', 'Platinum'], n),
    'points': np.random.randint(0, 100000, n),
    'satisfaction': np.random.choice([1, 2, 3, 4, 5, np.nan], n, p=[0.05,0.1,0.2,0.3,0.3,0.05])
})

# 2. Analyze before optimization
print("=== BEFORE Optimization ===")
print(df.dtypes)
mem_before = df.memory_usage(deep=True).sum() / 1024 / 1024
print(f"Total memory: {mem_before:.1f} MB")

# 3. Optimize
# Low-cardinality strings → category
for col in ['city', 'membership']:
    df[col] = df[col].astype('category')

# Integer downcast
for col in ['customer_id', 'age', 'points']:
    df[col] = pd.to_numeric(df[col], downcast='integer')

# Float with NaN → nullable Int64
df['satisfaction'] = df['satisfaction'].astype('Int8')

# 4. Analyze after optimization
print("\n=== AFTER Optimization ===")
print(df.dtypes)
mem_after = df.memory_usage(deep=True).sum() / 1024 / 1024
print(f"Total memory: {mem_after:.1f} MB")
print(f"Saved: {(1 - mem_after/mem_before)*100:.0f}%")

# 5. Verify data integrity
print(f"\n=== Integrity Check ===")
print(f"Rows: {len(df)}")
print(f"Satisfaction with NaN: {df['satisfaction'].isna().sum()}")
print(f"Unique cities: {df['city'].nunique()}")

TEXT 📖 Display only

> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed — install it locally (`pip install pandas`) to follow along. Actual values may vary slightly depending on your pandas version.

❓ FAQ

Q What's the difference between object and string?

A object is an array of Python object pointers — each element is a separate Python string object with significant memory overhead. string (StringDtype) is a dedicated string type introduced in Pandas 1.0+ that uses pd.NA for missing values and offers more consistent operation semantics. For new projects, prefer string over object.

Q What are the benefits of Nullable types?

A They solve the long-standing problem where "an integer column with missing values gets forced into float." Int64 can hold both integers and pd.NA while preserving integer semantics. For example, a customer age column with missing values stays an integer type (Int8) instead of becoming float64.

Q When should I use category?

A When a column's unique values are far fewer than its row count (rule of thumb: unique values < 50% of rows). Typical cases: finite-category columns like city, gender, status, or tier. category replaces repeated strings with integer codes plus a lookup table, cutting memory by 90%+. It's not suitable for high-cardinality columns (like user IDs).

Q What's the difference between convert_dtypes and astype?

A astype requires you to manually specify the target type (e.g. astype('int64')), while convert_dtypes automatically infers the best Nullable type (object→string, int64→Int64, float64→Float64). astype is more precise; convert_dtypes is more convenient.

Q How do I check memory usage?

A df.memory_usage(deep=True) returns the byte count for each column. deep=True computes the actual string memory for object columns (not just pointer sizes). df.info(memory_usage='deep') also shows total memory.

Q Is downcast safe?

A Yes — Pandas' pd.to_numeric(downcast=...) automatically selects the smallest type that can hold the data range. For example, integers from 0-100 will pick uint8 (0-255) with no overflow. But if you later add data outside that range, it may be silently truncated or upcast. It's best to run downcast as the final step after data cleaning is complete.

Q Why is memory_usage so large for string columns?

A Each element in an object/string column is a separate Python object with 50+ bytes of object-header overhead. 10,000 rows of strings can take up 600 KB, while the same number of int64 rows takes only 80 KB. Solution: use category for low-cardinality data, and consider chunked processing for high-cardinality data.

📖 Summary

The Pandas dtype system is richer than NumPy's: it adds Nullable types, category, and StringDtype
object is a memory hog — low-cardinality string columns should be converted to category (90%+ memory reduction)
Nullable types (Int64/Float64/boolean/string) use pd.NA to unify missing values, solving the "integer columns can't hold NaN" problem
astype converts types manually, convert_dtypes infers Nullable types automatically, and infer_objects infers types for object columns
Three steps to memory optimization: low-cardinality strings→category → numeric downcast → float64→float32
Use memory_usage(deep=True) to see real memory usage — the actual memory of object columns far exceeds pointer size
Run downcast after data cleaning is complete to avoid later data exceeding the type range

📝 Exercises

Basic (Difficulty ⭐): Create a DataFrame with 3 columns (string/integer/float). Check the types with dtypes, then use astype to convert the integer column to int8 and the float column to float32. Compare the change in memory_usage.
Intermediate (Difficulty ⭐⭐): Create an integer Series containing NaN. Handle it both the traditional way (which auto-converts to float64) and with the Nullable Int64 type, then compare the dtype and operation behavior. Next, optimize a string column with 5 unique values using the category type.
Challenge (Difficulty ⭐⭐⭐): Create a simulated 10K-row DataFrame with 4 columns (low-cardinality string / high-cardinality string / integer / float). Run the full memory optimization pipeline: check → low-cardinality → category → downcast → Nullable → verify. Print the memory change at each step and the final savings percentage.

← Previous: Index System · Next: Data Selection →

Web-Tutorial Tech Team

A team of developers maintaining programming tutorials. Each tutorial is written and reviewed by developers with expertise in that field. We work to keep our content accurate and reliable — if you spot an issue, please let us know.

About Us

Pandas: Data Types

1. What You'll Learn

2. Bob's Memory Blowup Crisis

(1) The Pain Point: A 500 MB Customer Table

(2) The Solution: 10x Compression with the category Type

▶ Example

(3) The Payoff: 3 Key Optimization Techniques

3. The Pandas dtype System

(1) The Full Type Landscape

(2) Pandas dtype vs NumPy dtype

4. object vs StringDtype vs category

(1) Comparing the Three

▶ Example

5. Type Conversion Methods

(1) astype: Explicit Conversion

▶ Example

(2) convert_dtypes: Automatically Infer the Best Type

▶ Example

(3) infer_objects: Infer the Best Type for object Columns

▶ Example

(4) Conversion Method Comparison

6. Nullable Types: Solving the Integer Missing-Value Problem

(1) Why Nullable Types Are Needed

▶ Example

(2) Nullable Types at a Glance

7. Memory Optimization in Practice

(1) Checking Memory Usage

▶ Example

(2) Optimization Strategies

▶ Example

(3) Downcast Reference Table

8. Complete Example: Optimizing a Large Customer Table

▶ Example

❓ FAQ

📖 Summary

📝 Exercises

🙏 帮我们做得更好