Pandas: Data Selection

Last updated: 2026-08-26

Data selection is the most frequent operation in data analysis—"show me the data that meets these conditions." Pandas offers a rich set of selection methods: by label, by position, by condition, by expression, and even SQL-like query strings. This section walks you through every selection method and helps you avoid the most common "chained indexing" pitfall.

⚠️ Note: The code below must be run in a local Python environment.

1. What You Will Learn


2. Charlie's Product Filtering Needs

(1) The Pain Point: Combining Multiple Conditions Is Too Cumbersome

Charlie needs to filter "Electronics products with price > 50 and rating > 4.0" out of 1000 products. He first wrote a long string of conditions using boolean indexing:

PYTHON
import pandas as pd

df = pd.DataFrame({
    'name': ['Laptop', 'Phone', 'Tablet', 'Monitor', 'Keyboard',
             'Mouse', 'Headset', 'Speaker', 'Camera', 'Charger'],
    'category': ['Electronics', 'Electronics', 'Electronics', 'Electronics',
                 'Accessories', 'Accessories', 'Accessories', 'Accessories',
                 'Electronics', 'Accessories'],
    'price': [999.99, 699.99, 349.99, 449.99, 79.99,
              29.99, 149.99, 89.99, 599.99, 19.99],
    'stock': [50, 120, 80, 35, 200, 300, 150, 180, 25, 500],
    'rating': [4.7, 4.5, 4.3, 4.6, 4.2, 4.0, 4.4, 4.1, 4.8, 3.9]
})

# Boolean indexing — works but verbose
result = df[(df['price'] > 50) & (df['rating'] > 4.0) & (df['category'] == 'Electronics')]
print(result[['name', 'price', 'rating']])
#       name    price  rating
# 0   Laptop   999.99     4.7
# 1    Phone   699.99     4.5
# 2   Tablet   349.99     4.3
# 3  Monitor   449.99     4.6
# 8   Camera   599.99     4.8
TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

(2) The Solution: The query Method Is More Readable

▶ Example

TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

: Readable queries with query (Difficulty ⭐⭐)

PYTHON
# Same result, much more readable
result = df.query('price > 50 and rating > 4.0 and category == "Electronics"')
print(result[['name', 'price', 'rating']])

# With variable
min_price = 50
min_rating = 4.0
result2 = df.query('price > @min_price and rating > @min_rating')
TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

(3) The Benefit: Four Selection Methods, Each with Its Strengths

Method Syntax Best For
loc df.loc[row, col] Precise label selection
iloc df.iloc[r, c] Position-based selection
Boolean indexing df[condition] Simple conditions
query df.query('expr') Complex multi-condition filters

3. loc: Label Selection

(1) loc Syntax

df.loc[row_label, col_label] — selects by label; slices include the endpoint.

▶ Example

TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

: loc label selection (Difficulty ⭐)

PYTHON
import pandas as pd

df = pd.DataFrame({
    'name': ['Laptop', 'Phone', 'Tablet', 'Monitor', 'Keyboard'],
    'price': [999.99, 699.99, 349.99, 449.99, 79.99],
    'stock': [50, 120, 80, 35, 200]
}, index=['A001', 'A002', 'A003', 'A004', 'A005'])

# Single row by label
print(df.loc['A001'])
# name      Laptop
# price     999.99
# stock         50

# Multiple rows
print(df.loc[['A001', 'A003']])

# Row slice + column slice (labels, INCLUDES endpoint)
print(df.loc['A001':'A003', 'name':'price'])
#          name    price
# A001   Laptop   999.99
# A002    Phone   699.99
# A003   Tablet   349.99

# Specific rows + specific columns
print(df.loc[['A001', 'A004'], ['name', 'stock']])
TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

(2) Conditional Selection with loc

▶ Example

TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

: loc + condition (Difficulty ⭐⭐)

PYTHON
import pandas as pd

df = pd.DataFrame({
    'name': ['Laptop', 'Phone', 'Tablet', 'Monitor', 'Keyboard'],
    'price': [999.99, 699.99, 349.99, 449.99, 79.99],
    'stock': [50, 120, 80, 35, 200]
})

# Filter rows + select columns in one operation
expensive = df.loc[df['price'] > 400, ['name', 'price']]
print(expensive)
#       name    price
# 0   Laptop   999.99
# 1    Phone   699.99
# 3  Monitor   449.99

# With assignment (safe — no chained indexing)
df.loc[df['stock'] < 50, 'stock'] = 50  # restock low items
TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

4. iloc: Position Selection

(1) iloc Syntax

df.iloc[row_pos, col_pos] — selects by integer position; slices exclude the endpoint (Python style).

▶ Example

TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

: iloc position selection (Difficulty ⭐)

PYTHON
import pandas as pd

df = pd.DataFrame({
    'name': ['Laptop', 'Phone', 'Tablet', 'Monitor', 'Keyboard'],
    'price': [999.99, 699.99, 349.99, 449.99, 79.99],
    'stock': [50, 120, 80, 35, 200]
})

# Single row by position
print(df.iloc[0])      # first row (Laptop)

# Multiple rows by position list
print(df.iloc[[0, 2, 4]])

# Row slice + column slice (positions, EXCLUDES endpoint)
print(df.iloc[0:3, 0:2])
#      name    price
# 0  Laptop   999.99
# 1   Phone   699.99
# 2  Tablet   349.99

# Last 2 rows
print(df.iloc[-2:])

# Every other row
print(df.iloc[::2])
TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

(2) Key Differences Between loc and iloc

Feature loc iloc
Index basis Label Integer position
Slice endpoint Includes endpoint Excludes endpoint
String labels
Negative indices
Conditional filtering
Recommended scenario When you know the name When you know the position
🔥 Common Mistake: df.loc[0:3] includes rows 0,1,2,3 (4 rows), while df.iloc[0:3] includes rows 0,1,2 (3 rows). This is the most common point of confusion between loc and iloc.


5. Conditional Filtering (Boolean Indexing)

(1) Single and Multiple Conditions

▶ Example

TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

: Conditional filtering (Difficulty ⭐⭐)

PYTHON
import pandas as pd

df = pd.DataFrame({
    'name': ['Laptop', 'Phone', 'Tablet', 'Monitor', 'Keyboard'],
    'category': ['Electronics', 'Electronics', 'Electronics', 'Electronics', 'Accessories'],
    'price': [999.99, 699.99, 349.99, 449.99, 79.99],
    'stock': [50, 120, 80, 35, 200],
    'rating': [4.7, 4.5, 4.3, 4.6, 4.2]
})

# Single condition
print(df[df['price'] > 400])

# Multiple conditions: & (AND), | (OR), ~ (NOT)
# MUST use parentheses around each condition!
high_price_good_rating = df[(df['price'] > 400) & (df['rating'] >= 4.5)]
print(high_price_good_rating[['name', 'price', 'rating']])

# OR condition
hot_items = df[(df['price'] > 500) | (df['stock'] > 100)]
print(hot_items[['name', 'price', 'stock']])

# NOT condition
not_electronics = df[~(df['category'] == 'Electronics')]
print(not_electronics)
TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
⚠️ Note: When combining multiple conditions, you must use & / | / ~, not and / or / not. Each condition must be wrapped in parentheses, otherwise operator precedence will cause an error.

(2) String Conditions

▶ Example

TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

: String condition filtering (Difficulty ⭐)

PYTHON
import pandas as pd

df = pd.DataFrame({
    'name': ['Wireless Mouse', 'USB Keyboard', 'Bluetooth Headset',
             'Wireless Speaker', 'USB Camera'],
    'price': [29.99, 79.99, 149.99, 89.99, 599.99]
})

# Contains substring
wireless = df[df['name'].str.contains('Wireless')]
print(wireless)

# Starts with
usb = df[df['name'].str.startswith('USB')]
print(usb)
TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

6. The query Method

(1) Basic Syntax

df.query('expression') — filters rows using a string expression, offering readability closer to SQL.

▶ Example

TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

: query expression queries (Difficulty ⭐⭐)

PYTHON
import pandas as pd

df = pd.DataFrame({
    'name': ['Laptop', 'Phone', 'Tablet', 'Monitor', 'Keyboard'],
    'category': ['Electronics', 'Electronics', 'Electronics', 'Electronics', 'Accessories'],
    'price': [999.99, 699.99, 349.99, 449.99, 79.99],
    'stock': [50, 120, 80, 35, 200],
    'rating': [4.7, 4.5, 4.3, 4.6, 4.2]
})

# Simple condition
print(df.query('price > 400'))

# Multiple conditions (and/or instead of &/|)
print(df.query('price > 400 and rating >= 4.5'))

# String comparison
print(df.query('category == "Electronics"'))

# Use @ to reference Python variables
threshold = 100
print(df.query('stock > @threshold'))

# Column names with spaces (backtick)
# df.query('`unit price` > 100')
TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

(2) query vs. Boolean Indexing

Feature Boolean Indexing query
Syntax df[(df.a>1) & (df.b<2)] df.query('a>1 and b<2')
Readability Nested parentheses Close to SQL
Variable reference Used directly Requires the @ prefix
Column names with spaces No problem Requires backticks
Performance Slightly slower (large DF) Slightly faster (engine optimized)
Flexibility Highest Limited by expression syntax

7. isin Filtering

(1) Multi-Value Matching

▶ Example

TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

: isin multi-value filtering (Difficulty ⭐)

PYTHON
import pandas as pd

df = pd.DataFrame({
    'name': ['Laptop', 'Phone', 'Tablet', 'Monitor', 'Keyboard'],
    'category': ['Electronics', 'Electronics', 'Electronics', 'Electronics', 'Accessories'],
    'price': [999.99, 699.99, 349.99, 449.99, 79.99]
})

# Filter by multiple category values
target_cats = ['Electronics', 'Audio']
# Note: Audio not in data → no match, no error
result = df[df['category'].isin(target_cats)]
print(result[['name', 'category']])

# Negation: NOT in list
other = df[~df['category'].isin(['Electronics'])]
print(other)
# Only Accessories

# isin with numeric values
target_prices = [999.99, 79.99]
print(df[df['price'].isin(target_prices)])
# Laptop and Keyboard
TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

(2) isin vs. == for Multiple Values

Method Syntax Use Case
isin df[col].isin([v1,v2,v3]) Multi-value matching
== df[col] == v1 Single-value matching
| combination (df[col]==v1) | (df[col]==v2) A few multiple values
query df.query('col in ["v1","v2"]') Readable multi-value matching

8. The Chained Indexing Pitfall

(1) What Is Chained Indexing

▶ Example

TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

: The hidden danger of chained indexing (Difficulty ⭐⭐)

PYTHON
import pandas as pd

df = pd.DataFrame({
    'name': ['Laptop', 'Phone', 'Tablet'],
    'price': [999.99, 699.99, 349.99],
    'category': ['Electronics', 'Electronics', 'Electronics']
})

# ❌ Chained indexing — may trigger SettingWithCopyWarning
# df[df['price'] > 400]['price'] = 0  # Unpredictable!

# ✅ Single operation with loc
df.loc[df['price'] > 400, 'price'] = 0  # Always works

# ❌ Another chained form
# df[df['category'] == 'Electronics']['stock'] = 100

# ✅ Correct approach
df.loc[df['category'] == 'Electronics', 'stock'] = 100
TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
🔥 Common Mistake: Chained indexing df[condition][col] = value may not modify the original DataFrame (the intermediate result is a copy). The correct way to write it is: df.loc[condition, col] = value—a single operation that specifies both rows and columns at once.

(2) Safe Selection Decision Tree

100%
graph TB
    Q["Need to select data?"] -->|"Read only"| LOC["loc / iloc / boolean"]
    Q -->|"Read + Assign"| SAFE["loc[row_cond, col] = value"]
    Q -->|"Complex filter"| QUERY["query()"]
    Q -->|"Multi-value match"| ISIN["isin()"]
    SAFE --> WARN["❌ Never: df[cond][col] = val"]
TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

9. Complete Example: Multi-Dimension Product Filtering for E-Commerce

▶ Example

TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

: The full product filtering workflow (Difficulty ⭐⭐⭐)

PYTHON
import pandas as pd
import numpy as np

# ============================================
# Comprehensive example: E-commerce product
# multi-dimension selection workflow
# ============================================

# 1. Create product catalog
np.random.seed(42)
df = pd.DataFrame({
    'name': [f'Product_{i:03d}' for i in range(1, 51)],
    'category': np.random.choice(['Electronics', 'Clothing', 'Home', 'Sports'], 50),
    'price': np.round(np.random.uniform(10, 500, 50), 2),
    'stock': np.random.randint(0, 300, 50),
    'rating': np.round(np.random.uniform(3.0, 5.0, 50), 1),
    'brand': np.random.choice(['Alpha', 'Beta', 'Gamma', 'Delta'], 50)
})

# 2. loc: precise selection
print("=== Electronics with price > 200 ===")
elec_expensive = df.loc[
    (df['category'] == 'Electronics') & (df['price'] > 200),
    ['name', 'price', 'rating']
]
print(elec_expensive.head())

# 3. iloc: position-based selection
print("\n=== First 3 rows, first 4 columns ===")
print(df.iloc[:3, :4])

# 4. Boolean: multi-condition
print("\n=== High rating, in stock, under $100 ===")
bargains = df[(df['rating'] >= 4.5) & (df['stock'] > 0) & (df['price'] < 100)]
print(bargains[['name', 'price', 'rating', 'stock']])

# 5. query: readable complex filter
print("\n=== Alpha brand Electronics or Home, rating >= 4.0 ===")
query_result = df.query(
    'brand == "Alpha" and category in ["Electronics", "Home"] and rating >= 4.0'
)
print(query_result[['name', 'category', 'rating']])

# 6. isin: multi-value filter
print("\n=== Selected brands ===")
top_brands = df[df['brand'].isin(['Alpha', 'Beta'])]
print(f"Alpha + Beta products: {len(top_brands)}")

# 7. Safe assignment
df.loc[df['stock'] == 0, 'status'] = 'Out of Stock'
df.loc[df['stock'] > 0, 'status'] = 'In Stock'
print(f"\n=== Stock Status ===")
print(df['status'].value_counts())
TEXT 📖 Display only
> **Output:** Run this in a local Python environment (pandas 2.x). The Piston server does not have pandas preinstalled, so please install it on your machine (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

❓ FAQ

Q What is the difference between loc and iloc?
A loc selects by label (row name/column name), and slices include the endpoint. iloc selects by integer position, and slices exclude the endpoint (Python style). Use loc when you know the name, and iloc when you know the position. When the Index is the default RangeIndex(0,1,2...), loc[0] and iloc[0] return the same result, but their semantics differ.
Q Why do conditional filters need parentheses?
A In Python, the & / | / ~ operators have higher precedence than comparison operators. df['a'] > 1 & df['b'] < 2 is parsed as df['a'] > (1 & df['b']) < 2, which raises an error. You must write (df['a'] > 1) & (df['b'] < 2). You cannot use and/or/not—they do not support element-wise operations at the array level.
Q What is the problem with chained indexing?
A df[condition][col] = value is two separate operations: the first may return either a view or a copy, and assigning to a copy in the second step will not affect the original DataFrame. This leads to the subtle bug of "I changed it but it didn't take effect." The correct way to write it is: df.loc[condition, col] = value, a single operation that specifies both rows and columns at once.
Q Does query support variables?
A Yes. Use @ to reference external variables: df.query('price > @min_price'). For column names with spaces, use backticks: df.query('unit price > 100'). Inside query, keywords like and/or/not/in are supported, making it closer to natural language.
Q What is the difference between isin and ==?
A == can only match a single value, while isin can match multiple values at once. df[col].isin([v1,v2,v3]) is equivalent to (df[col]==v1) | (df[col]==v2) | (df[col]==v3), but is more concise and efficient. Use ~df[col].isin(...) when you need negation.
Q What are the pitfalls of assigning after selection?
A The biggest pitfall is chained indexing (see FAQ #3). Another pitfall is that df[df.a > 0] = 1 sets the entire row to 1 (not just column a). The correct form is: df.loc[df.a > 0, 'a'] = 1. Always use loc to specify both the row condition and the target column at the same time.
Q Why does a loc slice include the endpoint?
A This is a design choice by Pandas—when slicing by label, the endpoint means "from label A to label B," and including both is more intuitive (just like a key range in a dictionary). Slicing by position with iloc follows the Python convention (excluding the endpoint). Remember: loc is "label semantics," and iloc is "Python semantics."

📖 Summary


📝 Exercises

  1. Basic (Difficulty ⭐): Create a student DataFrame (name / math / english / science). Use loc to select the name and math columns of students with math > 80, and use iloc to select the first 3 rows.
  2. Intermediate (Difficulty ⭐⭐): Filter the same data in 3 ways (boolean indexing / query / isin): price between 50 and 200, and category is Electronics or Home. Compare the amount of code and readability.
  3. Challenge (Difficulty ⭐⭐⭐): Create a 50-row product DataFrame and complete 5 selection exercises: loc single row / iloc slice / multi-condition boolean / query with variable / isin multi-value. After each selection, safely assign a new marker column using loc, and finally count the number of each marker.

← Previous Lesson: Data Types · Next Lesson: Data I/O →

Web-Tutorial.com

Web-Tutorial Tech Team

A team of developers maintaining programming tutorials. Each tutorial is written and reviewed by developers with expertise in that field. We work to keep our content accurate and reliable — if you spot an issue, please let us know.

100%

🙏 帮我们做得更好

我们是刚上线的编程教程站,几个人的小团队,精力有限。页面虽经检查,难免还有疏漏——链接失效、排版错乱、内容有误、语言生硬……

如果您发现了,麻烦告诉我们,我们会在收到反馈后第一时间进行修复,再次感谢您的光临 🙏