R: R EDA

Last updated: 2026-08-26

In the previous six lessons, we covered descriptive statistics, probability distributions, hypothesis testing, and regression—all of which are "analytical" tools. However, the first step in data analysis is not modeling, but rather EDA (Exploratory Data Analysis)—exploring what the data "looks like." In this lesson, we’ll learn the complete R EDA methodology.

After completing this lesson, you’ll be able to use R to thoroughly explore any dataset—including its shape, distribution, outliers, missing values, and correlations—laying the groundwork for future modeling.

1. What You'll Learn

The Complete EDA Process (5 Steps)
Data shape: str / dim / nrow / ncol
Distribution: summary / skimr
Missing values: md.pattern / naniar
Distribution Visualization: Histogram / Box Plot / Density Plot
Relationships: Scatter Plot / Correlation Matrix
Outliers: Box-and-Whisker Plot + IQR
Automated EDA (DataExplorer / SmartEDA)

2. Challenges with an Unfamiliar Dataset

(1) Challenge: Where do I start with a 100-column CSV file?

Bob received a dataset of customer data consisting of 100 columns and 10,000 rows, and his manager asked, "Take a look at this data first." He stared at the Excel file for five minutes, not knowing where to start.

(2) Solution using R

library(DataExplorer)

# One line EDA Report
create_report(df, output_file = "eda_report.html")

1 line of code → Complete EDA HTML report (30+ pages of charts and statistics).

3. The Complete EDA Process (5 Steps)

(1) The 5-Step Method

graph TB
    A[1. Data Shape] --> B[2. Data Types]
    B --> C[3. Missing Value Analysis]
    C --> D[4. Distribution Exploration]
    D --> E[5. Exploring Relationships]
    
    A --> A1[dim/str/head]
    B --> B1[summary/skimr]
    C --> C1[md.pattern/naniar]
    D --> D1[Histogram/Box plot]
    E --> E1[Scatter/Correlation Matrix]
    
    style A fill:#fff3cd
    style B fill:#cce5ff
    style C fill:#d4edda
    style D fill:#f8d7da
    style E fill:#e1d4ff

(2) 5-Step Quick Reference

Step	Objective	Key Functions
1. Shape	How big is the data?	`dim()` `nrow()` `ncol()` `str()`
2. Type	What is the column type?	`sapply(df, class)` `summary()`
3. Missing	How many are missing?	`colSums(is.na())` `skimr` `naniar`
4. Distribution	Numerical range?	`hist()` `boxplot()` `geom_density()`
5. Relationships	Are the columns related?	`pairs()` `cor()` `corrplot`

4. Step 1: Data Shape

# Prepare the data
df <- iris  # Use iris as demo

# Basic Information
dim(df)         # [1] 150 5  ← 150 row 5 col
nrow(df)        # [1] 150
ncol(df)        # [1] 5
names(df)       # Listed
str(df)         # Structure
# 'data.frame':	150 obs. of  5 variables:
#  $ Sepal.Length: num  5.1 4.9 4.7 4.6 5 ...
#  $ Sepal.Width : num  3.5 3 3.2 3.1 3.6 ...
#  $ Petal.Length: num  1.4 1.4 1.3 1.5 1.4 ...
#  $ Petal.Width : num  0.2 0.2 0.2 0.2 0.2 ...
#  $ Species     : Factor w/ 3 levels "setosa",...

# Data Preview
head(df, 3)    # First 3 rows
tail(df, 3)    # Last 3 rows

5. Step 2: Type and Abstract

(1) Numeric Columns vs. Categorical Columns

# Column Types
sapply(df, class)
# Sepal.Length  Sepal.Width Petal.Length  Petal.Width      Species
#   "numeric"    "numeric"   "numeric"   "numeric"    "factor"

# Automatic Recognition
library(dplyr)
df |> summarise(across(everything(), class))

(2) summary()—a one-line summary

summary(iris)
#   Sepal.Length    Sepal.Width     Petal.Length    Petal.Width
#  Min.   :4.30   Min.   :2.00   Min.   :1.00   Min.   :0.1
#  1st Qu.:5.10   1st Qu.:2.80   1st Qu.:1.60   1st Qu.:0.3
#  Median :5.80   Median :3.00   Median :4.35   Median :1.3
#  Mean   :5.84   Mean   :3.05   Mean   :3.76   Mean   :1.2
#  3rd Qu.:6.40   3rd Qu.:3.30   3rd Qu.:5.10   3rd Qu.:1.8
#  Max.   :7.90   Max.   :4.40   Max.   :6.90   Max.   :2.5
#       Species
#  setosa    :50
#  versicolor:50
#  virginica :50

(3) skimr Advanced Summary

install.packages("skimr")
library(skimr)

skim(iris)
# ── Data Summary ────────────────────────
# Values                           
# Number of rows                   150
# Number of columns                  5
# ── Variable type: factor ──
# Species: 1 unique, 50 each
# ── Variable type: numeric ──
# Sepal.Length: mean=5.84, sd=0.83, p0=4.3, p25=5.1, p50=5.8, p75=6.4, p100=7.9
# ... (Each variable 20+ metrics)

6. Step 3: Analysis of Missing Values

(1) Statistics on Missing Values

# Simulating Data with Missing Values
df <- data.frame(
  a = c(1, 2, NA, 4),
  b = c("x", NA, "z", "w"),
  c = c(NA, 2, 3, NA)
)

# Number of missing values
colSums(is.na(df))
# a b c
# 1 1 2

# Proportion of Missing Values
colMeans(is.na(df)) * 100
# a b c
# 25 25 50

# Entire rows with NA
sum(!complete.cases(df))  # [1] 3

(2) naniar: Advanced Visualization of Missing Values

install.packages("naniar")
library(naniar)

# Missing Value Patterns
vis_miss(iris)  # iris has no missing values, can draw empty chart
# Simulation with Missing Values
df_with_na <- iris
df_with_na[sample(150, 20), 1] <- NA
df_with_na[sample(150, 10), 3] <- NA
vis_miss(df_with_na)

# Missing Correlation
gg_miss_upset(df_with_na)

# Missing vs Variable Relationships
ggplot(df_with_na, aes(x = Sepal.Length, y = Petal.Length)) +
  geom_miss_point()  # The red dot indicates a missing item.

7. Step 4: Visualizing the Distribution

(1) Univariate Distribution

library(ggplot2)

# Histogram
ggplot(iris, aes(x = Sepal.Length)) +
  geom_histogram(bins = 30, fill = "skyblue", color = "white") +
  labs(title = "Sepal.Length Distribution")

# Density Plot
ggplot(iris, aes(x = Sepal.Length, fill = Species)) +
  geom_density(alpha = 0.5) +
  labs(title = "Sepal.Length Distribution by Species")

# Box-and-Whisker Plot (Identifying Outliers)
ggplot(iris, aes(y = Sepal.Length)) +
  geom_boxplot(fill = "lightblue", outlier.color = "red") +
  labs(title = "Sepal.Length Box-and-Whisker Plot")

# Box-and-Whisker Plot by Species
ggplot(iris, aes(x = Species, y = Sepal.Length, fill = Species)) +
  geom_boxplot() +
  labs(title = "Sepal.Length Distribution by Species")

(2) Multivariate Distributions

# Scatter Plot + Return
ggplot(iris, aes(x = Sepal.Length, y = Petal.Length, color = Species)) +
  geom_point(size = 3) +
  geom_smooth(method = "lm", se = FALSE) +
  labs(title = "Sepal vs Petal Length")

# Pair Graphs (Scatter Plot Matrix)
library(GGally)
ggpairs(iris, aes(color = Species))

(3) Correlation Matrix

# CalculateCorrelation Coefficient
cor_matrix <- cor(iris |> select(-Species))
print(round(cor_matrix, 2))

# Visualization of Related Matrices
library(corrplot)
corrplot(cor_matrix, method = "circle", type = "upper")

# ggplot2 Version
library(reshape2)
melted <- melt(cor_matrix)
ggplot(melted, aes(Var1, Var2, fill = value)) +
  geom_tile() +
  scale_fill_gradient2(low = "blue", high = "red", mid = "white",
                       midpoint = 0) +
  theme_minimal() +
  theme(axis.text.x = element_text(angle = 45, hjust = 1))

8. Step 5: Outlier Detection

(1) IQR Method (Most Robust)

detect_outliers <- function(x) {
  q1 <- quantile(x, 0.25, na.rm = TRUE)
  q3 <- quantile(x, 0.75, na.rm = TRUE)
  iqr <- q3 - q1
  lower <- q1 - 1.5 * iqr
  upper <- q3 + 1.5 * iqr
  x < lower | x > upper
}

# Applications
iris |>
  mutate(across(where(is.numeric), detect_outliers, .names = "{.col}_out")) |>
  select(ends_with("_out")) |>
  summarise(across(everything(), sum))

# Sepal.Length_out Sepal.Width_out Petal.Length_out  Petal.Width_out
#        0                4               0                0

(2) 3σ Method (for normally distributed data only)

detect_outliers_z <- function(x, threshold = 3) {
  z <- (x - mean(x, na.rm = TRUE)) / sd(x, na.rm = TRUE)
  abs(z) > threshold
}

9. Automated EDA

(1) DataExplorer

install.packages("DataExplorer")
library(DataExplorer)

# 1. One-Line Report
create_report(iris, output_file = "eda_report.html")

# 2. Report Contents
plot_intro(iris)        # Data Overview
plot_missing(iris)      # Missing values
plot_histogram(iris)    # Histogram of All Numeric Columns
plot_density(iris)      # Density Plot
plot_bar(iris)          # Bar Chart by Category
plot_boxplot(iris)      # Box-and-Whisker Plot
plot_scatterplot(iris)  # Scatter Plot Matrix
plot_correlation(iris)  # Related Matrices

(2) SmartEDA

install.packages("SmartEDA")
library(SmartEDA)

ExpReport(iris, op_file = "smarteda_report.html")
ExpNumStat(iris)         # Statistical Data
ExpCatStat(iris)         # Categorical Statistics

10. Complete Example: EDA Using the Iris and MPG Datasets

Below is an example of a complete workflow that ties together all the EDA concepts covered in this lesson.

▶ Example: Complete EDA for the Iris and MPG Datasets

R 📖 Display only

# ============================================
# iris + mpg Both Datasets Complete EDA
# Features: 5-step EDA Complete Process
# ============================================

library(ggplot2)
library(dplyr)
library(skimr)
library(naniar)
library(GGally)

# 1. Step 1: Data Shape
cat("=== Step 1: Data Shape ===\n")
cat("iris:", nrow(iris), "rows x", ncol(iris), "cols\n")
cat("mpg:", nrow(mpg), "rows x", ncol(mpg), "cols\n\n")

# 2. Step 2: Type + Abstract
cat("=== Step 2: Type and Abstract ===\n")
cat("iris Column Types:\n")
print(sapply(iris, class))
cat("\nmpg Column Types:\n")
print(sapply(mpg, class))

cat("\niris Summary:\n")
print(summary(iris))

cat("\nmpg Summary:\n")
print(summary(mpg |> select(displ, year, cyl, cty, hwy)))

# 3. Step 3: Missing Value Analysis
cat("\n=== Step 3: Missing values ===\n")
cat("iris Missing values:", sum(is.na(iris)), "\n")
cat("mpg Missing values:", sum(is.na(mpg)), "\n\n")

# Simulation with Missing Data
set.seed(42)
iris_na <- iris
iris_na[sample(150, 20), 1] <- NA
iris_na[sample(150, 10), 3] <- NA

vis_miss(iris_na, cluster = TRUE)
gg_miss_upset(iris_na)

# 4. Step 4: Distribution Visualization
cat("\n=== Step 4: Distribution Visualization ===\n")

# 4.1 Distribution of Numeric Variables
iris |>
  select(-Species) |>
  pivot_longer(everything(), names_to = "variable", values_to = "value") |>
  ggplot(aes(x = value)) +
  geom_histogram(bins = 20, fill = "skyblue", color = "white") +
  facet_wrap(~ variable, scales = "free") +
  labs(title = "iris 4 Distribution of a Numeric Variable")

# 4.2 By Component
ggplot(iris, aes(x = Species, y = Sepal.Length, fill = Species)) +
  geom_boxplot() +
  labs(title = "Sepal.Length Box-and-Whisker Plot by Species")

# 4.3 Scatter Plot Matrix
ggpairs(iris, aes(color = Species))

# 5. Step 5: Correlation Matrix
cat("\n=== Step 5: Correlation Matrix ===\n")
cor_matrix <- cor(iris |> select(-Species))
print(round(cor_matrix, 2))

library(corrplot)
corrplot(cor_matrix, method = "circle", type = "upper",
         addCoef.col = "black", number.cex = 0.8)

# 6. Outlier Detection
cat("\n=== Step 6: Outlier Detection ===\n")
detect_outliers <- function(x) {
  q1 <- quantile(x, 0.25, na.rm = TRUE)
  q3 <- quantile(x, 0.75, na.rm = TRUE)
  iqr <- q3 - q1
  lower <- q1 - 1.5 * iqr
  upper <- q3 + 1.5 * iqr
  sum(x < lower | x > upper, na.rm = TRUE)
}

iris |>
  summarise(across(where(is.numeric), detect_outliers)) |>
  print()

# 7. Automated Reports
cat("\n=== Step 7: Automated Reports ===\n")
library(DataExplorer)
create_report(iris, output_file = "iris_eda_report.html")
cat("iris_eda_report.html Generated\n")
create_report(mpg, output_file = "mpg_eda_report.html")
cat("mpg_eda_report.html Generated\n")

# 8. Summary of Business Insights
cat("\n=== Business Insights ===\n")
cat("1. iris 4 None of the numerical variables have missing values.\n")
cat("2. Setosa Species' Petal.Length Significantly smaller than the other two\n")
cat("3. Petal.Length and Petal.Width Highly correlated (0.96)\n")
cat("4. Sepal.Width The distribution is approximately normal\n")
cat("5. mpg 11 Numeric variables, 38 Vehicle models\n")

68 logic lines (exceeds 40-line limit, display only)

Expected Output: 30+ pages of EDA HTML reports + multiple statistical charts.

❓ FAQ

Q What is the difference between EDA and formal analysis?

A EDA stands for "exploration" (no preconceived assumptions; let the data speak for itself), while formal analysis stands for "validation" (start with a hypothesis, then use statistical tests). Perform EDA first, then build a model.

Q How should I handle missing values?

A There are three strategies: ① Delete (rows/columns containing NA) ② Imputation (mean/median/model-based prediction) ③ Flag (add an “is_missing” column). There is no standard answer; it depends on the importance of the data.

Q How should outliers be handled?

A There are four strategies: ① Delete (if confirmed to be an error) ② Replace (with the median) ③ Truncate (Winsorize) ④ Retain (using a robust method). First, determine whether it is a "business anomaly."

Q Correlation ≠ Causation!

A Correlation only indicates that "two variables change together"; it does not indicate which one causes the other. Causation requires experimentation (A/B testing) or in-depth analysis.

Q How large are DataExplorer reports?

A 30–50 columns of data generate 20–50 pages of HTML, including all charts and statistics. A complete report takes 5–10 seconds.

Q When should a logarithmic transformation be used?

A When the distribution is skewed (right-skewed)

📖 Summary

The 5-step EDA method: Shape → Type → Missing → Distribution → Relationships
Shape: dim nrow ncol str Depending on the data size
Type: summary skimr View statistics for each variable
Missing: colSums(is.na) Count + naniar::vis_miss Visualization
Distribution: hist / boxplot / density (univariate) + pairs / ggpairs (multivariate)
Related: cor() + corrplot Visualization of the correlation matrix
Outliers: IQR method (robust) < Q1 - 1.5×IQR or > Q3 + 1.5×IQR
Automation: DataExplorer::create_report() Generate a complete HTML report in a single line
EDA is a must before modeling—no EDA, no modeling
Correlation ≠ Causation—Only experiments can prove causation

📝 Exercises

Basic Exercise: Perform a complete EDA on the built-in mtcars dataset in R: ① Shape ② Summary ③ Missing values ④ Histograms for 4 numerical variables ⑤ Correlation matrix. Save a screenshot.
Basic Exercise: Simulate a data frame (1,000 rows, 5 columns) containing 20% missing values. Use naniar::vis_miss() to visualize the missing value patterns and compare the results of three handling strategies (deletion, mean imputation, and median imputation).
Basic Exercise: Use iris to plot the correlation matrix (corrplot) for the four numerical variables, identify the two most highly correlated pairs of variables, and verify the results using a scatter plot.
Advanced Exercise: Perform a complete EDA using the mpg dataset: ① Shape ② Summary ③ Box plots grouped by class ④ Scatter plot matrix by class ⑤ Correlation matrix ⑥ Automated report. Save screenshots.
Challenge: Construct a mixed dataset containing 10,000 rows and 20 columns (including numerical, categorical, missing, and outlier values), use DataExplorer::create_report() to generate a comprehensive EDA report (30+ pages in HTML), and write a summary of business insights (500 characters) based on the report.

Web-Tutorial Tech Team

A team of developers maintaining programming tutorials. Each tutorial is written and reviewed by developers with expertise in that field. We work to keep our content accurate and reliable — if you spot an issue, please let us know.

About Us