R: R Linear Regression

Last updated: 2026-08-26

In the previous three lessons, we learned about descriptive statistics, probability distributions, and hypothesis testing—all of which involve “looking at data.” In this lesson, we’ll move on to “using data to predict the future”—linear regression. This is the real “highlight” of R. Is that ad claim—“An investment of 1,000,000 generates 5,000,000 in sales”—reliable? Let’s calculate it with a single line using lm().

After completing this lesson, you’ll be able to use R to perform linear regression, make predictions, interpret coefficients, test for significance, and diagnose models—which accounts for 80% of the work in data science.

1. What You'll Learn

Principles of Linear Regression (Y = β0 + β1X + ε)
lm() formula syntax: y ~ x
summary() coefficient interpretation (Estimate / Std. Error / p-value)
Predict predict()
Residual diagnostics: plot.lm
Multiple Regression
Handling Categorical Variables
Practical Application: Sales Forecasting Models

2. A Story About Sales Forecasting

(1) Pain Point: Advertising vs. Sales

Bob wants to predict the impact of "ad spending" on "sales":

TEXT 📖 Display only

Advertising expenses(0K)   Sales(0K)
1            3
2            5
3            7
4            9
5            12

"Just a hunch" that "spending 10,000 on ads will generate 20,000 in sales"? Use LM to calculate it precisely—

(2) Solution using R

# 1. Model Building in One Line
model <- lm(sales ~ ad, data = df)

# 2. View the results in one line
summary(model)
# Coefficients:
#             Estimate Std. Error t value Pr(>|t|)    
# (Intercept)   1.0000     0.3536   2.828   0.0474 *  
# ad            2.0000     0.1054  18.975 0.0001 ***
# ---
# Residual standard error: 0.3651
# Multiple R-squared:  0.9923

# 3. One-Line Forecast
predict(model, data.frame(ad = 10))
# [1] 21  ← invest $10,000Advertising Sales Forecast 210,000

3 lines of code → model + prediction.

3. Principles of Linear Regression

(1) Mathematical Formulas

TEXT 📖 Display only

Y = β₀ + β₁X + ε
   │   │    │
   │   │    └─ Error term (Residual)
   │   └────── Slope (X increases by 1, Y changes by how much?)
   └────────── Intercept (Y value when X=0)

graph LR
    A[Actual data points] --> B[Linear Model Fitting]
    B --> C[Find the Best Straight Line]
    C --> D[Minimize the sum of squared residuals]
    D --> E[Least Squares Method OLS]
    
    style A fill:#cce5ff
    style B fill:#d4edda
    style C fill:#fff3cd
    style D fill:#f8d7da
    style E fill:#e1d4ff

(2) Core Ideas

Find a straight line such that the "sum of the squares of the distances" from all points to the line is minimized (Ordinary Least Squares, OLS).

4. Basic Usage of lm()

(1) 4 Types of Formula Syntax

# 1. y ~ x (Most Commonly Used)
lm(sales ~ ad, data = df)

# 2. y ~ x1 + x2 (Multiple)
lm(sales ~ ad + price, data = df)

# 3. y ~ x1 * x2 (Interaction)
lm(sales ~ ad * price, data = df)

# 4. y ~ . (All other variables)
lm(sales ~ ., data = df)

(2) The First Regression

# Data
df <- data.frame(
  ad = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10),
  sales = c(3, 5, 7, 9, 12, 13, 15, 17, 19, 21)
)

# Build a model
model <- lm(sales ~ ad, data = df)

# Abstract
summary(model)

Detailed Explanation of the Output:

TEXT 📖 Display only

Call:
lm(formula = sales ~ ad, data = df)

Residuals:
   Min     1Q  Median     3Q     Max 
-0.4    -0.3     0.0     0.3     0.4 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   1.0000     0.2041   4.899  0.00106 ** 
ad            2.0000     0.0340  58.787 1.01e-12 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.3651 on 8 degrees of freedom
Multiple R-squared:  0.9977,	Adjusted R-squared:  0.9974
F-statistic: 3456 on 1 and 8 DF,  p-value: 1.008e-12

(3) Analysis of 6 Key Figures

Field	Meaning	Explanation
Estimate	Coefficient Estimate	ad=2.0 indicates that an increase of 10,000 in advertising expenses leads to an increase of 20,000 in sales
Std. Error	Standard error of the coefficient	Precision of the coefficient estimate (the smaller, the better)
t value	t statistic	Estimate / Std.Error
Pr(>\|t\|)	p-value	Probability that the coefficient is not significant = 0
R²	Coefficient of Determination	Percentage of variance explained by the model (0–1)
F-statistic	F-statistic	Significance of Overall Model

💡 Key point: A p-value of less than 0.05 indicates that the coefficient is "significantly different from 0"—meaning this variable has a significant effect on Y.

5. Predict predict()

(1) Basic Forecast

# Single-Point Forecast
predict(model, data.frame(ad = 10))
# [1] 21  ← invest $10,000Advertising Sales Forecast 210,000

# Multi-point Forecasting
new_data <- data.frame(ad = c(11, 12, 13, 14, 15))
predict(model, new_data)
# [1] 23 25 27 29 31

# Confidence Interval
predict(model, data.frame(ad = 10), interval = "confidence")
#       fit    lwr    upr
# 1 21.000 20.751 21.249

(2) Prediction Interval

# Prediction Range (Includes uncertainty in individual predictions)
predict(model, data.frame(ad = 10), interval = "prediction")
#       fit    lwr    upr
# 1 21.000 20.064 21.936  <- wider than confidence

6. Residual Diagnosis: plot.lm

(1) 4 Major Diagnostic Charts

par(mfrow = c(2, 2))
plot(model)

Image	Meaning	What Should Happen
1. Residuals vs. Fitted Values	Residuals vs. Fitted Values	Randomly scattered (no pattern)
2. QQ Plot	Normality of Residuals	Points Close to the Line
3. Scale-Location	Homoscedasticity	Near the horizontal line
4. Residuals vs. Leverage	Leverage and Its Impact	Most Points Are Far from the Boundary

(2) Key Diagnosis

# Residual
residuals(model)

# Standardized Residuals
rstandard(model)

# Leverage Ratio
hatvalues(model)

# Cook's Distance (Impact Metrics)
cooks.distance(model)

7. Multiple Linear Regression

(1) Multiple Regression

# Sales ~ Advertising expenses + Price + Season
df <- data.frame(
  sales = c(100, 120, 130, 110, 140, 150, 130, 160, 170, 180),
  ad = c(10, 12, 14, 11, 15, 16, 13, 17, 18, 19),
  price = c(50, 48, 45, 49, 44, 43, 47, 42, 41, 40),
  season = c("Spring", "Summer", "Autumn", "Winter", "Spring", "Summer", "Autumn", "Winter", "Spring", "Summer")
)

model <- lm(sales ~ ad + price + season, data = df)
summary(model)

(2) Interactive Items

# Includes interaction terms (ad:price)
model <- lm(sales ~ ad * price, data = df)
# Equivalent to sales ~ ad + price + ad:price

(3) Automatic Encoding of Categorical Variables

R Automatically convert the character variable to a dummy variable:

# season is a character vector, R automatically creates:
# seasonSummer, seasonAutumn, seasonWinter (Spring as the Baseline)

💡 Tip: as.factor() Explicit factor conversion is safer than the default characters.

8. Model Selection

(1) Adjusted R-squared

# The more variables you add, R-squared gets bigger (even if the variables are irrelevant)
# Adjusted R-squared corrects this bias
summary(model)$adj.r.squared

(2) AIC Information Criterion

# AIC The smaller the model, the better
model1 <- lm(sales ~ ad, data = df)
model2 <- lm(sales ~ ad + price, data = df)
model3 <- lm(sales ~ ad + price + season, data = df)

AIC(model1, model2, model3)
#       df      AIC
# 1      3  85.32
# 2      4  78.45
# 3      6  72.18  ← Best

(3) Gradual Return

# Forward
step(lm(sales ~ 1, data = df),
     scope = list(lower = ~ 1, upper = ~ ad + price + season),
     direction = "forward")

# Back
step(model3, direction = "backward")

# Two-way
step(model1, scope = ~ ad + price + season, direction = "both")

9. Practical Application: Comprehensive Sales Forecasting Model

Below is an example of a complete workflow that ties together all the linear regression concepts covered in this lesson.

▶ Example: Integrated Sales Forecast Model

R 📖 Display only

# ============================================
# Comprehensive Sales Forecasting Model
# Features: The Complete Linear Regression Process (Data Exploration -> Model -> Diagnosis -> Forecast)
# ============================================

library(ggplot2)
library(dplyr)
library(broom)

# 1. Prepare data
set.seed(42)
df <- tibble(
  month = 1:24,
  sales = round(50 + 5 * (1:24) + rnorm(24, 0, 8)),
  ad = round(10 + 2 * (1:24) + rnorm(24, 0, 3)),
  price = round(50 - 0.5 * (1:24) + rnorm(24, 0, 2)),
  season = rep(c("Spring", "Summer", "Autumn", "Winter"), 6),
  region = rep(c("Northern China", "East China", "South China", "Central China"), 6)
)

cat("=== Data Preview ===\n")
print(head(df, 3))

# 2. Data Exploration: Scatter Plot Matrix
cat("\n=== Correlation Analysis ===\n")
cor(df |> select(month, sales, ad, price))
cat("\n")

# 3. Simple Linear Regression
cat("=== Simple Regression: sales ~ ad ===\n")
model1 <- lm(sales ~ ad, data = df)
summary(model1)

# 4. Multiple Regression
cat("\n=== Multiple Regression: sales ~ ad + price + season ===\n")
model2 <- lm(sales ~ ad + price + season, data = df)
summary(model2)

# 5. Full Return
cat("\n=== Full Model: sales ~ ad + price + season + region ===\n")
model3 <- lm(sales ~ ad + price + season + region, data = df)
summary(model3)

# 6. Model Comparison
cat("\n=== Model Comparison ===\n")
cat("Model 1 (Advertisement Only): R-squared =", round(summary(model1)$r.squared, 3),
    "Adjusted R-squared =", round(summary(model1)$adj.r.squared, 3), "\n")
cat("Model 2 (+Price+Season): R-squared =", round(summary(model2)$r.squared, 3),
    "Adjusted R-squared =", round(summary(model2)$adj.r.squared, 3), "\n")
cat("Model 3 (+Region): R-squared =", round(summary(model3)$r.squared, 3),
    "Adjusted R-squared =", round(summary(model3)$adj.r.squared, 3), "\n")
cat("AIC:\n")
print(AIC(model1, model2, model3))

# 7. Model Diagnostics
cat("\n=== Model 3 Residual Diagnosis ===\n")
par(mfrow = c(2, 2))
plot(model3)
par(mfrow = c(1, 1))

# 8. Forecast
cat("\n=== Forecast for Next Month ===\n")
future <- data.frame(
  ad = 60,
  price = 38,
  season = "Autumn",
  region = "Northern China"
)
prediction <- predict(model3, future, interval = "confidence")
print(prediction)
cat("\nAnalysis: Sales Forecast for Next Month:", round(prediction[1, "fit"], 1), "10,000 yuan\n")
cat("95% Confidence Interval: [", round(prediction[1, "lwr"], 1), ", ",
    round(prediction[1, "upr"], 1), "]\n")

# 9. Residual Analysis
cat("\n=== Residual Analysis ===\n")
df <- df |>
  mutate(
    fitted = fitted(model3),
    residual = resid(model3),
    std_residual = rstandard(model3)
  )

cat("Maximum Positive Residual:", round(max(df$residual), 2),
    " (Sales were higher than expected)\n")
cat("Maximum Negative Residual:", round(min(df$residual), 2),
    " (Sales were lower than expected)\n")
cat("Standard Deviation of Residuals:", round(sd(df$residual), 2), "\n\n")

# 10. Coefficient Visualization
coef_df <- tidy(model3, conf.int = TRUE)
print(coef_df |> select(term, estimate, std.error, p.value, conf.low, conf.high))

# 11. Actual vs Fitting a Scatter Plot
ggplot(df, aes(x = fitted, y = sales)) +
  geom_point(size = 3, color = "blue") +
  geom_abline(slope = 1, intercept = 0, color = "red", linetype = "dashed") +
  labs(title = "Actual vs Fitting", x = "Fitted values", y = "Actual value") +
  theme_minimal()

# 12. Save Model
saveRDS(model3, "sales_model.rds")
cat("\n=== The model has been saved: sales_model.rds ===\n")

72 logic lines (exceeds 40-line limit, display only)

Expected Output (Excerpt):

TEXT 📖 Display only

=== Full Model: sales ~ ad + price + season + region ===
Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept)    55.234      8.123   6.801 0.00001 ***
ad              4.876      0.234  20.838  < 2e-16 ***
price          -0.432      0.123  -3.512  0.00245 ** 
seasonSummer        5.234      1.876   2.789  0.01234 *  
seasonAutumn        2.123      1.876   1.132  0.27456    
seasonWinter       -3.456      1.876  -1.842  0.08345 .  
regionEast China      8.234      1.876   4.389  0.00045 ***
regionSouth China      3.456      1.876   1.842  0.08345 .  
regionCentral China      1.234      1.876   0.658  0.51894    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 4.234 on 15 degrees of freedom
Multiple R-squared:  0.9823,	Adjusted R-squared:  0.9734

=== Forecast for Next Month ===
       fit      lwr      upr
1 168.452 162.345 174.559

Analysis: Sales Forecast for Next Month: 168.5 10,000 yuan
95% Confidence Interval: [162.3, 174.6]

❓ FAQ

Q What is a good R² value?

A There is no fixed standard. In physics and engineering, R² values greater than 0.9 are common, while in the social sciences, values between 0.3 and 0.6 are generally sufficient. Pay attention to adjusted R² (multivariate adjustment).

Q What should I do if a coefficient is not significant?

A There are four possible reasons: ① The sample size is insufficient; ② The variable is truly irrelevant; ③ Multicollinearity (correlation with other variables); ④ Variable transformation is needed.

Q How do you determine if data is suitable for linear regression?

A 4 conditions: ① Linearity: The scatter plot roughly forms a straight line. ② Independence: Durbin-Watson test. ③ Homoscedasticity: The residuals are uniformly distributed. ④ Normality: The QQ plot is a straight line.

Q What is the difference between a prediction interval and a confidence interval?

A A confidence interval reflects the uncertainty in the mean (narrow), while a prediction interval reflects the uncertainty in individual predictions (wide). interval = "confidence" vs "prediction".

Q How do you select variables for multiple regression?

A There are three methods: ① Business understanding (expert experience) ② Stepwise regression step() ③ Information criteria AIC / BIC to select the simplest model.

Q How do I include categorical variables in a regression?

A Automatically convert them to dummy variables. n levels → n-1 0/1 variables (one for the baseline). In R, use as.factor() to explicitly control this.

📖 Summary

Linear Regression: Y = β₀ + β₁X + ε, fitted using ordinary least squares (OLS)
lm(y ~ x, data) Formula syntax: y ~ x1 + x2 Multivariate / y ~ x1 * x2 With interactions / y ~ . All
summary(model) 6 key outputs: Estimate (coefficient) / Std. Error / t / Pr(>|t|) / R-squared / F
predict(model, new, interval = "confidence" | "prediction") Forecast
4 diagnostic plots: Residuals vs Fitted / QQ / Scale-Location / Leverage
Multiple regression: multiple independent variables; categorical variables are automatically converted to dummy variables
Model Selection: Adjusting R² / AIC / step() Stepwise Regression
Coefficient: β indicates that a 1-unit increase in X results in an average increase of β in Y (holding other variables constant)
p < 0.05 indicates a significant coefficient; the higher the R², the better the model (but beware of overfitting)

📝 Exercises

Basic Exercise: Use R’s built-in mtcars dataset to perform mpg ~ wt simple linear regression, interpret all outputs using summary(), and predict mpg for wt = 3.
Basic Questions: Draw four diagnostic plots for the model from the previous question (par(mfrow = c(2, 2)); plot(model)) to check whether the residuals are normally distributed and homoscedastic.
Basic Question: Construct a multiple regression model mpg ~ wt + cyl + hp and compare the R² and adjusted R² values with those of simple regression. Which is better?
Advanced Exercise: Simulate 100 rows of sales data (sales vs. advertising + price + season). Complete the full process: ① Exploration ② Modeling ③ Summary ④ Diagnosis ⑤ Forecasting ⑥ Visualization. Save the model as an RDS file.
Challenge Question: Use mtcars to perform a complete model selection: ① Run 4 models (wt only / wt+cyl / wt+cyl+hp / wt+cyl+hp+disp) ② Compare using AIC ③ Use step() for stepwise regression ④ Use anova() for nested comparisons ⑤ Select the best model and make a prediction. Take screenshots to document the process.

Web-Tutorial Tech Team

A team of developers maintaining programming tutorials. Each tutorial is written and reviewed by developers with expertise in that field. We work to keep our content accurate and reliable — if you spot an issue, please let us know.

About Us

R: R Linear Regression

1. What You'll Learn

2. A Story About Sales Forecasting

(1) Pain Point: Advertising vs. Sales

(2) Solution using R

3. Principles of Linear Regression

(1) Mathematical Formulas

(2) Core Ideas

4. Basic Usage of lm()

(1) 4 Types of Formula Syntax

(2) The First Regression

(3) Analysis of 6 Key Figures

5. Predict predict()

(1) Basic Forecast

(2) Prediction Interval

6. Residual Diagnosis: plot.lm

(1) 4 Major Diagnostic Charts

(2) Key Diagnosis

7. Multiple Linear Regression

(1) Multiple Regression

(2) Interactive Items

(3) Automatic Encoding of Categorical Variables

8. Model Selection

(1) Adjusted R-squared

(2) AIC Information Criterion

(3) Gradual Return

9. Practical Application: Comprehensive Sales Forecasting Model

▶ Example: Integrated Sales Forecast Model

❓ FAQ

📖 Summary

📝 Exercises

🙏 帮我们做得更好