R: R Logistic Regression

Q: Logistic regression vs. linear regression?

If Y is continuous, use lm() ; if Y is 0/1, use glm(family = binomial) .

Q: How do you interpret the odds ratio?

OR = exp(coefficient). An OR > 1 indicates an increased odds (more likely to drop out), while an OR < 1 indicates a decreased odds (less likely). An OR of 1.5 means the odds are increased by 50%.

Q: How do I choose a threshold?

The default is 0.5, but it should be adjusted based on business needs. Use a low threshold (0.2–0.3) when the cost of false negatives is high (e.g., healthcare), and a high threshold (0.7–0.8) when the cost of false positives is high (e.g., spam).

Q: What is a good AUC value?

0.5 = random; 0.7–0.8 = average; 0.8–0.9 = good; 0.9+ = excellent.

Q: What is the recommended split between the training and test sets?

Common splits are 70/30 or 80/20. For large datasets, a 90/10 split is also acceptable; for small datasets, use cross-validation (caret’s trainControl ).

Q: What should I use for multi-class classification?

Use nnet::multinom() (multi-class logistic regression) or caret (auto-model selection). For three or more classes, consider Random Forest.

Last updated: 2026-08-26

In the previous lesson, we learned about linear regression—predicting continuous values (sales, temperature). But 80% of real-world problems involve “whether” questions—will a user make a purchase? Will an email be clicked? Will a loan go into default? In this lesson, we’ll learn about logistic regression—predicting probabilities and categories.

After completing this lesson, you’ll be able to use R to perform: customer churn prediction, email click-through rate prediction, credit scoring, and A/B testing modeling—covering 60% of the classification problems in data science.

1. What You'll Learn

Principles of Logistic Regression (logit transformation + sigmoid)
glm() binomial distribution family
Interpretation of Coefficients (Odds Ratio)
predict type="response" predicted probability
ROC Curve + AUC Evaluation
Caret training/test set split
Threshold selection (default 0.5)
Practical Application: Customer Churn Prediction

2. A Story About Customer Churn Prediction

(1) Pain Point: Which customers are likely to churn?

Xiao Zhao is a data analyst at a SaaS company. His manager asked, "Which customers are likely to churn next month? How can we retain them before that happens?"

Data: 1,000 customers; features = age, duration of use, number of customer service interactions; label = whether the customer has churned.

(2) Solution using R

# 1. Model Building in One Line
model <- glm(churn ~ age + usage + support_calls,
             data = df, family = binomial)

# 2. Probability per row
df$prob <- predict(model, type = "response")

# 3. One-Line Evaluation
library(pROC)
roc_obj <- roc(df$churn, df$prob)
auc(roc_obj)  # [1] 0.85  ← Model Quality

3 lines of code → Churn prediction + model quality.

3. Principles of Logistic Regression

(1) Why is logistic regression needed?

graph TB
    A[Y Category 0/1] --> B[Linear regression is not applicable]
    B --> C[We need to map continuous values to 0-1]
    C --> D[Logit Transformation + Sigmoid]
    D --> E[Logistic Regression]
    
    style A fill:#fff3cd
    style B fill:#f8d7da
    style C fill:#d4edda
    style D fill:#cce5ff
    style E fill:#e1d4ff

Linear regression predicts continuous values, while logistic regression predicts probabilities (0–1).

(2) Mathematical Formulas

TEXT 📖 Display only

P(Y=1) = 1 / (1 + e^(-z))
       │      │      │
       │      │      └─ z = β₀ + β₁X₁ + β₂X₂ + ...
       │      └─ e Index
       └─ Sigmoid Function (Range 0-1)

Equivalent notation (logit transformation):

TEXT 📖 Display only

log(P/(1-P)) = β₀ + β₁X₁ + β₂X₂ + ...

odds = P / (1-P) = probability of success / probability of failure.

(3) Sigmoid Function

sigmoid <- function(x) 1 / (1 + exp(-x))

# Examples
sigmoid(0)     # 0.5
sigmoid(2)     # 0.881
sigmoid(-2)    # 0.119
sigmoid(10)    # 0.99995

4. Basic Usage of glm()

(1) Syntax

glm(formula, data, family = binomial)

Parameter	Meaning
`formula`	y ~ x1 + x2 (same as lm)
`data`	Data Frame
`family`	`binomial` (binary classification) / `gaussian` (linear) / `poisson` (counting)

(2) The First Logistic Regression

# Data
df <- data.frame(
  churn = c(0, 0, 0, 1, 1, 1, 0, 1, 0, 1),
  age = c(25, 30, 35, 50, 55, 60, 28, 58, 32, 65),
  usage = c(80, 70, 60, 30, 20, 10, 75, 15, 65, 5)
)

# Modeling
model <- glm(churn ~ age + usage, data = df, family = binomial)
summary(model)

Output:

TEXT 📖 Display only

Coefficients:
             Estimate Std. Error z value Pr(>|z|)    
(Intercept)   8.452      4.123   2.050   0.0403 *  
age           0.087      0.043   2.023   0.0431 *  
usage        -0.156      0.058  -2.690   0.0072 ** 
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(3) Analysis of the 6 Major Outputs

Field	Meaning	Explanation
Estimate	Coefficient (in logit space)	A positive coefficient → An increase in this variable increases the probability of churn
Std. Error	Standard Error	Coefficient Estimation Accuracy
z value	z statistic	Estimate / Std.Error
Pr(>\|z\|)	p-value	Probability that the coefficient is significantly ≠ 0
Null deviance	Zero-model deviance	Intercept-only model
Residual deviance	Residual deviance	The smaller the value, the better the fit

💡 Key: z value replaces the t-value (large-sample normal approximation); p < 0.05 indicates that the coefficient is significant.

5. Interpretation of Coefficients: Odds Ratio

(1) Why use the odds ratio?

The coefficients of logistic regression are in logit space, which is not intuitive. Convert them to odds ratios (OR) for interpretation:

# odds ratio = exp(Coefficient)
exp(coef(model))
# (Intercept)         age         usage
#   4670.123      1.091      0.856

(2) OR Interpretation

OR Value	Meaning
OR = 1	Variable has no effect
OR > 1	An increase in the variable increases the odds (the event is more likely to occur)
OR < 1	An increase in the variable reduces the odds (the event is less likely to occur)

# age's OR = 1.09
# Analysis: For each 1 year increase in age, churn odds increase by 9%

# usage's OR = 0.86
# Analysis: For each 1 unit increase in usage, churn odds decrease by 14%

(3) 95% Confidence Interval

exp(confint(model))
#                  2.5 %    97.5 %
# (Intercept)  12.345  12345.678
# age           1.002      1.198
# usage         0.745      0.978

CI does not include 1 → The coefficient is significant.

6. Forecasts

(1) The `type` parameter

# type = "response"  <- Probability (0-1) * Most Commonly Used
predict(model, type = "response")

# type = "link"  <- logit space (Default)
predict(model, type = "link")

(2) Real-World Forecasting

# Training Set Predictions
df$prob <- predict(model, type = "response")

# Threshold 0.5 to convert to category
df$pred <- ifelse(df$prob > 0.5, 1, 0)

# New Data Forecasts
new_customers <- data.frame(
  age = c(40, 55, 30),
  usage = c(50, 15, 80)
)
predict(model, new_customers, type = "response")
# [1] 0.123 0.876 0.045
# Analysis: Age 55, Usage 15, probability of customer churn 87.6%

(3) Threshold Selection

Threshold	Applicable Scenarios
0.5	Default, balancing precision and recall
0.3	Reduce false negatives (medical diagnoses, fraud)
0.7	Reduce false positives (spam)

# Custom Thresholds
df$pred <- ifelse(df$prob > 0.3, 1, 0)

7. Model Evaluation: ROC + AUC

(1) Confusion Matrix

library(caret)

# True vs Forecast
confusionMatrix(
  factor(df$pred),
  factor(df$churn),
  positive = "1"
)

#           Reference
# Prediction  0  1
#         0  50  5
#         1  10 35
# Accuracy: 0.85
# Sensitivity (Recall): 0.875
# Specificity: 0.833

(2) 4 Key Indicators

Metric	Formula	Meaning
Accuracy	(TP + TN) / Total	Overall Accuracy
Precision	TP / (TP+FP)	The proportion of cases predicted to be positive that are actually positive
Recall (Sensitivity)	TP / (TP+FN)	Proportion of actual positives correctly predicted
F1	2×P×R/(P+R)	Harmonic mean of P and R

(3) ROC Curve + AUC

library(pROC)

# Calculate ROC
roc_obj <- roc(df$churn, df$prob)
auc(roc_obj)
# [1] 0.92  ← The closer we get to 1 The better

# Draw ROC
plot(roc_obj, main = "ROC Curve", col = "blue", lwd = 2)
abline(a = 0, b = 1, lty = 2, col = "gray")  # Random Classification Baseline

# AUC Analysis
# 0.5-0.7: Poor
# 0.7-0.8: Fair
# 0.8-0.9: Good
# 0.9-1.0: Excellent

8. Training/Test Set Split

library(caret)

# Division 70% Training / 30% Test
set.seed(42)
train_index <- createDataPartition(df$churn, p = 0.7, list = FALSE)
train_data <- df[train_index, ]
test_data <- df[-train_index, ]

# Train a Model
model <- glm(churn ~ age + usage, data = train_data, family = binomial)

# Test Set Predictions
test_data$prob <- predict(model, test_data, type = "response")
test_data$pred <- ifelse(test_data$prob > 0.5, 1, 0)

# Evaluation
confusionMatrix(factor(test_data$pred), factor(test_data$churn), positive = "1")

# True AUC
roc_obj <- roc(test_data$churn, test_data$prob)
auc(roc_obj)

9. Multi-class (multinom)

# For multi-class classification multinom (requires nnet package)
install.packages("nnet")
library(nnet)

# 3 Categories
df <- data.frame(
  y = c("A", "A", "B", "B", "C", "C"),
  x = c(1, 2, 3, 4, 5, 6)
)

model <- multinom(y ~ x, data = df)
summary(model)

# Forecast
predict(model, type = "class")      # Category
predict(model, type = "probs")      # Probability

10. Hands-On: The Complete Customer Churn Prediction Process

Below is an example of a complete workflow that ties together all the logistic regression concepts covered in this lesson.

▶ Example: Predicting the Loss of 1,000 Customers

R 📖 Display only

# ============================================
# 1000 Customer Churn Prediction
# Features: The Complete Logistic Regression Workflow
# ============================================

library(ggplot2)
library(dplyr)
library(caret)
library(pROC)

# 1. Prepare data
set.seed(42)
n <- 1000
df <- tibble(
  age = round(rnorm(n, 40, 12)),
  usage = round(rnorm(n, 50, 20)),
  support_calls = sample(0:10, n, replace = TRUE),
  plan = sample(c("Basics", "Advanced", "Company"), n, replace = TRUE,
                prob = c(0.5, 0.3, 0.2))
) |>
  mutate(
    # Probability of attrition: Older, Use less, Many customer service calls -> Prone to churn
    logit_p = -3 + 0.03 * age - 0.05 * usage + 0.3 * support_calls,
    p = 1 / (1 + exp(-logit_p)),
    churn = rbinom(n, 1, p)
  ) |>
  select(-logit_p, -p)

cat("=== Data Preview ===\n")
print(head(df, 3))
cat("\nChurn rate:", round(mean(df$churn) * 100, 2), "%\n")

# 2. Training/Test Set Partitioning
set.seed(42)
train_index <- createDataPartition(df$churn, p = 0.7, list = FALSE)
train_data <- df[train_index, ]
test_data <- df[-train_index, ]

cat("\nTraining Set:", nrow(train_data), "rows\n")
cat("Test Set:", nrow(test_data), "rows\n\n")

# 3. Train a Model
model <- glm(churn ~ age + usage + support_calls + plan,
             data = train_data, family = binomial)
cat("=== Model Summary ===\n")
summary(model)

# 4. Explanation of Coefficients
cat("\n=== Coefficient odds ratio ===\n")
or_df <- tidy(model, conf.int = TRUE, exponentiate = TRUE)
print(or_df |> select(term, estimate, std.error, p.value, conf.low, conf.high))

# 5. Training Set Predictions
train_data$prob <- predict(model, type = "response")
train_data$pred <- ifelse(train_data$prob > 0.5, 1, 0)

# 6. Test Set Predictions
test_data$prob <- predict(model, test_data, type = "response")
test_data$pred <- ifelse(test_data$prob > 0.5, 1, 0)

# 7. Training Set Evaluation
cat("\n=== Training Set Evaluation ===\n")
train_cm <- confusionMatrix(factor(train_data$pred), factor(train_data$churn),
                             positive = "1")
print(train_cm)

# 8. Test Set Evaluation
cat("\n=== Test Set Evaluation ===\n")
test_cm <- confusionMatrix(factor(test_data$pred), factor(test_data$churn),
                            positive = "1")
print(test_cm)

# 9. ROC + AUC
cat("\n=== AUC ===\n")
roc_train <- roc(train_data$churn, train_data$prob)
roc_test <- roc(test_data$churn, test_data$prob)
cat("Training Set AUC:", round(auc(roc_train), 3), "\n")
cat("Test Set AUC:", round(auc(roc_test), 3), "\n")

# 10. Draw ROC
plot(roc_test, main = "ROC Curve", col = "blue", lwd = 2)
abline(a = 0, b = 1, lty = 2, col = "gray")
legend("bottomright",
       legend = paste0("AUC = ", round(auc(roc_test), 3)),
       col = "blue", lwd = 2)

# 11. Threshold Analysis
cat("\n=== Threshold Analysis ===\n")
thresholds <- seq(0.1, 0.9, by = 0.1)
threshold_results <- lapply(thresholds, function(t) {
  pred <- ifelse(test_data$prob > t, 1, 0)
  cm <- confusionMatrix(factor(pred), factor(test_data$churn), positive = "1")
  data.frame(
    threshold = t,
    precision = cm$byClass["Precision"],
    recall = cm$byClass["Sensitivity"],
    f1 = cm$byClass["F1"]
  )
}) |> bind_rows()
print(threshold_results)

# 12. Identify high-risk customers
cat("\n=== High-Risk Clients Top 5 (Probability of attrition > 70%) ===\n")
high_risk <- test_data |>
  filter(prob > 0.7) |>
  arrange(desc(prob)) |>
  head(5) |>
  select(age, usage, support_calls, plan, prob, pred)
print(high_risk)

# 13. Probability Distribution Chart
ggplot(test_data, aes(x = prob, fill = factor(churn))) +
  geom_histogram(bins = 30, alpha = 0.7, position = "identity") +
  labs(title = "Probability Distribution of Attrition (Sorted by True label)",
       x = "Predict the Probability of Churn", y = "Frequency", fill = "True label") +
  scale_fill_brewer(palette = "Set1", labels = c("Not churned", "Churned")) +
  theme_minimal()

# 14. Save Model
saveRDS(model, "churn_model.rds")
cat("\n=== The model has been saved: churn_model.rds ===\n")

85 logic lines (exceeds 40-line limit, display only)

Expected Output (Excerpt):

TEXT 📖 Display only

=== Training Set Evaluation ===
Confusion Matrix:
          Reference
Prediction   0   1
         0 525  42
         1  18 115
Accuracy: 0.91
Sensitivity: 0.733
Specificity: 0.967

=== Test Set Evaluation ===
Accuracy: 0.89
AUC = 0.92

❓ FAQ

Q Logistic regression vs. linear regression?

A If Y is continuous, use lm(); if Y is 0/1, use glm(family = binomial).

Q How do you interpret the odds ratio?

A OR = exp(coefficient). An OR > 1 indicates an increased odds (more likely to drop out), while an OR < 1 indicates a decreased odds (less likely). An OR of 1.5 means the odds are increased by 50%.

Q How do I choose a threshold?

A The default is 0.5, but it should be adjusted based on business needs. Use a low threshold (0.2–0.3) when the cost of false negatives is high (e.g., healthcare), and a high threshold (0.7–0.8) when the cost of false positives is high (e.g., spam).

Q What is a good AUC value?

A 0.5 = random; 0.7–0.8 = average; 0.8–0.9 = good; 0.9+ = excellent.

Q What is the recommended split between the training and test sets?

A Common splits are 70/30 or 80/20. For large datasets, a 90/10 split is also acceptable; for small datasets, use cross-validation (caret’s trainControl).

Q What should I use for multi-class classification?

A Use nnet::multinom() (multi-class logistic regression) or caret (auto-model selection). For three or more classes, consider Random Forest.

📖 Summary

Logistic Regression: Y is a 0/1 binary classification, and the output is a probability (not 0 or 1)
Mathematics: log(P/(1-P)) = β₀ + β₁X + ..., equivalent to P = sigmoid(z)
glm(y ~ x, family = binomial) The formula syntax is the same as lm
Interpretation of Coefficients: OR = exp(β); an OR > 1 indicates an increased odds of the event, while an OR < 1 indicates a decreased odds.
predict(model, type = "response") Predicted probability; type = "link" Predicted logit
ROC + AUC Evaluation: AUC 0.5–1.0 (0.5 = random, 1.0 = perfect)
Threshold: Default 0.5, adjust as needed (balancing false negatives vs. false positives)
Split the training and test sets 70/30 using caret::createDataPartition
Multiple Categories: nnet::multinom(y ~ x)

📝 Exercises

Basic Exercise: Create a data frame (200 rows, 4 columns: churn + age + usage + support_calls), build a model using glm, interpret all outputs using summary, and calculate the odds ratio using exp(coef(model)).
Basic Exercise: Using the model from the previous question, predict the training set using predict(type = "response"), plot a histogram of actual labels versus predicted probabilities, and verify that the logistic regression outputs probabilities (0–1).
Basic Exercise: Using the model from the previous question, plot the ROC curve using pROC::roc() and calculate the AUC. Verify that an AUC of 0.7–0.9 indicates a good model.
Advanced Problem: Use the mtcars dataset to construct a binary classification problem (vs represents the 0/1 engine type), build a model using glm(vs ~ mpg + wt, family = binomial), split the data into a 70/30 split, and calculate the AUC and confusion matrix for the test set.
Challenge: Complete Customer Churn Prediction—Simulate 1,000 customers (4 features + 1 label). Complete workflow: ① Exploration ② Partitioning ③ Modeling ④ Evaluation (Confusion Matrix + ROC + AUC) ⑤ Threshold Analysis ⑥ High-Risk Customer Identification ⑦ Probability Distribution Plot ⑧ Save Model. Save screenshots of the process.

Web-Tutorial Tech Team

A team of developers maintaining programming tutorials. Each tutorial is written and reviewed by developers with expertise in that field. We work to keep our content accurate and reliable — if you spot an issue, please let us know.

About Us

R: R Logistic Regression

1. What You'll Learn

2. A Story About Customer Churn Prediction

(1) Pain Point: Which customers are likely to churn?

(2) Solution using R

3. Principles of Logistic Regression

(1) Why is logistic regression needed?

(2) Mathematical Formulas

(3) Sigmoid Function

4. Basic Usage of glm()

(1) Syntax

(2) The First Logistic Regression

(3) Analysis of the 6 Major Outputs

5. Interpretation of Coefficients: Odds Ratio

(1) Why use the odds ratio?

(2) OR Interpretation

(3) 95% Confidence Interval

6. Forecasts

(1) The type parameter

(2) Real-World Forecasting

(3) Threshold Selection

7. Model Evaluation: ROC + AUC

(1) Confusion Matrix

(2) 4 Key Indicators

(3) ROC Curve + AUC

8. Training/Test Set Split

9. Multi-class (multinom)

10. Hands-On: The Complete Customer Churn Prediction Process

▶ Example: Predicting the Loss of 1,000 Customers

❓ FAQ

📖 Summary

📝 Exercises

🙏 帮我们做得更好

(1) The `type` parameter