R: R JSON and XML

Q: How do I convert null in JSON to R?

null → NA , true → TRUE , false → FALSE ; arrays → lists/data frames. jsonlite handles this automatically.

Last updated: 2026-08-26

In the previous lesson, we learned about CSV and Excel—but in the Web era, data is increasingly in JSON format (API responses, configuration files, NoSQL databases). In this lesson, we’ll learn the standard methods for reading and writing JSON in R: jsonlite, as well as how to handle XML data: xml2.

After completing this lesson, you’ll be able to flatten nested JSON returned by an API into a data frame in 5 seconds and write your own JSON configuration files.

1. What You'll Learn

What are JSON and XML, and when should they be used?
Installing the jsonlite package and 4 core functions
fromJSON: Reads JSON (flattens nested data)
toJSON outputs JSON
Hands-On Guide to Nested JSON (API Responses)
xml2: Read an XML file
JSON ↔ data.frame conversion
Hands-On: Batch API Data Extraction

2. The Story of an API Data Set

(1) Pain Point: The API returns nested JSON

Bob is building a weather app and needs to retrieve data from a weather API:

JSON

{
  "status": "ok",
  "data": {
    "city": "Beijing",
    "date": "2024-01-15",
    "forecast": [
      {"day": "Monday", "high": 5, "low": -3, "weather": "Sunny"},
      {"day": "Tuesday", "high": 7, "low": -1, "weather": "Cloudy"},
      {"day": "Wednesday", "high": 3, "low": -5, "weather": "Snowy"}
    ]
  }
}

He wanted to extract the "7-day weather forecast" and save it in R for analysis. In Python, it took 10 lines of code requests + json; in R, it was done in a single line—

(2) Solution using R

library(jsonlite)

# 1. Reading nested data with a single line of code JSON
weather <- fromJSON("https://api.weather.com/forecast?city=Beijing")

# 2. Extract 7 Weather Forecast(Nested → Data Frame)
forecast <- weather$data$forecast  # It's a data frame!

# 3. Write JSON Layout
config <- list(api_key = "xxx", cities = c("Beijing", "Shanghai"))
write_json(config, "config.json", pretty = TRUE)

Nested JSON in just 3 lines of code. That’s the “magic” of jsonlite.

graph LR
    A[API URL<br/>weather.com/forecast] --> B[jsonlite::fromJSON]
    B --> C{Nested JSON}
    C --> D[forecast Array<br/>Auto-rotate data.frame]
    C --> E[current Object<br/>Auto-rotate list]
    D --> F[dplyr Analysis<br/>7 Weather Forecast]
    D --> G[ggplot2 Drawing<br/>Trend Visualization]
    E --> H[Real-time Weather Data]

    style A fill:#cce5ff
    style B fill:#d4edda
    style C fill:#fff3cd
    style D fill:#f8d7da
    style E fill:#e1d4ff
    style F fill:#ffe1d4
    style G fill:#cce5ff
    style H fill:#d4edda

3. JSON vs. XML: When to Use Which?

graph TB
    subgraph JSON[JSON Faction]
        J1[Grammar: Key-value pairs Concise]
        J2[Type: Numbers/String/Boolean/Array/Object]
        J3[Analysis: Fast C Language Implementation]
        J4[R Package: jsonlite]
    end

    subgraph XML[XML Faction]
        X1[Grammar: Nested Tags Long-winded]
        X2[Type: All strings]
        X3[Analysis: slower]
        X4[R Package: xml2]
    end

    JSON --> J5[Web API / NoSQL / Profile]
    XML --> X5[Legacy Enterprise Systems / SOAP / RSS / SVG]

    style JSON fill:#d4edda
    style XML fill:#f8d7da

(1) Comparison of the Two Formats

Property	JSON	XML
Full name	JavaScript Object Notation	eXtensible Markup Language
Syntax	`{"key": "value"}`	`<key>value</key>`
Readability	Concise	Verbose
Data Type	Number, String, Boolean, null, Array, Object	All strings
Analysis Speed	Fast	Slower
Primary Use Cases	Web APIs, NoSQL, configuration files	Legacy enterprise systems, SOAP, documents (RSS, SVG)
R package	`jsonlite`	`xml2`

(2) JSON Example

JSON

{
  "name": "Alice",
  "age": 25,
  "is_student": true,
  "scores": [85, 90, 92],
  "address": {
    "city": "Beijing",
    "zip": "100000"
  }
}

(3) XML Example

XML

<student>
  <name>Alice</name>
  <age>25</age>
  <scores>
    <item>85</item>
    <item>90</item>
    <item>92</item>
  </scores>
  <address>
    <city>Beijing</city>
    <zip>100000</zip>
  </address>
</student>

💡 Tip: Almost all new projects use JSON—it’s concise, easy to read, and parses quickly. Only legacy enterprise systems and government data still use XML.

4. Four Core Functions of jsonlite

(1) Function Quick Reference Table

Function	Purpose
`fromJSON()`	JSON → R object (list / data.frame)
`toJSON()`	R object → JSON string
`read_json()`	Read JSON files/URLs
`write_json()`	Write a JSON file

(2) A Detailed Explanation of fromJSON

library(jsonlite)

# 1. Read JSON String
json_str <- '{"name": "Alice", "age": 25}'
fromJSON(json_str)
# $name
# [1] "Alice"
# 
# $age
# [1] 25

# 2. Read JSON Documents
df <- fromJSON("data.json")

# 3. Read URL(API)
weather <- fromJSON("https://api.example.com/weather?city=beijing")

# 4. Expand directly into a data frame
json_array <- '[{"name": "A", "age": 20}, {"name": "B", "age": 25}]'
fromJSON(json_array)
#   name age
# 1    A  20
# 2    B  25  ← Automatically Rotate Data Frames!

(3) Automatic flattening of nested JSON

nested <- '{
  "status": "ok",
  "data": {
    "city": "Beijing",
    "forecast": [
      {"day": "Monday", "high": 5, "low": -3},
      {"day": "Tuesday", "high": 7, "low": -1}
    ]
  }
}'

result <- fromJSON(nested)
str(result)
# List of 2
#  $ status: chr "ok"
#  $ data  : List of 2
#   ..$ city     : chr "Beijing"
#   ..$ forecast :'data.frame':	2 obs. of  3 variables:
#   .. ..$ day  : chr [1:2] "Monday" "Tuesday"
#   .. ..$ high: num [1:2] 5 7
#   .. ..$ low : num [1:2] -3 -1

# Automatically Convert Nested Arrays to DataFrames!
result$data$forecast
#   day high low
# 1 Monday    5  -3
# 2 Tuesday    7  -1

💡 Tip: The magic of jsonlite lies in its ability to automatically flatten nested arrays into data frames—in other languages (such as Python), you have to do this manually.

5. A Detailed Explanation of toJSON

(1) Basic Syntax

toJSON(x, pretty = FALSE, auto_unbox = FALSE, dataframe = "columns")

(2) Practical Application

# 1. Data Frame → JSON
df <- data.frame(name = c("A", "B"), age = c(20, 25))
toJSON(df)
# [{"name":"A","age":20},{"name":"B","age":25}]

# 2. Format the output(Indented)
toJSON(df, pretty = TRUE)
# [
#   {
#     "name": "A",
#     "age": 20
#   },
#   {
#     "name": "B",
#     "age": 25
#   }
# ]

# 3. List → JSON
config <- list(
  api_key = "secret123",
  cities = c("Beijing", "Shanghai", "Guangzhou"),
  options = list(timeout = 30, retry = 3)
)
toJSON(config, pretty = TRUE)
# {
#   "api_key": "secret123",
#   "cities": ["Beijing", "Shanghai", "Guangzhou"],
#   "options": {
#     "timeout": 30,
#     "retry": 3
#   }
# }

(3) Key Parameters

Parameter	Function	Default
`pretty`	Formatting (with indentation)	FALSE
`auto_unbox`	Automatically remove the array from vectors of length 1	FALSE
`dataframe`	Data frame display mode	"columns"

# auto_unbox:Prevent a single value from becoming an array
toJSON(list(name = "Alice", scores = c(85)))  # Default
# {"name":["Alice"],"scores":[85]}

toJSON(list(name = "Alice", scores = c(85)), auto_unbox = TRUE)
# {"name":"Alice","scores":85}  ← Do not convert single values to arrays

(4) write_json: Write to a file

# Write JSON Document
write_json(config, "config.json", pretty = TRUE)

# Write concisely JSON(No indentation)
write_json(data, "data.json")

# Append to the file(Read first, Merge, then Write)
old <- read_json("data.json", simplifyVector = FALSE)
new <- c(old, list(updated_at = Sys.time()))
write_json(new, "data.json", pretty = TRUE)

6. Hands-On: Batch API Data Extraction

(1) Actual API Calls

library(jsonlite)
library(dplyr)

# Call a public API(GitHub User Information)
user_info <- fromJSON("https://api.github.com/users/hadley")
str(user_info)
# List of 50+
#  $ login              : chr "hadley"
#  $ id                 : int 4192
#  $ name               : chr "Hadley Wickham"
#  $ company            : chr "@posit-pbc"
#  $ location           : chr "Houston, TX"
#  $ public_repos       : int 50
#  ...

# Extract Key Fields
info <- tibble(
  name = user_info$name,
  company = user_info$company,
  repos = user_info$public_repos,
  followers = user_info$followers
)
print(info)

(2) Batch capture of multiple users

# Retrieve multiple items in bulk GitHub Number of repositories per user
users <- c("hadley", "yihui", "jtleek", "rstudio")

repos_data <- lapply(users, function(user) {
  info <- fromJSON(paste0("https://api.github.com/users/", user))
  tibble(
    user = user,
    repos = info$public_repos,
    followers = info$followers,
    created_at = as.Date(info$created_at)
  )
}) |> bind_rows()

print(repos_data)

7. xml2: Reading XML Files

(1) Installation and Core Functions

install.packages("xml2")
library(xml2)

# 4 Core Functions
read_xml()        # Read XML Document
xml_find_all()    # XPath Query Node
xml_text()        # Extract Node Text
xml_attr()        # Retrieve Node Properties

(2) Hands-On: Reading RSS Feeds

# 1. Read RSS(XML Format)
rss <- read_xml("https://www.r-bloggers.com/feed")

# 2. Extract All <item> Node
items <- xml_find_all(rss, "//item")
cat("Found", length(items), "Articles\n")

# 3. Extract the title of each article,Link,Date
articles <- tibble(
  title = xml_text(xml_find_all(items, "./title")),
  link = xml_attr(xml_find_all(items, "./link"), "href"),
  pub_date = xml_text(xml_find_all(items, "./pubDate"))
)

print(head(articles, 3))

(3) Hands-On: Reading SVG (XML Images)

# SVG is an XML format
svg <- read_xml("logo.svg")
# Extract All <circle> cx, cy, r attributes
circles <- xml_find_all(svg, "//circle")
data.frame(
  cx = as.numeric(xml_attr(circles, "cx")),
  cy = as.numeric(xml_attr(circles, "cy")),
  r = as.numeric(xml_attr(circles, "r"))
)

8. Hands-On: Reading and Writing JSON Configuration Files

(1) Project Structure

TEXT 📖 Display only

project/
├── config.json          # Profile
├── R/
│   ├── main.R           # Main Program
│   └── utils.R          # Utility Functions
└── data/
    └── input.json       # Input Data

(2) Example of config.json

JSON

{
  "api": {
    "key": "your-api-key",
    "endpoint": "https://api.example.com",
    "timeout": 30
  },
  "cities": ["Beijing", "Shanghai", "Guangzhou", "Shenzhen"],
  "options": {
    "log_level": "info",
    "max_retries": 3
  }
}

(3) Loading the R program configuration

# Load Configuration
config <- read_json("config.json", simplifyVector = FALSE)
print(config$cities)
# [1] "Beijing" "Shanghai" "Guangzhou" "Shenzhen"

# Using the Configuration
for (city in config$cities) {
  url <- paste0(config$api$endpoint, "?city=", city, "&key=", config$api$key)
  data <- fromJSON(url)
  # Processing data...
}

# Update Configuration
config$options$log_level <- "debug"
write_json(config, "config.json", pretty = TRUE, auto_unbox = TRUE)

9. Complete Example: API Data Extraction + JSON Persistence

Below is an example of a complete workflow that ties together all the concepts covered in this lesson.

▶ Example: Scraping and Persisting Weather API Data

R 📖 Display only

# ============================================
# Weather API Data Extraction + Persistence
# Features:Simulated Crawling 4 City Weather,Save as JSON
# ============================================

library(jsonlite)
library(dplyr)

# 1. Simulation API Response(For use in actual projects fromJSON(url))
mock_api_response <- function(city) {
  set.seed(match(city, c("Beijing", "Shanghai", "Guangzhou", "Shenzhen")))
  list(
    status = "ok",
    data = list(
      city = city,
      date = Sys.Date(),
      current = list(
        temp = sample(0:30, 1),
        humidity = sample(40:90, 1),
        weather = sample(c("Sunny", "Cloudy", "Rainy", "Snowy"), 1)
      ),
      forecast = data.frame(
        day = c("Today", "Tomorrow", "The day after tomorrow", "The day after tomorrow"),
        high = sample(5:30, 4),
        low = sample(-5:15, 4),
        weather = sample(c("Sunny", "Cloudy", "Rainy", "Snowy"), 4, replace = TRUE),
        stringsAsFactors = FALSE
      )
    )
  )
}

# 2. Batch Scraping 4 City
cities <- c("Beijing", "Shanghai", "Guangzhou", "Shenzhen")
cat("=== Fetching 4 City Weather ===\n")
all_weather <- lapply(cities, mock_api_response)
names(all_weather) <- cities

# 3. Retrieve All Forecasts(Nested → Data Frame)
all_forecast <- bind_rows(lapply(all_weather, function(w) {
  fc <- w$data$forecast
  fc$city <- w$data$city
  fc
}))

cat("\n=== 4 The Future of Cities 4 Weather Forecast ===\n")
print(all_forecast)

# 4. Get the current weather(Nested → Data Frame)
current_weather <- bind_rows(lapply(all_weather, function(w) {
  cw <- w$data$current
  tibble(
    city = w$data$city,
    temp = cw$temp,
    humidity = cw$humidity,
    weather = cw$weather
  )
}))

cat("\n=== Current Weather ===\n")
print(current_weather)

# 5. Save as JSON
write_json(all_weather, "all_weather.json", pretty = TRUE, auto_unbox = TRUE)
cat("\n=== Saved all_weather.json ===\n")

# 6. Save as a compressed file JSON(Remove Indentation)
write_json(all_weather, "all_weather_compact.json", auto_unbox = TRUE)

# 7. Read back JSON for verification
reloaded <- read_json("all_weather.json", simplifyVector = FALSE)
cat("\n=== Read-Back Verification ===\n")
cat("Number of cities:", length(reloaded), "\n")
cat("Current Temperature in Beijing:", reloaded$Beijing$data$current$temp, "°C\n")

# 8. Write a concise version for a single city JSON Layout
config <- list(
  api_key = "secret-key-xxx",
  default_city = "Beijing",
  update_interval = 3600,
  enabled_cities = cities
)
write_json(config, "config.json", pretty = TRUE, auto_unbox = TRUE)
cat("\n=== The configuration file has been generated config.json ===\n")

# 9. Read the configuration file
loaded_config <- read_json("config.json", simplifyVector = FALSE)
cat("Default City:", loaded_config$default_city, "\n")
cat("Enable City:", paste(loaded_config$enabled_cities, collapse = ", "), "\n")

# 10. Use jsonlite Processing API Error
cat("\n=== Error Handling Examples ===\n")
error_response <- '{"status": "error", "message": "Invalid API key"}'
result <- tryCatch(
  {
    parsed <- fromJSON(error_response)
    if (parsed$status == "error") stop(parsed$message)
    parsed
  },
  error = function(e) {
    cat("API Error:", e$message, "\n")
    NULL
  }
)

77 logic lines (exceeds 40-line limit, display only)

Expected Output (Excerpt):

TEXT 📖 Display only

=== 4 The Future of Cities 4 Weather Forecast ===
   day high low weather city
1 Today    12  -3      Sunny  Beijing
2 Tomorrow    15   0     Cloudy  Beijing
3 The day after tomorrow     8  -5      Snowy  Beijing
...

=== Current Weather ===
# A tibble: 4 × 4
  city   temp humidity weather
  <chr> <int>    <int> <chr>  
1 Beijing     18       65 Sunny     
2 Shanghai     22       78 Cloudy   
3 Guangzhou     28       85 Rainy     
4 Shenzhen     26       72 Cloudy

❓ FAQ

Q How do I choose between JSON and CSV?

A CSV is suitable for structured tables (one record per row), while JSON is suitable for nested data (including arrays, objects, and hierarchies) or Web APIs.

Q How do I flatten deeply nested JSON?

A jsonlite automatically converts arrays to data frames. For multi-level nesting, use purrr::flatten() to flatten it step by step, or tidyr::unnest() to break it down.

Q How do I convert a list returned by fromJSON into a data.frame?

A Nested arrays are automatically converted to data.frames; for plain lists, use as.data.frame() or dplyr::bind_rows() to force the conversion.

Q Does write_json produce garbled Chinese characters?

A The default encoding is UTF-8. In R 4.x, Chinese characters typically do not appear garbled. If they do appear garbled, add Encoding = "UTF-8" or use saveRDS() (R's native format).

Q Which should I choose, the xml2 or XML package?

A Use xml2 for new code (C++ backend, simple API). The XML package can still be used for legacy code, but it is no longer maintained.

Q How do I convert null in JSON to R?

A null → NA, true → TRUE, false → FALSE; arrays → lists/data frames. jsonlite handles this automatically.

📖 Summary

JSON is the Web API standard format, while XML is still used in legacy enterprise systems, RSS, and SVG.
jsonlite is the standard for handling JSON in R: relies solely on R, automatically flattens nested data, and offers good performance
4 core functions: fromJSON / toJSON / read_json / write_json
The Magic of jsonlite: Nested arrays are automatically converted to data.frames (in other languages, you have to flatten them manually)
toJSON() Key parameters: pretty (formatting), auto_unbox (convert single values to non-array format)
xml2 Read XML: 4 functions read_xml xml_find_all xml_text xml_attr, with XPath
JSON is suitable for storing configuration files (human-readable + versatile); R uses saveRDS internally (type-preserving)

📝 Exercises

Basic Problem: Construct a nested JSON string (containing the arrays name, age, and scores), parse it using fromJSON(), and then use toJSON(pretty = TRUE) to output a formatted version, verifying that the nested data has been correctly flattened.
Basic Exercise: Convert a data frame (5 rows, 3 columns) to JSON, then use read_json() to read it back in and verify that the data is complete.
Basic Problem: Use xml2 to read a simple XML string, extract the name attributes and text content of all <item> nodes, and output the data into a data frame.
Advanced Problem: Simulate an API response (containing a nested structure of status, data, and forecast), parse it using fromJSON, and then use bind_rows to merge the forecast array into a data frame.
Challenge: Write a complete program that: ① creates a JSON file containing weather data for 3 cities; ② reads and extracts the current temperature for all cities; ③ calculates the national average temperature; ④ writes the result to a new file using write_json. Take a screenshot of the console output and save it.

Web-Tutorial Tech Team

A team of developers maintaining programming tutorials. Each tutorial is written and reviewed by developers with expertise in that field. We work to keep our content accurate and reliable — if you spot an issue, please let us know.

About Us

R: R JSON and XML

1. What You'll Learn

2. The Story of an API Data Set

(1) Pain Point: The API returns nested JSON

(2) Solution using R

3. JSON vs. XML: When to Use Which?

(1) Comparison of the Two Formats

(2) JSON Example

(3) XML Example

4. Four Core Functions of jsonlite

(1) Function Quick Reference Table

(2) A Detailed Explanation of fromJSON

(3) Automatic flattening of nested JSON

5. A Detailed Explanation of toJSON

(1) Basic Syntax

(2) Practical Application

(3) Key Parameters

(4) write_json: Write to a file

6. Hands-On: Batch API Data Extraction

(1) Actual API Calls

(2) Batch capture of multiple users

7. xml2: Reading XML Files

(1) Installation and Core Functions

(2) Hands-On: Reading RSS Feeds

(3) Hands-On: Reading SVG (XML Images)

8. Hands-On: Reading and Writing JSON Configuration Files

(1) Project Structure

(2) Example of config.json

(3) Loading the R program configuration

9. Complete Example: API Data Extraction + JSON Persistence

▶ Example: Scraping and Persisting Weather API Data

❓ FAQ

📖 Summary

📝 Exercises

🙏 帮我们做得更好