The History of AI
AI didn’t emerge overnight—it has gone through more than 70 years of ups and downs, including two “winters” and three waves. Only by understanding the history of AI can we determine whether this current wave is a true revolution or just another bubble.
1. What You'll Learn
- The Three Waves and Two Winters of AI
- The Significance and Limitations of the Turing Test
- Expert Systems → Statistical Learning → The Evolution of Deep Learning
- Why Was AlexNet in 2012 a Turning Point?
- Reasons for the Boom in Generative AI in 2022–2023
2. A True Story from a Job Candidate
(1) Pain Point: Knowing Nothing About the History of AI
During his interview, Charlie was asked, “Do you think the development of AI will go through a ‘winter’?” He only knew that ChatGPT was all the rage and knew nothing about the history of AI. The interviewer followed up with, “Are you familiar with expert systems?”—Charlie had absolutely no answer, and the interview ended abruptly.
(2) Insights from the AI Chronicle
After returning, Charlie flipped through the history of AI and discovered that AI had already “died twice and come back to life”: it had experienced low points in both the 1970s and the 1990s, marked by funding cuts and researchers switching careers. The current third wave, however, is the one that has truly made its way into millions of households.
(3) Returns: Judgment from a Historical Perspective
After completing this course, Charlie will be able to confidently say during an interview: “There have been two ‘winters’ in the history of AI, but each recovery has brought about even more powerful technology. The current wave of generative AI is based on Transformers, big data, and GPUs, and is fundamentally different from the previous wave of expert systems—its use cases are real-world, not just proof-of-concept.”
3. The Turing Test: The Intellectual Starting Point for AI
In 1950, Alan Turing published his paper "Computing Machinery and Intelligence," in which he proposed the famous Turing test:
graph LR
A[Human Judge] -->|asks questions| B[Hidden Entity A<br/>could be human or machine]
A -->|asks questions| C[Hidden Entity B<br/>could be human or machine]
B -->|answers| A
C -->|answers| A
A --> D{Can the judge<br/>tell which is machine?}
D -->|No| E[Machine passes<br/>Turing Test]
D -->|Yes| F[Machine fails]
(1) The Core Concept of the Turing Test
If a machine's responses make it impossible for human evaluators to distinguish whether it is a human or a machine, then that machine can be considered intelligent.
(2) Limitations of the Turing Test
| Limitations | Description | Examples |
|---|---|---|
| Measures behavior but not understanding | Machines can "simulate" intelligence without truly "understanding" | ELIZA (1966) simulated a psychotherapist using pattern matching |
| Focuses on linguistic intelligence | Ignores other intelligences such as visual, kinesthetic, and emotional | Cannot assess the intelligence required for autonomous driving |
| Subjectivity of Human Reviewers | Different reviewers have different criteria | Some people are easily deceived by "chatbots" |
| Can be "tricked" | The machine intentionally makes mistakes or feigns hesitation to "pretend to be human" | Eugene Goostman (2014) pretended to be a 13-year-old Ukrainian boy |
▶ Example: A Case of Deception in the Turing Test (Difficulty: ⭐⭐)
💡 Tip: The Turing Test is an important thought experiment in the philosophy of AI, but modern AI research no longer aims to "pass the Turing Test." Today's AI research focuses more on outperforming humans in specific tasks (such as image recognition, Go, and protein folding prediction).
4. The Three Waves of AI
timeline
title AI Development Timeline
1950s : Turing Test : Dartmouth Conference (1956)
1960s-70s : Expert Systems Era : First AI Winter (1974)
1980s : Expert System Revival : Backpropagation (1986)
1987-93 : Second AI Winter
2012 : AlexNet / Deep Learning Revolution
2017 : Transformer Architecture
2022 : ChatGPT / Generative AI Era
(1) The First Wave (1956–1974): Reasoning and Search
| Dimension | Content |
|---|---|
| Core Idea | Simulating human reasoning using symbolic logic |
| Notable Achievements | Logic reasoning programs, chess programs, the ELIZA chatbot |
| Key Events | 1956 Dartmouth Conference—The Birth of the Field of AI |
| Promise | "Within 20 years, machines will be able to do everything humans can do" |
| Reasons for Failure | Overoptimism, insufficient computing power, and the fact that real-world problems are far more complex than those in the laboratory |
▶ Example: Representative Achievements in the First Wave of Symbolic Reasoning (Difficulty: ⭐)
(2) The Second Wave (1980–1987): Expert Systems
| Dimension | Content |
|---|---|
| Core Concept | Encoding expert knowledge into if-then rules |
| Notable Achievements | MYCIN (Medical Diagnostics), XCON (Computer Configuration) |
| Business Success | DEC Saves $40 million Annually with XCON |
| Reasons for Failure | Knowledge acquisition bottlenecks (experts find it difficult to codify all their experience into rules), skyrocketing maintenance costs, inability to handle uncertainty |
(3) The Third Wave (2012–present): Deep Learning + Generative AI
| Dimension | Content |
|---|---|
| Core Concept | Using multilayer neural networks to automatically learn features from massive amounts of data |
| Notable Achievements | AlexNet, AlphaGo, the GPT series, Stable Diffusion |
| Driving Forces | Big Data + GPU Computing Power + Algorithmic Breakthroughs (Transformer) |
| Differences from the Previous Two Approaches | Does not rely on human-defined coding rules; the model automatically learns from the data; application scenarios are real-world and quantifiable |
5. Comparison of the Three Waves
| Dimension | First Wave (Symbolic AI) | Second Wave (Expert Systems) | Third Wave (Deep Learning) |
|---|---|---|---|
| Time | 1956–1974 | 1980–1987 | 2012–present |
| Core Methods | Symbolic reasoning, search | Knowledge base + reasoning engine | Neural networks + data-driven |
| Knowledge Source | Manually Coded Logic | Manually Coded Rules | Automatically Learned from Data |
| Representative Applications | Mathematical Proofs, Chess | Medical Diagnosis, Configuration | Image Recognition, NLP, Generation |
| Major Limitations | Inability to handle ambiguity | Bottlenecks in knowledge acquisition | Data dependency, poor interpretability |
| Failures/Risks | Overpromising | High maintenance costs | Illusions, biases, energy consumption |
6. Two Key Turning Points
(1) AlexNet (2012) — The Deep Learning Revolution
In 2012, Alex Krizhevsky and his colleagues won the ImageNet image recognition competition using deep convolutional neural networks, achieving an error rate more than 10 percentage points lower than that of the runner-up:
| Metric | Before AlexNet | AlexNet |
|---|---|---|
| ImageNet Top-5 Error Rate | ~26% | 15.3% |
| Number of model layers | 1–3 layers | 8 layers |
| Training Method | Manual features + shallow classifiers | End-to-end deep learning |
| Hardware | CPU | GPU (2× GTX 580) |
AlexNet demonstrated three things: (1) Deep networks outperform shallow ones; (2) GPUs can significantly speed up training; (3) The more data, the better the results.
(2) Transformer (2017) — A Paradigm Shift in NLP
Google's paper "Attention Is All You Need" introduced the Transformer architecture, which revolutionized NLP:
▶ Example: Comparison of RNN/LSTM vs. Transformer Approaches (Difficulty: ⭐⭐)
TEXT
Before Transformer:
RNN/LSTM → Sequential processing → Slow + Forget long-range dependencies
After Transformer:
Self-Attention → Parallel processing → Fast + Understand long-range context
| Dimension | RNN/LSTM | Transformer |
|---|---|---|
| Processing Method | Sequential (word-by-word) | Parallel (all at once) |
| Long-range dependencies | Poor (vanishing gradients) | Good (direct correlation via self-attention) |
| Training Speed | Slow (cannot be parallelized) | Fast (GPU-friendly) |
| Representative Models | LSTM, GRU | GPT, BERT, T5 |
7. The Boom in Generative AI (2022–2026)
Why did ChatGPT “skyrocket to fame” in 2022? This was due to the convergence of three key factors:
| Condition | Status | Description |
|---|---|---|
| Algorithm | ✅ Mature | Transformer architecture + RLHF alignment |
| Data | ✅ Abundant | Vast amounts of text accumulated on the Internet |
| Computing Power | ✅ Available | Large-scale deployment of GPU clusters (A100/H100) |
(3) The Pace of Development of Generative AI
▶ Example: Timeline of Generative AI Development (Difficulty: ⭐)
TEXT
2022-11: ChatGPT (GPT-3.5) — 100M users in 2 months
2023-03: GPT-4 — Multimodal (text + image input)
2023-07: Llama 1 — Open-source LLM revolution
2024-01: GPT-4 Turbo / Gemini — Longer context
2024-07: GPT-4o — Real-time voice + vision
2025-xx: Agent era — AI autonomously uses tools
8. Complete Example: From Expert Systems to Machine Learning
Implement a minimalist "expert system" in Python to diagnose diseases, and then discuss why it will be replaced by machine learning:
▶ Example: From Expert Systems to Machine Learning—Disease Diagnosis (Difficulty: ⭐⭐⭐)
PYTHON
# ============================================
# From Expert System to Machine Learning
# A medical diagnosis example
# ============================================
# --- Expert System: hand-crafted rules ---
def diagnose_expert(symptoms):
"""Rule-based medical diagnosis (expert system style)"""
if "fever" in symptoms and "cough" in symptoms:
return "Flu"
elif "headache" in symptoms and "fever" in symptoms:
return "Malaria"
elif "sneezing" in symptoms and "runny_nose" in symptoms:
return "Common Cold"
else:
return "Unknown — no matching rule"
# --- Test the expert system ---
test_cases = [
["fever", "cough"], # Expected: Flu
["headache", "fever"], # Expected: Malaria
["sneezing", "runny_nose"], # Expected: Common Cold
["fever", "fatigue"], # Expected: Unknown (rule gap!)
["cough", "sore_throat"], # Expected: Unknown (rule gap!)
]
print("=== Expert System Diagnosis ===")
for symptoms in test_cases:
result = diagnose_expert(symptoms)
print(f"Symptoms: {symptoms} → Diagnosis: {result}")
print()
print("Problem: 2 out of 5 cases return 'Unknown'")
print("Reason: Human experts cannot write rules for ALL symptom combinations")
print("Solution: Let ML learn patterns from thousands of real patient records")
💻 Output:
TEXT
=== Expert System Diagnosis ===
Symptoms: ['fever', 'cough'] → Diagnosis: Flu
Symptoms: ['headache', 'fever'] → Diagnosis: Malaria
Symptoms: ['sneezing', 'runny_nose'] → Diagnosis: Common Cold
Symptoms: ['fever', 'fatigue'] → Diagnosis: Unknown — no matching rule
Symptoms: ['cough', 'sore_throat'] → Diagnosis: Unknown — no matching rule
Problem: 2 out of 5 cases return 'Unknown'
Reason: Human experts cannot write rules for ALL symptom combinations
Solution: Let ML learn patterns from thousands of real patient records
💡 Tip: This is precisely the core issue of the second AI winter—expert systems cannot cover all possible scenarios, and maintenance costs grow exponentially with the number of rules. Machine learning automatically discovers rules from data, thereby resolving the "knowledge acquisition bottleneck."
❓ FAQ
Q Will AI experience another "winter"?
A It’s possible, but unlikely. The previous two "winters" were caused by "unfulfilled promises"—excessively high maintenance costs for expert systems and insufficient computing power. The current wave is fundamentally different: ① Real-world application scenarios (recommendations, translation, and code generation all have commercial value); ② A mature technology stack (Transformers + GPUs + big data); ③ A rich open-source ecosystem (Llama, PyTorch, Hugging Face). Even if investment dries up, the technology itself will not disappear.
Q Is the Turing Test still relevant today?
A It is relevant as a philosophical thought experiment, but it is outdated as a standard for evaluating AI. Modern AI evaluation focuses more on task-specific benchmarks (such as MMLU and HumanEval) rather than on whether an AI can "fool humans."
Q Why was AlexNet so important in 2012?
A Because it marked the first time deep learning outperformed traditional methods on large-scale real-world tasks. Previously, neural networks were considered “academic toys,” but AlexNet—using GPUs, big data, and deep networks—demonstrated the practical value of deep learning, directly sparking the deep learning revolution.
Q What is the fundamental difference between generative AI and previous forms of AI?
A Previous AI was primarily “discriminative”—determining whether an email was spam or whether an image showed a cat or a dog. Generative AI can “create” new content—writing articles, drawing images, and writing code. This represents a qualitative shift from “understanding” to “creation.”
Q When will AGI be achieved?
A No one knows. AGI (Artificial General Intelligence) refers to AI that can match or surpass human levels of performance on any intellectual task. Current AI systems are all “narrow AI,” meaning they are powerful only at specific tasks. Experts predict it could take anywhere from 5 to 50 years, while others say it will never be achieved. Stay informed, but don’t worry.
Q Should I study traditional AI (symbolic reasoning) or jump straight into deep learning?
A It depends on your goals. If you’re working on NLP, computer vision, or generative AI, go straight to deep learning. If you’re working on knowledge graphs, logical reasoning, or expert systems, traditional AI is still valuable. This tutorial focuses primarily on deep learning, but it will briefly introduce the concepts of traditional AI.
📖 Summary
- AI has gone through three waves: symbolic reasoning (1950s) → expert systems (1980s) → deep learning (2012–present)
- The root causes of the two "winters": overpromising + technical bottlenecks (computing power/knowledge acquisition)
- AlexNet (2012) demonstrated the power of deep learning, GPUs, and big data, sparking the third wave
- Transformer (2017) addressed the long-range dependency issue in RNNs and became a cornerstone of modern NLP
- The boom in generative AI is the inevitable result of the simultaneous maturation of three key factors: algorithms (Transformers), data (Internet corpora), and computing power (GPUs).
- Only by understanding history can we predict the future—the current trend has real commercial value and is fundamentally different from the previous two bubbles.
📝 Exercises
- Basic Question (Difficulty: ⭐): Draw a timeline of AI development and label at least 8 key milestones (including two "winters").
- Advanced Question (Difficulty: ⭐⭐): List three advantages and three disadvantages of expert systems and machine learning, respectively, and present them in a table.
- Challenge Question (Difficulty: ⭐⭐⭐): Write 200 characters—What do you think is the fundamental difference between this AI wave and the previous two? Is it possible that we’ll face another “winter”? Explain your reasoning.
