Generative AI and Large Models
Generative AI is currently the hottest technology trend—ChatGPT, Midjourney, and GitHub Copilot are all products of this field. This chapter will help you understand: What is generative AI? How do large language models work? What are tokens, context windows, and temperature parameters? How can you set up an LLM API with just five lines of Python code?
1. What You'll Learn
- The Difference Between Generative AI and Discriminative AI
- How Large Language Models (LLMs) Work
- Tokens and Context Windows
- Overview of the GPT/Claude/Llama Model Family
- Calling the LLM API with Python
2. Story: An AI Assistant in 5 Lines of Code
(1) Pain Point: Intelligent Upgrades to the Customer Service System
Alice works as a backend developer at an e-commerce company. Her boss has asked her to add an AI assistant to the customer service system that can automatically answer common user questions. Alice has heard of GPT and ChatGPT, but doesn’t know how to use them—“Isn’t that something for algorithm experts? Can someone like me, who writes CRUD code, really handle this?”
(2) Bob's 5-line solution
Bob pulled up his keyboard and, in five minutes, wrote five lines of Python code to get the OpenAI API working:
from openai import OpenAI
client = OpenAI(api_key="your-api-key")
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "How to return a product?"}]
)
print(response.choices[0].message.content)
Bob said, "Look, a large language model is just a super-powerful text generator. You give it a prompt, and it gives you an answer. It's no different from calling a REST API."
▶ Example: Connect to the OpenAI API in 5 lines of code (Difficulty: ⭐)
# Minimal OpenAI API call
from openai import OpenAI
client = OpenAI() # Uses OPENAI_API_KEY env variable
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Say hello in 3 languages."}]
)
print(response.choices[0].message.content)
Hello! / ¡Hola! / Hello!
(3) Benefits: Breaking Through with an API Mindset
Alice discovered that—calling an LLM doesn’t require an understanding of the mathematical derivations behind Transformers, just as calling a database doesn’t require an understanding of how B+ trees work. Understanding the principles helps you make better choices, but all you need to get started is the API.
3. Distinguishing Between AI and Generative AI
(1) Two AI Paradigms
AI models are classified into two major paradigms based on their output objectives:
| Dimension | Discriminative AI | Generative AI |
|---|---|---|
| Objective | Categorize/classify existing data | Generate new data |
| Learning Content | Decision Boundary (Classification Line) | Data Distribution (How to Generate) |
| Input → Output | Features → Class/Value | Prompt → New Text/Image/Code |
| Typical Tasks | Classification, Regression, Detection | Text Generation, Image Generation, Code Generation |
| Representative Models | ResNet, SVM, BERT (Encoder) | GPT, Stable Diffusion, Midjourney |
| Typical Applications | Spam filtering, facial recognition | ChatGPT, AI-generated art, code completion |
| Output Determinism | Determinism (same input, same output) | Randomness (different outputs for the same prompt) |
Intuitive understanding: Discriminative AI answers "What is this?", while generative AI answers "Create one for me."
(2) Why is generative AI so popular?
Generative AI Exploded in 2022–2023: Three Reasons:
- The Transformer architecture (2017) enables models to process long text
- Massive Data + High Computing Power have enabled model scale to leap from the hundreds of millions to the hundreds of billions.
- RLHF (Reinforcement Learning with Human Feedback) aligns the model’s output with human preferences
(3) Can the discriminant and the generative models be combined?
Yes. Modern AI applications often combine the two—first using a discriminative model for intent classification, then using a generative model to generate a response. ChatGPT’s “function call” feature is essentially a combination of discrimination (determining which function to call) and generation (generating function parameters and a natural language response).
4. How Large Language Models (LLMs) Work
(1) Core Mechanism: Predicting the Next Token
The essence of an LLM is extremely simple—given the preceding text, predict the next most likely token. By repeating this prediction process, a complete response is generated.
Input: "The cat sat on the"
Step 1: "The cat sat on the" → predict "mat" (most likely next token)
Step 2: "The cat sat on the mat" → predict "."
Step 3: "The cat sat on the mat." → predict "<END>"
Output: "The cat sat on the mat."
This isn’t about “understanding” the text; rather, it involves learning the statistical patterns of token occurrence within a massive corpus. However, because the training data is sufficiently large (on the scale of the internet), these statistical patterns exhibit astonishing “intelligence.”
(2) LLM Generation Process
graph LR
A[Prompt<br/>Input Text] --> B[Tokenizer<br/>Tokenize]
B --> C[Transformer<br/>Model Inference]
C --> D[Next Token<br/>Probability Dist.]
D --> E[Sampling<br/>Temp/Top-P Sample]
E --> F[Decode<br/>Decode to Text]
F --> G{Done?}
G -->|No| C
G -->|Yes| H[Output<br/>Full Output Text]
(3) Intuition Behind the Transformer Architecture
The Transformer is the core engine of large language models (LLMs) and was proposed by Google in 2017 in the paper "Attention Is All You Need." Its key innovation is the self-attention mechanism—which allows the model to "see" all other tokens in the input and calculate their relevance when processing each token.
| Component | Function | Intuition |
|---|---|---|
| Self-Attention | Calculates the correlation between tokens | Automatically focuses on key information while reading |
| Feed-Forward | Performs a nonlinear transformation on each token | Understands and processes information |
| Layer Norm | Stabilizes training | Keeps values stable |
| Positional Encoding | Annotates positional information | Understands word order ("The cat eats the fish" ≠ "The fish eats the cat") |
5. Tokens and Context Windows
(1) What is a token?
A token is the smallest unit of text processed by an LLM. It is neither a character nor a word, but rather a "subword"—something between the two.
Text: "Hello, world!"
Tokens: ["Hello", ",", " world", "!"] → 4 tokens
Text: "artificial intelligence"
Tokens: ["art", "ific", "ial"] → 3 tokens (subword tokenization)
English: On average, 1 token ≈ 4 characters (about 0.75 words). Chinese: On average, 1 token ≈ 1–2 Chinese characters, so token consumption in Chinese is typically 2–3 times that of English.
▶ Example: Calculating the number of tokens using tiktoken (Difficulty: ⭐)
# Calculate token count using tiktoken
import tiktoken
encoding = tiktoken.encoding_for_model("gpt-4o-mini")
texts = [
"Hello, world!",
"The quick brown fox jumps over the lazy dog.",
"Artificial intelligence is changing the world.",
"Generative AI is transforming the world."
]
for text in texts:
tokens = encoding.encode(text)
print(f"Text: {text}")
print(f" Tokens: {len(tokens)} | Decoded: {encoding.decode(tokens[:3])}...")
print()
Text: Hello, world!
Tokens: 3 | Decoded: Hello...
Text: The quick brown fox jumps over the lazy dog.
Tokens: 10 | Decoded: The quick brown...
Text: Artificial intelligence is changing the world.
Tokens: 8 | Decoded: Artifici...
Text: Generative AI is transforming the world.
Tokens: 8 | Decoded: Generative AI is...
(2) Context Window
The context window is the maximum number of tokens an LLM can process at one time. It determines how much information the model can "remember"—the total number of tokens in the input and output combined cannot exceed this limit.
| Model | Context Window | Approximate Capacity |
|---|---|---|
| GPT-4o-mini | 128K tokens | ~300 pages of an English book |
| GPT-4o | 128K tokens | ~300 pages of an English book |
| Claude 3.5 Sonnet | 200K tokens | ~500 pages of English books |
| Llama 3.1 405B | 128K tokens | ~300 pages of an English book |
(3) Comparison of Token-Based Billing Methods
The LLM API is billed based on token usage, with different prices for inputs and outputs:
| Model | Input Price (per 1M tokens) | Output Price (per 1M tokens) | Description |
|---|---|---|---|
| GPT-4o-mini | $0.15 | $0.60 | Best value for money, top choice for everyday use |
| GPT-4o | $2.50 | $10.00 | High-quality, complex tasks |
| Claude 3.5 Sonnet | $3.00 | $15.00 | Strong at long text and code |
| Llama 3.1 70B (Hosted) | ~$0.20 | ~$0.20 | Open-source model; free for self-hosting |
6. Temperature and Sampling Parameters
(1) Temperature
Temperature controls the randomness of LLM outputs. The lower the temperature, the more deterministic the output; the higher the temperature, the more diverse the output.
Temperature = 0 → Always pick the most likely token (deterministic)
Temperature = 0.7 → Mostly likely but with some variation (default for chat)
Temperature = 1.0 → Natural variation, follows the learned distribution
Temperature = 2.0 → Very random, often nonsensical
(2) Top-P (K-means clustering)
Top-P is another parameter that controls randomness. It samples only from the first P tokens in the probability distribution:
Top-P = 0.1 → Only consider top 10% most likely tokens (conservative)
Top-P = 0.9 → Consider tokens covering 90% probability (moderate)
Top-P = 1.0 → Consider all tokens (no filtering)
| Parameter Scenario | Temperature | Top-P | Results |
|---|---|---|---|
| Code Generation | 0 | 1 | High certainty, correct code |
| Creative Writing | 0.8 | 0.9 | Creative but not outlandish |
| Brainstorming | 1.2 | 0.95 | High diversity, may yield surprising results |
| Data Extraction | 0 | 1 | Consistent format, reliable JSON |
▶ Example: Comparison of Outputs for Different Temperature Parameters (Difficulty: ⭐⭐)
# Compare outputs at different temperatures
from openai import OpenAI
client = OpenAI()
prompt = "Write a one-sentence description of a sunset."
temperatures = [0, 0.5, 1.0, 1.5]
for temp in temperatures:
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": prompt}],
temperature=temp,
max_tokens=50
)
text = response.choices[0].message.content
print(f"Temperature={temp}: {text}")
print()
Temperature=0: The sun dipped below the horizon, painting the sky in shades of orange and pink.
Temperature=0.5: Golden light spilled across the horizon as the sun melted into a sea of amber clouds.
Temperature=1.0: The day's final breath set the clouds ablaze, draping the world in a warm, fading embrace of crimson and gold.
Temperature=1.5: Colors—copper, violet, impossible rose—bloomed like dying flowers across the fractured sky as the sun surrendered.
7. Evolution of the GPT Series and Model Families
(1) Evolution of the GPT Series
| Model | Year | Number of Parameters | Key Breakthrough |
|---|---|---|---|
| GPT-1 | 2018 | 117 million | Demonstrated the viability of the pre-training + fine-tuning paradigm |
| GPT-2 | 2019 | 1.5 billion | Demonstrated scalability, but deemed "too dangerous" to release in full |
| GPT-3 | 2020 | 175 billion | In-context learning, few-shot learning |
| Codex | 2021 | 1.2 billion (Code Initiative) | Code generation, the engine behind GitHub Copilot |
| ChatGPT | 2022 | GPT-3.5 | With the addition of RLHF, conversational AI goes mainstream |
| GPT-4 | 2023 | Undisclosed (estimated to be in the trillions) | Multimodal (text + images), with a significant leap in reasoning capabilities |
| GPT-4o | 2024 | Not disclosed | Native multimodal (text/image/speech), twice as fast |
(2) Comparison of Major LLMs
| Dimension | GPT-4o | Claude 3.5 Sonnet | Llama 3.1 405B | Qwen 2.5 72B |
|---|---|---|---|---|
| Developer | OpenAI | Anthropic | Meta | Alibaba |
| Type | Proprietary | Proprietary | Open Source | Open Source |
| Context | 128K | 200K | 128K | 128K |
| Multimodal | Text/Image/Audio | Text/Image | Text | Text/Image |
| Strengths | Well-rounded | Long text/code/security | Self-hosted/customizable | Chinese/multilingual |
| API Pricing | Mid-to-High | Mid-to-High | Free for self-hosted | Free for self-hosted |
| Suitable for | General production environments | Enterprise-level applications | Private/research use | Chinese-language scenarios |
(3) Comparison of Open-Source vs. Closed-Source Models
| Dimension | Proprietary Models (GPT/Claude) | Open-Source Models (Llama/Qwen/Mistral) |
|---|---|---|
| Weight | Not disclosed | Publicly available for download |
| Deployment | API-only | Self-deployable (requires a GPU) |
| Data Privacy | Data is routed through third-party servers | Data remains local |
| Customization | Prompt/Fine-tuning API Only | Full Fine-tuning/LoRA/RLHF |
| Cost | Billed by token; higher costs for larger volumes | Fixed GPU cost; lower costs for larger volumes |
| Ease of Use | Low (just call the API) | High (requires a GPU and deployment knowledge) |
| Model Capabilities | Best (GPT-4o-level) | Close but slightly weaker (Llama 3.1 405B is close to GPT-4) |
8. API Call Patterns
(1) Basic Call Patterns
There are only three core parameters for calling the LLM API: model, messages, and the optional temperature.
▶ Example: Basic OpenAI API Calls (Difficulty: ⭐)
# Basic OpenAI API call with system prompt
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": "You are a concise technical assistant."},
{"role": "user", "content": "What is a Transformer in 2 sentences?"}
],
temperature=0.3
)
print(response.choices[0].message.content)
print(f"\nTokens used: {response.usage.total_tokens}")
A Transformer is a neural network architecture based on self-attention, allowing it to process all input tokens in parallel rather than sequentially. Introduced in 2017, it became the foundation for modern LLMs like GPT and BERT.
Tokens used: 58
(2) Multi-round dialogue mode
The Chat API maintains the conversation history using the messages array, appending one message per round:
▶ Example: Multi-round conversation API call (Difficulty: ⭐⭐)
# Multi-turn conversation with OpenAI API
from openai import OpenAI
client = OpenAI()
conversation = [
{"role": "system", "content": "You are a helpful Python tutor."}
]
questions = [
"What is a list comprehension?",
"Can you show me an example?",
"What if I want to filter even numbers?"
]
for q in questions:
conversation.append({"role": "user", "content": q})
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=conversation,
temperature=0.3
)
reply = response.choices[0].message.content
conversation.append({"role": "assistant", "content": reply})
print(f"User: {q}")
print(f"Bot: {reply[:120]}...")
print()
User: What is a list comprehension?
Bot: A list comprehension is a concise way to create lists in Python using a single line of syntax, combining a for loop and optional conditions...
User: Can you show me an example?
Bot: Sure! Here's a basic example: squares = [x**2 for x in range(10)] This creates a list [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]....
User: What if I want to filter even numbers?
Bot: Add a condition: evens = [x for x in range(20) if x % 2 == 0] This produces [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]....
(3) Stream Output Mode
Streaming output returns tokens one by one, providing a better user experience (like a typewriter effect) and displaying the first token faster:
▶ Example: Streaming Output (stream=True) Experience (Difficulty: ⭐⭐)
# Streaming output with OpenAI API
from openai import OpenAI
client = OpenAI()
stream = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Explain quantum computing in 3 sentences."}],
temperature=0.5,
stream=True
)
print("Streaming response: ", end="")
for chunk in stream:
if chunk.choices[0].delta.content is not None:
print(chunk.choices[0].delta.content, end="", flush=True)
print("\n\n[Stream complete]")
Streaming response: Quantum computing uses quantum bits (qubits) that can exist in superposition, representing 0 and 1 simultaneously. This allows quantum computers to process many calculations in parallel, exponentially speeding up certain problems. Key applications include cryptography, drug discovery, and optimization.
[Stream complete]
stream=True returns an iterator rather than a complete response, making it suitable for displaying content one character at a time in web applications. However, the total number of tokens cannot be determined in advance; it must be obtained via the usage field in the last chunk.
9. Comprehensive Example: AI Copywriting Assistant
Build an "AI Copywriting Assistant": Enter a product description, call the OpenAI API to generate ad copy in three different styles, and print them out for comparison.
▶ Example: AI Copywriting Assistant—Generating Ad Copy in 3 Different Styles (Difficulty: ⭐⭐⭐)
# ============================================
# AI Copywriting Assistant
# Input: product description
# Output: 3 styles of ad copy
# ============================================
from openai import OpenAI
client = OpenAI()
product = "Wireless noise-cancelling headphones, 40-hour battery, $199"
styles = [
("Professional", "Write a professional, feature-focused ad copy."),
("Emotional", "Write an emotional, lifestyle-focused ad copy."),
("Humorous", "Write a humorous, witty ad copy.")
]
print("=" * 60)
print(f"Product: {product}")
print("=" * 60)
for style_name, style_prompt in styles:
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": f"You are an expert copywriter. {style_prompt}"},
{"role": "user", "content": f"Write ad copy for: {product}. Keep it under 80 words."}
],
temperature=0.8,
max_tokens=150
)
copy = response.choices[0].message.content
tokens = response.usage.total_tokens
cost = (response.usage.prompt_tokens * 0.15 + response.usage.completion_tokens * 0.60) / 1_000_000
print(f"\n[{style_name}]")
print(f"{copy}")
print(f"\n(Tokens: {tokens}, Cost: ${cost:.6f})")
print("\n" + "=" * 60)
============================================================
Product: Wireless noise-cancelling headphones, 40-hour battery, $199
============================================================
[Professional]
Experience premium noise-cancelling technology with 40 hours of uninterrupted battery life. Our wireless headphones deliver crystal-clear audio and comfortable design for $199. Perfect for professionals who demand focus and quality.
(Tokens: 72, Cost: $0.000032)
[Emotional]
Escape the noise. Slip on these headphones and let the world fade away—40 hours of pure, uninterrupted bliss. Whether it's your morning commute or a quiet evening, your soundtrack awaits. Just $199 for peace you can hear.
(Tokens: 68, Cost: $0.000030)
[Humorous]
Tired of hearing your neighbor's karaoke? These noise-cancelling headphones block the chaos and deliver 40 hours of sweet silence. At $199, it's cheaper than moving. Your ears will thank you.
(Tokens: 58, Cost: $0.000025)
============================================================
❓ FAQ
tiktoken library to calculate this precisely.📖 Summary
- Generative AI learns from data distributions to create new content, while discriminative AI learns decision boundaries to classify—the two can be used in combination
- The core of an LLM is "next-token prediction": repeatedly predicting the most likely next token and concatenating them to form a complete output
- A token is the smallest unit processed by an LLM; Chinese tokens consume 2–3 times as many resources as English tokens; the context window determines how much text the model can process at a time.
- Temperature control output randomness: 0 = deterministic, 0.7 = natural, 1.5+ = creative; Top-P is another randomness filter
- The GPT series has evolved from 100 million parameters to the trillion-level; GPT-4o is currently OpenAI’s flagship model. In the open-source camp, there are Llama, Qwen, and Mistral.
- Calling the LLM API requires only three core parameters: model, messages, and temperature; multi-turn conversations require maintaining a complete history; streaming output enhances the user experience
📝 Exercises
- Basic Problem (Difficulty ⭐): Write a small translation program using the OpenAI API—it should take Chinese input and output an English translation. Requirements: Specify the role as “Translation Assistant” in the system prompt, and set
temperature=0to ensure consistent translations. - Advanced Exercise (Difficulty ⭐⭐): Using the same prompt (e.g., "Write a poem about autumn"), call the model 5 times each with
temperature=0andtemperature=1. Record and compare the differences in the output, then summarize the patterns of how temperature affects the output. - Challenge (Difficulty: ⭐⭐⭐): Use the API to build a simple chatbot: Support multi-turn conversations (maintain a
messagesarray), and exit the loop when it encounters "bye"; include a prompt asking the bot to play a specific role (such as "Python programming tutor"), and try to print the responses word-for-word using stream output.



