404 Not Found

404 Not Found


nginx

Generative AI and Large Models

Generative AI is currently the hottest technology trend—ChatGPT, Midjourney, and GitHub Copilot are all products of this field. This chapter will help you understand: What is generative AI? How do large language models work? What are tokens, context windows, and temperature parameters? How can you set up an LLM API with just five lines of Python code?

1. What You'll Learn


2. Story: An AI Assistant in 5 Lines of Code

(1) Pain Point: Intelligent Upgrades to the Customer Service System

Alice works as a backend developer at an e-commerce company. Her boss has asked her to add an AI assistant to the customer service system that can automatically answer common user questions. Alice has heard of GPT and ChatGPT, but doesn’t know how to use them—“Isn’t that something for algorithm experts? Can someone like me, who writes CRUD code, really handle this?”

(2) Bob's 5-line solution

Bob pulled up his keyboard and, in five minutes, wrote five lines of Python code to get the OpenAI API working:

PYTHON
from openai import OpenAI
client = OpenAI(api_key="your-api-key")
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "How to return a product?"}]
)
print(response.choices[0].message.content)

Bob said, "Look, a large language model is just a super-powerful text generator. You give it a prompt, and it gives you an answer. It's no different from calling a REST API."

▶ Example: Connect to the OpenAI API in 5 lines of code (Difficulty: ⭐)

PYTHON
# Minimal OpenAI API call
from openai import OpenAI
client = OpenAI()  # Uses OPENAI_API_KEY env variable
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Say hello in 3 languages."}]
)
print(response.choices[0].message.content)
💻 Output:

TEXT
Hello! / ¡Hola! / Hello!

(3) Benefits: Breaking Through with an API Mindset

Alice discovered that—calling an LLM doesn’t require an understanding of the mathematical derivations behind Transformers, just as calling a database doesn’t require an understanding of how B+ trees work. Understanding the principles helps you make better choices, but all you need to get started is the API.


3. Distinguishing Between AI and Generative AI

(1) Two AI Paradigms

AI models are classified into two major paradigms based on their output objectives:

Dimension Discriminative AI Generative AI
Objective Categorize/classify existing data Generate new data
Learning Content Decision Boundary (Classification Line) Data Distribution (How to Generate)
Input → Output Features → Class/Value Prompt → New Text/Image/Code
Typical Tasks Classification, Regression, Detection Text Generation, Image Generation, Code Generation
Representative Models ResNet, SVM, BERT (Encoder) GPT, Stable Diffusion, Midjourney
Typical Applications Spam filtering, facial recognition ChatGPT, AI-generated art, code completion
Output Determinism Determinism (same input, same output) Randomness (different outputs for the same prompt)

Intuitive understanding: Discriminative AI answers "What is this?", while generative AI answers "Create one for me."

Generative AI Exploded in 2022–2023: Three Reasons:

  1. The Transformer architecture (2017) enables models to process long text
  2. Massive Data + High Computing Power have enabled model scale to leap from the hundreds of millions to the hundreds of billions.
  3. RLHF (Reinforcement Learning with Human Feedback) aligns the model’s output with human preferences

(3) Can the discriminant and the generative models be combined?

Yes. Modern AI applications often combine the two—first using a discriminative model for intent classification, then using a generative model to generate a response. ChatGPT’s “function call” feature is essentially a combination of discrimination (determining which function to call) and generation (generating function parameters and a natural language response).


4. How Large Language Models (LLMs) Work

(1) Core Mechanism: Predicting the Next Token

The essence of an LLM is extremely simple—given the preceding text, predict the next most likely token. By repeating this prediction process, a complete response is generated.

TEXT
Input:  "The cat sat on the"
Step 1: "The cat sat on the" → predict "mat"  (most likely next token)
Step 2: "The cat sat on the mat" → predict "."
Step 3: "The cat sat on the mat." → predict "<END>"
Output: "The cat sat on the mat."

This isn’t about “understanding” the text; rather, it involves learning the statistical patterns of token occurrence within a massive corpus. However, because the training data is sufficiently large (on the scale of the internet), these statistical patterns exhibit astonishing “intelligence.”

(2) LLM Generation Process

100%
graph LR
    A[Prompt<br/>Input Text] --> B[Tokenizer<br/>Tokenize]
    B --> C[Transformer<br/>Model Inference]
    C --> D[Next Token<br/>Probability Dist.]
    D --> E[Sampling<br/>Temp/Top-P Sample]
    E --> F[Decode<br/>Decode to Text]
    F --> G{Done?}
    G -->|No| C
    G -->|Yes| H[Output<br/>Full Output Text]

(3) Intuition Behind the Transformer Architecture

The Transformer is the core engine of large language models (LLMs) and was proposed by Google in 2017 in the paper "Attention Is All You Need." Its key innovation is the self-attention mechanism—which allows the model to "see" all other tokens in the input and calculate their relevance when processing each token.

Component Function Intuition
Self-Attention Calculates the correlation between tokens Automatically focuses on key information while reading
Feed-Forward Performs a nonlinear transformation on each token Understands and processes information
Layer Norm Stabilizes training Keeps values stable
Positional Encoding Annotates positional information Understands word order ("The cat eats the fish" ≠ "The fish eats the cat")
💡 Tip: You don’t need to understand the mathematical details of Transformers to use LLMs effectively. It’s like how you don’t need to understand how an engine works to drive a car well—but knowing that “the accelerator controls the engine speed” helps you drive better.


5. Tokens and Context Windows

(1) What is a token?

A token is the smallest unit of text processed by an LLM. It is neither a character nor a word, but rather a "subword"—something between the two.

TEXT
Text:   "Hello, world!"
Tokens: ["Hello", ",", " world", "!"]     → 4 tokens

Text:   "artificial intelligence"
Tokens: ["art", "ific", "ial"]               → 3 tokens (subword tokenization)

English: On average, 1 token ≈ 4 characters (about 0.75 words). Chinese: On average, 1 token ≈ 1–2 Chinese characters, so token consumption in Chinese is typically 2–3 times that of English.

▶ Example: Calculating the number of tokens using tiktoken (Difficulty: ⭐)

PYTHON
# Calculate token count using tiktoken
import tiktoken

encoding = tiktoken.encoding_for_model("gpt-4o-mini")

texts = [
    "Hello, world!",
    "The quick brown fox jumps over the lazy dog.",
    "Artificial intelligence is changing the world.",
    "Generative AI is transforming the world."
]

for text in texts:
    tokens = encoding.encode(text)
    print(f"Text: {text}")
    print(f"  Tokens: {len(tokens)} | Decoded: {encoding.decode(tokens[:3])}...")
    print()
💻 Output:

TEXT
Text: Hello, world!
  Tokens: 3 | Decoded: Hello...

Text: The quick brown fox jumps over the lazy dog.
  Tokens: 10 | Decoded: The quick brown...

Text: Artificial intelligence is changing the world.
  Tokens: 8 | Decoded: Artifici...

Text: Generative AI is transforming the world.
  Tokens: 8 | Decoded: Generative AI is...

(2) Context Window

The context window is the maximum number of tokens an LLM can process at one time. It determines how much information the model can "remember"—the total number of tokens in the input and output combined cannot exceed this limit.

Model Context Window Approximate Capacity
GPT-4o-mini 128K tokens ~300 pages of an English book
GPT-4o 128K tokens ~300 pages of an English book
Claude 3.5 Sonnet 200K tokens ~500 pages of English books
Llama 3.1 405B 128K tokens ~300 pages of an English book

(3) Comparison of Token-Based Billing Methods

The LLM API is billed based on token usage, with different prices for inputs and outputs:

Model Input Price (per 1M tokens) Output Price (per 1M tokens) Description
GPT-4o-mini $0.15 $0.60 Best value for money, top choice for everyday use
GPT-4o $2.50 $10.00 High-quality, complex tasks
Claude 3.5 Sonnet $3.00 $15.00 Strong at long text and code
Llama 3.1 70B (Hosted) ~$0.20 ~$0.20 Open-source model; free for self-hosting
💡 Tip: 1M tokens ≈ 750K English words ≈ approximately 500K Chinese characters. Writing a 500-character Chinese request consumes approximately 500–1,000 tokens, at a cost of less than $0.001.


6. Temperature and Sampling Parameters

(1) Temperature

Temperature controls the randomness of LLM outputs. The lower the temperature, the more deterministic the output; the higher the temperature, the more diverse the output.

TEXT
Temperature = 0    → Always pick the most likely token (deterministic)
Temperature = 0.7  → Mostly likely but with some variation (default for chat)
Temperature = 1.0  → Natural variation, follows the learned distribution
Temperature = 2.0  → Very random, often nonsensical

(2) Top-P (K-means clustering)

Top-P is another parameter that controls randomness. It samples only from the first P tokens in the probability distribution:

TEXT
Top-P = 0.1  → Only consider top 10% most likely tokens (conservative)
Top-P = 0.9  → Consider tokens covering 90% probability (moderate)
Top-P = 1.0  → Consider all tokens (no filtering)
Parameter Scenario Temperature Top-P Results
Code Generation 0 1 High certainty, correct code
Creative Writing 0.8 0.9 Creative but not outlandish
Brainstorming 1.2 0.95 High diversity, may yield surprising results
Data Extraction 0 1 Consistent format, reliable JSON
💡 Tip: Generally, you only need to adjust Temperature; leave Top-P at its default value (1.0). Avoid making significant adjustments to both at the same time—since both Temperature and Top-P control randomness, adjusting them together can lead to unpredictable results.

▶ Example: Comparison of Outputs for Different Temperature Parameters (Difficulty: ⭐⭐)

PYTHON
# Compare outputs at different temperatures
from openai import OpenAI
client = OpenAI()

prompt = "Write a one-sentence description of a sunset."
temperatures = [0, 0.5, 1.0, 1.5]

for temp in temperatures:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
        temperature=temp,
        max_tokens=50
    )
    text = response.choices[0].message.content
    print(f"Temperature={temp}: {text}")
    print()
💻 Output (varies each time; for reference only):

TEXT
Temperature=0: The sun dipped below the horizon, painting the sky in shades of orange and pink.

Temperature=0.5: Golden light spilled across the horizon as the sun melted into a sea of amber clouds.

Temperature=1.0: The day's final breath set the clouds ablaze, draping the world in a warm, fading embrace of crimson and gold.

Temperature=1.5: Colors—copper, violet, impossible rose—bloomed like dying flowers across the fractured sky as the sun surrendered.

7. Evolution of the GPT Series and Model Families

(1) Evolution of the GPT Series

Model Year Number of Parameters Key Breakthrough
GPT-1 2018 117 million Demonstrated the viability of the pre-training + fine-tuning paradigm
GPT-2 2019 1.5 billion Demonstrated scalability, but deemed "too dangerous" to release in full
GPT-3 2020 175 billion In-context learning, few-shot learning
Codex 2021 1.2 billion (Code Initiative) Code generation, the engine behind GitHub Copilot
ChatGPT 2022 GPT-3.5 With the addition of RLHF, conversational AI goes mainstream
GPT-4 2023 Undisclosed (estimated to be in the trillions) Multimodal (text + images), with a significant leap in reasoning capabilities
GPT-4o 2024 Not disclosed Native multimodal (text/image/speech), twice as fast

(2) Comparison of Major LLMs

Dimension GPT-4o Claude 3.5 Sonnet Llama 3.1 405B Qwen 2.5 72B
Developer OpenAI Anthropic Meta Alibaba
Type Proprietary Proprietary Open Source Open Source
Context 128K 200K 128K 128K
Multimodal Text/Image/Audio Text/Image Text Text/Image
Strengths Well-rounded Long text/code/security Self-hosted/customizable Chinese/multilingual
API Pricing Mid-to-High Mid-to-High Free for self-hosted Free for self-hosted
Suitable for General production environments Enterprise-level applications Private/research use Chinese-language scenarios

(3) Comparison of Open-Source vs. Closed-Source Models

Dimension Proprietary Models (GPT/Claude) Open-Source Models (Llama/Qwen/Mistral)
Weight Not disclosed Publicly available for download
Deployment API-only Self-deployable (requires a GPU)
Data Privacy Data is routed through third-party servers Data remains local
Customization Prompt/Fine-tuning API Only Full Fine-tuning/LoRA/RLHF
Cost Billed by token; higher costs for larger volumes Fixed GPU cost; lower costs for larger volumes
Ease of Use Low (just call the API) High (requires a GPU and deployment knowledge)
Model Capabilities Best (GPT-4o-level) Close but slightly weaker (Llama 3.1 405B is close to GPT-4)

8. API Call Patterns

(1) Basic Call Patterns

There are only three core parameters for calling the LLM API: model, messages, and the optional temperature.

▶ Example: Basic OpenAI API Calls (Difficulty: ⭐)

PYTHON
# Basic OpenAI API call with system prompt
from openai import OpenAI
client = OpenAI()

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You are a concise technical assistant."},
        {"role": "user", "content": "What is a Transformer in 2 sentences?"}
    ],
    temperature=0.3
)

print(response.choices[0].message.content)
print(f"\nTokens used: {response.usage.total_tokens}")
💻 Output:

TEXT
A Transformer is a neural network architecture based on self-attention, allowing it to process all input tokens in parallel rather than sequentially. Introduced in 2017, it became the foundation for modern LLMs like GPT and BERT.

Tokens used: 58

(2) Multi-round dialogue mode

The Chat API maintains the conversation history using the messages array, appending one message per round:

▶ Example: Multi-round conversation API call (Difficulty: ⭐⭐)

PYTHON
# Multi-turn conversation with OpenAI API
from openai import OpenAI
client = OpenAI()

conversation = [
    {"role": "system", "content": "You are a helpful Python tutor."}
]

questions = [
    "What is a list comprehension?",
    "Can you show me an example?",
    "What if I want to filter even numbers?"
]

for q in questions:
    conversation.append({"role": "user", "content": q})
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=conversation,
        temperature=0.3
    )
    reply = response.choices[0].message.content
    conversation.append({"role": "assistant", "content": reply})
    print(f"User: {q}")
    print(f"Bot:  {reply[:120]}...")
    print()
💻 Output:

TEXT
User: What is a list comprehension?
Bot:  A list comprehension is a concise way to create lists in Python using a single line of syntax, combining a for loop and optional conditions...

User: Can you show me an example?
Bot:  Sure! Here's a basic example: squares = [x**2 for x in range(10)] This creates a list [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]....

User: What if I want to filter even numbers?
Bot:  Add a condition: evens = [x for x in range(20) if x % 2 == 0] This produces [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]....
💡 Tip: Multi-round conversations require sending the entire conversation history back to the API (since LLMs are stateless), so the longer the conversation, the more tokens are consumed. In production environments, you should limit the length of conversations or use summarization to truncate them.

(3) Stream Output Mode

Streaming output returns tokens one by one, providing a better user experience (like a typewriter effect) and displaying the first token faster:

▶ Example: Streaming Output (stream=True) Experience (Difficulty: ⭐⭐)

PYTHON
# Streaming output with OpenAI API
from openai import OpenAI
client = OpenAI()

stream = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Explain quantum computing in 3 sentences."}],
    temperature=0.5,
    stream=True
)

print("Streaming response: ", end="")
for chunk in stream:
    if chunk.choices[0].delta.content is not None:
        print(chunk.choices[0].delta.content, end="", flush=True)
print("\n\n[Stream complete]")
💻 Output:

TEXT
Streaming response: Quantum computing uses quantum bits (qubits) that can exist in superposition, representing 0 and 1 simultaneously. This allows quantum computers to process many calculations in parallel, exponentially speeding up certain problems. Key applications include cryptography, drug discovery, and optimization.

[Stream complete]
💡 Tip: stream=True returns an iterator rather than a complete response, making it suitable for displaying content one character at a time in web applications. However, the total number of tokens cannot be determined in advance; it must be obtained via the usage field in the last chunk.


9. Comprehensive Example: AI Copywriting Assistant

Build an "AI Copywriting Assistant": Enter a product description, call the OpenAI API to generate ad copy in three different styles, and print them out for comparison.

▶ Example: AI Copywriting Assistant—Generating Ad Copy in 3 Different Styles (Difficulty: ⭐⭐⭐)

PYTHON
# ============================================
# AI Copywriting Assistant
# Input: product description
# Output: 3 styles of ad copy
# ============================================
from openai import OpenAI
client = OpenAI()

product = "Wireless noise-cancelling headphones, 40-hour battery, $199"

styles = [
    ("Professional", "Write a professional, feature-focused ad copy."),
    ("Emotional", "Write an emotional, lifestyle-focused ad copy."),
    ("Humorous", "Write a humorous, witty ad copy.")
]

print("=" * 60)
print(f"Product: {product}")
print("=" * 60)

for style_name, style_prompt in styles:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": f"You are an expert copywriter. {style_prompt}"},
            {"role": "user", "content": f"Write ad copy for: {product}. Keep it under 80 words."}
        ],
        temperature=0.8,
        max_tokens=150
    )
    copy = response.choices[0].message.content
    tokens = response.usage.total_tokens
    cost = (response.usage.prompt_tokens * 0.15 + response.usage.completion_tokens * 0.60) / 1_000_000
    print(f"\n[{style_name}]")
    print(f"{copy}")
    print(f"\n(Tokens: {tokens}, Cost: ${cost:.6f})")

print("\n" + "=" * 60)
💻 Output:

TEXT
============================================================
Product: Wireless noise-cancelling headphones, 40-hour battery, $199
============================================================

[Professional]
Experience premium noise-cancelling technology with 40 hours of uninterrupted battery life. Our wireless headphones deliver crystal-clear audio and comfortable design for $199. Perfect for professionals who demand focus and quality.

(Tokens: 72, Cost: $0.000032)

[Emotional]
Escape the noise. Slip on these headphones and let the world fade away—40 hours of pure, uninterrupted bliss. Whether it's your morning commute or a quiet evening, your soundtrack awaits. Just $199 for peace you can hear.

(Tokens: 68, Cost: $0.000030)

[Humorous]
Tired of hearing your neighbor's karaoke? These noise-cancelling headphones block the chaos and deliver 40 hours of sweet silence. At $199, it's cheaper than moving. Your ears will thank you.

(Tokens: 58, Cost: $0.000025)

============================================================

❓ FAQ

Q What is the relationship between ChatGPT and GPT-4?
A ChatGPT is the product name (OpenAI’s conversational AI application), while GPT-4 is the model name (the underlying engine). ChatGPT initially used GPT-3.5 and was later upgraded to GPT-4/GPT-4o. To use an analogy: ChatGPT is a car, and GPT-4 is the engine. The same car can be equipped with different engines.
Q Are tokens the same as words?
A No, they are not. In English, approximately 1 token = 0.75 words (4 characters); in Chinese, approximately 1 token = 1–2 Chinese characters. Therefore, 1,000 Chinese characters consume approximately 500–1,000 tokens, which is more than the equivalent length in English. You can use the tiktoken library to calculate this precisely.
Q What is the appropriate setting for the temperature parameter?
A General recommendations—use 0 (deterministic) for code/data extraction/JSON generation; 0.7 (natural and varied) for everyday conversation/writing; and 1.0–1.5 (more varied) for brainstorming/creativity. If you’re unsure, 0.7 is the safest default value.
Q Are open-source models sufficient?
A It depends on the use case. Llama 3.1 (405B) and Qwen 2.5 (72B) perform at a level close to GPT-4 on many tasks and are suitable for: ① scenarios with strict data privacy requirements; ② tasks requiring custom fine-tuning; and ③ cases with high token usage where cost control is a priority. However, if you’re seeking the highest performance and don’t mind your data passing through a third party, proprietary APIs still hold the lead.
Q Is it expensive to call the LLM API?
A It’s not expensive for everyday use. GPT-4o-mini costs about $0.15 per 1 million input tokens, so 1,000 Chinese requests (500 characters each) would cost about $0.10. However, if you’re doing batch processing (such as classifying 100,000 records), the costs can add up quickly. Recommendations: ① Use mini models for small tasks; ② Test batch tasks on a small scale first; ③ For high-volume usage, consider deploying an open-source model yourself.
Q Why do LLMs “hallucinate” (make up facts)?
A Because LLMs are essentially “next-token predictors,” not “fact retrievers.” They learn the statistical patterns of language, but there is no guarantee that the facts they generate are correct. When the model is uncertain, it generates content that “seems reasonable” but is actually incorrect. Mitigation strategies: ① Lower the temperature ② Provide reference materials (RAG) ③ Instruct the model to “say ‘I don’t know’ if it’s uncertain.”

📖 Summary


📝 Exercises

  1. Basic Problem (Difficulty ⭐): Write a small translation program using the OpenAI API—it should take Chinese input and output an English translation. Requirements: Specify the role as “Translation Assistant” in the system prompt, and set temperature=0 to ensure consistent translations.
  2. Advanced Exercise (Difficulty ⭐⭐): Using the same prompt (e.g., "Write a poem about autumn"), call the model 5 times each with temperature=0 and temperature=1. Record and compare the differences in the output, then summarize the patterns of how temperature affects the output.
  3. Challenge (Difficulty: ⭐⭐⭐): Use the API to build a simple chatbot: Support multi-turn conversations (maintain a messages array), and exit the loop when it encounters "bye"; include a prompt asking the bot to play a specific role (such as "Python programming tutor"), and try to print the responses word-for-word using stream output.
Web-Tutorial.com

Web-Tutorial Tech Team

A team of developers maintaining programming tutorials. Each tutorial is written and reviewed by developers with expertise in that field. We work to keep our content accurate and reliable — if you spot an issue, please let us know.

100%

🙏 帮我们做得更好

我们是刚上线的编程教程站,几个人的小团队,精力有限。页面虽经检查,难免还有疏漏——链接失效、排版错乱、内容有误、语言生硬……

如果您发现了,麻烦告诉我们,我们会在收到反馈后第一时间进行修复,再次感谢您的光临 🙏