AI Technology

Anyone who has used AI knows the feeling: with the same request, a well-written prompt and a badly written one produce wildly different results. The problem is that most people simply aren’t good at writing prompts — not for lack of talent, but for lack of a systematic optimization process. I recently came across an open-source project, Prompt Optimizer — nearly 29k stars on GitHub — built to solve exactly this.

PDF2Markdown - An Intelligent Article Extraction Tool for Large PDF Documents # Project Overview # PDF2Markdown is an intelligent content extraction tool built specifically for large scanned PDF files. It combines traditional OCR technology with modern AI large language models to intelligently extract pure article content from documents, automatically filtering out non-article elements such as images and tables. Mixed Chinese-English documents are fully supported.

Building a Local RAG with LangChain + Ollama + pgvector: A Complete Hands-On Walkthrough from 0 to 1 (with uv Dependency Management and an Interview Guide) # This post is a directly actionable document. Follow it top to bottom and you will build a local RAG (retrieval-augmented generation) system from scratch and understand the key concepts and code. All core scripts come with comments to make learning and interview review easier.

AI Tutorial: CPU/GPU and Large Model Training # This is a highly condensed reference: clearly structured, right to the point — covering CPU/GPU fundamentals, tensors and numerical precision, CUDA and PyTorch in practice, hardware selection, common interview questions, and a debugging checklist. 0. Quick Overview (30 Seconds) # CPU vs GPU: CPUs excel at general-purpose/sequential work; GPUs excel at massive parallelism (matrices/vectors). Large models need GPUs: training/inference is fundamentally matrix multiplication and parallelization — exactly what a GPU’s high concurrency + high-bandwidth memory deliver. Tensors and precision: all data becomes tensors; precision (FP16/FP8) and quantization (INT8/INT4) trade speed/VRAM against quality. The PyTorch GPU mantra: device = "cuda" if ...; model.to(device); data.to(device) Pick a GPU by VRAM first: VRAM first, then bandwidth/compute; for production, prefer full-strength high-quality models or cloud-hosted APIs. 1. CPU vs GPU: Differences, Workloads, and Analogies # 1.1 The One-Line Comparison # Dimension CPU GPU Architecture Few cores, complex control flow Massive small cores, SIMT parallelism Excels at Branching/system tasks/small-scale compute Matrix multiplication, convolution, attention, graphics rendering Task model Time-sliced, low-latency switching Batch- and throughput-oriented Typical use Business logic, scheduling, I/O Main training/inference operators (GEMM, Conv, etc.) 1.2 An Intuitive Analogy # CPU = a veteran expert: meticulous thinking, does one thing at a time with fast switching. GPU = a massive army: hordes of soldiers working simultaneously — built for parallel homogeneous small tasks. 1.3 Optional Mermaid Diagram (CPU Execution vs GPU Parallelism) # 1 2 3 4 5 6 7 8 9 10 11 flowchart LR subgraph CPU["CPU (sequential/few cores)"] A1[Task1-SliceA] --> A2[Task2-SliceB] --> A3[Task3-SliceC] end subgraph GPU["GPU (parallel/many cores)"] B1[Element1 compute]:::p B2[Element2 compute]:::p B3[Element3 compute]:::p B4[Element4 compute]:::p end classDef p fill:#e9f5ff,stroke:#3b82f6,stroke-width:1px; 2. Tensors, Precision, and Quantization (with Examples) # 2.1 Tensor Hierarchy # 0D: scalar 3.14 1D: vector [1,2,3] 2D: matrix (e.g. a 3×3 table) 3D+: still called a tensor (e.g. batch×channel×height×width) Image example: a batch of 32 224×224 RGB images → 32×3×224×224 (or N×H×W×C, depending on the framework).

AI Tutorial: Prompt Engineering # Prompt engineering focuses on designing, optimizing, and strategizing prompts — helping users mobilize the capabilities of large language models more effectively, and pushing their adoption across real-world scenarios and research domains. 1. Basic Concepts # 1.1 What Is Prompt Engineering # A prompt is simply this: you use natural language to tell the model what to do, how to do it, what it may do, and what it must not do. That is all there is to it.

The physical barrier modern large language models confront when scaling context length is rarely raw floating-point throughput (FLOPs). It is the memory-access wall between high-bandwidth memory (HBM) and on-chip static random-access memory (SRAM). The standard Scaled Dot-Product Attention operator established by Vaswani et al. (2017) incurs an $O(N^2)$ memory-traffic overhead as sequence length $N$ grows, reducing self-attention primarily to a memory-bound workload. Understanding the Transformer requires stripping away anthropomorphic cognitive metaphors and grounding the discussion in the geometry of matrix projections alongside the hardware constraints of modern GPUs.

AI Technical Glossary # This reference gathers the core terminology of the LLM field, from basic concepts to advanced technical architecture, to help you build a systematic understanding of the AI technology landscape. 📚 Fundamentals # Term Technical definition Plain-language explanation Example AGI (Artificial General Intelligence) An AI system with human-level intelligence An all-capable AI that can think, learn, and create like a person A robot that can write poetry, code, cook, and chat at the same time LLM (Large Language Model) A large neural network model trained on massive data A “super brain” that understands and generates human language GPT-4, Claude, ERNIE Bot, and the like are all LLMs Training The process of fitting neural network parameters on large data The AI’s “study phase” — like a person absorbing knowledge from books Training a model on all text on the internet until it learns language Inference A trained model generating output from input The AI’s “application phase” — like a person answering questions from what they learned The model generating an answer after you ask it a question Token The smallest unit of text a model processes, a fragment split by a tokenization algorithm The particles of AI language, processed one at a time "我喜欢苹果" → ["我", "喜欢", "苹果"] 🏗️ Architecture # Term Technical definition Plain-language explanation Example Transformer A deep learning architecture based on self-attention, proposed by Google in 2017 The “neural skeleton” of modern AI that lets models understand language efficiently GPT, BERT, and every other large model is built on Transformer Encoder A neural network component that encodes an input sequence into semantic representations The AI’s “understanding unit” — turns text into vectors machines understand BERT uses an encoder for text understanding tasks Decoder A neural network component that generates output token by token based on context The AI’s “writing unit” — generates answers from what it understood The GPT series are all decoder-only models Self-Attention A mechanism that computes how much each element in a sequence relates to the others The AI automatically “focuses on what matters”, like a person picking out key points while reading In “deposit money at the bank”, “bank” attends to “money”; in “fish by the river bank”, it attends to “river” Multi-Head Attention Several self-attention mechanisms run in parallel to capture different types of dependencies The AI understands text from multiple angles at once One head tracks syntax while another tracks semantics Positional Encoding Vector representations that add position information to each token Lets the model know “who comes first, who comes later” “The dog bit the man” and “the man bit the dog” mean different things Query The vector that actively asks for related information — what the current word needs The numeric expression of “what am I looking for” “Apple” queries attributes like taste and color Key The identifier vector for information being queried — what each word can offer The label of “what I can provide” “Sweet” serves as the Key for a taste feature, waiting to be queried Value The representation vector holding the actual content and true semantic information “My actual content”, in numbers The actual semantic representation of “sweet”: [0.8, 0.2, -0.1] Attention Weight Importance scores expressing how much to attend, usually normalized via softmax “How much to pay attention”, quantified 0.8 means strong attention, 0.1 weak; all weights sum to 1 Cross-Attention Attention across two sequences — Query comes from one, Key/Value from another Cross-modal information exchange In image-text matching, text Queries attend to image Keys/Values Causal Attention Attention restricted to the current position and earlier, preventing future information leaks Attention that can “only look backward” When GPT generates the 5th word it can only see the previous 4 Softmax Function An activation function that turns any real-valued vector into a probability distribution Converts scores into “importance percentages” [2,1,0] → [0.67,0.24,0.09], preserving relative magnitudes 🔢 Mathematical Representation # Term Technical definition Plain-language explanation Example Vector A mathematical object with magnitude and direction; an ordered list of numbers A “numeric ID card” that describes a thing with numbers [25, 180, 70] can represent a person’s age, height, and weight Embedding The technique of mapping discrete symbols into a continuous vector space Turns words into “numeric coordinates” "king"→[0.25, -0.12, 0.78, ...] Query / Key / Value The three core vector matrices in self-attention: what is asked, what is labeled, what is delivered Query = what I want, Key = what I can offer, Value = my actual content Query=[0.1,0.2] asks about taste, Key=[0.8,0.1] labels sweetness, Value=[0.9,0.05] is the actual representation of sweetness Feed-Forward Network Applies an independent nonlinear transform at each position Deepens the model’s understanding of each word From “spring” the model further associates “warmth, growth” Layer Normalization Standardizes a layer’s inputs A “stabilizer” for training Prevents gradient explosion or divergence Residual Connection A cross-layer connection that preserves the original information An “express lane” for information, preventing loss Like a shortcut path that keeps deep networks from degrading 🔄 Processing Pipeline # Term Technical definition Plain-language explanation Example Tokenizer Converts text into a sequence of tokens A “knife for chopping text” "Hello world" → ["Hello", " world"] Context Window The maximum number of tokens a model can process The AI’s “memory limit” GPT-4 has a 128K context Decoding Generates text token by token from a probability distribution The AI’s “writing process” Starts generating from the most probable word Temperature A parameter controlling generation randomness A “creativity dial” High temperature is more creative, low more stable Top-p Sampling A sampling strategy based on cumulative probability An “essence filter” Only considers candidates whose cumulative probability reaches 90% Max Tokens Caps the length of generated output A “word-count limiter” Keeps the AI from answering too long 🛠️ Engineering Practice # Term Technical definition Plain-language explanation Example RAG (Retrieval-Augmented Generation) An AI approach combining retrieval and generation An “open-book exam” AI Look up references first, then answer the question Prompt Engineering The craft of designing and optimizing prompts “The art of asking” Helping the AI understand your needs better Fine-tuning Training a pretrained model on a specific task “Targeted job training” Turning a general model into a medical assistant BPE (Byte Pair Encoding) A common tokenization algorithm A “text compression technique” "unhappiness" → ["un","happi","ness"] Detokenization Turns a token sequence back into readable text “Reassembling the pieces” ["我","喜欢","苹果"]→"我喜欢苹果" Streaming Generates output token by token in real time The “typewriter effect” A chatbot thinking while it types 🧠 Classic Models Compared # Term Technical definition Plain-language explanation Example RNN (Recurrent Neural Network) A neural network that processes sequences step by step A “read-one-word-at-a-time AI” Translating "我爱你" word by word LSTM (Long Short-Term Memory) An improved RNN that handles long-range dependencies “A better memory” Can remember content from the beginning CNN (Convolutional Neural Network) A neural network that excels at image patterns An “image specialist” Recognizing cats, dogs, and faces Encoder-Decoder Architecture A model containing both understanding and generation modules An “all-round AI” Machine translation models 📊 Application Scenarios # Term Technical definition Plain-language explanation Example Chat product A user-facing AI application interface An “AI chat shell” ChatGPT, Claude API call An interface for program-to-program communication The “AI phone line” An application calling the OpenAI API Context management The technique of maintaining conversation history The “AI’s memory” A chatbot remembers what you said Multi-turn dialogue A continuous human-machine interaction mode “Ongoing conversation” Ask about the weather, then what to wear Function Calling The model invoking external APIs to perform tasks The “AI’s ability to act” The AI checks the weather or searches automatically 🧩 Model Optimization and Training Techniques # Term Technical definition Plain-language explanation Example LoRA (Low-Rank Adaptation) Fine-tunes model parameters via low-rank matrices “Lightweight fine-tuning” Lets an LLM quickly adapt to a new domain Quantization Represents model parameters at lower precision “Slimming the model down” FP32→INT8 speeds up inference Pruning Removes redundant neurons or connections “Trimming the branches” Cutting useless parameters Distillation (Knowledge Distillation) A large model teaches a small one “Teacher trains the student” GPT-4 teaching a small model Checkpoint A saved intermediate state during model training A “training save point” Prevents losing progress on a power cut 🔍 Vector Retrieval and Knowledge Integration # Term Technical definition Plain-language explanation Example Embedding Model A model that converts text into semantic vectors A “semantic coordinate machine” text-embedding-3-large Vector Database A database supporting vector retrieval A “semantic warehouse” Milvus, Pinecone, FAISS Cosine Similarity Measures how similar two vectors’ directions are A “semantic similarity meter” A cat is sleeping ≈ The cat is resting Knowledge Graph Stores knowledge as nodes and relationships A “knowledge map” apple → is a → fruit Hybrid Search Combines semantic retrieval with keyword matching “Belt-and-suspenders search” Searching both cat and pet animal at once 🧩 Multimodal and Agents # Term Technical definition Plain-language explanation Example Multimodal Model Handles text, images, audio, and other modalities at once A “full-senses AI” GPT-4V, Gemini VLM (Vision-Language Model) Vision-Language Model An AI that can “see pictures” A visual question-answering AI Speech Recognition Converts speech to text A “dictation AI” Voice input methods TTS (Text-to-Speech) Converts text to speech An “AI announcer” The AI reads its answer aloud AI Agent An AI capable of autonomous action and decision-making An AI assistant “that can act” Devin, AutoGPT ⚙️ Model Evaluation and Safety # Term Technical definition Plain-language explanation Example Hallucination The model generating false information “Confident nonsense” Inventing papers or facts Alignment Bringing the model in line with human values “Values training” Tuning a model with RLHF RLHF (Reinforcement Learning from Human Feedback) Optimizes a model with human preferences “Humans teaching AI to speak” How ChatGPT was trained Red Teaming Adversarial testing of model safety A “security penetration test” Testing whether the model leaks secrets Bias Systematic prejudice in model outputs “The AI plays favorites” Preference for a gender or language 🧰 Emerging Trends and Future Directions # Term Technical definition Plain-language explanation Example Mixture of Experts A structure with multiple sub-models activated dynamically A “panel-of-experts AI” The Gemini 1.5 Pro architecture Context Compression Compresses conversation history to save tokens “Memory compression” Summarizing long conversations Memory-Augmented Model An AI combined with long-term memory mechanisms An AI “with a memory” ChatGPT long-term memory Autonomous Agent An AI that can plan and execute tasks on its own A “self-managing AI” AutoGPT, Devin Synthetic Data Virtual training data generated by AI “AI-made textbooks” Expanding a training set with AI 💡 Study Advice # 🎯 Priority for Mastering Core Concepts # Beginner (must know): Token, Embedding, Transformer, LLM Intermediate (important): Self-Attention, RAG, Context Window Advanced (optional): LoRA, Mixture of Experts, Red Teaming 📖 Suggested Learning Path # Understand the basics: what a token is, and why vector representation is needed Master the core architecture: the Transformer encoder-decoder structure Practice application techniques: combining prompt engineering with RAG Go deeper into technical details: attention mechanisms and alignment training 🔗 Concept Map # 1 2 3 4 Fundamentals → Math → Architecture → Processing → Engineering → Optimization → Retrieval → Agents ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ Token → Vector → Transformer → Decoding → RAG → LoRA → Embedding → Agent LLM → Q/K/V → Attention → Context → Prompt → Quant → Knowledge → Memory 🚀 Tip: the AI technology landscape is vast but tightly interconnected. A good approach is to study it systematically along four dimensions: understand → build → optimize → secure.

AI Tutorial: A Guide to AI LLMs, from Basics to Depth # This article takes you deep into the core concepts of large language models — from basic principles to vector representation — building a complete knowledge system step by step. 1. Foundations of AI Application Development # 1.1 Basic Principles and Concepts # In Plain Words # Core mechanism: predict the next word from the previous one — like a word-chain game How it works: output is generated token by token A Deeper Look # A large AI model involves two key stages: