flykhan
2536c937e3
翻译自英文原版 maths-cs-ai-compendium，共 20 章全部完成。 第01章 向量 | 第02章 矩阵 | 第03章 微积分 第04章 统计学 | 第05章 概率论 | 第06章 机器学习 第07章 计算语言学 | 第08章 计算机视觉 | 第09章 音频与语音 第10章 多模态学习 | 第11章 自主系统 | 第12章 图神经网络 第13章 计算与操作系统 | 第14章 数据结构与算法 第15章 生产级软件工程 | 第16章 SIMD与GPU编程 第17章 AI推理 | 第18章 ML系统设计 第19章 应用人工智能 | 第20章 前沿人工智能 翻译说明： - 所有数学公式 $...$ / $$...$$、代码块、图片引用完整保留 - mkdocs.yml 配置中文导航 + language: zh - README.md 已翻译为中文（兼 docs/index.md） - docs/ 目录包含指向各章文件的 symlink - 约 29,000 行中文内容，排除 .cache/ 构建缓存
157 lines
17 KiB
Plaintext
157 lines
17 KiB
Plaintext
# Maths, CS & AI Compendium
|
|
|
|
> An open, intuition-first textbook covering mathematics, computer science, and artificial intelligence from the ground up. Written for curious practitioners, not exam survivors.
|
|
|
|
## About
|
|
- Author: Henry Ndubuaku
|
|
- Repository: https://github.com/HenryNdubuaku/maths-cs-ai-compendium
|
|
- Website: https://henryndubuaku.github.io/maths-cs-ai-compendium/
|
|
- License: Open source educational resource
|
|
- Audience: AI/ML engineers, researchers, and students who want deep understanding
|
|
|
|
## Chapters
|
|
|
|
### Chapter 1: Vectors
|
|
- [Vector Spaces](chapter 01: vectors/01. vector spaces.md): Vector spaces, dimensions, subspaces, closure properties, and their role as the foundation of ML
|
|
- [Vector Properties](chapter 01: vectors/02. vector properties.md): Magnitude, direction, parallelism, orthogonality, linear independence, sparsity, unit vectors
|
|
- [Norms and Metrics](chapter 01: vectors/03. norms and metrics.md): L1, L2, Lp, infinity norms, distance metrics, cosine similarity
|
|
- [Products](chapter 01: vectors/04. products.md): Dot product, cross product, outer product, triple products, cosine similarity
|
|
- [Basis and Duality](chapter 01: vectors/05. basis and duality.md): Basis vectors, change of basis, dual spaces, covectors
|
|
|
|
### Chapter 2: Matrices
|
|
- [Matrix Properties](chapter 02: matrices/01. matrix properties.md): Transpose, trace, rank, determinant, inverse, condition number, norms, positive definiteness
|
|
- [Matrix Types](chapter 02: matrices/02. matrix types.md): Identity, diagonal, symmetric, orthogonal, sparse, Toeplitz, circulant, Hermitian, permutation
|
|
- [Operations](chapter 02: matrices/03. operations.md): Matrix multiplication, Hadamard product, outer product, systems of equations, pseudo-inverse
|
|
- [Linear Transformations](chapter 02: matrices/04. linear transformations.md): Rotation, reflection, scaling, shearing, affine transforms, homogeneous coordinates
|
|
- [Decompositions](chapter 02: matrices/05. decompositions.md): Gaussian elimination, LU, Cholesky, eigendecomposition, QR, SVD, PCA, NMF
|
|
|
|
### Chapter 3: Calculus
|
|
- [Differential Calculus](chapter 03: calculus/01. differential calculus.md): Limits, derivatives, chain rule, differentiation rules, L'Hopital's rule
|
|
- [Integral Calculus](chapter 03: calculus/02. integral calculus.md): Definite and indefinite integrals, fundamental theorem, u-substitution, integration by parts
|
|
- [Multivariate Calculus](chapter 03: calculus/03. multivariate calculus.md): Partial derivatives, gradients, Jacobians, Hessians, directional derivatives
|
|
- [Function Approximation](chapter 03: calculus/04. function approximation.md): Taylor series, Maclaurin series, Fourier series, polynomial approximation
|
|
- [Optimisation](chapter 03: calculus/05. optimisation.md): Critical points, convexity, Newton's method, gradient descent, Lagrange multipliers, KKT
|
|
|
|
### Chapter 4: Statistics
|
|
- [Fundamentals](chapter 04: statistics/01. fundamentals.md): Random variables, distributions, expectation, moments, variance
|
|
- [Measures](chapter 04: statistics/02. measures.md): Dispersion, quartiles, z-scores, correlation, outlier detection
|
|
- [Sampling](chapter 04: statistics/03. sampling.md): Sampling methods, central limit theorem, bootstrap
|
|
- [Hypothesis Testing](chapter 04: statistics/04. hypothesis testing.md): t-tests, p-values, confidence intervals, ANOVA
|
|
- [Inference](chapter 04: statistics/05. inference.md): Statistical inference, MLE, MAP estimation
|
|
|
|
### Chapter 5: Probability
|
|
- [Counting](chapter 05: probability/01. counting.md): Permutations, combinations, factorial, multinomial, inclusion-exclusion
|
|
- [Probability Concepts](chapter 05: probability/02. probability concepts.md): Axioms, conditional probability, Bayes theorem, independence
|
|
- [Distributions](chapter 05: probability/03. distributions.md): Bernoulli, binomial, Poisson, Gaussian, exponential, beta, Dirichlet
|
|
- [Bayesian Methods](chapter 05: probability/04. bayesian.md): Prior, posterior, likelihood, conjugate priors, MCMC
|
|
- [Information Theory](chapter 05: probability/05. information theory.md): Entropy, cross-entropy, KL divergence, mutual information
|
|
|
|
### Chapter 6: Machine Learning
|
|
- [Classical ML](chapter 06: machine learning/01. classical machine learning.md): Naive Bayes, decision trees, random forests, SVM, K-means, GMM
|
|
- [Gradient ML](chapter 06: machine learning/02. gradient machine learning.md): Linear/logistic regression, SGD, Adam, loss functions, regularisation, evaluation
|
|
- [Deep Learning](chapter 06: machine learning/03. deep learning.md): MLPs, CNNs, RNNs, LSTMs, attention, transformers, ViT, autoencoders, VAEs
|
|
- [Reinforcement Learning](chapter 06: machine learning/04. reinforcement learning.md): MDPs, Q-learning, policy gradient, actor-critic, PPO, RLHF, DPO
|
|
- [Distributed Training](chapter 06: machine learning/05. distributed deep learning.md): Data/model/pipeline/tensor parallelism, mixed precision, scaling laws, MoE
|
|
|
|
### Chapter 7: Computational Linguistics
|
|
- [Linguistic Foundations](chapter 07: computational linguistics/01. linguistic foundations.md): Morphology, syntax, semantics, pragmatics, phonology
|
|
- [Text Processing](chapter 07: computational linguistics/02. text processing and classic NLP.md): Tokenisation, TF-IDF, n-grams, NER, POS tagging
|
|
- [Embeddings and Sequence Models](chapter 07: computational linguistics/03. embeddings and sequence models.md): Word2Vec, GloVe, RNNs, LSTMs, seq2seq
|
|
- [Transformers and LMs](chapter 07: computational linguistics/04. transformers and language models.md): Self-attention, BERT, GPT, T5, positional encoding
|
|
- [Advanced Text Generation](chapter 07: computational linguistics/05. advanced text generation.md): MoE, SSMs, Mamba, GQA, MLA, modern LLM architectures, text diffusion
|
|
|
|
### Chapter 8: Computer Vision
|
|
- [Image Fundamentals](chapter 08: computer vision/01. image fundamentals.md): Pixels, colour spaces, filtering, edge detection, SIFT, feature extraction
|
|
- [Convolutional Networks](chapter 08: computer vision/02. convolutional networks.md): Convolution, pooling, LeNet, AlexNet, VGG, ResNet, EfficientNet, MobileNet
|
|
- [Detection and Segmentation](chapter 08: computer vision/03. object detection and segmentation.md): YOLO, SSD, Faster R-CNN, RetinaNet, U-Net, Mask R-CNN, panoptic
|
|
- [Vision Transformers and Generation](chapter 08: computer vision/04. vision transformers and generation.md): ViT, DeiT, Swin, DINO, MAE, GANs, diffusion, flow matching
|
|
- [Video and 3D Vision](chapter 08: computer vision/05. video and 3D vision.md): Optical flow, SlowFast, tracking, stereo depth, NeRF, 3D Gaussian Splatting, SLAM
|
|
|
|
### Chapter 9: Audio & Speech
|
|
- [Digital Signal Processing](chapter 09: audio and speech/01. digital signal processing.md): Waveforms, sampling, Fourier transform, spectrograms, MFCCs
|
|
- [Automatic Speech Recognition](chapter 09: audio and speech/02. automatic speech recognition.md): CTC, RNN-T, Conformer, Whisper, wav2vec
|
|
- [Text to Speech](chapter 09: audio and speech/03. text to speech and voice.md): WaveNet, Tacotron, VITS, voice conversion
|
|
- [Speaker and Audio Analysis](chapter 09: audio and speech/04. speaker and audio analysis.md): Speaker recognition, diarisation, VAD
|
|
- [Source Separation](chapter 09: audio and speech/05. source separation and noise.md): Conv-TasNet, active noise cancellation
|
|
|
|
### Chapter 10: Multimodal Learning
|
|
- [Multimodal Representations](chapter 10: multimodal learning/01. multimodal representations.md): Fusion strategies, CLIP, ALIGN, SigLIP, contrastive learning, InfoNCE
|
|
- [Vision Language Models](chapter 10: multimodal learning/02. vision language models.md): VQA, Flamingo, LLaVA, PaLI, grounding, document understanding
|
|
- [Image and Video Tokenisation](chapter 10: multimodal learning/03. image and video tokenisation.md): VQ-VAE, VQ-GAN, residual quantisation, video tokenisers
|
|
- [Cross-Modal Generation](chapter 10: multimodal learning/04. cross-modal generation.md): DALL-E, Stable Diffusion, Imagen, text-to-video, text-to-audio
|
|
- [Unified Architectures](chapter 10: multimodal learning/05. unified multimodal architectures.md): Gemini, GPT-4o, multimodal agents, world models
|
|
|
|
### Chapter 11: Autonomous Systems
|
|
- [Perception](chapter 11: autonomous systems/01. perception.md): Sensors, sensor fusion, BEVFusion, 3D detection, depth estimation, occupancy networks
|
|
- [Robot Learning](chapter 11: autonomous systems/02. robot learning.md): Kinematics, dynamics, PID/MPC/impedance control, imitation learning, sim-to-real, world models
|
|
- [Vision-Language-Action Models](chapter 11: autonomous systems/03. vision-language-action models.md): VLAs, RT-2, Octo, OpenVLA, Pi-0, action tokenisation
|
|
- [Self-Driving](chapter 11: autonomous systems/04. self-driving.md): Driving stack, motion prediction, planning, end-to-end driving, world models, safety, SAE levels
|
|
- [Space and Extreme Robotics](chapter 11: autonomous systems/05. space and extreme robotics.md): Planetary rovers, communication constraints, underwater, swarm robotics, HRI
|
|
|
|
### Chapter 12: Graph Neural Networks
|
|
- [Geometric Deep Learning](chapter 12: graph neural networks/01. geometric deep learning.md): Symmetry groups, invariance, equivariance, five geometric domains
|
|
- [Graph Theory](chapter 12: graph neural networks/02. graph theory.md): Adjacency, Laplacian, spectral theory, community detection
|
|
- [Graph Neural Networks](chapter 12: graph neural networks/03. graph neural networks.md): Message passing, GCN, GraphSAGE, GIN, over-smoothing, heterogeneous graphs, link prediction
|
|
- [Graph Attention Networks](chapter 12: graph neural networks/04. graph attention networks.md): GAT, GATv2, Graph Transformers, Graphormer, GPS, temporal graphs
|
|
- [3D Graph Networks](chapter 12: graph neural networks/05. 3d graph networks.md): SE(3)-equivariance, SchNet, DimeNet, EGNN, MACE, graph generation, drug discovery
|
|
|
|
### Chapter 13: Computing and OS
|
|
- [Discrete Maths](chapter 13: computing and OS/01. discrete maths.md): Logic, proofs, sets, relations, functions, graph theory, recurrences, computability, P vs NP
|
|
- [Computer Architecture](chapter 13: computing and OS/02. computer architecture.md): Number systems, IEEE 754, logic gates, CPU, ISAs, pipelining, memory hierarchy, virtual memory, I/O, DMA
|
|
- [Operating Systems](chapter 13: computing and OS/03. operating systems.md): Processes, threads, scheduling, memory management, file systems, networking, containers, security
|
|
- [Concurrency and Parallelism](chapter 13: computing and OS/04. concurrency and parallelism.md): Synchronisation, deadlock, lock-free, OpenMP, MPI, async/await, Amdahl's law
|
|
- [Programming Languages](chapter 13: computing and OS/05. programming languages.md): Paradigms, type systems, memory management, compilation, JIT, closures, pattern matching
|
|
|
|
### Chapter 14: Data Structures and Algorithms
|
|
- [Foundations](chapter 14: data structures and algorithms/00. foundations.md): Big O notation, recursion, backtracking, dynamic programming, pattern recognition for interviews
|
|
- [Arrays and Hashing](chapter 14: data structures and algorithms/01. arrays and hashing.md): Arrays, hash tables, two pointers, sliding window, prefix sums (with NeetCode problems)
|
|
- [Linked Lists, Stacks, and Queues](chapter 14: data structures and algorithms/02. linked lists, stacks, and queues.md): Fast/slow pointers, monotonic stack, heaps (with NeetCode problems)
|
|
- [Trees](chapter 14: data structures and algorithms/03. trees.md): BSTs, tries, Union-Find, segment/Fenwick trees (with NeetCode problems)
|
|
- [Graphs](chapter 14: data structures and algorithms/04. graphs.md): BFS, DFS, Dijkstra, topological sort, SCCs (with NeetCode problems)
|
|
- [Sorting and Search](chapter 14: data structures and algorithms/05. sorting and search.md): Merge/quick sort, binary search, greedy, DP, backtracking (with NeetCode problems)
|
|
|
|
### Chapter 15: Production Software Engineering
|
|
- [Linux and CMD](chapter 15: production software engineering/01. linux and CMD.md): Shell, file system, permissions, processes, SSH, essential ML commands
|
|
- [Git and Version Control](chapter 15: production software engineering/02. git and repository management.md): Branching, merging, PRs, git for ML, experiment tracking
|
|
- [Codebase Design](chapter 15: production software engineering/03. codebase design.md): Project structure, clean code, design patterns, config management, API design, AI coding agents
|
|
- [Testing and QA](chapter 15: production software engineering/04. testing and quality assurance.md): pytest, mocking, testing ML code, CI/CD, linting, code review
|
|
- [Deployment and DevOps](chapter 15: production software engineering/05. deployment and devops.md): Docker, model serving, experiment tracking, reproducibility, monitoring, feature stores, orchestration
|
|
|
|
### Chapter 16: SIMD and GPU Programming
|
|
- [Why C++ and How ML Frameworks Work](chapter 16: SIMD and GPU programming/00. why C++ and how ML frameworks work.md): Python/C++ architecture, NumPy/PyTorch/JAX internals, C++ fundamentals, pybind11
|
|
- [Hardware Fundamentals](chapter 16: SIMD and GPU programming/01. hardware fundamentals.md): Moore's law, SIMD concept, roofline model, chip architectures
|
|
- [ARM and NEON](chapter 16: SIMD and GPU programming/02. ARM and NEON.md): NEON intrinsics, I8MM, SME2, SVE, Apple Silicon, auto-vectorisation, Cactus engine
|
|
- [x86 and AVX](chapter 16: SIMD and GPU programming/03. x86 and AVX.md): AVX2/AVX-512/AMX intrinsics, alignment, profiling
|
|
- [GPU Architecture and CUDA](chapter 16: SIMD and GPU programming/04. GPU architecture and CUDA.md): GPU design, CUDA C++, warps, shared memory tiling, advanced optimisations, GPU generations
|
|
- [Triton, TPUs, and Pallas](chapter 16: SIMD and GPU programming/05. triton, TPUs and pallax.md): Triton kernels, Flash Attention, TPU architecture, JAX/Pallas
|
|
- [RISC-V and Embedded Systems](chapter 16: SIMD and GPU programming/06. RISC-V and embedded systems.md): RISC-V V extension, TinyML, TFLM, edge deployment constraints
|
|
- [Vulkan Compute](chapter 16: SIMD and GPU programming/07. vulkan compute and cross-platform GPU.md): Vulkan pipeline, GLSL compute shaders, Kompute, WebGPU
|
|
|
|
### Chapter 17: AI Inference
|
|
- [Quantisation](chapter 17: AI inference/01. quantisation.md): PTQ, QAT, GPTQ, AWQ, QuIP#, HQQ, AQLM, BitNet, FP8, MX formats, KV-cache quantisation
|
|
- [Efficient Architectures](chapter 17: AI inference/02. efficient architectures.md): StreamingLLM, sparse/linear attention, MQA/GQA/MLA, Mamba, Flash Attention, Ring Attention, pruning
|
|
- [Serving and Batching](chapter 17: AI inference/03. serving and batching.md): Prefill vs decode, continuous batching, PagedAttention/vLLM, constrained generation, request routing
|
|
- [Edge Inference](chapter 17: AI inference/04. edge inference.md): On-device runtimes, compiler stack, NPUs, on-device LLMs, federated learning, Cactus engine
|
|
- [Scaling and Deployment](chapter 17: AI inference/05. scaling and deployment.md): Tensor/pipeline parallelism, speculative decoding, prefix caching, KV-cache eviction, inference frameworks, cost optimisation
|
|
|
|
### Chapter 18: ML Systems Design
|
|
- [Systems Design Fundamentals](chapter 18: ML systems design/01. systems design fundamentals.md): Client-server, networking, DNS, load balancing, caching, databases, CAP theorem, message queues, API design
|
|
- [Cloud Computing](chapter 18: ML systems design/02. cloud computing.md): IaaS/PaaS/SaaS, Kubernetes, storage, networking, serverless, cost management, multi-region, IaC
|
|
- [Large Scale Infrastructure](chapter 18: ML systems design/03. large scale infrastructure.md): Distributed systems, microservices, data pipelines, GPU clusters, InfiniBand, fault tolerance, vector search, observability
|
|
- [ML Systems Design](chapter 18: ML systems design/04. ML systems design.md): ML lifecycle, data management, feature stores, A/B testing, feedback loops, fairness, monitoring
|
|
- [ML Design Examples](chapter 18: ML systems design/05. ML design examples.md): Recommendation systems, search ranking, ads, fraud detection, content moderation, RAG chatbot, image search
|
|
|
|
### Chapter 19: Applied AI
|
|
- [AI for Finance](chapter 19: applied AI/01. AI for finance.md): Time series, algorithmic trading, portfolio optimisation, risk modelling
|
|
- [Protein Design](chapter 19: applied AI/02. protein design.md): AlphaFold, protein structure prediction, inverse folding
|
|
- [Drug Discovery](chapter 19: applied AI/03. drug discovery.md): Molecular representations, virtual screening, binding affinity prediction
|
|
- [Agentic Systems](chapter 19: applied AI/04. agentic systems.md): AI agents, tool use, planning, multi-agent systems
|
|
- [Healthcare](chapter 19: applied AI/05. healthcare.md): Medical imaging, clinical NLP, drug safety, health monitoring
|
|
|
|
### Chapter 20: Bleeding Edge AI
|
|
- [Quantum Machine Learning](chapter 20: bleeding edge AI/01. quantum machine learning.md): Qubits, quantum gates, variational circuits, quantum kernels
|
|
- [Neuromorphic Computing](chapter 20: bleeding edge AI/02. neuromorphic computing.md): Spiking neural networks, neuromorphic hardware, event-driven vision
|
|
- [Datacentres in Space](chapter 20: bleeding edge AI/03. datacentres in space.md): Orbital computing, latency, radiation, power constraints
|
|
- [Decentralised AI](chapter 20: bleeding edge AI/04. decentralised AI.md): Federated learning, blockchain ML, distributed training
|
|
- [Brain Machine Interfaces](chapter 20: bleeding edge AI/05. brain machine interfaces.md): Neural decoding, BCI architectures, neural signal processing
|