在这篇文章中，我们将探讨大语言模型的发展史，以及如何训练出一个强大的LLM。我们从数据的收集和预处理开始，逐步介绍模型架构，训练方法，后训练，评估，部署，以及应用。之后我们将探讨LLM的应用，包括LLM-Based Agents，等等。最后我们将展望LLM的未来发展趋势。
在这篇文章中，我们将深入探讨LLM的高性能推理（Inference）技术，包括量化（Quantization），剪枝（Pruning），知识蒸馏（Knowledge Distillation）等方法。
随着OpenClaw在2026年初的爆火，Context Engineering和Harness Engineering的概念被提出并且重新被人关注。在这篇文章中，我们将深入探讨这两个概念在LLM应用中的重要性和实践方法。
Detailed exploration of LLM architectures from Transformer to Mamba, RetNet, and beyond, covering attention mechanisms, MoE, and the complete training pipeline.
In this blog, we explore how reinforcement learning (RL) is used to post-train large language models (LLMs) to improve performance, robustness, and alignment. We cover RLHF, PPO, DPO, RLVR, GRPO, GSPO, DAPO, SAPO, and GDPO.
KL Divergence is a fundamental concept in information theory and statistics. We explore entropy as the foundation, then delve into KL Divergence, its mathematical formulation, Monte Carlo Estimators, and applications in classification, generative modeling, knowledge distillation, reinforcement learning, and LLMs.
In this blog, I will cover all kinds of learning paradigms in Deep Learning, including supervised learning, unsupervised learning, self-supervised learning, contrastive learning, curriculum learning, and more.
In this blog, I will go through different position encoding mechanisms used in Transformer-based neural network architectures, from sinusoidal to RoPE, and extend to the visual domain.
Comprehensive overview of diffusion models: DDPM, DDIM, Score Matching, ODE/SDE, Flow Matching, Stable Diffusion, and related architectures.
Training large neural networks can be time-consuming and resource-intensive. This blog explores various techniques to speed up training, including single-GPU optimization and multi-GPU parallelization strategies.
