https://pillumina.github.io/tags/gpu/ Recent content in Gpu on CctoctoFX https://pillumina.github.io/imgs/icon_head.png https://pillumina.github.io/imgs/icon_head.png Hugo -- 0.148.2 en Mon, 22 Jun 2026 09:04:00 +0800 https://pillumina.github.io/posts/aiinfra/llm-computation-methodology/part-5/ Mon, 22 Jun 2026 09:04:00 +0800 https://pillumina.github.io/posts/aiinfra/llm-computation-methodology/part-5/ 训练显存完整估算：从单卡四笔账（权重/优化器/梯度/激活）出发，叠加 TP/PP/DP/CP/EP 并行折扣，结合 ZeRO/FSDP、Gradient Checkpointing、Offload 建立训练显存体系。含 M3 完整案例和多模态/LoRA 微调场景。