Chao Ma (@ickma2311) “Efficient AI Lecture 14: LLM Post-Training PEFT is one of the most practical ide” — TopicDigg
Chao Ma
@ickma2311
Applied Math × AI × Chip Architectures
Exploring the foundations behind modern intelligence systems.
Sharing clear technical insights & visual explanations.
加入 November 2023
73 正在关注 2.7K 粉丝
Efficient AI Lecture 14: LLM Post-Training
PEFT is one of the most practical ideas in LLM post-training.
Instead of updating the whole model, train a tiny targeted part:
- Adapters: small inserted modules
- Prompt tuning: soft prompt vectors
- Prefix tuning: KV prefixes
- LoRA: low-rank updates
- QLoRA: 4-bit base + LoRA
-Bit-Delta: 1-bit fine-tune delta
Small changes, big adaptation.
My note:
显示更多
