MSCH: Microbatch-Based Selective Activation Checkpointing With Recomputation Hidden for Efficient Training of LLM Models
Activation checkpointing is a widely-used technique to reduce GPU memory consumption during model training. While it helps to conserve memory, it introduces additional computational load. Existing solutions such as selective activation checkpointing (SAC) and microbatch-based selective recomputation...
Kaydedildi:
| Asıl Yazarlar: | , , , , , , , , , , |
|---|---|
| Materyal Türü: | Artigo |
| Dil: | Inglês |
| Baskı/Yayın Bilgisi: |
IEEE
2024-01-01
|
| Seri Bilgileri: | IEEE Access |
| Konular: | |
| Online Erişim: | https://ieeexplore.ieee.org/document/10670415/ |
| Etiketler: |
Etiket eklenmemiş, İlk siz ekleyin!
|