Efficient Fine-Tuning of Large Language Models via a Low-Rank Gradient Estimator
In this paper, we present a Low-Rank Gradient Estimator (LoGE) to accelerate the finetune-time computation of transformers, especially large language models (LLMs). Unlike Parameter-Efficient Fine-Tuning (PEFT) methods, which primarily aim to minimize the number of fine-tuning parameters, LoGE also...
Tallennettuna:
| Päätekijät: | , , , , |
|---|---|
| Aineistotyyppi: | Artigo |
| Kieli: | Inglês |
| Julkaistu: |
MDPI AG
2024-12-01
|
| Sarja: | Applied Sciences |
| Aiheet: | |
| Linkit: | https://www.mdpi.com/2076-3417/15/1/82 |
| Tagit: |
Ei tageja, Lisää ensimmäinen tagi!
|