Efficient Fine-Tuning of Large Language Models via a Low-Rank Gradient Estimator
In this paper, we present a Low-Rank Gradient Estimator (LoGE) to accelerate the finetune-time computation of transformers, especially large language models (LLMs). Unlike Parameter-Efficient Fine-Tuning (PEFT) methods, which primarily aim to minimize the number of fine-tuning parameters, LoGE also...
保存先:
| 主要な著者: | , , , , |
|---|---|
| フォーマット: | Artigo |
| 言語: | Inglês |
| 出版事項: |
MDPI AG
2024-12-01
|
| シリーズ: | Applied Sciences |
| 主題: | |
| オンライン･アクセス: | https://www.mdpi.com/2076-3417/15/1/82 |
| タグ: |
タグなし, このレコードへの初めてのタグを付けませんか!
|