Autocorrelation Matrix Knowledge Distillation: A Task-Specific Distillation Method for BERT Models

Pre-trained language models perform well in various natural language processing tasks. However, their large number of parameters poses significant challenges for edge devices with limited resources, greatly limiting their application in practical deployment. This paper introduces a simple and effici...

Ausführliche Beschreibung

Bibliografische Detailangaben

Hauptverfasser:	Kai Zhang, Jinqiu Li, Bingqian Wang, Haoran Meng
Format:	Artigo
Sprache:	Inglês
Veröffentlicht:	MDPI AG 2024-10-01
Schriftenreihe:	Applied Sciences
Schlagworte:	BERT knowledge distillation model compression
Online-Zugang:	https://www.mdpi.com/2076-3417/14/20/9180
Tags:	Tag hinzufügen Keine Tags, Fügen Sie das erste Tag hinzu!