Layer-Wise Attention with Pivot Layers for Effective Fine-Tuning of Encoder-Based Language Models

Fine-tuning pre-trained encoder-based language models for down-stream tasks is typically performed by exploiting the output of the last encoder layer. However, an alternative line of research suggests that leveraging representations from multiple encoder layers may yield richer linguistic informatio...

Ausführliche Beschreibung

Bibliografische Detailangaben

Hauptverfasser:	Seung-Dong Lee, Jun-Ha Hwang, Miseo Kim, Young-Seob Jeong
Format:	Artigo
Sprache:	Inglês
Veröffentlicht:	MDPI AG 2026-04-01
Schriftenreihe:	Applied Sciences
Schlagworte:	attention mechanism encoder-based models fine-tuning GLUE benchmark layer-wise attention natural language understanding
Online-Zugang:	https://www.mdpi.com/2076-3417/16/9/4278
Tags:	Tag hinzufügen Keine Tags, Fügen Sie das erste Tag hinzu!