2024 | OriginalPaper | Buchkapitel
DALLMi
: Domain Adaption for LLM-Based Multi-label Classifier
verfasst von : Miruna Bețianu, Abele Mălan, Marco Aldinucci, Robert Birke, Lydia Chen
Erschienen in: Advances in Knowledge Discovery and Data Mining
Verlag: Springer Nature Singapore
Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.
Wählen Sie Textabschnitte aus um mit Künstlicher Intelligenz passenden Patente zu finden. powered by
Markieren Sie Textabschnitte, um KI-gestützt weitere passende Inhalte zu finden. powered by
Abstract
DALLMi
, Domain Adaptation Large Language Model interpolator, a first-of-its-kind semi-supervised domain adaptation method for text data models based on LLMs, specifically BERT. The core of DALLMi
is the novel variation loss and MixUp regularization, which jointly leverage the limited positively labeled and large quantity of unlabeled text and, importantly, their interpolation from the BERT word embeddings. DALLMi
also introduces a label-balanced sampling strategy to overcome the imbalance between labeled and unlabeled data. We evaluate DALLMi
against the partial-supervised and unsupervised approach on three datasets under different scenarios of label availability for the target domain. Our results show that DALLMi
achieves higher mAP than unsupervised and partially-supervised approaches by 19.9% and 52.2%, respectively.