|
Title
|
|
|
|
BART for knowledge grounded conversations
| |
|
Author
|
|
|
|
| |
|
Abstract
|
|
|
|
| Transformers have shown incredible capabilities for conversation modeling, however, they store factual knowledge in their learned parameters, which is costly to update with new knowledge since it requires retraining. Models trained before the Coronavirus pandemic do not know about COVID-19. In this paper, we investigate how a BART model can be adapted to a knowledge grounded conversational setup. We introduce the notion of ๐๐๐ฆ and ๐๐ข๐๐๐ฆ tokens to retrieve knowledge stored in an external database, that can easily be updated with new knowledge. As factual knowledge can hardly be reduced to a single sentence or vector, we allow the model to retrieve multiple sentences from the memory. Our analysis shows perplexity decreases with the number of passages retrieved from memory. Second, our analysis shows a shared encoder for knowledge retrieval, and conversation understanding reduces the model size and perform as well as a specialized module. |
| |
|
Language
|
|
|
|
English
| |
|
Source (book)
|
|
|
|
Proceedings of KDD Workshop on Conversational Systems Towards Mainstream Adoption (KDD Converseโ20). ACM, New York, NY, USA
| |
|
Publication
|
|
|
|
2020
| |
|
Volume/pages
|
|
|
|
2666
(2020)
, p. 1-6
| |
|
Full text (open access)
|
|
|
|
| |
|