Alert button

BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model

Add code
Bookmark button
Alert button
Feb 11, 2019
Alex Wang, Kyunghyun Cho

Figure 1 for BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model
Figure 2 for BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model
Figure 3 for BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model
Figure 4 for BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon

We show that BERT (Devlin et al., 2018) is a Markov random field language model. Formulating BERT in this way gives way to a natural procedure to sample sentence from BERT. We sample sentences from BERT and find that it can produce high-quality, fluent generations. Compared to the generations of a traditional left-to-right language model, BERT generates sentences that are more diverse but of slightly worse quality.

* technical report  
View paper onarxiv icon

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon