Nathan Barry (@nathanrs) “BERT is just a Single Text Diffusion Step! (1/n) When I first read about languag” — TopicDigg
Nathan Barry
@nathanrs
I like to work on cool things. Edit prediction models @zeddotdev, previously @Apple, @zfellows, CS + Math @utaustin
加入 June 2020
360 正在关注 2.8K 粉丝
BERT is just a Single Text Diffusion Step! (1/n)
When I first read about language diffusion models, I was surprised to find that their training objective was just a generalization of masked language modeling (MLM), something we’ve been doing since BERT from 2018.
The first thought I had was, “can we finetune a BERT-like model to do text generation?”
显示更多
