We have a weekly gathering to discuss recent papers in NLP and AI. These are very informal and relaxed reading meetings, so slides are not required although presenters usually make them.
José Camargo de Souza
Nearest Neighbor Machine Translation
Alon Lavie, Craig Stewart and Amin Farajian
Ricardo Rei and Catarina Farinha
COMET: A Neural Framework for MT Evaluation + some findings from participating in the WMT20 Metrics shared task.
Daan Van Stigt and Jose Camargo de Souza
Learning to summarize from human feedback
Ricardo Rei
An Overview on Adapters: What they are and how can we use them for NLG
- Parameter-Efficient Transfer Learning for NLP
- Simple, Scalable Adaptation for Neural Machine Translation
- The Adapter-Bot: All-In-One Controllable Conversational Model
Rita Costa
End-to-End Neural Pipeline for Goal-Oriented Dialogue Systems using GPT-2
Daan Van Stigt
- SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings multilingual-BERT
- Multilingual Alignment of Contextual Word Representations
Pedro Mota
End-to-End Neural Word Alignment Outperforms GIZA++ Slides
Miguel Vera
ACL 2020 Best paper award: Beyond Accuracy: Behavioral Testing of NLP Models with CheckList
Lena Voita
Note: The paper presented by Lena is still under review, thus it is not publicly available. Meanwhile check out her page: Lena Voita published work
Kyunghyun Cho
LxMLS 2020 talk: Question Answering and Generation for Evaluating Summarization
Daan Van Stigt
Posterior Control of Blackbox Generation
Ricardo and Catarina F.
Recent trends in Evaluation: ACL notes on Natural Language Generation Evaluation (focus on MT)
Invited talk from Sean Welleck. where he talked about his current work on Natural Language Generation: Some interesting work by Sean:
- Consistency of a Recurrent Language Model With Respect to Incomplete Decoding
- Neural Text deGeneration with Unlikelihood Training
- Dialogue Natural Language Inference
Patrick Fernandes
Catarina F. and Ricardo
Paper: [2006.06264] Tangled up in BLEU
Austin
Paper: Mirror-Generative Neural Machine Translation
Craig
@jose.souza
Paper: [2004.12681] Lexically Constrained Neural Machine Translation with Levenshtein Transformer
@Ricardo
Paper: [2004.13637] Recipes for building an open-domain chatbot
Slides: retrival vs generative chatbots
@Daan and @Fabio
Paper: [1905.00076] Ensemble Distribution Distillation and [2002.11531] A general framework for ensemble distribution distillation
Slides: RG - Ensemble Distribution Distillation
@Patrick
Paper: [2004.03061] Information-Theoretic Probing for Linguistic Structure
Slides: Information-Theoretic Probing for Linguistic Structure
@Daan
Paper: [2003.12298] Information-Theoretic Probing with Minimum Description Length
Blog: Information-Theoretic Probing with MDL
Slides: Information-Theoretic Probing
Pedro Lobato
Paper: ELECTRA: Pre-Training Text Encoders as Discriminators rather than Generators
Slides: Google Slides
@Katya
Paper: Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models and Mix-review: Alleviate Forgetting in the Pretrain-Finetune Framework for Neural Language Generation Models
Slides: mixout and mix-review
@Daan
Paper: Torch-Struct: Deep Structured Prediction Library
Slides: Torch-Struct
@Nuno Miguel G and Pedro Lobato
Paper: Reformer: The Efficient Transformer (PDF)
@Amin and @António
Paper:
@Ekaterina
Paper: Optimizing data usage via differentiable rewards
Slides: “Optimizing data usage via differentiable rewards” by Xinyi Wang et al.
@Catarina F
Paper: Gmail Smart Compose: Real-Time Assisted Writing
@Fabio and @Amin
Paper: Neural Machine Translation with Soft Prototype
@Miguel and @Nuno Miguel G
Paper: Improving Conditioning in Context-Aware Sequence to Sequence Models
@Ekaterina and @Fabio
Paper: On NMT Search Errors and Model Errors: Cat Got Your Tongue?
@António
Paper: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Pedro Lobato
Paper: Mask-Predict: Parallel Decoding of Conditional Masked Language Models
@Fabio
Paper: Improving Back-Translation with Uncertainty-based Confidence Estimation
@Sérgio
Paper: CORRECTION OF AUTOMATIC SPEECH RECOGNITION WITH TRANSFORMER SEQUENCE-TO-SEQUENCE MODEL
@Ricardo
Paper: 75 Languages, 1 Model: Parsing Universal Dependencies Universally
Rafaela Saraiva
Paper: Training Neural Response Selection for Task-Oriented Dialogue Systems
Pedro Lobato
Paper: TinyBERT
Pedro Lobato
Paper: Large Memory Layers with Product Keys
Invited speaker: Patrick Fernandes
Paper: Structured Neural Summarization (ICLR 2019)
@Ricardo
Paper: Do Neural Dialog Systems Use Conversation History Effectively? An Empirical Study (And if we have time: Pretraining Methods for Dialog Context Representation Learning)
ACL compilation and digest.
@Daan
Title: Neural language models with latent syntax
Description: In this talk I will present my thesis work at the University of Amsterdam under supervision of Wilker Aziz. In the work I investigate semi-supervised and unsupervised learning of the recurrent neural network grammar (RNNG) (Dyer et al. 2016). I will also briefly describe concurrent work by Kim et al. (2019) who (unbeknownst to me) worked on an almost identical approach.
Slides: https://github.com/daandouwe/thesis/blob/master/doc/presentation-unbabel.pdf
Links:
- Thesis https://msclogic.illc.uva.nl/theses/archive/publication/4811/Neural-language-models-with-latent-syntax and code https://github.com/daandouwe/thesis
- Original RNNG (Dyer et al. 2016) https://www.aclweb.org/anthology/N16-1024
- Unsupervised RNNG (Kim et al. 2019) https://www.aclweb.org/anthology/N19-1114
Invited speaker: Daniel Loureiro
He’s going to talk about his accepted paper at ACL:
“Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation”
Antonio Gois
Resuming XLnet and related papers
Slides: non-sequential overview
@Tsvetomila and @Marcos
Paper: [1906.08237] XLNet: Generalized Autoregressive Pretraining for Language Understanding
Slides: XLNet
@Amin and @António
2nd part
@Amin and @António
Papers: TBD