Abstract:

Recent progress in text generation technologies have been transformative in the way people create, access, and share information online and/or through mobile apps.  

In this talk, I will present our themed effort towards controllable text generation – generating natural language utterances with desirable properties. We propose a generic paradigm of constrained text generation via the Monte-Carlo Markov chain sampling. Unlike previous single-pass generation, this paradigm is based on idea of iterative editing and refining text. In addition, the formulation allows flexible specification of desired target in terms of keyword, category, and logical rule constraints. We will present efficient and provably convergent algorithms for the inference problem under constraints. We have deployed several algorithms along this theme into large-scale mobile platforms with significant gain. Finally, I will discuss future challenges in natural language generation.

Short Bio:

Dr. Lei Li is Director of ByteDance AI Lab, leading the research and product development for machine translation, machine writing, AI drug discovery, and intelligent robot. His research interest is on machine learning, data mining, and natural language processing. Lei received his B.S. in Computer Science and Engineering from Shanghai Jiao Tong University (ACM class) and Ph.D. in Computer Science from Carnegie Mellon University, respectively. His dissertation work on fast algorithms for mining co-evolving time series was awarded ACM KDD best dissertation (runner up). His recent work on AI writer Xiaomingbot received 2^nd-class award of Wu Wen-tsün AI in 2017. He is a recipient of CCF distinguished speaker in 2017, and CCF Young Elite award in 2019 (only 6 recipients). Before ByteDance, he worked at EECS department of UC Berkeley and Baidu's Institute of Deep Learning in Silicon Valley. He has served as 2017 KDD Cup co-Chair, 2018 KDD hands-on tutorial co-chair, 2019/2020 KDD sponsorship co-chair, and Area Chairs/SPCs for EMNLP 2019/2020, AACL 2020, CIKM 2019, AAAI 2019/2020, IJCAI 2017. He has published over 70 technical papers and holds more than 10 patents.