Language Inclusivity with Foundation Models

Speaker: Trevor Cohn

Affiliation: Google/Unimelb

Abstract

Large language models (LLM) models are having enormous impact in a myriad of application environments. Initially progress was limited to the English language, but is now increasingly multilingual. In this talk, I will outline the progress in LLM research, and the opportunities they afford to multilingual settings. In particular, I will outline two research methods developed in Google, namely multilingual prompting which exploits in-context learning and multilingual language understanding to improve translation, and model composition, a means by which trained models can be cheaply combined to realise synergistic benefits from their ensemble, providing a means to move towards more reusable and modular systems. I will finish by discussing some open research problems in this critical area.

Biography

Trevor Cohn is a Research Scientist at Google Research Australia, where he leads the language research team developing generative AI technologies, with a focus on better multilingual and multimodal models. He also is a Professor at the University of Melbourne, where he has worked since 2014, having previously worked at the University of Sheffield and the University of Edinburgh. His research interests focus on development of probabilistic and statistical machine learning methods for modelling natural language text, with particular interests in machine translation, multilingual transfer and parsing, reducing cultural bias and stereotyping in NLP systems, and improving the robustness of NLP systems to adversarial attacks. He has been awarded an ARC Future Fellowship, among several other academic grants, and serves on the board of Transactions of ACL and senior programme committee for several leading conferences in NLP, AI and machine learning.