SwissText 2024: Lewis Tunstall (Huggingface) on "Large Language Models"

Recording of Lewis Tunstall’s keynote at the 9th SwissText Conference in Chur, Switzerland on "Large Language Models".
About SwissText: The Swiss Text Analytics Conference (SwissText) is an annual conference in Switzerland that brings together experts from industry and academia in the fields of Natural Language Processing (NLP), Computational Linguistics and Text Analytics. LINK: www.swisstext.org
Most foundation models typically undergo two stages of training: pre-training and post-training. Pre-training involves training on large-scale datasets to develop a broad understanding of language and other modalities, while post-training tailors the model for specific tasks and improves alignment with human values and objectives. In this talk, I will provide an overview of the primary methods for alignment, emphasizing practical insights and ongoing challenges. These methods include Reinforcement Learning from Human Feedback (RLHF) as well as more recent, popular approaches such as Direct Policy Optimization (DPO) and Kahneman-Tversky Optimization (KTO). Additionally, the talk will explore the role of synthetic data in alignment, where a strong model like GPT-4 is used to
distill its capabilities into small, yet performant models. I will also discuss the challenges associated with evaluating aligned models and highlight the use and limitations of LLMs as proxies for human evaluation.