CMU Advanced NLP Fall 2024 (5): Pre-training and Pre-trained Models

This lecture (by Xiang yue) for CMU CS 11-711, Advanced NLP (Fall 2024) covers:
Overview of pre-training
Pre-training objectives
Pre-training data
Open vs. closed models
Representative pre-trained models
Class Site:
phontron.com/c...

[@sammygasana]

I really love the general setups explained by xiang , sometimes especially when starting it helps to have a general heuristic idea of what works then experiment from there.

[@AdityaSharma-ld2ed]

Hi Graham, when will the new video of the course come out?

[@AbidineVall]

Hello, when will the new course be available?