In this course, we’ll read and discuss the latest language modeling and representation learning methods in natural language processing. This includes prominent deep learning architectures including transformers, methods of self-supervised learning and transfer learning, contrastive learning, large language models and the power of scale, emergent properties of large language models, parameter efficient fine-tuning methods, learning from few training examples and task instructions, methods for making large language models more efficient, applications to other fields, and other recent topics in contemporary NLP.

Prerequisite Knowledge:

Students must have experience with machine learning and deep learning including necessary mathematical background. Experience with natural language processing is a plus.

Students should also feel comfortable with implementing basic machine learning algorithms and understanding/running open source machine learning code.

Students should also have experience with reading machine learning papers and developing a decent understanding of the main concepts/ideas presented in the paper.

At least one of the following courses is recommended as prerequisite:

  • CPSC 477/577 Natural Language Processing
  • CPSC 452 Deep Learning Theory and Applications
  • CPSC 552 Deep Learning Theory and Applications
  • CPSC 677 Advanced NLP

Note: Students who haven’t taken any of these courses but feel comfortable with deep learning and modern NLP, and students from other relevant departments such as statistics, linguistics, neuroscience and biomedical science are welcome to participate but please contact the instructor for approval.

  • Lectures: Tue, Thu 2:30 PM - 3:45 PM
  • Lecture Location: WHL 120