Natural Language Processing (6.806-864)

Generating and understanding human language remains one of the most exciting (and challenging) frontiers in artificial intelligence research. In this class, we'll survey contemporary prediction problems involving human language data, and introduce probabilistic modeling and representation learning tools that can be used to tackle them.

Course Staff

Instructors
Jacob Andreas (jda@mit.edu)
Jim Glass (glass@csail.mit.edu)

TAs
Abby Bertics
Dylan Doblar
Ekin Akyurek
Evan Hernandez
Harini Suresh
Hongyin Luo
Pranav Krishna
Wei Fang

Admin

Homework, announcements, etc. will be distributed on this Canvas page.

Class will meet on Tuesdays and Thursdays from 1 to 2:30 PM ET via Zoom (see Canvas for link).

Grading

You are encouraged to work together on homework assignments, but all submitted writeups and code should be done on your own.

50% homework, 50% midterm (students in 6.864 will complete extra problems).
Grade scale: A [90, 100]; B [80, 90); C [70, 80); D [60, 70); F [0, 60).

Homework assignments lose 10\% for every late day. Final projects will not be accepted late.

This is not a normal semester! We want everyone to learn from this class, and we can almost certainly find a way to accommodate any issues that arise. If you're struggling, please reach out to Jacob or Jim as soon as possible.

Assignments

• HW1: classification and representation
• HW2: dataset creation
• HW3: sequences and trees
• HW4: question answering

Syllabus

Tu 16 Feb	Introduction	[sp20 slides]
Th 18 Feb	Text classification	[sp20 slides]
Tu 23 Feb	Distributional semantics	[sp20 slides]
Th 25 Feb	Word embeddings	[sp20 slides]
Tu 2 Mar	Classical models of sequences (n-grams and HMMs)	[sp20 slides]
Th 4 Mar	Conditional random fields	[sp20 slides]
Tu 9 Mar	(no class)
Th 11 Mar	Neural sequence models (recurrent NNs)	[sp20 slides]
Tu 16 Mar	Attention mechanisms	[sp20 slides]
Th 18 Mar	Transformers	[sp20 slides]
Tu 23 Mar	(no class)
Th 25 Mar	Pretraining	[sp20 slides]
Tu 30 Mar	Classical models of trees (PCFGs and tree CRFs)	[sp20 slides]
Th 1 Apr	Neural models of trees
Tu 6 Apr	More fun with language models
Th 8 Apr	Machine reading and question answering	[sp20 slides]
Tu 13 Apr	Speech (part 1)	[sp20 slides]
Th 15 Apr	Speech (part 2)
Tu 20 Apr	(no class)
Th 22 Apr	Interpretability
Tu 27 Apr	Dialog	[sp20 slides]
Th 29 Apr	NLP and human language processing
Tu 4 May	Grammar induction
Th 6 May	Language and vision	[sp20 slides]
Tu 11 May	Language and action
Th 13 May	Social and ethical considerations	[sp20 slides]
Tu 18 May	Project presentations
Th 20 May	Project presentations