Linguis 159: Language Processing

1 Course Information

Lecture times Tuesdays & Thursdays 12:30-1:50pm
Lecture Location SST 442
Canvas site

2 Instructor Information

Instructor Richard Futrell (
Instructor's office SSPB 2215
Instructor's office hours Mondays 4-5pm

3 Course Description

This course is on human language processing: what is the process in the human mind that converts language to meaning and meaning to language. We will cover experimental studies on human language understanding, as well as models and approaches from computational linguistics. Students will learn how to formulate and test precise theories of how language processing works by discussing and evaluating state-of-the-art research papers.

Detailed topics: Bayesian inference and information theory as underlying principles of language processing, speech perception, noisy channel models of sentence understanding, human languages as efficient codes for meaning, (probabilistic) context-free grammars for modeling syntactic structure, working memory effects on sentence processing, language evolution and how constraints on language processing shape languages, distributional vector-space methods for modeling word meanings, modern neural network methods for modeling language and deriving meaning from language.

4 Course Format

Class time will be spent on a mixture of lectures and seminar-style discussions about research papers. Homework will consist of short paper responses. The major assessments will be two review papers in which you review, evaluate, and propose extensions to a research paper of your choice.

Students may bring laptops to class as long as they are closed during lectures and discussions, unless we are using them as part of exercises.

5 Intended audience

This course is intended for advanced undergraduates studying language science, cognitive science, computer science, psychology, languages, and related fields. Some background in linguistics, such as Linguis 3, will make the class easier, but we will be reviewing the necessary concepts from linguistics as we go. We will be developing models using some probability theory: some background in probability will make the class and readings easier, but we will introduce/review the necessary concepts early on in the class. We will not do any math that is advanced beyond high-school algebra.

Here is a survey for students beginning the class.

6 Readings

We will have two kinds of readings: background readings and primary-literature readings. There is no course textbook. You don't need to buy anything for this course. All readings are provided as pdf documents either here or on the Canvas site. Some of the pdf documents are password-protected. You can find the password in the announcements on the Canvas site.

  • Background readings. These are readings taken from textbooks which provide context and orientation to a problem we are studying. You will not be directly assessed on your knowledge of these background readings, but you will find that reading them makes lectures and the primary-literature readings dramatically more comprehensible.
  • Primary-literature readings. These are research articles taken from scholarly journals, intended for a scientific audience of other researchers. For each primary-literature reading, you will be completing a paper response, as described below. In addition to the assigned readings, I am also providing a large list of further research articles which will form the basis for your Review Papers.

The background readings are drawn from these books:

  • J&M — Dan Jurafsky & James Martin (2018). Speech and Language Processing, 3rd edition.
  • Sedivy — Julie Sedivy (2018). Language in Mind: An Introduction to Psycholinguistics. Oxford University Press.
  • Gleick — James Gleick (2011). The Information: A History, a Theory, a Flood. Pantheon Books.

7 Syllabus (subject to modification)

Day Topic Background reading Primary-literature reading Deadlines
9/27 Introduction      
10/2 Probability and inference Intro to Bayes' Rule    
10/4 Speech perception Sedivy 4.3    
10/9 Noisy channel models   Gibson et al. (2013)  
10/11 Efficient Coding I Gleick Ch. 7    
10/16 Efficient Coding II   Mahowald et al. (2013)  
10/18 Ambiguity I Sedivy 8-8.2    
10/23 Ambiguity II   Tanenhaus et al. (1995)  
10/25 Syntactic structure I Sedivy 8.3 J&M 10-10.3   Decide on target article for Review Paper 1
10/30 Syntactic structure II      
11/1 Syntactic structure III / Prediction I Sedivy 8.4    
11/6 Prediction II   Altmann & Kamide (1999)  
11/8 Working memory I Sedivy 8.5, J&M 13-13.4   Review Paper 1 due
11/13 Working memory II   Futrell et al. (2015)  
11/15 Working memory III / Local coherence I      
11/20 Local coherence II   Kamide & Kukona (2018)  
11/22 Thanksgiving break      
11/27 Language evolution I Tamariz & Kirby (2016)    
11/29 Language evolution II   Fedzechkina et al. (2012) Decide on target article for Review Paper 2
12/4 Word meaning I Sedivy 7-7.1, J&M 6-6.3    
12/6 Word meaning II   Caliskan et al. (2017)  
12/14 (Finals week)     Review Paper 2 due

8 Requirements & Grading

  • Grade breakdown

    Work Grade percentage
    Paper responses 35%
    Review paper 1 25%
    Review paper 2 30%
    Participation 10%
  • Description of requirements
    • Paper responses. For each primary-literature reading, you will be required to produce a paper response with your reactions and thoughts about the article. The paper response consists of answers to three discussion questions that I will provide for each article. Paper responses should be completed 1 hour before class, so that I can review your responses ahead of the classroom discussion. Discussion questions about a paper will be made available 4 days before the class where we discuss that paper.
    • Review papers. You will be required to write two review papers (6-8 pages) about original research articles. These are like extended paper responses, with a special focus on critically evaluating the paper, developing new predictions, and proposing experiments to test those predictions. I will provide a pool of research papers that you can choose from for this project. If you wish, you may work in groups of 2 for these projects and turn in joint writeups.

      More info on review papers, including the list of papers you can choose from.

  • Assignment late policy

    Assignments (other than paper responses) can be turned in up to 7 days late; 10% of your score will be deducted for each 24 hours of lateness (rounded up). For example, if an assignment is worth 80 points, you turn it in 3 days late, and earn a 70 before lateness is taken into account, your score will be (1-0.3)*70=49.

  • Working together

    You may work together on homework, but the final writeups that you turn in must be written by you alone. For the Review Papers, you may work together and turn in a joint writeup.

  • Mapping of class score to letter grade

    I grade the course on a curve, but I guarantee minimum grades based on these thresholds:

    Threshold Guaranteed minimum grade
    >= 90% A
    >= 80% B
    >= 70% C
    >= 60% D

    So for example a score of 90.0001% guarantees you an A-, but you could end up with a higher grade due to the curve.

9 Academic Integrity

We will be adhering fully to the standards and practices set out in UCI's policy on academic integrity.

Author: Richard Futrell

Created: 2018-12-04 Tue 14:06