# Information-Theoretic Models of Language Processing

## 1. Course Information

Lecture times | MTWRF 1-2:30pm |

Lecture Location | MWRF: SBSG 1517; T: SSPB 1222 |

Syllabus | http://socsci.uci.edu/~rfutrell/teaching/itlp2022 |

## 2. Instructor Information

Instructor | Richard Futrell (rfutrell@uci.edu) |

## 3. Course Description

Human language processing is the process by which a person transforms an input linguistic signal (audio, text, etc.) into a representation of meaning during comprehension, or transforms a representation of meaning into linguistic output during production. Information theory is the mathematical theory of information and communication. We will develop simple information-theoretic models that can explain some aspects of human language comprehension, with a focus on explaining eyetracking data in which eye movements are taken as an index of the mental processes that comprehenders are performing.

## 4. Course Format

Course time will be spent on lectures, discussions, exercises, and demos. You will follow along and perform small group exercises in CoLab notebooks.

## 5. Intended audience

This course is intended for linguists and cognitive scientists of all backgrounds. The course assumes familiarity with the Python programming language, but only minimal familiarity with statistics, probability, information theory, etc.

## 6. Schedule (subject to modification)

## 7. Resources

**On information theory**There is a Khan Academy video course on information theory, which is highly recommended.

James Gleick wrote a popular book about information theory. The Information: A History, A Theory, a Flood.

The comprehensive textbook on information theory is Cover & Thomas (2006). Prof. Cover's lectures based on the book are online. If you have a strong math background, this is the book to work through.

A more accessible introduction is given in MacKay (2003).

**On probability**If you would like to brush up on probability theory, I recommend watching John Tsitiklis's lectures.