Logistic Regression, Maximum Entropy, Discrete Choice, and All That

Like many powerful techniques, polytomous logistic regression models (PLRMs) have been rediscovered numerous times. One rediscovery, by computational linguists in the early 1990's, led to the popular "maximum entropy" (MaxEnt) approach to statistical language modeling. Novel jargon and notation, along with quirky philosophical argumentation by early authors, has led to confusion about the relationship between PRLMs and MaxEnt. Parameter tying is the key, and computational linguists, it turns out, use logistic regression in the same way as econometricians, in work on discrete choice analysis which led to a Nobel Prize. After comparing views of PLRMs from statistics, linguistics, and economics, I will describe an application with connections to all three: the use of a 6034-class PLRM in evaluating the confusability of drug names.

Back to the schedule