Date: April 27th, 2023 (12:30 pm – 1:30 pm)

Speaker: Annie Liang

Paper Title: “The Transfer Performance of Economic Models” (joint with Isaiah Andrews, Drew Fudenberg, Lihua Lei, and Chaofeng Wu)

Abstract: Economists often estimate models using data from a particular setting, e.g. estimating risk preferences in a specific subject pool. Whether a model’s predictions extrapolate well across settings depends on whether the estimated model has captured generalizable structure. We provide a tractable formulation for this out-of-domain prediction problem, and define the transfer error of a model to be its performance on data from a new domain. We derive finite-sample forecast intervals that are guaranteed to cover realized transfer errors with a user-selected probability when domains are iid, and use these intervals to compare the transferability of economic models and black box algorithms for predicting certainty equivalents.  We find that in this application, black box algorithms outperform the economic models when estimated and tested on different data from the same domain,  but models motivated by economic theory generalize across domains better than the  black-box algorithms do. 

Bio: Annie Liang is an assistant professor of economics and of computer science (by courtesy) at Northwestern University. Her research is in economic theory—in particular, learning and information—and the application of machine learning methods for model building and evaluation. Prior to joining Northwestern, she was an assistant professor of economics at the University of Pennsylvania and a post-doctoral researcher at Microsoft Research.