Date: May 4th, 2023 (12:30 pm – 1:30 pm)

Speaker: Daniel Martin

Paper Title: “Labeling and Training with Elicited Beliefs”

Abstract: We introduce the use of incentive-compatible belief elicitation for labeling data and training machine learning models.  Eliciting beliefs truthfully through proper scoring rules is now standard in experiments and surveys, but has not yet been applied to labeling or training.  We conduct an online experiment in which participants were incentivized to truthfully report their belief that a white blood cell was cancerous for a series of cell images and propose methods for labeling each image based on participant reports.  We evaluate these methods by training a convolutional neural net on the labels they generate and find that they outperform standard labeling methods in terms of both accuracy and calibration.

Bio: Daniel Martin is a behavioral, cognitive, and experimental economist who studies attention and perception (how information is processed) and information disclosure (how information is communicated). His current research explores many ways in which human and AI interactions are impacted by attention, perception, and information disclosure. He is currently the Wilcox Family Chair in Entrepreneurial Economics at UCSB, and before receiving a PhD in Economics from NYU, he was the co-founder of a small business that is now one of the leading providers of IT services to small and medium-sized businesses in the Carolinas. At UCSB he teaches undergraduate courses in entrepreneurship and PhD courses on attention and perception.