Daniel Martin, PhD (UCSB)

Associate Professor of Economics 
Wilcox Family Chair in Entrepreneurial Economics

Department of Economics

University of California, Santa Barbara

“Labeling and Training with Elicited Beliefs”

19 W 4th St, Rm 517, New York, NY 10003

or

Zoom Link: https://nyu.zoom.us/j/95898284580

Meeting ID: 958 9828 4580

Abstract

We introduce the use of incentive-compatible belief elicitation for labeling data and training machine learning models.  Eliciting beliefs truthfully through proper scoring rules is now standard in experiments and surveys, but has not yet been applied to labeling or training.  We conduct an online experiment in which participants were incentivized to truthfully report their belief that a white blood cell was cancerous for a series of cell images and propose methods for labeling each image based on participant reports.  We evaluate these methods by training a convolutional neural net on the labels they generate and find that they outperform standard labeling methods in terms of both accuracy and calibration.

Speaker Bio