National Science Foundation, Award IIS-2040898
FAI: End-to-End Fairness for Algorithm-in-the-Loop Decision Making in the Public Sector
Supported by the NSF Program on Fairness in Artificial Intelligence in Collaboration with Amazon
Project details:
PI: Daniel B. Neill
Co-PIs: Ravi Shroff, Constantine Kontokosta, Edward McFowland III
Project duration: February 1, 2021- January 31, 2025
Funding amount: $1,000,000 (total, including $625,000 NSF grant and $375,000 unrestricted gifts from Amazon)
Project personnel:
Daniel B. Neill (Associate Professor of Computer Science, Public Service, and Urban Analytics, NYU) (PI)
Ravi Shroff (Associate Professor of Applied Statistics, NYU) (co-PI)
Constantine Kontokosta (Associate Professor of Urban Science and Planning, NYU) (co-PI)
Edward McFowland III (Assistant Professor of Technology and Operations Management, Harvard Business School) (co-PI)
Samrachana Adhikari (Associate Professor of Biostatistics, NYU)
Rozalen Adous (high school student, NYU ARISE program)
Chris Anzalone (MS student, NYU)
Tamanna Begum (high school student, NYU ARISE program)
Isaac Bohart (MD student, NYU Langone School of Medicine)
Kate Boxer (PhD student, NYU)
Jared Burke (undergraduate student, NYU)
Julie Cestaro (MS student, NYU)
Peter Chang (research assistant, Harvard Business School)
Stacy Chao (MS student, NYU)
Eric Corbett (Smart Cities/Provost’s Postdoctoral Fellow, NYU)
Gordon Dai (undergraduate student, NYU)
Sophia Deng (MS student, NYU)
Ellie Haber (undergraduate student, NYU)
Bo Hong (postdoctoral researcher, NYU)
Betty Hou (PhD student, NYU)
Benjamin Jakubowski (graduate student, NYU)
Samah Karim (senior research software engineer, Harvard Business School)
Sasha Lefevre (MS student, NYU)
Erica Liberman (high school student, NYU ARISE program)
Gabe Lora (undergraduate student, NYU)
Neil Menghani (MS student, NYU)
Serene Mo (undergraduate student, NYU)
Omotara Oloye (undergraduate student, UC Berkeley)
John Pamplin (Assistant Professor of Epidemiology, Columbia University Mailman School of Public Health)
Pragya Parthasarathy (undergraduate student, NYU)
Pavan Ravishankar (PhD student, NYU)
Rushabh Shah (MS student, NYU)
Michelle Vaccaro (PhD student, MIT)
Sheng Wang (Smart Cities Postdoctoral Fellow, NYU)
Ryan Zhang (high school student, NYU ARISE program)
Project abstract:
The goal of this project is to develop methods and tools that assist public sector organizations with fair and equitable policy interventions. In areas such as housing and criminal justice, critical decisions that impact lives, families, and communities are made by a variety of actors, including city officials, police, and court judges. In these high-stakes contexts, human decision makers’ implicit biases can lead to disparities in outcomes across racial, gender, and socioeconomic lines. While artificial intelligence (AI) offers great promise for identifying and potentially correcting these sorts of biases, a rapidly growing literature has shown that automated decision tools can also worsen existing disparities or create new biases. To help bridge this gap between the promise and practice of AI, the interdisciplinary team of investigators will develop an integrated framework and new methodological approaches to support fair and equitable decision-making. This framework is motivated by three main ideas: (1) identifying and mitigating the impacts of biases on downstream decisions and their impacts, instead of simply measuring biases in data and in predictive models; (2) enabling the combination of an algorithmic decision support tool and a human decision maker to make fairer and more equitable decisions than either human or algorithm alone; and (3) developing operational definitions of fairness and quantitative assessments of bias, guided by stakeholder discussions, that are directly relevant and applicable to the housing and criminal justice domains. The ultimate impact of this work is to advance social justice for those who live in cities, and who rely on city services or are involved with the justice system, by assessing and mitigating biases in decision-making processes and reducing disparities.
The project team will address both the risks and the benefits of algorithmic decision-making through transformative technical contributions. First, they will develop a new, pipelined conceptualization of fairness consisting of seven distinct stages: data, models, predictions, recommendations, decisions, impacts, and outcomes. This end-to-end fairness pipeline will account for multiple sources of bias, model how biases propagate through the pipeline to result in inequitable outcomes, and assess sensitivity to unmeasured biases. Second, they will build a general methodological framework for identifying and correcting biases at each stage of this pipeline, assessing intersectional and contextual biases across multiple data dimensions, and incorporating new ideas for model assessment and analysis of heterogeneous treatment effects. This generalized bias scan will provide essential information throughout the end-to-end fairness pipeline, informing not only what human and algorithmic biases exist, but what interventions are likely to mitigate these biases. Third, the project addresses algorithm-in-the-loop decision processes, in which an algorithmic decision support tool provides recommendations to a human decision-maker. The investigators will develop approaches for modeling systematic biases in human decisions, identifying possible explanatory factors for those biases, and optimizing individualized algorithmic “nudges” to guide human decisions toward fairness. Finally, the project team will create new metrics for measuring the presence and extent of bias. The outputs of the project will be designed for integration into the operational decision-making of city agencies responsible for making fair and equitable decisions in the criminal justice and housing domains. The investigators will assess the fairness of existing practices, and create open source tools for assessing and correcting biases, for users in each domain. They will develop tools which can be used to (a) reduce incarceration by equitably providing supportive interventions to justice involved populations; (b) prioritize housing inspections and repairs; (c) assess and improve the fairness of civil and criminal court proceedings; and (d) analyze the disparate health impacts of adverse environmental exposures, including poor-quality housing and aggressive, unfair policing practices. Operational deployments of the developed tools will be regularly and comprehensively evaluated to assess impacts and to avoid unintended consequences, both maximizing the benefits and minimizing potential harms from both algorithmic and human decisions.
Publications:
Sam Corbett-Davies, Johann D. Gaebler, Hamed Nilforoshan, Ravi Shroff, and Sharad Goel. The measure and mismeasure of fairness. Journal of Machine Learning Research 24(312): 1−117, 2023. (pdf)
Charles R. Doss and Edward McFowland III. Nonparametric subset scanning for detection of heteroscedasticity. Journal of Computational and Graphical Statistics 31(3): 813-823, 2022. doi:10.1080/10618600.2022.2026779.
Constantine E. Kontokosta, Boyeong Hong, and Bartosz J. Bonczak. Measuring sensitivity to social distancing behavior during the COVID-19 pandemic. Scientific Reports 12: 16350, 2022. doi:10.1038/s41598-022-20198-4.
Constantine E. Kontokosta, Boyeong Hong, and Bartosz J. Bonczak. Socio-spatial inequality and the effects of density on COVID-19 transmission in U.S. cities. Nature Cities, 2024, in press.
Edward McFowland III. Commentary on “Causal decision making and causal effect estimation are not the same… and why it matters”. INFORMS Journal on Data Science 1(1): 21-22, 2022. doi:10.1287/ijds.2021.0010. (pdf)
Hamed Nilforoshan, Johann D. Gaebler, Ravi Shroff, and Sharad Goel. Causal conceptions of fairness and their consequences. Proc. 39th International Conference on Machine Learning, PMLR 162: 16848-16887, 2022. (pdf)
Charles A. Pehlivanian and Daniel B. Neill. Efficient optimization of partition scan statistics via the Consecutive Partitions Property. Journal of Computational and Graphical Statistics 32(2): 712-729, 2023. (pdf)
Pavan Ravishankar, Qingyu Mo, Edward McFowland III, and Daniel B. Neill. Provable detection of propagating sampling bias in prediction models. Proc. 37th AAAI Conf. on Artificial Intelligence, 9562-9569, 2023. (pdf) (supplement)
Ravi Shroff and Konstantinos Vamvourellis. Pretrial release judgments and decision fatigue. Judgment and Decision Making 17(6): 1176-1207, 2022. (pdf)
Neil Menghani. Insufficiently Justified Disparate Impact: A New Criterion for Fair Recommendations. New York University, Courant Institute of Mathematical Sciences, Department of Computer Science. MS Thesis, 2022.
Qingyu Serene Mo. Modeling Stage-Specified Propagation of Bias with COMPAS Data. New York University, Courant Institute of Mathematical Sciences. Undergraduate Honors Thesis, 2022.
Working papers:
I. C. Bohart, J. R. Caldwell, J. L. Swartz, P. Rosen, N. Genes, C. A. Koziatek, D. B. Neill*, and D. C. Lee*. Fairness and bias of machine learning approaches for diabetes screening in the emergency department. Submitted for publication.
Kate Boxer, Boyeong Hong, Constantine Kontokosta, and Daniel B. Neill. Estimating reporting bias in 311 complaint data. Submitted for publication.
Kate Boxer, Edward McFowland III, and Daniel B. Neill. Auditing predictive models for intersectional biases. Submitted for publication. (arXiv:2306.13064)
Neil Menghani, Edward McFowland III, and Daniel B. Neill. Insufficiently justified disparate impact: a new criterion for fair recommendations. Submitted for publication. (arXiv:2306.11181)
This material is based upon work supported by the National Science Foundation Program on Fairness in Artificial Intelligence in Collaboration with Amazon, grants IIS-2040898 (primary funding source) and IIS-1926470. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation or Amazon.
Last update: 12/19/2023.