arXiv subjects

Harry Cheon

Publications and source records attributed to Harry Cheon.

4 recordsLinked to original sources

arXiv2026

Measuring What Matters: Synthetic Benchmarks for Concept Bottleneck Models

Concept bottleneck models predict outcomes from high-level concepts detected in inputs. Although concepts provide a simple way to reap benefits from interpretability, very few datasets include concept labels. This limits researchers' ability to determine which problems are suitable for these models, isolate the factors that drive their performance or lead to failures, or uncover which algorithms perform well. In this paper, we develop synthetic benchmarks for concept-bottleneck models, focusing on their two main use cases: decision support, in which models assist humans in making better decisions, and automation, in which models handle routine tasks without supervision. Our benchmarks can generate labeled datasets while controlling for properties that affect performance, including data modality, concept choice, annotation quality, and completeness. We demonstrate how the benchmarks can be used to evaluate representative classes of concept bottleneck models. Our demonstrations show how the benchmarks can diagnose failure modes and guide follow-up testing.

cs.LG↗

arXiv2026

Power Consumption Patterns Using Telemetry Data

This paper examines the analysis of package power consumption using Intel's telemetry data. It challenges the prevailing belief that hardware choice is the primary determinant of a device's power consumption and instead emphasizes the significant role of user behavior. The paper includes two sections: Exploratory Data Analysis (EDA) and a linear model for power consumption. The EDA section provides valuable insights from Intel's telemetry data, comparing power consumption across countries, with a specific focus on power consumption patterns in the US and China. Our simple linear model affirms those patterns and highlight the possible importance of user behavior and its influence on power consumption. Ultimately, the paper underscores the need to understand power consumption patterns and identifies areas where stakeholders like Intel can make improvements to reduce environmental impact effectively and efficiently.

cs.CY↗

arXiv2025

Responsiveness Verification: Will Predictions Change? How Much? How Often?

Machine learning models are often used in applications where their inputs change due to routine interactions, strategic manipulation, or noise. In such settings, models can undermine safety as these changes lead them to predict over regions of input space they have not seen. We propose to address these challenges by measuring responsiveness---the probability that a model output changes when its inputs change under an interaction model. We develop algorithms to estimate responsiveness for any machine learning model, and a framework to specify broad classes of interaction models. We pair these algorithms with statistical guarantees that support practical validation. We demonstrate how our tools can promote safety and reliability across domains by detecting preclusion in recidivism prediction, estimating the cost of gaming in content moderation, and testing the robustness of benchmarks for LLMs.

cs.LG↗

arXiv2024

Feature Responsiveness Scores: Model-Agnostic Explanations for Recourse

Consumer protection rules require companies that deploy models to automate decisions in high-stakes settings to explain predictions to decision subjects. These rules are motivated, in part, by the belief that explanations can promote recourse by revealing information that decision subjects can use to contest or overturn their predictions. In practice, companies provide individuals with a list of principal reasons based on feature importance derived from methods like SHAP and LIME. In this work, we show how common practices can fail to provide recourse and propose to highlight features based on their responsiveness -- the probability that a decision subject can attain a target prediction through an arbitrary intervention on the feature. We develop efficient methods to compute responsiveness scores for any model and actionability constraints. We show that standard practices in lending can undermine decision subjects by highlighting unresponsive features and explaining predictions that are fixed.

stat.ML↗