arXiv subjects

Ke Wan

Publications and source records attributed to Ke Wan.

16 recordsLinked to original sources

arXiv2026

Federated Rule Ensemble Method in Medical Data

Machine learning has become integral to medical research and is increasingly applied in clinical settings to support diagnosis and decision-making; however, its effectiveness depends on access to large, diverse datasets, which are limited within single institutions. Although integrating data across institutions can address this limitation, privacy regulations and data ownership constraints hinder these efforts. Federated learning enables collaborative model training without sharing raw data; however, most methods rely on complex architectures that lack interpretability, limiting clinical applicability. Therefore, we proposed a federated RuleFit framework to construct a unified and interpretable global model for distributed environments. It integrates three components: preprocessing based on differentially private histograms to estimate shared cutoff values, enabling consistent rule definitions and reducing heterogeneity across clients; local rule generation using gradient boosting decision trees with shared cutoffs; and coefficient estimation via $\ell_1$-regularized optimization using a Federated Dual Averaging algorithm for sparse and consistent variable selection. In simulation studies, the proposed method achieved a performance comparable to that of centralized RuleFit while outperforming existing federated approaches. Real-world analysis demonstrated its ability to provide interpretable insights with competitive predictive accuracy. Therefore, the proposed framework offers a practical and effective solution for interpretable and reliable modeling in federated learning environments.

cs.LG↗

arXiv2026

GIST: Targeted Data Selection for Instruction Tuning via Coupled Optimization Geometry

Targeted data selection has emerged as a crucial paradigm for efficient instruction tuning, aiming to identify a small yet influential subset of training examples for a specific target task. In practice, influence is often measured through the effect of an example on parameter updates. To make selection scalable, many approaches leverage optimizer statistics (e.g., Adam states) as an axis-aligned surrogate for update geometry (i.e., diagonal precondition), implicitly treating parameters as coordinate-wise independent. We show that this assumption breaks down in parameter-efficient fine-tuning (PEFT) methods such as LoRA. In this setting, the induced optimization geometry exhibits strong cross-parameter coupling with non-trivial off-diagonal interactions, while the task-relevant update directions are confined to a low-dimensional subspace. Motivated by this mismatch, we propose GIST (Gradient Isometric Subspace Transformation), a simple yet principled alternative that replaces axis-aligned scaling with robust subspace alignment. GIST recovers a task-specific subspace from validation gradients via singular value decomposition (SVD), projects training gradients into this coupled subspace, and scores examples by their alignment with target directions. Extensive experiments have demonstrated that GIST matches or outperforms the state-of-the-art baseline with only 0.29% of the storage and 25% of the computational time under the same selection budget.

cs.LG↗

arXiv2025

From Preferences to Prejudice: The Role of Alignment Tuning in Shaping Social Bias in Video Diffusion Models

Recent advances in video diffusion models have significantly enhanced text-to-video generation, particularly through alignment tuning using reward models trained on human preferences. While these methods improve visual quality, they can unintentionally encode and amplify social biases. To systematically trace how such biases evolve throughout the alignment pipeline, we introduce VideoBiasEval, a comprehensive diagnostic framework for evaluating social representation in video generation. Grounded in established social bias taxonomies, VideoBiasEval employs an event-based prompting strategy to disentangle semantic content (actions and contexts) from actor attributes (gender and ethnicity). It further introduces multi-granular metrics to evaluate (1) overall ethnicity bias, (2) gender bias conditioned on ethnicity, (3) distributional shifts in social attributes across model variants, and (4) the temporal persistence of bias within videos. Using this framework, we conduct the first end-to-end analysis connecting biases in human preference datasets, their amplification in reward models, and their propagation through alignment-tuned video diffusion models. Our results reveal that alignment tuning not only strengthens representational biases but also makes them temporally stable, producing smoother yet more stereotyped portrayals. These findings highlight the need for bias-aware evaluation and mitigation throughout the alignment process to ensure fair and socially responsible video generation.

cs.CL↗

arXiv2025

R-KV: Redundancy-aware KV Cache Compression for Reasoning Models

Reasoning models have demonstrated impressive performance in self-reflection and chain-of-thought reasoning. However, they often produce excessively long outputs, leading to prohibitively large key-value (KV) caches during inference. While chain-of-thought inference significantly improves performance on complex reasoning tasks, it can also lead to reasoning failures when deployed with existing KV cache compression approaches. To address this, we propose Redundancy-aware KV Cache Compression for Reasoning models (R-KV), a novel method specifically targeting redundant tokens in reasoning models. Our method preserves nearly 100% of the full KV cache performance using only 10% of the KV cache, substantially outperforming existing KV cache baselines, which reach only 60% of the performance. Remarkably, R-KV even achieves 105% of full KV cache performance with 16% of the KV cache. This KV-cache reduction also leads to a 90% memory saving and a 6.6X throughput over standard chain-of-thought reasoning inference. Experimental results show that R-KV consistently outperforms existing KV cache compression baselines across two mathematical reasoning datasets.

cs.CL↗

arXiv2025

Causal rule ensemble approach for multi-arm data

Heterogeneous treatment effect (HTE) estimation is critical in medical research. It provides insights into how treatment effects vary among individuals, which can provide statistical evidence for precision medicine. While most existing methods focus on binary treatment situations, real-world applications often involve multiple interventions. However, current HTE estimation methods are primarily designed for binary comparisons and often rely on black-box models, which limit their applicability and interpretability in multi-arm settings. To address these challenges, we propose an interpretable machine learning framework for HTE estimation in multi-arm trials. Our method employs a rule-based ensemble approach consisting of rule generation, rule ensemble, and HTE estimation, ensuring both predictive accuracy and interpretability. Through extensive simulation studies and real data applications, the performance of our method was evaluated against state-of-the-art multi-arm HTE estimation approaches. The results indicate that our approach achieved lower bias and higher estimation accuracy compared with those of existing methods. Furthermore, the interpretability of our framework allows clearer insights into how covariates influence treatment effects, facilitating clinical decision making. By bridging the gap between accuracy and interpretability, our study contributes a valuable tool for multi-arm HTE estimation, supporting precision medicine.

stat.ML↗

arXiv2024

A comparison on constrain encoding methods for quantum approximate optimization algorithm

The Quantum Approximate Optimization Algorithm (QAOA) represents a significant opportunity for practical quantum computing applications, particularly in the era before error correction is fully realized. This algorithm is especially relevant for addressing constraint satisfaction problems (CSPs), which are critical in various fields such as supply chain management, energy distribution, and financial modeling. In our study, we conduct a numerical comparison of three different strategies for incorporating linear constraints into QAOA: transforming them into an unconstrained format, introducing penalty dephasing, and utilizing the quantum Zeno effect. We assess the efficiency and effectiveness of these methods using the knapsack problem as a case study. Our findings provide insights into the potential applicability of different encoding methods for various use cases.

quant-ph↗

arXiv2024

Online Drift Detection with Maximum Concept Discrepancy

Continuous learning from an immense volume of data streams becomes exceptionally critical in the internet era. However, data streams often do not conform to the same distribution over time, leading to a phenomenon called concept drift. Since a fixed static model is unreliable for inferring concept-drifted data streams, establishing an adaptive mechanism for detecting concept drift is crucial. Current methods for concept drift detection primarily assume that the labels or error rates of downstream models are given and/or underlying statistical properties exist in data streams. These approaches, however, struggle to address high-dimensional data streams with intricate irregular distribution shifts, which are more prevalent in real-world scenarios. In this paper, we propose MCD-DD, a novel concept drift detection method based on maximum concept discrepancy, inspired by the maximum mean discrepancy. Our method can adaptively identify varying forms of concept drift by contrastive learning of concept embeddings without relying on labels or statistical properties. With thorough experiments under synthetic and real-world scenarios, we demonstrate that the proposed method outperforms existing baselines in identifying concept drifts and enables qualitative analysis with high explainability.

cs.LG↗

arXiv2024

Interpretable Transformer Hawkes Processes: Unveiling Complex Interactions in Social Networks

Social networks represent complex ecosystems where the interactions between users or groups play a pivotal role in information dissemination, opinion formation, and social interactions. Effectively harnessing event sequence data within social networks to unearth interactions among users or groups has persistently posed a challenging frontier within the realm of point processes. Current deep point process models face inherent limitations within the context of social networks, constraining both their interpretability and expressive power. These models encounter challenges in capturing interactions among users or groups and often rely on parameterized extrapolation methods when modelling intensity over non-event intervals, limiting their capacity to capture intricate intensity patterns, particularly beyond observed events. To address these challenges, this study proposes modifications to Transformer Hawkes processes (THP), leading to the development of interpretable Transformer Hawkes processes (ITHP). ITHP inherits the strengths of THP while aligning with statistical nonlinear Hawkes processes, thereby enhancing its interpretability and providing valuable insights into interactions between users or groups. Additionally, ITHP enhances the flexibility of the intensity function over non-event intervals, making it better suited to capture complex event propagation patterns in social networks. Experimental results, both on synthetic and real data, demonstrate the effectiveness of ITHP in overcoming the identified limitations. Moreover, they highlight ITHP's applicability in the context of exploring the complex impact of users or groups within social networks.

cs.SI↗

arXiv2024

Extention of Bagging MARS with Group LASSO for Heterogeneous Treatment Effect Estimation

Recent years, large scale clinical data like patient surveys and medical record data are playing an increasing role in medical data science. These large-scale clinical data, collectively referred to as "real-world data (RWD)". It is expected to be widely used in large-scale observational studies of specific diseases, personal medicine or precise medicine, finding the responder of drugs or treatments. Applying RWD for estimating heterogeneous treat ment effect (HTE) has already been a trending topic. HTE has the potential to considerably impact the development of precision medicine by helping doctors make more informed precise treatment decisions and provide more personalized medical care. The statistical models used to estimate HTE is called treatment effect models. Powers et al. proposed a some treatment effect models for observational study, where they pointed out that the bagging causal MARS (BCM) performs outstanding compared to other models. While BCM has excellent performance, it still has room for improvement. In this paper, we proposed a new treatment effect model called shrinkage causal bagging MARS method to improve their shared basis conditional mean regression framework based on the following points: first, we estimated basis functions using transformed outcome, then applied the group LASSO method to optimize the model and estimate parameters. Besides, we are focusing on pursing better interpretability of model to improve the ethical acceptance. We designed simulations to verify the performance of our proposed method and our proposed method superior in mean square error and bias in most simulation settings. Also we applied it to real data set ACTG 175 to verify its usability, where our results are supported by previous studies.

stat.ME↗

arXiv2023

Survival causal rule ensemble method considering the main effect for estimating heterogeneous treatment effects

With an increasing focus on precision medicine in medical research, numerous studies have been conducted in recent years to clarify the relationship between treatment effects and patient characteristics. The treatment effects for patients with different characteristics are always heterogeneous, and various heterogeneous treatment effect machine learning estimation methods have been proposed owing to their flexibility and high prediction accuracy. However, most machine learning methods rely on black-box models, preventing direct interpretation of the relationship between patient characteristics and treatment effects. Moreover, most of these studies have focused on continuous or binary outcomes, although survival outcomes are also important in medical research. To address these challenges, we propose a heterogeneous treatment effect estimation method for survival data based on RuleFit, an interpretable machine learning method. Numerical simulation results confirmed that the prediction performance of the proposed method was comparable to that of existing methods. We also applied a dataset from an HIV study, the AIDS Clinical Trials Group Protocol 175 dataset, to illustrate the interpretability of the proposed method using real data. Consequently, the proposed method established an interpretable model with sufficient prediction accuracy.

stat.ME↗

arXiv2023

Causal rule ensemble method for estimating heterogeneous treatment effect with consideration of main effects

This study proposes a novel framework based on the RuleFit method to estimate Heterogeneous Treatment Effect (HTE) in a randomized clinical trial. To achieve this, we adopted S-learner of the metaalgorithm for our proposed framework. The proposed method incorporates a rule term for the main effect and treatment effect, which allows HTE to be interpretable form of rule. By including a main effect term in the proposed model, the selected rule is represented as an HTE that excludes other effects. We confirmed a performance equivalent to that of another ensemble learning methods through numerical simulation and demonstrated the interpretation of the proposed method from a real data application.

stat.ME↗

arXiv2023

Hybrid Quantum Algorithms integrating QAOA, Penalty Dephasing and Zeno Effect for Solving Binary Optimization Problems with Multiple Constraints

When tackling binary optimization problems using quantum algorithms, the conventional Ising representation and Quantum Approximate Optimization Algorithm (QAOA) encounter difficulties in efficiently handling errors for large-scale problems involving multiple constraints. To address these challenges, this paper presents a hybrid framework that combines the use of standard Ising Hamiltonians to solve a subset of the constraints, while employing non-Ising formulations to represent and address the remaining constraints. The resolution of these non-Ising constraints is achieved through either penalty dephasing or the quantum Zeno effect. This innovative approach leads to a collection of quantum circuits with adaptable structures, depending on the chosen representation for each constraint. Furthermore, this paper introduces a novel technique that utilizes the quantum Zeno effect by frequently measuring the constraint flag, enabling the resolution of any optimization constraint. Theoretical properties of these algorithms are discussed, and their performance in addressing practical aircraft loading problems is highly promising, showcasing significant potential for a wide range of industrial applications.

quant-ph↗

arXiv2023

A General Model of Vehicle Routing Guidance Systems based on Distributive Learning Scheme

Dynamic traffic assignment and vehicle route guidance have been important problems in ITS for some time. This paper proposes a new model for VRGS, which takes into consideration of the information propagation, user selection and information reaction. Parameter p is then defined as the updating weight for computing cost of traffic based on a distributive learning scheme. p is calculated through a function which denotes information propagation over time and space and the function needs further optimization. Comparison to static traffic assignment, DTA and feasible strategies are given, and future work is also stated.

cs.NI↗

arXiv2023

Market Making and Pricing of Financial Derivatives based on Road Travel Times

Travel time derivatives are financial instruments that derive their value from road travel times, serving as an underlying asset that cannot be directly traded. Within the transportation domain, these derivatives are proposed as a more comprehensive approach to value pricing. They enable road pricing based not only on the level of travel time but also its volatility. In the financial market, travel time derivatives are introduced as innovative hedging instruments to mitigate market risk, particularly in light of recent stress experienced by the crypto market and traditional banking sector. The paper focuses on three main aspects: (1) the motivation behind the introduction of these derivatives, driven by the demand for hedging; (2) exploring the potential market for these instruments; and (3) delving into the product design and pricing schemes associated with them. The pricing schemes are devised by utilizing real-time travel time data captured by sensors. These data are modeled using Ornstein-Uhlenbeck processes and, more broadly, continuous time autoregressive moving average (CARMA) models. The calibration of these models is achieved through a hidden factor model, which describes the dynamics of travel time processes. The risk-neutral pricing principle is then employed to determine the prices of the derivatives, employing well-designed procedures to identify the market value of risk.

stat.AP↗

arXiv2023

On the properties of Gaussian Copula Mixture Models

This paper investigates Gaussian copula mixture models (GCMM), which are an extension of Gaussian mixture models (GMM) that incorporate copula concepts. The paper presents the mathematical definition of GCMM and explores the properties of its likelihood function. Additionally, the paper proposes extended Expectation Maximum algorithms to estimate parameters for the mixture of copulas. The marginal distributions corresponding to each component are estimated separately using nonparametric statistical methods. In the experiment, GCMM demonstrates improved goodness-of-fitting compared to GMM when using the same number of clusters. Furthermore, GCMM has the ability to leverage un-synchronized data across dimensions for more comprehensive data analysis.

cs.LG↗

arXiv2022

Rules Ensemble Method with Group Lasso for Heterogeneous Treatment Effect Estimation

The increasing scientific attention given to precision medicine based on real-world data has led many recent studies to clarify the relationships between treatment effects and patient characteristics. However, this is challenging because of ubiquitous heterogeneity in the treatment effect for individuals and the real-world data on their background being complex and noisy. Because of their flexibility, various heterogeneous treatment effect (HTE) machine learning (ML) estimation methods have been proposed. However, most ML methods incorporate black-box models that hamper direct interpretation of the interrelationships between individuals' characteristics and the treatments' effects. This study proposes an ML method for estimating HTE based on the rule ensemble method termed RuleFit. The main advantage of RuleFit are interpretability and accuracy. However, HTEs are always defined in the potential outcome framework, and RuleFit cannot be applied directly. Thus, we modified RuleFit and proposed a method to estimate HTEs that directly interpret the interrelationships among the individuals' features from the model.

stat.ME↗