arXiv subjects

Yi Zhang

Publications and source records attributed to Yi Zhang.

At least 631 records · Page 35Linked to original sources

arXiv2024

MedSAM-U: Uncertainty-Guided Auto Multi-Prompt Adaptation for Reliable MedSAM

The Medical Segment Anything Model (MedSAM) has shown remarkable performance in medical image segmentation, drawing significant attention in the field. However, its sensitivity to varying prompt types and locations poses challenges. This paper addresses these challenges by focusing on the development of reliable prompts that enhance MedSAM's accuracy. We introduce MedSAM-U, an uncertainty-guided framework designed to automatically refine multi-prompt inputs for more reliable and precise medical image segmentation. Specifically, we first train a Multi-Prompt Adapter integrated with MedSAM, creating MPA-MedSAM, to adapt to diverse multi-prompt inputs. We then employ uncertainty-guided multi-prompt to effectively estimate the uncertainties associated with the prompts and their initial segmentation results. In particular, a novel uncertainty-guided prompts adaptation technique is then applied automatically to derive reliable prompts and their corresponding segmentation outcomes. We validate MedSAM-U using datasets from multiple modalities to train a universal image segmentation model. Compared to MedSAM, experimental results on five distinct modal datasets demonstrate that the proposed MedSAM-U achieves an average performance improvement of 1.7\% to 20.5\% across uncertainty-guided prompts.

cs.CV↗

arXiv2024

Ptychographic non-line-of-sight imaging for depth-resolved visualization of hidden objects

Non-line-of-sight (NLOS) imaging enables the visualization of objects hidden from direct view, with applications in surveillance, remote sensing, and light detection and ranging. Here, we introduce a NLOS imaging technique termed ptychographic NLOS (pNLOS), which leverages coded ptychography for depth-resolved imaging of obscured objects. Our approach involves scanning a laser spot on a wall to illuminate the hidden objects in an obscured region. The reflected wavefields from these objects then travel back to the wall, get modulated by the wall's complex-valued profile, and the resulting diffraction patterns are captured by a camera. By modulating the object wavefields, the wall surface serves the role of the coded layer as in coded ptychography. As we scan the laser spot to different positions, the reflected object wavefields on the wall translate accordingly, with the shifts varying for objects at different depths. This translational diversity enables the acquisition of a set of modulated diffraction patterns referred to as a ptychogram. By processing the ptychogram, we recover both the objects at different depths and the modulation profile of the wall surface. Experimental results demonstrate high-resolution, high-fidelity imaging of hidden objects, showcasing the potential of pNLOS for depth-aware vision beyond the direct line of sight.

eess.IV↗

arXiv2024

Parameters Inference for Nonlinear Wave Equations with Markovian Switching

Traditional partial differential equations with constant coefficients often struggle to capture abrupt changes in real-world phenomena, leading to the development of variable coefficient PDEs and Markovian switching models. Recently, research has introduced the concept of PDEs with Markov switching models, established their well-posedness and presented numerical methods. However, there has been limited discussion on parameter estimation for the jump coefficients in these models. This paper addresses this gap by focusing on parameter inference for the wave equation with Markovian switching. We propose a Bayesian statistical framework using discrete sparse Bayesian learning to establish its convergence and a uniform error bound. Our method requires fewer assumptions and enables independent parameter inference for each segment by allowing different underlying structures for the parameter estimation problem within each segmented time interval. The effectiveness of our approach is demonstrated through three numerical cases, which involve noisy spatiotemporal data from different wave equations with Markovian switching. The results show strong performance in parameter estimation for variable coefficient PDEs.

stat.ML↗

arXiv2024

Inference-Time Rule Eraser: Fair Recognition via Distilling and Removing Biased Rules

Machine learning models often make predictions based on biased features such as gender, race, and other social attributes, posing significant fairness risks, especially in societal applications, such as hiring, banking, and criminal justice. Traditional approaches to addressing this issue involve retraining or fine-tuning neural networks with fairness-aware optimization objectives. However, these methods can be impractical due to significant computational resources, complex industrial tests, and the associated CO2 footprint. Additionally, regular users often fail to fine-tune models because they lack access to model parameters In this paper, we introduce the Inference-Time Rule Eraser (Eraser), a novel method designed to address fairness concerns by removing biased decision-making rules from deployed models during inference without altering model weights. We begin by establishing a theoretical foundation for modifying model outputs to eliminate biased rules through Bayesian analysis. Next, we present a specific implementation of Eraser that involves two stages: (1) distilling the biased rules from the deployed model into an additional patch model, and (2) removing these biased rules from the output of the deployed model during inference. Extensive experiments validate the effectiveness of our approach, showcasing its superior performance in addressing fairness concerns in AI systems.

cs.LG↗

arXiv2024

Semi-Supervised Learning with Multi-Head Co-Training

Co-training, extended from self-training, is one of the frameworks for semi-supervised learning. Without natural split of features, single-view co-training works at the cost of training extra classifiers, where the algorithm should be delicately designed to prevent individual classifiers from collapsing into each other. To remove these obstacles which deter the adoption of single-view co-training, we present a simple and efficient algorithm Multi-Head Co-Training. By integrating base learners into a multi-head structure, the model is in a minimal amount of extra parameters. Every classification head in the unified model interacts with its peers through a "Weak and Strong Augmentation" strategy, in which the diversity is naturally brought by the strong data augmentation. Therefore, the proposed method facilitates single-view co-training by 1). promoting diversity implicitly and 2). only requiring a small extra computational overhead. The effectiveness of Multi-Head Co-Training is demonstrated in an empirical study on standard semi-supervised learning benchmarks.

cs.LG↗

arXiv2024

A Survey of Embodied Learning for Object-Centric Robotic Manipulation

Embodied learning for object-centric robotic manipulation is a rapidly developing and challenging area in embodied AI. It is crucial for advancing next-generation intelligent robots and has garnered significant interest recently. Unlike data-driven machine learning methods, embodied learning focuses on robot learning through physical interaction with the environment and perceptual feedback, making it especially suitable for robotic manipulation. In this paper, we provide a comprehensive survey of the latest advancements in this field and categorize the existing work into three main branches: 1) Embodied perceptual learning, which aims to predict object pose and affordance through various data representations; 2) Embodied policy learning, which focuses on generating optimal robotic decisions using methods such as reinforcement learning and imitation learning; 3) Embodied task-oriented learning, designed to optimize the robot's performance based on the characteristics of different tasks in object grasping and manipulation. In addition, we offer an overview and discussion of public datasets, evaluation metrics, representative applications, current challenges, and potential future research directions. A project associated with this survey has been established at https://github.com/RayYoh/OCRM_survey.

cs.RO↗

arXiv2024

The hot circumgalactic medium in the eROSITA All-Sky Survey I. X-ray surface brightness profiles

The circumgalactic medium (CGM) provides the material needed for galaxy formation and influences galaxy evolution. The hot ($T>10^6K$) CGM is poorly detected around galaxies with stellar masses ($M_*$) lower than $3\times10^{11}M_\odot$ due to the low surface brightness. We used the X-ray data from the first four SRG/eROSITA All-Sky Surveys (eRASS:4). Based on the SDSS spectroscopic survey and halo-based group finder algorithm, we selected central galaxies with spectroscopic redshifts of $z_{\rm spec}<0.2$ and stellar masses of $10.0<\log(M_*/M_\odot)<11.5$ (85,222 galaxies) -- or halo masses of $11.5<\log(M_{\rm 200m}/M_\odot)<14.0$ (125,512 galaxies). By stacking the X-ray emission around galaxies, masking the detected X-ray point sources and carefully modeling the X-ray emission from the unresolved active galactic nuclei (AGN) and X-ray binaries (XRB), we obtain the X-ray emission from the hot CGM. We detected the X-ray emission around MW-mass and more massive central galaxies extending up to the virial radius ($R_{\rm vir}$). We used a $β$ model to describe the X-ray surface brightness profile and found $β=0.43^{+0.10}_{-0.06}\,(0.37^{+0.04}_{-0.02})$ for MW-mass (M31-mass) galaxies.We estimated the baryon budget of the hot CGM and obtained a value that is lower than the prediction of $Λ$CDM cosmology, indicating significant gas depletion in these halos. We extrapolated the hot CGM profile measured within $R_{\rm vir}$ to larger radii and found that within $\approx 3 R_{\rm vir}$, the baryon budget is close to the $Λ$CDM cosmology prediction. Our results set a firm footing for the presence of the hot CGM around such galaxies. These measurements constitute a new benchmark for galaxy evolution models and possible implementations of feedback processes therein.

astro-ph.GA↗

arXiv2024

The hot circumgalactic medium in the eROSITA All-Sky Survey II. Scaling relations between X-ray luminosity and galaxies' mass

Understanding how the properties of galaxies relate to the properties of the hot circum-galactic medium (CGM) around them can constrain galaxy evolution models. We measured the X-ray luminosity of the hot CGM based on the surface brightness profiles of central galaxy samples measured from Spectrum Roentgen Gamma (SRG)/eROSITA all-sky survey data. We related the X-ray luminosity to the galaxies' stellar and halo mass, and we compared the observed relations to the self-similar model and intrinsic (i.e., not forward-modeled) output of the IllustrisTNG, EAGLE, and SIMBA simulations. The average hot CGM X-ray luminosity ($L_{\rm X,CGM}$) correlates with the galaxy's stellar mass ($M_*$). It increases from $(1.6 \pm 2.1)\times10^{39} \rm erg\,s^{-1}$ to $(3.4 \pm 0.3)\times10^{41} \rm erg\,s^{-1}$, when $\log(M_*)$ increases from 10.0 to 11.5. A power law describes the correlation as $\log(L_{\rm X,CGM})= (2.4\pm 0.1)\times \log(M_*)+(14.6\pm1.5)$. The hot CGM X-ray luminosity as a function of halo mass is measured within $\log(M_{\rm 500c})=11.3-13.7$, extending our knowledge of the scaling relation by more than two orders of magnitude. $L_{\rm X,CGM}$ increases with $M_{\rm 500c}$ from $(3.0 \pm 1.6)\times10^{39}\ \rm erg\,s^{-1}$ at $\log(M_{\rm 500c})=11.3$ to $(1.3 \pm 0.1)\times10^{42}\ \rm erg\,s^{-1}$ at $\log(M_{\rm 500c})=13.7$. The relation follows a power law of $\log(L_{\rm X,CGM})= (1.32\pm 0.05)\times \log(M_{\rm 500c})+(24.1\pm0.7)$. Our observations highlight the necessity of non-gravitational processes at the galaxy group scale while suggesting these processes are sub-dominant at the galaxy scale. We show that the outputs of current cosmological galaxy simulations generally align with the observational results uncovered here but with possibly important deviations in selected mass ranges.

astro-ph.GA↗

arXiv2024

Rota-Baxter operators, differential operators, pre- and Novikov structures on groups and Lie algebras

Rota-Baxter operators on various structures have found important applications in diverse areas, from renormalization of quantum field theory to Yang-Baxter equations. Relative Rota-Baxter operators on Lie algebras are closely related to pre-Lie algebras and post-Lie algebras. Some of their group counterparts have been introduced to study post-groups, skew left braces and set-theoretic solutions of Yang-Baxter equations, but searching suitable notions of relative Rota-Baxter operators on groups with weight zero and pre-groups has been challenging and has been the focus of recent studies, by provisionally imposing an abelian condition. Arising from the works of Balinsky-Novikov and Gelfand-Dorfman, Novikov algebras and their constructions from differential commutative algebras have led to broad applications. Finding their suitable counterparts for groups and Lie algebras has also attracted quite much recent attention. This paper uses one-sided-inverse pairs of maps to give a perturbative approach to a general notion of relative Rota-Baxter operators and differential operators on a group and a Lie algebra with limit-weight. With the extra condition of limit-abelianess on the group or Lie algebra, we give an interpretation of relative Rota-Baxter and differential operators with weight zero. These operators motivate us to define pre-groups and Novikov groups respectively as the induced structures. The tangent maps of these operators are shown to give Rota-Baxter and differential operators with weight zero on Lie algebras. The tangent spaces of the pre-Lie and Novikov Lie groups are pre-Lie algebras and Novikov Lie algebras, fulfilling the expected property. Furthermore, limit-weight relative Rota-Baxter operators on groups give rise to skew left braces and then set-theoretic solutions of the Yang-Baxter equation.

math.QA↗

arXiv2024

Intent-guided Heterogeneous Graph Contrastive Learning for Recommendation

Contrastive Learning (CL)-based recommender systems have gained prominence in the context of Heterogeneous Graph (HG) due to their capacity to enhance the consistency of representations across different views. However, existing frameworks often neglect the fact that user-item interactions within HG are governed by diverse latent intents (e.g., brand preferences or demographic characteristics of item audiences), which are pivotal in capturing fine-grained relations. The exploration of these underlying intents, particularly through the lens of meta-paths in HGs, presents us with two principal challenges: i) How to integrate CL with intents; ii) How to mitigate noise from meta-path-driven intents. To address these challenges, we propose an innovative framework termed Intent-guided Heterogeneous Graph Contrastive Learning (IHGCL), which designed to enhance CL-based recommendation by capturing the intents contained within meta-paths. Specifically, the IHGCL framework includes: i) a meta-path-based Dual Contrastive Learning (DCL) approach to effectively integrate intents into the recommendation, constructing intent-intent contrast and intent-interaction contrast; ii) a Bottlenecked AutoEncoder (BAE) that combines mask propagation with the information bottleneck principle to significantly reduce noise perturbations introduced by meta-paths. Empirical evaluations conducted across six distinct datasets demonstrate the superior performance of our IHGCL framework relative to conventional baseline methods. Our model implementation is available at https://github.com/wangyu0627/IHGCL.

cs.IR↗

arXiv2024

Feature Re-Embedding: Towards Foundation Model-Level Performance in Computational Pathology

Multiple instance learning (MIL) is the most widely used framework in computational pathology, encompassing sub-typing, diagnosis, prognosis, and more. However, the existing MIL paradigm typically requires an offline instance feature extractor, such as a pre-trained ResNet or a foundation model. This approach lacks the capability for feature fine-tuning within the specific downstream tasks, limiting its adaptability and performance. To address this issue, we propose a Re-embedded Regional Transformer (R$^2$T) for re-embedding the instance features online, which captures fine-grained local features and establishes connections across different regions. Unlike existing works that focus on pre-training powerful feature extractor or designing sophisticated instance aggregator, R$^2$T is tailored to re-embed instance features online. It serves as a portable module that can seamlessly integrate into mainstream MIL models. Extensive experimental results on common computational pathology tasks validate that: 1) feature re-embedding improves the performance of MIL models based on ResNet-50 features to the level of foundation model features, and further enhances the performance of foundation model features; 2) the R$^2$T can introduce more significant performance improvements to various MIL models; 3) R$^2$T-MIL, as an R$^2$T-enhanced AB-MIL, outperforms other latest methods by a large margin.The code is available at: https://github.com/DearCaat/RRT-MIL.

cs.CV↗

arXiv2024

Doubly Robust Conditional Independence Testing with Generative Neural Networks

This article addresses the problem of testing the conditional independence of two generic random vectors $X$ and $Y$ given a third random vector $Z$, which plays an important role in statistical and machine learning applications. We propose a new non-parametric testing procedure that avoids explicitly estimating any conditional distributions but instead requires sampling from the two marginal conditional distributions of $X$ given $Z$ and $Y$ given $Z$. We further propose using a generative neural network (GNN) framework to sample from these approximated marginal conditional distributions, which tends to mitigate the curse of dimensionality due to its adaptivity to any low-dimensional structures and smoothness underlying the data. Theoretically, our test statistic is shown to enjoy a doubly robust property against GNN approximation errors, meaning that the test statistic retains all desirable properties of the oracle test statistic utilizing the true marginal conditional distributions, as long as the product of the two approximation errors decays to zero faster than the parametric rate. Asymptotic properties of our statistic and the consistency of a bootstrap procedure are derived under both null and local alternatives. Extensive numerical experiments and real data analysis illustrate the effectiveness and broad applicability of our proposed test.

stat.ME↗

arXiv2024

MACM: Utilizing a Multi-Agent System for Condition Mining in Solving Complex Mathematical Problems

Recent advancements in large language models, such as GPT-4, have demonstrated remarkable capabilities in processing standard queries. Despite these advancements, their performance substantially declines in \textbf{advanced mathematical problems requiring complex, multi-step logical reasoning}. To enhance their inferential capabilities, current research has delved into \textit{prompting engineering}, exemplified by methodologies such as the Tree of Thought and Graph of Thought. Nonetheless, these existing approaches encounter two significant limitations. Firstly, their effectiveness in tackling complex mathematical problems is somewhat constrained. Secondly, the necessity to design distinct prompts for individual problems hampers their generalizability. In response to these limitations, this paper introduces the \textit{Multi-Agent System for conditional Mining} (\textbf{MACM}) prompting method. It not only resolves intricate mathematical problems but also demonstrates strong generalization capabilities across various mathematical contexts. With the assistance of MACM, the accuracy of GPT-4 Turbo on the most challenging level five mathematical problems in the MATH dataset increase from $\mathbf{54.68\%} \text{ to } \mathbf{76.73\%}$. The code is available in \url{https://github.com/bin123apple/MACM}.

cs.AI↗

arXiv2024

On a $d$-degree Erdős-Ko-Rado Theorem

A family of subsets $\mathcal{F}$ is intersecting if $A \cap B \neq \emptyset$ for any $A, B \in \mathcal{F}$. In this paper, we show that for given integers $k > d \ge 2$ and $n \ge 2k+2d-3$, and any intersecting family $\mathcal{F}$ of $k$-subsets of $\{1, \cdots, n\}$, there exists a $d$-subset of $[n]$ contained in at most $\binom{n-d-1}{k-d-1}$ subsets of $\mathcal{F}$. This result, proved using spectral graph theory, gives a $d$-degree generalization of the celebrated Erdős-Ko-Rado Theorem, improving a theorem of Kupavskii.

math.CO↗

arXiv2024

Learning Visual Conditioning Tokens to Correct Domain Shift for Fully Test-time Adaptation

Fully test-time adaptation aims to adapt the network model based on sequential analysis of input samples during the inference stage to address the cross-domain performance degradation problem of deep neural networks. This work is based on the following interesting finding: in transformer-based image classification, the class token at the first transformer encoder layer can be learned to capture the domain-specific characteristics of target samples during test-time adaptation. This learned token, when combined with input image patch embeddings, is able to gradually remove the domain-specific information from the feature representations of input samples during the transformer encoding process, thereby significantly improving the test-time adaptation performance of the source model across different domains. We refer to this class token as visual conditioning token (VCT). To successfully learn the VCT, we propose a bi-level learning approach to capture the long-term variations of domain-specific characteristics while accommodating local variations of instance-specific characteristics. Experimental results on the benchmark datasets demonstrate that our proposed bi-level visual conditioning token learning method is able to achieve significantly improved test-time adaptation performance by up to 1.9%.

cs.CV↗

arXiv2024

Conceptual Codebook Learning for Vision-Language Models

In this paper, we propose Conceptual Codebook Learning (CoCoLe), a novel fine-tuning method for vision-language models (VLMs) to address the challenge of improving the generalization capability of VLMs while fine-tuning them on downstream tasks in a few-shot setting. We recognize that visual concepts, such as textures, shapes, and colors are naturally transferable across domains and play a crucial role in generalization tasks. Motivated by this interesting finding, we learn a conceptual codebook consisting of visual concepts as keys and conceptual prompts as values, which serves as a link between the image encoder's outputs and the text encoder's inputs. Specifically, for a given image, we leverage the codebook to identify the most relevant conceptual prompts associated with the class embeddings to perform the classification. Additionally, we incorporate a handcrafted concept cache as a regularization to alleviate the overfitting issues in low-shot scenarios. We observe that this conceptual codebook learning method is able to achieve enhanced alignment between visual and linguistic modalities. Extensive experimental results demonstrate that our CoCoLe method remarkably outperforms the existing state-of-the-art methods across various evaluation settings, including base-to-new generalization, cross-dataset evaluation, and domain generalization tasks. Detailed ablation studies further confirm the efficacy of each component in CoCoLe.

cs.CV↗

arXiv2024

When Pedestrian Detection Meets Multi-Modal Learning: Generalist Model and Benchmark Dataset

Recent years have witnessed increasing research attention towards pedestrian detection by taking the advantages of different sensor modalities (e.g. RGB, IR, Depth, LiDAR and Event). However, designing a unified generalist model that can effectively process diverse sensor modalities remains a challenge. This paper introduces MMPedestron, a novel generalist model for multimodal perception. Unlike previous specialist models that only process one or a pair of specific modality inputs, MMPedestron is able to process multiple modal inputs and their dynamic combinations. The proposed approach comprises a unified encoder for modal representation and fusion and a general head for pedestrian detection. We introduce two extra learnable tokens, i.e. MAA and MAF, for adaptive multi-modal feature fusion. In addition, we construct the MMPD dataset, the first large-scale benchmark for multi-modal pedestrian detection. This benchmark incorporates existing public datasets and a newly collected dataset called EventPed, covering a wide range of sensor modalities including RGB, IR, Depth, LiDAR, and Event data. With multi-modal joint training, our model achieves state-of-the-art performance on a wide range of pedestrian detection benchmarks, surpassing leading models tailored for specific sensor modality. For example, it achieves 71.1 AP on COCO-Persons and 72.6 AP on LLVIP. Notably, our model achieves comparable performance to the InternImage-H model on CrowdHuman with 30x smaller parameters. Codes and data are available at https://github.com/BubblyYi/MMPedestron.

cs.CV↗

arXiv2024

ProTIP: Probabilistic Robustness Verification on Text-to-Image Diffusion Models against Stochastic Perturbation

Text-to-Image (T2I) Diffusion Models (DMs) have shown impressive abilities in generating high-quality images based on simple text descriptions. However, as is common with many Deep Learning (DL) models, DMs are subject to a lack of robustness. While there are attempts to evaluate the robustness of T2I DMs as a binary or worst-case problem, they cannot answer how robust in general the model is whenever an adversarial example (AE) can be found. In this study, we first introduce a probabilistic notion of T2I DMs' robustness; and then establish an efficient framework, ProTIP, to evaluate it with statistical guarantees. The main challenges stem from: i) the high computational cost of the generation process; and ii) determining if a perturbed input is an AE involves comparing two output distributions, which is fundamentally harder compared to other DL tasks like classification where an AE is identified upon misprediction of labels. To tackle the challenges, we employ sequential analysis with efficacy and futility early stopping rules in the statistical testing for identifying AEs, and adaptive concentration inequalities to dynamically determine the "just-right" number of stochastic perturbations whenever the verification target is met. Empirical experiments validate the effectiveness and efficiency of ProTIP over common T2I DMs. Finally, we demonstrate an application of ProTIP to rank commonly used defence methods.

cs.CV↗