arXiv subjects

Yi Zhang

Publications and source records attributed to Yi Zhang.

At least 649 records · Page 36Linked to original sources

arXiv2024

Lite-SAM Is Actually What You Need for Segment Everything

This paper introduces Lite-SAM, an efficient end-to-end solution for the SegEvery task designed to reduce computational costs and redundancy. Lite-SAM is composed of four main components: a streamlined CNN-Transformer hybrid encoder (LiteViT), an automated prompt proposal network (AutoPPN), a traditional prompt encoder, and a mask decoder. All these components are integrated within the SAM framework. Our LiteViT, a high-performance lightweight backbone network, has only 1.16M parameters, which is a 23% reduction compared to the lightest existing backbone network Shufflenet. We also introduce AutoPPN, an innovative end-to-end method for prompt boxes and points generation. This is an improvement over traditional grid search sampling methods, and its unique design allows for easy integration into any SAM series algorithm, extending its usability. we have thoroughly benchmarked Lite-SAM across a plethora of both public and private datasets. The evaluation encompassed a broad spectrum of universal metrics, including the number of parameters, SegEvery execution time, and accuracy. The findings reveal that Lite-SAM, operating with a lean 4.2M parameters, significantly outpaces its counterparts, demonstrating performance improvements of 43x, 31x, 20x, 21x, and 1.6x over SAM, MobileSAM, Edge-SAM, EfficientViT-SAM, and MobileSAM-v2 respectively, all the while maintaining competitive accuracy. This underscores Lite-SAM's prowess in achieving an optimal equilibrium between performance and precision, thereby setting a new state-of-the-art(SOTA) benchmark in the domain.

cs.CV↗

arXiv2024

NODE-Adapter: Neural Ordinary Differential Equations for Better Vision-Language Reasoning

In this paper, we consider the problem of prototype-based vision-language reasoning problem. We observe that existing methods encounter three major challenges: 1) escalating resource demands and prolonging training times, 2) contending with excessive learnable parameters, and 3) fine-tuning based only on a single modality. These challenges will hinder their capability to adapt Vision-Language Models (VLMs) to downstream tasks. Motivated by this critical observation, we propose a novel method called NODE-Adapter, which utilizes Neural Ordinary Differential Equations for better vision-language reasoning. To fully leverage both visual and textual modalities and estimate class prototypes more effectively and accurately, we divide our method into two stages: cross-modal prototype construction and cross-modal prototype optimization using neural ordinary differential equations. Specifically, we exploit VLM to encode hand-crafted prompts into textual features and few-shot support images into visual features. Then, we estimate the textual prototype and visual prototype by averaging the textual features and visual features, respectively, and adaptively combine the textual prototype and visual prototype to construct the cross-modal prototype. To alleviate the prototype bias, we then model the prototype optimization process as an initial value problem with Neural ODEs to estimate the continuous gradient flow. Our extensive experimental results, which cover few-shot classification, domain generalization, and visual reasoning on human-object interaction, demonstrate that the proposed method significantly outperforms existing state-of-the-art approaches.

cs.CV↗

arXiv2024

Enhancing Mobile "How-to" Queries with Automated Search Results Verification and Reranking

Many people use search engines to find online guidance to solve computer or mobile device problems. Users frequently encounter challenges in identifying effective solutions from search results, often wasting time trying ineffective solutions that seem relevant yet fail to solve real problems. This paper introduces a novel approach to improving the accuracy and relevance of online technical support search results through automated search results verification and reranking. Taking "How-to" queries specific to on-device execution as a starting point, we developed the first solution that allows an AI agent to interpret and execute step-by-step instructions in the search results in a controlled Android environment. We further integrated the agent's findings into a reranking mechanism that orders search results based on the success indicators of the tested solutions. The paper details the architecture of our solution and a comprehensive evaluation of the system through a series of tests across various application domains. The results demonstrate a significant improvement in the quality and reliability of the top-ranked results. Our findings suggest a paradigm shift in how search engine ranking for online technical support help can be optimized, offering a scalable and automated solution to the pervasive challenge of finding effective and reliable online help.

cs.IR↗

arXiv2024

Towards Energy-Aware Federated Learning via MARL: A Dual-Selection Approach for Model and Client

Although Federated Learning (FL) is promising in knowledge sharing for heterogeneous Artificial Intelligence of Thing (AIoT) devices, their training performance and energy efficacy are severely restricted in practical battery-driven scenarios due to the ``wooden barrel effect'' caused by the mismatch between homogeneous model paradigms and heterogeneous device capability. As a result, due to various kinds of differences among devices, it is hard for existing FL methods to conduct training effectively in energy-constrained scenarios, such as battery constraints of devices. To tackle the above issues, we propose an energy-aware FL framework named DR-FL, which considers the energy constraints in both clients and heterogeneous deep learning models to enable energy-efficient FL. Unlike Vanilla FL, DR-FL adopts our proposed Muti-Agents Reinforcement Learning (MARL)-based dual-selection method, which allows participated devices to make contributions to the global model effectively and adaptively based on their computing capabilities and energy capacities in a MARL-based manner. Experiments conducted with various widely recognized datasets demonstrate that DR-FL has the capability to optimize the exchange of knowledge among diverse models in large-scale AIoT systems while adhering to energy limitations. Additionally, it improves the performance of each individual heterogeneous device's model.

cs.LG↗

arXiv2024

Evolution of High-energy Electron Distribution in Pulsar Wind Nebulae

In this paper, we analyze the spectral energy distributions (SEDs) of 17 powerful (with a spin-down luminosity greater than $10^{35}$ erg s$^{-1}$) young (with an age less than 15000 yrs) pulsar wind nebulae (PWNe) using a simple time-independent one-zone emission model. Our aim is to investigate correlations between model parameters and the ages of the corresponding PWNe, thereby revealing the evolution of high-energy electron distributions within PWNe. Our findings are as follows: (1) The electron distributions in PWNe can be characterized by a double power-law with a superexponential cutoff; (2) As PWNe evolve, the high-energy end of the electron distribution spectrum becomes harder with the index decreasing from approximately 3.5 to 2.5, while the low-energy end spectrum index remains constant near 1.5; (3) There is no apparent correlation between the break energy or cutoff energy and the age of PWNe. (4) The average magnetic field within PWNe decreases with age, leading to a positive correlation between the energy loss timescale of electrons at the break energy or the high-energy cutoff, and the age of the PWN. (5) The total electron energy within PWNe remains constant near $2 \times 10^{48}$ erg, while the total magnetic energy decreases with age.

astro-ph.HE↗

arXiv2024

Prospects for the detection of very-high-energy pulsars with LHAASO and SWGO

Pulsations from the Crab pulsar have been detected by the MAGIC telescopes at energies up to 1.5 TeV, and the pulsed emission from the Vela pulsar was detected by H.E.S.S., reaching tens of TeV. These discoveries, along with the proposed additional emission due to inverse Compton scattering at TeV energies, lead us to consider suitable candidates for detection with current and future extensive air show (EAS) experiments at very-high-energy (VHE; 0.1 $-$ 100 TeV) ranges. Leveraging energy spectrum data from pulsars as observed by Fermi and Imaging Atmospheric Cherenkov Telescopes (IACTs) and considering the sensitivities of both LHAASO and SWGO, this study evaluates their detectability and estimates the time required for their significant detection. Our results indicate that LHAASO could detect the Crab's pulsed signal within six years, while SWGO might detect Vela's signal within one year. Observations of the most energetic Fermi pulsars with EAS experiments will provide insight into the nature of VHE pulsar emissions, helping to clarify the primary characteristics of VHE pulsars.

astro-ph.HE↗

arXiv2024

Deep-learning-based groupwise registration for motion correction of cardiac $T_1$ mapping

Quantitative $T_1$ mapping by MRI is an increasingly important tool for clinical assessment of cardiovascular diseases. The cardiac $T_1$ map is derived by fitting a known signal model to a series of baseline images, while the quality of this map can be deteriorated by involuntary respiratory and cardiac motion. To correct motion, a template image is often needed to register all baseline images, but the choice of template is nontrivial, leading to inconsistent performance sensitive to image contrast. In this work, we propose a novel deep-learning-based groupwise registration framework, which omits the need for a template, and registers all baseline images simultaneously. We design two groupwise losses for this registration framework: the first is a linear principal component analysis (PCA) loss that enforces alignment of baseline images irrespective of the intensity variation, and the second is an auxiliary relaxometry loss that enforces adherence of intensity profile to the signal model. We extensively evaluated our method, termed ``PCA-Relax'', and other baseline methods on an in-house cardiac MRI dataset including both pre- and post-contrast $T_1$ sequences. All methods were evaluated under three distinct training-and-evaluation strategies, namely, standard, one-shot, and test-time-adaptation. The proposed PCA-Relax showed further improved performance of registration and mapping over well-established baselines. The proposed groupwise framework is generic and can be adapted to applications involving multiple images.

eess.IV↗

arXiv2024

MDCR: A Dataset for Multi-Document Conditional Reasoning

The same real-life questions posed to different individuals may lead to different answers based on their unique situations. For instance, whether a student is eligible for a scholarship depends on eligibility conditions, such as major or degree required. ConditionalQA was proposed to evaluate models' capability of reading a document and answering eligibility questions, considering unmentioned conditions. However, it is limited to questions on single documents, neglecting harder cases that may require cross-document reasoning and optimization, for example, "What is the maximum number of scholarships attainable?" Such questions over multiple documents are not only more challenging due to more context having to understand, but also because the model has to (1) explore all possible combinations of unmentioned conditions and (2) understand the relationship between conditions across documents, to reason about the optimal outcome. To evaluate models' capability of answering such questions, we propose a new dataset MDCR, which can reflect real-world challenges and serve as a new test bed for complex conditional reasoning that requires optimization. We evaluate this dataset using the most recent LLMs and demonstrate their limitations in solving this task. We believe this dataset will facilitate future research in answering optimization questions with unknown conditions.

cs.CL↗

arXiv2024

Axial Anomalies of Maximally Supersymmetric Tensor Theories

We revisit anomalies of $(4,0)$ and $(3,1)$ maximally supersymmetric tensor theories in $d=6$. A $(4,0)$ on-shell tensor multiplet descends to that of the $d=5$ maximal supergravity upon a dimensional reduction, hypothesized to offer a strong-coupled UV completion of the latter in the same sense of $(2,0)$ theories as the UV completion of $d=5$ $\mathcal{N}=2$ pure Yang-Mills. The gravitational anomalies, found to be nonvanishing, had been computed, although its relevance in the absence of the $d=6$ metric is not obvious. We perform a comprehensive anomaly computation for $(4,0)$ and $(3,1)$ tensor supermultiplets, respectively, for $Sp(4)$ and $Sp(3)\times Sp(1)$ $R$-symmetry anomalies and the mixed $R$-gravitational anomaly thereof, and find that anomalies involving $R$-symmetries cancel out identically. We close with questions on how to address the anomaly in this class of theories with no general covariance.

hep-th↗

arXiv2024

Inverse scattering transform for the defocusing-defocusing coupled Hirota equations with non-zero boundary conditions: double-pole solutions

The inverse scattering transform for the defocusing-defocusing coupled Hirota equations with non-zero boundary conditions at infinity is thoroughly discussed. We delve into the analytical properties of the Jost eigenfunctions and scrutinize the characteristics of the scattering coefficients. To enhance our investigation of the fundamental eigenfunctions, we have derived additional auxiliary eigenfunctions with the help of the adjoint problem. Two symmetry conditions are studied to constrain the behavior of the eigenfunctions and scattering coefficients. Utilizing these symmetries, we precisely delineate the discrete spectrum and establish the associated symmetries of the scattering data. By framing the inverse problem within the context of the Riemann-Hilbert problem, we develop suitable jump conditions to express the eigenfunctions. Consequently, we deduce the pure soliton solutions from the defocusing-defocusing coupled Hirota equations, and the double-poles solutions are provided explicitly for the first time in this work.

nlin.SI↗

arXiv2024

A Fractal-based Complex Belief Entropy for Uncertainty Measure in Complex Evidence Theory

Complex Evidence Theory (CET), an extension of the traditional D-S evidence theory, has garnered academic interest for its capacity to articulate uncertainty through Complex Basic Belief Assignment (CBBA) and to perform uncertainty reasoning using complex combination rules. Nonetheless, quantifying uncertainty within CET remains a subject of ongoing research. To enhance decision-making, a method for Complex Pignistic Belief Transformation (CPBT) has been introduced, which allocates CBBAs of multi-element focal elements to subsets. CPBT's core lies in the fractal-inspired redistribution of the complex mass function. This paper presents an experimental simulation and analysis of CPBT's generation process along the temporal dimension, rooted in fractal theory. Subsequently, a novel Fractal-Based Complex Belief (FCB) entropy is proposed to gauge the uncertainty of CBBA. The properties of FCB entropy are examined, and its efficacy is demonstrated through various numerical examples and practical application.

cs.IT↗

arXiv2024

DIRECT-3D: Learning Direct Text-to-3D Generation on Massive Noisy 3D Data

We present DIRECT-3D, a diffusion-based 3D generative model for creating high-quality 3D assets (represented by Neural Radiance Fields) from text prompts. Unlike recent 3D generative models that rely on clean and well-aligned 3D data, limiting them to single or few-class generation, our model is directly trained on extensive noisy and unaligned `in-the-wild' 3D assets, mitigating the key challenge (i.e., data scarcity) in large-scale 3D generation. In particular, DIRECT-3D is a tri-plane diffusion model that integrates two innovations: 1) A novel learning framework where noisy data are filtered and aligned automatically during the training process. Specifically, after an initial warm-up phase using a small set of clean data, an iterative optimization is introduced in the diffusion process to explicitly estimate the 3D pose of objects and select beneficial data based on conditional density. 2) An efficient 3D representation that is achieved by disentangling object geometry and color features with two separate conditional diffusion models that are optimized hierarchically. Given a prompt input, our model generates high-quality, high-resolution, realistic, and complex 3D objects with accurate geometric details in seconds. We achieve state-of-the-art performance in both single-class generation and text-to-3D generation. We also demonstrate that DIRECT-3D can serve as a useful 3D geometric prior of objects, for example to alleviate the well-known Janus problem in 2D-lifting methods such as DreamFusion. The code and models are available for research purposes at: https://github.com/qihao067/direct3d.

cs.CV↗

arXiv2024

Textual Inversion and Self-supervised Refinement for Radiology Report Generation

Existing mainstream approaches follow the encoder-decoder paradigm for generating radiology reports. They focus on improving the network structure of encoders and decoders, which leads to two shortcomings: overlooking the modality gap and ignoring report content constraints. In this paper, we proposed Textual Inversion and Self-supervised Refinement (TISR) to address the above two issues. Specifically, textual inversion can project text and image into the same space by representing images as pseudo words to eliminate the cross-modeling gap. Subsequently, self-supervised refinement refines these pseudo words through contrastive loss computation between images and texts, enhancing the fidelity of generated reports to images. Notably, TISR is orthogonal to most existing methods, plug-and-play. We conduct experiments on two widely-used public datasets and achieve significant improvements on various baselines, which demonstrates the effectiveness and generalization of TISR. The code will be available soon.

cs.CV↗

arXiv2024

StatBot.Swiss: Bilingual Open Data Exploration in Natural Language

The potential for improvements brought by Large Language Models (LLMs) in Text-to-SQL systems is mostly assessed on monolingual English datasets. However, LLMs' performance for other languages remains vastly unexplored. In this work, we release the StatBot.Swiss dataset, the first bilingual benchmark for evaluating Text-to-SQL systems based on real-world applications. The StatBot.Swiss dataset contains 455 natural language/SQL-pairs over 35 big databases with varying level of complexity for both English and German. We evaluate the performance of state-of-the-art LLMs such as GPT-3.5-Turbo and mixtral-8x7b-instruct for the Text-to-SQL translation task using an in-context learning approach. Our experimental analysis illustrates that current LLMs struggle to generalize well in generating SQL queries on our novel bilingual dataset.

cs.CL↗

arXiv2024

Dynamically Expanding Capacity of Autonomous Driving with Near-Miss Focused Training Framework

The long-tail distribution of real driving data poses challenges for training and testing autonomous vehicles (AV), where rare yet crucial safety-critical scenarios are infrequent. And virtual simulation offers a low-cost and efficient solution. This paper proposes a near-miss focused training framework for AV. Utilizing the driving scenario information provided by sensors in the simulator, we design novel reward functions, which enable background vehicles (BV) to generate near-miss scenarios and ensure gradients exist not only in collision-free scenes but also in collision scenarios. And then leveraging the Robust Adversarial Reinforcement Learning (RARL) framework for simultaneous training of AV and BV to gradually enhance AV and BV capabilities, as well as generating near-miss scenarios tailored to different levels of AV capabilities. Results from three testing strategies indicate that the proposed method generates scenarios closer to near-miss, thus enhancing the capabilities of both AVs and BVs throughout training.

cs.RO↗

arXiv2024

Programmable Multi-input Buck-Boost Converter for Photovoltaics Arrays

This paper proposes a programmable multi-input buck-boost structure method, which can enhance the operation tolerance for the PV array under extremely harsh climatic conditions. The proposed structure based on a traditional two switches buck-boost converter can connect PV panels in parallel and cascade flexibly, and also enable the individual operation of each PV panel. The active switches can be programmed to change the connection structures as well as achieve the maximum power point track of PV panels simultaneously. The paper presents the programming method for an exemplified scalable structure converter for two PV panels. The simulation has been established in MATLAB/Simulink to validate the performance of the proposed converter in terms of multiplexing function, wide operating range of PV panels, and low switching stress.

eess.SY↗

arXiv2024

FairCLIP: Social Bias Elimination based on Attribute Prototype Learning and Representation Neutralization

The Vision-Language Pre-training (VLP) models like CLIP have gained popularity in recent years. However, many works found that the social biases hidden in CLIP easily manifest in downstream tasks, especially in image retrieval, which can have harmful effects on human society. In this work, we propose FairCLIP to eliminate the social bias in CLIP-based image retrieval without damaging the retrieval performance achieving the compatibility between the debiasing effect and the retrieval performance. FairCLIP is divided into two steps: Attribute Prototype Learning (APL) and Representation Neutralization (RN). In the first step, we extract the concepts needed for debiasing in CLIP. We use the query with learnable word vector prefixes as the extraction structure. In the second step, we first divide the attributes into target and bias attributes. By analysis, we find that both attributes have an impact on the bias. Therefore, we try to eliminate the bias by using Re-Representation Matrix (RRM) to achieve the neutralization of the representation. We compare the debiasing effect and retrieval performance with other methods, and experiments demonstrate that FairCLIP can achieve the best compatibility. Although FairCLIP is used to eliminate bias in image retrieval, it achieves the neutralization of the representation which is common to all CLIP downstream tasks. This means that FairCLIP can be applied as a general debiasing method for other fairness issues related to CLIP.

cs.CV↗

arXiv2024

Reverse Transition Kernel: A Flexible Framework to Accelerate Diffusion Inference

To generate data from trained diffusion models, most inference algorithms, such as DDPM, DDIM, and other variants, rely on discretizing the reverse SDEs or their equivalent ODEs. In this paper, we view such approaches as decomposing the entire denoising diffusion process into several segments, each corresponding to a reverse transition kernel (RTK) sampling subproblem. Specifically, DDPM uses a Gaussian approximation for the RTK, resulting in low per-subproblem complexity but requiring a large number of segments (i.e., subproblems), which is conjectured to be inefficient. To address this, we develop a general RTK framework that enables a more balanced subproblem decomposition, resulting in $\tilde O(1)$ subproblems, each with strongly log-concave targets. We then propose leveraging two fast sampling algorithms, the Metropolis-Adjusted Langevin Algorithm (MALA) and Underdamped Langevin Dynamics (ULD), for solving these strongly log-concave subproblems. This gives rise to the RTK-MALA and RTK-ULD algorithms for diffusion inference. In theory, we further develop the convergence guarantees for RTK-MALA and RTK-ULD in total variation (TV) distance: RTK-ULD can achieve $ε$ target error within $\tilde{\mathcal O}(d^{1/2}ε^{-1})$ under mild conditions, and RTK-MALA enjoys a $\mathcal{O}(d^{2}\log(d/ε))$ convergence rate under slightly stricter conditions. These theoretical results surpass the state-of-the-art convergence rates for diffusion inference and are well supported by numerical experiments.

stat.ML↗