arXiv subjects

Chi Zhang

Publications and source records attributed to Chi Zhang.

At least 505 records · Page 28Linked to original sources

arXiv2024

MeshXL: Neural Coordinate Field for Generative 3D Foundation Models

The polygon mesh representation of 3D data exhibits great flexibility, fast rendering speed, and storage efficiency, which is widely preferred in various applications. However, given its unstructured graph representation, the direct generation of high-fidelity 3D meshes is challenging. Fortunately, with a pre-defined ordering strategy, 3D meshes can be represented as sequences, and the generation process can be seamlessly treated as an auto-regressive problem. In this paper, we validate the Neural Coordinate Field (NeurCF), an explicit coordinate representation with implicit neural embeddings, is a simple-yet-effective representation for large-scale sequential mesh modeling. After that, we present MeshXL, a family of generative pre-trained auto-regressive models, which addresses the process of 3D mesh generation with modern large language model approaches. Extensive experiments show that MeshXL is able to generate high-quality 3D meshes, and can also serve as foundation models for various down-stream applications.

cs.CV↗

arXiv2024

Efficient and Long-Tailed Generalization for Pre-trained Vision-Language Model

Pre-trained vision-language models like CLIP have shown powerful zero-shot inference ability via image-text matching and prove to be strong few-shot learners in various downstream tasks. However, in real-world scenarios, adapting CLIP to downstream tasks may encounter the following challenges: 1) data may exhibit long-tailed data distributions and might not have abundant samples for all the classes; 2) There might be emerging tasks with new classes that contain no samples at all. To overcome them, we propose a novel framework to achieve efficient and long-tailed generalization, which can be termed as Candle. During the training process, we propose compensating logit-adjusted loss to encourage large margins of prototypes and alleviate imbalance both within the base classes and between the base and new classes. For efficient adaptation, we treat the CLIP model as a black box and leverage the extracted features to obtain visual and textual prototypes for prediction. To make full use of multi-modal information, we also propose cross-modal attention to enrich the features from both modalities. For effective generalization, we introduce virtual prototypes for new classes to make up for their lack of training images. Candle achieves state-of-the-art performance over extensive experiments on 11 diverse datasets while substantially reducing the training time, demonstrating the superiority of our approach. The source code is available at https://github.com/shijxcs/Candle.

cs.CV↗

arXiv2024

EMMA: Your Text-to-Image Diffusion Model Can Secretly Accept Multi-Modal Prompts

Recent advancements in image generation have enabled the creation of high-quality images from text conditions. However, when facing multi-modal conditions, such as text combined with reference appearances, existing methods struggle to balance multiple conditions effectively, typically showing a preference for one modality over others. To address this challenge, we introduce EMMA, a novel image generation model accepting multi-modal prompts built upon the state-of-the-art text-to-image (T2I) diffusion model, ELLA. EMMA seamlessly incorporates additional modalities alongside text to guide image generation through an innovative Multi-modal Feature Connector design, which effectively integrates textual and supplementary modal information using a special attention mechanism. By freezing all parameters in the original T2I diffusion model and only adjusting some additional layers, we reveal an interesting finding that the pre-trained T2I diffusion model can secretly accept multi-modal prompts. This interesting property facilitates easy adaptation to different existing frameworks, making EMMA a flexible and effective tool for producing personalized and context-aware images and even videos. Additionally, we introduce a strategy to assemble learned EMMA modules to produce images conditioned on multiple modalities simultaneously, eliminating the need for additional training with mixed multi-modal prompts. Extensive experiments demonstrate the effectiveness of EMMA in maintaining high fidelity and detail in generated images, showcasing its potential as a robust solution for advanced multi-modal conditional image generation tasks.

cs.CV↗

arXiv2024

DreamGaussian4D: Generative 4D Gaussian Splatting

4D content generation has achieved remarkable progress recently. However, existing methods suffer from long optimization times, a lack of motion controllability, and a low quality of details. In this paper, we introduce DreamGaussian4D (DG4D), an efficient 4D generation framework that builds on Gaussian Splatting (GS). Our key insight is that combining explicit modeling of spatial transformations with static GS makes an efficient and powerful representation for 4D generation. Moreover, video generation methods have the potential to offer valuable spatial-temporal priors, enhancing the high-quality 4D generation. Specifically, we propose an integral framework with two major modules: 1) Image-to-4D GS - we initially generate static GS with DreamGaussianHD, followed by HexPlane-based dynamic generation with Gaussian deformation; and 2) Video-to-Video Texture Refinement - we refine the generated UV-space texture maps and meanwhile enhance their temporal consistency by utilizing a pre-trained image-to-video diffusion model. Notably, DG4D reduces the optimization time from several hours to just a few minutes, allows the generated 3D motion to be visually controlled, and produces animated meshes that can be realistically rendered in 3D engines.

cs.CV↗

arXiv2024

Neural Networks with (Low-Precision) Polynomial Approximations: New Insights and Techniques for Accuracy Improvement

Replacing non-polynomial functions (e.g., non-linear activation functions such as ReLU) in a neural network with their polynomial approximations is a standard practice in privacy-preserving machine learning. The resulting neural network, called polynomial approximation of neural network (PANN) in this paper, is compatible with advanced cryptosystems to enable privacy-preserving model inference. Using ``highly precise'' approximation, state-of-the-art PANN offers similar inference accuracy as the underlying backbone model. However, little is known about the effect of approximation, and existing literature often determined the required approximation precision empirically. In this paper, we initiate the investigation of PANN as a standalone object. Specifically, our contribution is two-fold. Firstly, we provide an explanation on the effect of approximate error in PANN. In particular, we discovered that (1) PANN is susceptible to some type of perturbations; and (2) weight regularisation significantly reduces PANN's accuracy. We support our explanation with experiments. Secondly, based on the insights from our investigations, we propose solutions to increase inference accuracy for PANN. Experiments showed that combination of our solutions is very effective: at the same precision, our PANN is 10% to 50% more accurate than state-of-the-arts; and at the same accuracy, our PANN only requires a precision of 2^{-9} while state-of-the-art solution requires a precision of 2^{-12} using the ResNet-20 model on CIFAR-10 dataset.

cs.LG↗

arXiv2024

Tripartite quantum Rabi model with trapped Rydberg ions

We investigate a tripartite quantum Rabi model (TQRM) wherein a bosonic mode concurrently couples to two spin-$1/2$ particles through a spin-spin interaction, resulting in a spin-spin-boson coupling -- a departure from conventional quantum Rabi models featuring bipartite spin-boson couplings. The symmetries of the TQRM depend on the detuning parameter, representing the energy difference between the spin states. At zero detuning a parity symmetry renders the TQRM reducible to a quantum Rabi model. A subradiant to superradiant transition in the groundstate is predicted as the tripartite coupling strength increases. For non-zero detuning the total spin emerges as the sole conserved quantity in the TQRM. It is found that superradiance prevails in the groundstate as long as the tripartite coupling remains non-zero. We derive the Braak $\mathcal{G}$-function of the TQRM analytically, with which the eigenspectra are obtained. The TQRM can be realized in a viable trapped Rydberg ion quantum simulator, where the required tripartite couplings and single body interactions in the TQRM are naturally present. Our study opens opportunities to explore and create novel correlations and entanglement in the spin and motional degrees of freedoms with the TQRM.

quant-ph↗

arXiv2024

Nonreciprocal singularities dominated by the dissipative photon-magnon coupling in non-Hermitian systems

We investigated the magnon-photon coupling in an open cavity magnonic system, which leads to two different nonreciprocal singularities dominated by the dissipative coupling. One type of singularity is the exceptional point, which is just on the exceptional surface in parameter space. The other type of singularity is the bound state in the continuum discovered in the level-attraction-like coupling, which is above the exceptional surface. In experiment, we realized the two different singularities with nonreciprocity and selectivity in an open cavity magnonic system with suitable dissipation rating. Our results can be understood well with the pseudo-Hermitian theory of magnon-polariton system.

cond-mat.mes-hall↗

arXiv2024

The RoboDrive Challenge: Drive Anytime Anywhere in Any Condition

In the realm of autonomous driving, robust perception under out-of-distribution conditions is paramount for the safe deployment of vehicles. Challenges such as adverse weather, sensor malfunctions, and environmental unpredictability can severely impact the performance of autonomous systems. The 2024 RoboDrive Challenge was crafted to propel the development of driving perception technologies that can withstand and adapt to these real-world variabilities. Focusing on four pivotal tasks -- BEV detection, map segmentation, semantic occupancy prediction, and multi-view depth estimation -- the competition laid down a gauntlet to innovate and enhance system resilience against typical and atypical disturbances. This year's challenge consisted of five distinct tracks and attracted 140 registered teams from 93 institutes across 11 countries, resulting in nearly one thousand submissions evaluated through our servers. The competition culminated in 15 top-performing solutions, which introduced a range of innovative approaches including advanced data augmentation, multi-sensor fusion, self-supervised learning for error correction, and new algorithmic strategies to enhance sensor robustness. These contributions significantly advanced the state of the art, particularly in handling sensor inconsistencies and environmental variability. Participants, through collaborative efforts, pushed the boundaries of current technologies, showcasing their potential in real-world scenarios. Extensive evaluations and analyses provided insights into the effectiveness of these solutions, highlighting key trends and successful strategies for improving the resilience of driving perception systems. This challenge has set a new benchmark in the field, providing a rich repository of techniques expected to guide future research in this field.

cs.CV↗

arXiv2024

Do Finetti: On Causal Effects for Exchangeable Data

We study causal effect estimation in a setting where the data are not i.i.d. (independent and identically distributed). We focus on exchangeable data satisfying an assumption of independent causal mechanisms. Traditional causal effect estimation frameworks, e.g., relying on structural causal models and do-calculus, are typically limited to i.i.d. data and do not extend to more general exchangeable generative processes, which naturally arise in multi-environment data. To address this gap, we develop a generalized framework for exchangeable data and introduce a truncated factorization formula that facilitates both the identification and estimation of causal effects in our setting. To illustrate potential applications, we introduce a causal Pólya urn model and demonstrate how intervention propagates effects in exchangeable data settings. Finally, we develop an algorithm that performs simultaneous causal discovery and effect estimation given multi-environment data.

stat.ME↗

arXiv2024

A Survey on Data-Centric Recommender Systems

Recommender systems (RSs) have become an essential tool for mitigating information overload in a range of real-world applications. Recent trends in RSs have revealed a major paradigm shift, moving the spotlight from model-centric innovations to data-centric efforts (e.g., improving data quality and quantity). This evolution has given rise to the concept of data-centric recommender systems (Data-Centric RSs), marking a significant development in the field. This survey provides the first systematic overview of Data-Centric RSs, covering 1) the foundational concepts of recommendation data and Data-Centric RSs; 2) three primary issues of recommendation data; 3) recent research developed to address these issues; and 4) several potential future directions of Data-Centric RSs.

cs.IR↗

arXiv2024

Gravity-induced entanglement between two massive microscopic particles in curved spacetime: I.The Schwarzschild background

The experiment involving the entanglement of two massive particles through gravitational fields has been devised to discern the quantum attributes of gravity. In this paper, we present a scheme to extend this experiment's applicability to more generalized curved spacetimes, with the objective of validating universal quantum gravity within broader contexts. Specifically, we direct our attention towards the quantum gravity induced entanglement of mass (QGEM) in astrophysical phenomena, such as particles traversing the interstellar medium. Notably, we ascertain that the gravitational field within curved spacetime can induce observable entanglement between particle pairs in both scenarios, even when dealing with particles significantly smaller than mesoscopic masses. Furthermore, we obtain the characteristic spectra of QGEM across diverse scenarios, shedding light on potential future experimental examinations. This approach not only establishes a more pronounced and extensive manifestation of the quantum influences of gravity compared to the original scheme but also opens avenues for prospective astronomical experiments. These experiments, aligned with our postulates, hold immense advantages and implications for the detection of quantum gravity and can be envisioned for future design.

gr-qc↗

arXiv2024

Gravity-induced entanglement between two massive microscopic particles in curved spacetime: II.Friedmann- Lemaître-Robertson-Walker universe

In our previous work, we have explored quantum gravity induced entanglement of mass (QGEM) in curved spacetime, observing entanglement formation between particles moving along geodesics in a Schwarzschild spacetime background. We find that long interaction time induces entanglement, even for particles with microscopic mass, addressing decoherence concerns. In this work, we build upon our previous work by extending our investigation to a time-dependent spacetime. Specifically, we explore the entanglement induced by the mutual gravitation of massive particles in the Friedmann-Lemaître-Robertson-Walker(FLRW) universe. Through calculations of the phase variation and the QGEM spectrum, our proposed scheme offers a potential method for observing the formation of entanglement caused by the quantum gravity of massive particles as they propagate in the FLRW universe. Consequently, our research provides fresh insights into the field of entanglement in cosmology.

gr-qc↗

arXiv2024

Super-Resolving Blurry Images with Events

Super-resolution from motion-blurred images poses a significant challenge due to the combined effects of motion blur and low spatial resolution. To address this challenge, this paper introduces an Event-based Blurry Super Resolution Network (EBSR-Net), which leverages the high temporal resolution of events to mitigate motion blur and improve high-resolution image prediction. Specifically, we propose a multi-scale center-surround event representation to fully capture motion and texture information inherent in events. Additionally, we design a symmetric cross-modal attention module to fully exploit the complementarity between blurry images and events. Furthermore, we introduce an intermodal residual group composed of several residual dense Swin Transformer blocks, each incorporating multiple Swin Transformer layers and a residual connection, to extract global context and facilitate inter-block feature aggregation. Extensive experiments show that our method compares favorably against state-of-the-art approaches and achieves remarkable performance.

cs.CV↗

arXiv2024

Learning Monocular Depth from Focus with Event Focal Stack

Depth from Focus estimates depth by determining the moment of maximum focus from multiple shots at different focal distances, i.e. the Focal Stack. However, the limited sampling rate of conventional optical cameras makes it difficult to obtain sufficient focus cues during the focal sweep. Inspired by biological vision, the event camera records intensity changes over time in extremely low latency, which provides more temporal information for focus time acquisition. In this study, we propose the EDFF Network to estimate sparse depth from the Event Focal Stack. Specifically, we utilize the event voxel grid to encode intensity change information and project event time surface into the depth domain to preserve per-pixel focal distance information. A Focal-Distance-guided Cross-Modal Attention Module is presented to fuse the information mentioned above. Additionally, we propose a Multi-level Depth Fusion Block designed to integrate results from each level of a UNet-like architecture and produce the final output. Extensive experiments validate that our method outperforms existing state-of-the-art approaches.

cs.CV↗

arXiv2024

An efficient truncation scheme for Eulerian and total Lagrangian SPH methods

In smoothed particle hydrodynamics (SPH) method, the particle-based approximations are implemented via kernel functions, and the evaluation of performance involves two key criteria: numerical accuracy and computational efficiency. In the SPH community, the Wendland kernel reigns as the prevailing choice due to its commendable accuracy and reasonable computational efficiency. Nevertheless, there exists an urgent need to enhance the computational efficiency of numerical methods while upholding accuracy. In this paper, we employ a truncation approach to limit the compact support of the Wendland kernel to 1.6h. This decision is based on the observation that particles within the range of 1.6h to 2h make negligible contributions, practically approaching zero, to the SPH approximation. To address integration errors stemming from the truncation, we incorporate the Laguerre-Gauss kernel for particle relaxation due to the fact that this kernel has been demonstrated to enable the attainment of particle distributions with reduced residue and integration errors \cite{wang2023fourth}. Furthermore, we introduce the kernel gradient correction to rectify numerical errors from the SPH approximation of kernel gradient and the truncated compact support. A comprehensive set of numerical examples including fluid dynamics in Eulerian formulation and solid dynamics in total Lagrangian formulation are tested and have demonstrated that truncated and standard Wendland kernels enable achieve the same level accuracy but the former significantly increase the computational efficiency.

cs.CE↗

arXiv2024

A characterization of wavelet sets on Vilenkin groups with its application to construction of MRA wavelets

Let $G$ be a Vilenkin group. In 2008, Y. A. Farkov constructed wavelets on $G$ via the multiresolution analysis method. In this article, a characterization of wavelet sets on $G$ is established, which provides another method for the construction of wavelets. As an application, the relation between multiresolution analyses and wavelets determined from wavelet sets is also presented. To some extent, these results positively answer a question mentioned by P. Mahapatra and D. Singh in [Bull. Sci. Math. 167 (2021), Paper No. 102945, 20 pp].

math.CA↗

arXiv2024

Resonances involving integer magnons and spin-1/2 excitations in a magnetism modulated two-dimensional electron gas

We conduct an experimental study of high-mobility two-dimensional electron gas (2DEG) in GaAs/AlGaAs quantum wells modulated by strong magnetism at an in-plane magnetic field ($B$). The modulated $B$-fields are performed via the single stripe and gratings which are made of the heavy rare earth metal Terbium (Tb) thin films on the sample surface. The robust ferromagnetic resonances (FMRs) persist to the temperature of 50-70 K in both the stripe and grating samples, for the ferromagnetism (FM) phase of Tb exists at above 100 K. The high-order (with integer numbers of $j \equiv \hbarω/gμ_{B}B = 1, 2,$...) magnetic resonances can also be observed in the stripe structure via the microwave (MW) photovoltaic detection and magnetoresistance under microwave irradiation. In addition, the resonance features around $j = 1/2$ are robust in the single-stripe modulated sample, which suggests the spinons with spin-1/2 collective excitations in a 1D Heisenberg model.

cond-mat.mes-hall↗

arXiv2024

Abusing Processor Exception for General Binary Instrumentation on Bare-metal Embedded Devices

Analyzing the security of closed-source drivers and libraries in embedded systems holds significant importance, given their fundamental role in the supply chain. Unlike x86, embedded platforms lack comprehensive binary manipulating tools, making it difficult for researchers and developers to effectively detect and patch security issues in such closed-source components. Existing works either depend on full-fledged operating system features or suffer from tedious corner cases, restricting their application to bare-metal firmware prevalent in embedded environments. In this paper, we present PIFER (Practical Instrumenting Framework for Embedded fiRmware) that enables general and fine-grained static binary instrumentation for embedded bare-metal firmware. By abusing the built-in hardware exception-handling mechanism of the embedded processors, PIFER can perform instrumentation on arbitrary target addresses. Additionally, We propose an instruction translation-based scheme to guarantee the correct execution of the original firmware after patching. We evaluate PIFER against real-world, complex firmware, including Zephyr RTOS, CoreMark benchmark, and a close-sourced commercial product. The results indicate that PIFER correctly instrumented 98.9% of the instructions. Further, a comprehensive performance evaluation was conducted, demonstrating the practicality and efficiency of our work.

cs.CR↗