arXiv subjects

Ziwei Wang

Publications and source records attributed to Ziwei Wang.

At least 163 records · Page 9Linked to original sources

arXiv2024

3D Small Object Detection with Dynamic Spatial Pruning

In this paper, we propose an efficient feature pruning strategy for 3D small object detection. Conventional 3D object detection methods struggle on small objects due to the weak geometric information from a small number of points. Although increasing the spatial resolution of feature representations can improve the detection performance on small objects, the additional computational overhead is unaffordable. With in-depth study, we observe the growth of computation mainly comes from the upsampling operation in the decoder of 3D detector. Motivated by this, we present a multi-level 3D detector named DSPDet3D which benefits from high spatial resolution to achieves high accuracy on small object detection, while reducing redundant computation by only focusing on small object areas. Specifically, we theoretically derive a dynamic spatial pruning (DSP) strategy to prune the redundant spatial representation of 3D scene in a cascade manner according to the distribution of objects. Then we design DSP module following this strategy and construct DSPDet3D with this efficient module. On ScanNet and TO-SCENE dataset, our method achieves leading performance on small object detection. Moreover, DSPDet3D trained with only ScanNet rooms can generalize well to scenes in larger scale. It takes less than 2s to directly process a whole building consisting of more than 4500k points while detecting out almost all objects, ranging from cups to beds, on a single RTX 3090 GPU. Project page: https://xuxw98.github.io/DSPDet3D/.

cs.CV↗

arXiv2024

Why does in-context learning fail sometimes? Evaluating in-context learning on open and closed questions

We measure the performance of in-context learning as a function of task novelty and difficulty for open and closed questions. For that purpose, we created a novel benchmark consisting of hard scientific questions, each paired with a context of various relevancy. We show that counter-intuitively, a context that is more aligned with the topic does not always help more than a less relevant context. This effect is especially visible for open questions and questions of high difficulty or novelty. This result reveals a fundamental difference between the treatment of close-form and open-form questions by large-language models and shows a need for a more robust evaluation of in-context learning on the variety of different types of questions. It also poses a new question of how to optimally select a context for large language models, especially in the context of Retrieval Augmented Generation (RAG) systems. Our results suggest that the answer to this question can be highly application-dependent and might be contingent on factors including the format of the question, the perceived difficulty level of the questions, and the novelty or popularity of the information we seek.

cs.CL↗

arXiv2024

Textured Exciton Insulators

We introduce and study new interacting topological states that arise in time-reversal symmetric bands with an underlying obstruction to forming localized states. If the $U(1)$ valley symmetry linked to independent charge conservation in each time-reversal sector is spontaneously broken, the corresponding `excitonic' order parameter is forced to form a topologically non-trivial texture across the Brillouin zone. We show that the resulting phase, which we dub a textured exciton insulator, cannot be given a local-moment description due to a form of delicate topology. Using toy models of bands with Chern or Euler obstructions to localization we construct explicit examples of the Chern or Euler texture insulators (CTIs or ETIs) they support, and demonstrate that these are generically competitive ground states at intermediate coupling. We construct field theories that capture the response properties of these new states. Finally, we identify the incommensurate Kekulé spiral phase observed in magic-angle bi- and trilayer graphene as a concrete realization of an ETI.

cond-mat.str-el↗

arXiv2024

Chern-Textured Exciton Insulators with Valley Spiral Order in Moiré Materials

We explore the phase diagrams of moiré materials in search of a new class of intervalley-coherent correlated insulating state: the Chern texture insulator (CTI). This phase of matter, proposed in a companion paper, breaks valley $U(1)$ symmetry in a nontrivial fashion wherein the valley order parameter is forced to texture in momentum space as a consequence of band topology. Using detailed Hartree-Fock studies, we establish that the CTI emerges as an energetically competitive intermediate-coupling ground state in several moiré systems which lack a twofold rotation symmetry that forbids the single-particle topology essential to the formation of the CTI valley texture.

cond-mat.str-el↗

arXiv2024

Towards Accurate Post-training Quantization for Diffusion Models

In this paper, we propose an accurate data-free post-training quantization framework of diffusion models (ADP-DM) for efficient image generation. Conventional data-free quantization methods learn shared quantization functions for tensor discretization regardless of the generation timesteps, while the activation distribution differs significantly across various timesteps. The calibration images are acquired in random timesteps which fail to provide sufficient information for generalizable quantization function learning. Both issues cause sizable quantization errors with obvious image generation performance degradation. On the contrary, we design group-wise quantization functions for activation discretization in different timesteps and sample the optimal timestep for informative calibration image generation, so that our quantized diffusion model can reduce the discretization errors with negligible computational overhead. Specifically, we partition the timesteps according to the importance weights of quantization functions in different groups, which are optimized by differentiable search algorithms. We also select the optimal timestep for calibration image generation by structural risk minimizing principle in order to enhance the generalization ability in the deployment of quantized diffusion model. Extensive experimental results show that our method outperforms the state-of-the-art post-training quantization of diffusion model by a sizable margin with similar computational cost.

cs.CV↗

arXiv2024

Spiral Complete Coverage Path Planning Based on Conformal Slit Mapping in Multi-connected Domains

The generation of smoother and shorter spiral complete coverage paths in multi-connected domains is a crucial research topic in path planning for robotic cavity machining and other related fields. Traditional methods for spiral path planning in multi-connected domains typically incorporate a subregion division procedure that leads to excessive subregion bridging, requiring longer, more sharply turning, and unevenly spaced spirals to achieve complete coverage. To address this issue, this paper proposes a novel spiral complete coverage path planning method using conformal slit mapping. It takes advantage of the fact that conformal slit mapping can transform multi-connected domains into regular disks or annuluses without the need for subregion division. Firstly, a slit mapping calculation technique is proposed for segmented cubic spline boundaries with corners. Secondly, a spiral path spacing control method is developed based on the maximum inscribed circle radius between adjacent conformal slit mapping iso-parameters. Thirdly, the spiral coverage path is derived by offsetting iso-parameters. Numerical experiments indicate that our method shares a comparable order of magnitude in computation time with the traditional PDE-based spiral complete coverage path method, but it excels in optimizing total path length, smoothness, and spacing consistency. Finally, we performed experiments on cavity milling and dry runs to compare the new method with the traditional PDE-based method in terms of machining duration and steering impact, respectively. The comparison reveals that, with both algorithms achieving complete coverage, the new algorithm reduces machining time and steering impact by 12.34% and 22.78%, respectively, compared with the traditional PDE-based method.

cs.RO↗

arXiv2024

Double-dome Unconventional Superconductivity in Twisted Trilayer Graphene

Graphene moiré systems are ideal environments for investigating complex phase diagrams and gaining fundamental insights into the mechanisms underlying exotic states of matter, as they permit controlled manipulation of electronic properties. Magic-angle twisted trilayer graphene (MATTG) has emerged as a key platform to explore moiré superconductivity, owing to the robustness of its superconducting order and the displacement-field tunability of its energy bands. Recent measurements strongly suggest that superconductivity in MATTG is unconventional. Here, we report the first direct observation of double-dome superconductivity in MATTG. The temperature, magnetic field, and bias current dependence of the superconductivity of doped holes collectively show that it is significantly suppressed near moiré filling $ν^* = -2.6$, leading to a double dome in the phase diagram within a finite window of the displacement field. The temperature dependence of the normal-state resistance and the $I-V$ curves straddling $ν^*$ are suggestive of a phase transition and the potentially distinct nature of superconductivity in the two domes. Hartree-Fock calculations incorporating mild strain yield an incommensurate Kekulé spiral state whose effective spin polarization peaks in the regime where superconductivity is suppressed in experiments. This allows us to draw conclusions about the normal state as well as the unconventional nature of the superconducting order parameter.

cond-mat.mes-hall↗

arXiv2024

Bethe/Gauge Correspondence for $A_N$ Spin Chains with Integrable Boundaries

We continue the survey initiated in arXiv:2012.14197 to explore the Bethe/Gauge correspondence between supersymmetric SO/Sp gauge theories in 2d/3d/4d and open spin chain with integrable boundaries. We collect the known Bethe ansatz equations of different types of spin chains with general boundaries that have been analyzed in the literature, and compare them with the vacua equations of the quiver gauge theories. It seems that not all the vacua equations of quiver gauge theory with BCD-type gauge groups can be realized as some known Bethe ansatz equations of integrable spin chain models.

hep-th↗

arXiv2024

Memory-based Adapters for Online 3D Scene Perception

In this paper, we propose a new framework for online 3D scene perception. Conventional 3D scene perception methods are offline, i.e., take an already reconstructed 3D scene geometry as input, which is not applicable in robotic applications where the input data is streaming RGB-D videos rather than a complete 3D scene reconstructed from pre-collected RGB-D videos. To deal with online 3D scene perception tasks where data collection and perception should be performed simultaneously, the model should be able to process 3D scenes frame by frame and make use of the temporal information. To this end, we propose an adapter-based plug-and-play module for the backbone of 3D scene perception model, which constructs memory to cache and aggregate the extracted RGB-D features to empower offline models with temporal learning ability. Specifically, we propose a queued memory mechanism to cache the supporting point cloud and image features. Then we devise aggregation modules which directly perform on the memory and pass temporal information to current frame. We further propose 3D-to-2D adapter to enhance image features with strong global context. Our adapters can be easily inserted into mainstream offline architectures of different tasks and significantly boost their performance on online tasks. Extensive experiments on ScanNet and SceneNN datasets demonstrate our approach achieves leading performance on three 3D scene perception tasks compared with state-of-the-art online methods by simply finetuning existing offline models, without any model and task-specific designs. \href{https://xuxw98.github.io/Online3D/}{Project page}.

cs.CV↗

arXiv2024

Online Training of Large Language Models: Learn while chatting

Large Language Models(LLMs) have dramatically revolutionized the field of Natural Language Processing(NLP), offering remarkable capabilities that have garnered widespread usage. However, existing interaction paradigms between LLMs and users are constrained by either inflexibility, limitations in customization, or a lack of persistent learning. This inflexibility is particularly evident as users, especially those without programming skills, have restricted avenues to enhance or personalize the model. Existing frameworks further complicate the model training and deployment process due to their computational inefficiencies and lack of user-friendly interfaces. To overcome these challenges, this paper introduces a novel interaction paradigm-'Online Training using External Interactions'-that merges the benefits of persistent, real-time model updates with the flexibility for individual customization through external interactions such as AI agents or online/offline knowledge bases.

cs.CL↗

arXiv2024

Primordial magnetic field as a common solution of nanohertz gravitational waves and the Hubble tension

The origin of interstellar and intergalactic magnetic fields remains largely unknown. One possibility is that they are related to the primordial magnetic fields (PMFs) produced by, for instance, the phase transitions of the early Universe. In this paper, we show that the PMF-induced turbulence generated at around the QCD phase transition epoch--the characteristic magnetic field strength $B_{\rm ch}^* \sim \mathcal{O}(1)~\rm{μG}$ and coherent length scale $\ell_{\rm ch}^* \sim \mathcal{O}(1)~\rm{pc}$--can naturally accommodate nanohertz gravitational waves reported by pulsar timing array (PTA) collaborations. Moreover, the evolution of the PMFs to the recombination era with the form of $B_{\rm ch}\sim \ell_{\rm ch}^{-α}$ can induce baryon density inhomogeneities, alter the recombination history, and alleviate the tension of the Hubble parameter $H_0$ and the matter clumpiness parameter $S_8$ between early- and late-time measurements for $0.88\leq α\leq 1.17$ (approximate 95\% credible region based on three PTA likelihoods). The further evolved PMFs may account for the $\sim {\cal O}(10^{-16})$ Gauss extragalactic magnetic field inferred with GRB 221009A.

astro-ph.HE↗

arXiv2024

On the importance of heavy fields in pseudo-scalar inflation

Pseudo-scalar inflation coupled with U(1) gauge fields through the Chern-Simons term has been extensively studied. However, new physics arising from UV theories may still influence the pseudo-scalar field at low-energy scales, potentially impacting predictions of inflation. In the realm of effective field theory (EFT), we investigated axion inflation, where operators from heavy fields are also present, in addition to the axion and gauge fields. The integrated out fields have two significant effects: the non-linear dispersion regime and coupling heavy modes to the Chern-Simons term. The first effect changes the propagation of the curvature fluctuation, while the second one results in additional operators that contribute to curvature fluctuation via inverse decay. We derived the power spectrum and magnitude of equilateral non-Gaussianity in this low-energy EFT. We found that the second effect could become significant as the mass of heavy fields approaches Hubble scale.

hep-th↗

arXiv2023

MCUFormer: Deploying Vision Transformers on Microcontrollers with Limited Memory

Due to the high price and heavy energy consumption of GPUs, deploying deep models on IoT devices such as microcontrollers makes significant contributions for ecological AI. Conventional methods successfully enable convolutional neural network inference of high resolution images on microcontrollers, while the framework for vision transformers that achieve the state-of-the-art performance in many vision applications still remains unexplored. In this paper, we propose a hardware-algorithm co-optimizations method called MCUFormer to deploy vision transformers on microcontrollers with extremely limited memory, where we jointly design transformer architecture and construct the inference operator library to fit the memory resource constraint. More specifically, we generalize the one-shot network architecture search (NAS) to discover the optimal architecture with highest task performance given the memory budget from the microcontrollers, where we enlarge the existing search space of vision transformers by considering the low-rank decomposition dimensions and patch resolution for memory reduction. For the construction of the inference operator library of vision transformers, we schedule the memory buffer during inference through operator integration, patch embedding decomposition, and token overwriting, allowing the memory buffer to be fully utilized to adapt to the forward pass of the vision transformer. Experimental results demonstrate that our MCUFormer achieves 73.62\% top-1 accuracy on ImageNet for image classification with 320KB memory on STM32F746 microcontroller. Code is available at https://github.com/liangyn22/MCUFormer.

cs.CV↗

arXiv2023

Context Matters: Data-Efficient Augmentation of Large Language Models for Scientific Applications

In this paper, we explore the challenges inherent to Large Language Models (LLMs) like GPT-4, particularly their propensity for hallucinations, logic mistakes, and incorrect conclusions when tasked with answering complex questions. The capacity of LLMs to present erroneous answers in a coherent and semantically rigorous manner further complicates the detection of factual inaccuracies. This issue is especially pronounced in fields that require specialized expertise. Our work delves into these challenges, aiming to enhance the understanding and mitigation of such errors, thereby contributing to the improvement of LLM accuracy and reliability in scientific and other specialized domains. Our findings reveal a non-linear relationship between the context's relevancy and the answers' measured quality. In addition, we demonstrate that with the correct calibration, it is possible to automate the grading procedure -- a finding suggesting that, at least to some degree, the LLMs can be used to self-examine the quality of their own performance. Finally, we describe an experimental platform that can be seen as a proof-of-concept of the techniques described in this work.

cs.CL↗

arXiv2023

ThinkBot: Embodied Instruction Following with Thought Chain Reasoning

Embodied Instruction Following (EIF) requires agents to complete human instruction by interacting objects in complicated surrounding environments. Conventional methods directly consider the sparse human instruction to generate action plans for agents, which usually fail to achieve human goals because of the instruction incoherence in action descriptions. On the contrary, we propose ThinkBot that reasons the thought chain in human instruction to recover the missing action descriptions, so that the agent can successfully complete human goals by following the coherent instruction. Specifically, we first design an instruction completer based on large language models to recover the missing actions with interacted objects between consecutive human instruction, where the perceived surrounding environments and the completed sub-goals are considered for instruction completion. Based on the partially observed scene semantic maps, we present an object localizer to infer the position of interacted objects for agents to achieve complex human goals. Extensive experiments in the simulated environment show that our ThinkBot outperforms the state-of-the-art EIF methods by a sizable margin in both success rate and execution efficiency.

cs.CV↗

arXiv2023

The nanohertz stochastic gravitational wave background from cosmic string loops and the abundant high redshift massive galaxies

Recently, pulsar timing array (PTA) experiments have provided compelling evidence for the existence of the nanohertz stochastic gravitational wave background (SGWB). In this work, we demonstrated that cosmic string loops generated from cosmic global strings offer a viable explanation for the observed nanohertz SGWB data, requiring a cosmic string tension parameter of $\log(Gμ) \sim -12$ and a loop number density of $\log N \sim 4$. Additionally, we revisited the impact of cosmic string loops on the abundance of massive galaxies at high redshifts. However, our analysis revealed challenges in identifying a consistent parameter space that can concurrently explain both the SGWB data and observations from the James Webb Space Telescope. This indicates the necessity for either extending the existing model employed in this research or acknowledging distinct physical origins for these two phenomena.

astro-ph.HE↗

arXiv2023

Kekulé spirals and charge transfer cascades in twisted symmetric trilayer graphene

We study the phase diagram of magic-angle twisted symmetric trilayer graphene in the presence of uniaxial heterostrain and interlayer displacement field. For experimentally reasonable strain, our mean-field analysis finds robust Kekulé spiral order whose doping-dependent ordering vector is incommensurate with the moiré superlattice, consistent with recent scanning tunneling microscopy experiments, and paralleling the behaviour of closely-related twisted bilayer graphene (TBG) systems. Strikingly, we identify a new possibility absent in TBG: the existence of $\textit{commensurate}$ Kekulé spiral order even at zero strain for experimentally realistic values of the interlayer potential in a trilayer. Our studies also reveal a complex pattern of charge transfer between weakly- and strongly-dispersive bands in strained trilayer samples as the density is tuned by electrostatic gating, that can be understood intuitively in terms of the `cascades' in the compressibility of magic-angle TBG.

cond-mat.str-el↗

arXiv2023

Robust Adversarial Attacks Detection for Deep Learning based Relative Pose Estimation for Space Rendezvous

Research on developing deep learning techniques for autonomous spacecraft relative navigation challenges is continuously growing in recent years. Adopting those techniques offers enhanced performance. However, such approaches also introduce heightened apprehensions regarding the trustability and security of such deep learning methods through their susceptibility to adversarial attacks. In this work, we propose a novel approach for adversarial attack detection for deep neural network-based relative pose estimation schemes based on the explainability concept. We develop for an orbital rendezvous scenario an innovative relative pose estimation technique adopting our proposed Convolutional Neural Network (CNN), which takes an image from the chaser's onboard camera and outputs accurately the target's relative position and rotation. We perturb seamlessly the input images using adversarial attacks that are generated by the Fast Gradient Sign Method (FGSM). The adversarial attack detector is then built based on a Long Short Term Memory (LSTM) network which takes the explainability measure namely SHapley Value from the CNN-based pose estimator and flags the detection of adversarial attacks when acting. Simulation results show that the proposed adversarial attack detector achieves a detection accuracy of 99.21%. Both the deep relative pose estimator and adversarial attack detector are then tested on real data captured from our laboratory-designed setup. The experimental results from our laboratory-designed setup demonstrate that the proposed adversarial attack detector achieves an average detection accuracy of 96.29%.

cs.CV↗