arXiv subjects

Zihao Wang

Publications and source records attributed to Zihao Wang.

At least 253 records · Page 14Linked to original sources

arXiv2024

Texture-guided Coding for Deep Features

With the rapid development of machine vision technology in recent years, many researchers have begun to focus on feature compression that is better suited for machine vision tasks. The target of feature compression is deep features, which arise from convolution in the middle layer of a pre-trained convolutional neural network. However, due to the large volume of data and high level of abstraction of deep features, their application is primarily limited to machine-centric scenarios, which poses significant constraints in situations requiring human-computer interaction. This paper investigates features and textures and proposes a texture-guided feature compression strategy based on their characteristics. Specifically, the strategy comprises feature layers and texture layers. The feature layers serve the machine, including a feature selection module and a feature reconstruction network. With the assistance of texture images, they selectively compress and transmit channels relevant to visual tasks, reducing feature data while providing high-quality features for the machine. The texture layers primarily serve humans and consist of an image reconstruction network. This image reconstruction network leverages features and texture images to reconstruct preview images for humans. Our method fully exploits the characteristics of texture and features. It eliminates feature redundancy, reconstructs high-quality preview images for humans, and supports decision-making. The experimental results demonstrate excellent performance when employing our proposed method to compress the deep features.

cs.CV↗

arXiv2024

Selecting Large Language Model to Fine-tune via Rectified Scaling Law

The ever-growing ecosystem of LLMs has posed a challenge in selecting the most appropriate pre-trained model to fine-tune amidst a sea of options. Given constrained resources, fine-tuning all models and making selections afterward is unrealistic. In this work, we formulate this resource-constrained selection task into predicting fine-tuning performance and illustrate its natural connection with Scaling Law. Unlike pre-training, we find that the fine-tuning scaling curve includes not just the well-known "power phase" but also the previously unobserved "pre-power phase". We also explain why existing Scaling Law fails to capture this phase transition phenomenon both theoretically and empirically. To address this, we introduce the concept of "pre-learned data size" into our Rectified Scaling Law, which overcomes theoretical limitations and fits experimental results much better. By leveraging our law, we propose a novel LLM selection algorithm that selects the near-optimal model with hundreds of times less resource consumption, while other methods may provide negatively correlated selection. The project page is available at rectified-scaling-law.github.io.

cs.LG↗

arXiv2024

Multi-frequency control and measurement of a spin-7/2 system encoded in a transmon qudit

Qudits hold great promise for efficient quantum computation and the simulation of high-dimensional quantum systems. Utilizing a local Hilbert space of dimension d > 2 is known to speed up certain quantum algorithms relative to their qubit counterparts given efficient local qudit control and measurement. However, the direct realization of high-dimensional rotations and projectors has proved challenging, with most experiments relying on decompositions of SU(d) operations into series of rotations between two-level subspaces of adjacent states and projective readout of a small number of states. Here we employ simultaneous multi-frequency drives to generate rotations and projections in an effective spin-7/2 system by mapping it onto the energy eigenstates of a superconducting circuit. We implement single-shot readout of the 8 states using a multi-tone dispersive readout (F_assignment = 88.3%) and exploit the strong nonlinearity in a high EJ/EC transmon to simultaneously address each transition and realize a spin displacement operator. By combining the displacement operator with a virtual SNAP gate, we realize arbitrary single-qudit unitary operations in O(d) physical pulses and extract spin displacement gate fidelities ranging from 0.997 to 0.989 for virtual spins of size j = 1 to j = 7/2. These native qudit operations could be combined with entangling operations to explore qudit-based error correction or simulations of lattice gauge theories with qudits. Our multi-frequency approach to qudit control and measurement can be readily extended to other physical platforms that realize a multi-level system coupled to a cavity and can become a building block for efficient qudit-based quantum computation and simulation.

quant-ph↗

arXiv2024

Non-Abelian Braiding of Topological Edge Bands

Braiding is a geometric concept that manifests itself in a variety of scientific contexts from biology to physics, and has been employed to classify bulk band topology in topological materials. Topological edge states can also form braiding structures, as demonstrated recently in a type of topological insulators known as Möbius insulators, whose topological edge states form two braided bands exhibiting a Möbius twist. While the formation of Möbius twist is inspiring, it belongs to the simple Abelian braid group $\mathbb{B}_2$. The most fascinating features about topological braids rely on the non-Abelianness in the higher-order braid group $\mathbb{B}_N$ ($N \geq 3$), which necessitates multiple edge bands, but so far it has not been discussed. Here, based on the gauge enriched symmetry, we develop a scheme to realize non-Abelian braiding of multiple topological edge bands. We propose tight-binding models of topological insulators that are able to generate topological edge states forming non-Abelian braiding structures. Experimental demonstrations are conducted in two acoustic crystals, which carry three and four braided acoustic edge bands, respectively. The observed braiding structure can correspond to the topological winding in the complex eigenvalue space of projective translation operator, akin to the previously established point-gap winding topology in the bulk of the Hatano-Nelson model. Our work also constitutes the realization of non-Abelian braiding topology on an actual crystal platform, but not based on the "virtual" synthetic dimensions.

cond-mat.mes-hall↗

arXiv2024

Numerical dissipation induced by the low-pass filtering in nonlinear gyrokinetic simulations

De-aliasing is an essential procedure for eliminating the aliasing error in nonlinear simulations, such as nonlinear gyrokinetic turbulence simulations. An ideal approach to de-aliasing in the periodic dimension is Fourier truncation. Finite difference low-pass filtering applied in the non-periodic direction strongly dampens aliasing modes. At the same time, it induces numerical dissipation in the region of the physically realistic solution. It is shown analytically that the long-wave dissipation coefficient is proportional to the (Np-3) power of the wavenumber under desirable constraints satisfying the highest order accuracy, where Np is the number of filter points. Numerical results after applying the optimised low-pass filtering to the nonlinear gyrokinetic turbulence simulation suggest that the nine-point format preserves intact mesoscopic zonal structures in Tokamak plasmas, and is therefore suitable for long-time nonlinear turbulence simulations.

physics.plasm-ph↗

arXiv2024

Chimera: A Lossless Decoding Method for Accelerating Large Language Models Inference by Fusing all Tokens

Large language models (LLMs) have demonstrated remarkable capabilities across various tasks. However, their widespread application is hindered by the resource-intensive decoding process. To address this challenge, current approaches have incorporated additional decoding heads to enable parallel prediction of multiple subsequent tokens, thereby achieving inference acceleration. Nevertheless, the accuracy of these decoding heads falls short of the auto-regressive decoding approach. In light of these limitations, we propose Chimera, a novel framework specifically designed for speculative sampling. Within this framework, we introduce a lightweight draft model that effectively utilizes previously generated tokens to predict subsequent words. To ensure both accuracy and efficiency, we present two strategies within the lightweight draft model. Firstly, we focus on capturing short-range dependencies at the bottom layer. Secondly, we leverage the readily available representations from the original LLM.Through empirical evaluation on the Vicuna and LlaMA-2 series, Chimera demonstrates impressive results, achieving an average latency speedup ratio of 2.7x compared to the vanilla auto-regressive decoding approach. This highlights the potential of our proposed framework in significantly improving the efficiency of large language models during the decoding process.

cs.CL↗

arXiv2024

Fast Gradient Computation for Gromov-Wasserstein Distance

The Gromov-Wasserstein distance is a notable extension of optimal transport. In contrast to the classic Wasserstein distance, it solves a quadratic assignment problem that minimizes the pair-wise distance distortion under the transportation of distributions and thus could apply to distributions in different spaces. These properties make Gromov-Wasserstein widely applicable to many fields, such as computer graphics and machine learning. However, the computation of the Gromov-Wasserstein distance and transport plan is expensive. The well-known Entropic Gromov-Wasserstein approach has a cubic complexity since the matrix multiplication operations need to be repeated in computing the gradient of Gromov-Wasserstein loss. This becomes a key bottleneck of the method. Currently, existing methods accelerate the computation focus on sampling and approximation, which leads to low accuracy or incomplete transport plan. In this work, we propose a novel method to accelerate accurate gradient computation by dynamic programming techniques, reducing the complexity from cubic to quadratic. In this way, the original computational bottleneck is broken and the new entropic solution can be obtained with total quadratic time, which is almost optimal complexity. Furthermore, it can be extended to some variants easily. Extensive experiments validate the efficiency and effectiveness of our method.

cs.LG↗

arXiv2024

Diffusion based Zero-shot Medical Image-to-Image Translation for Cross Modality Segmentation

Cross-modality image segmentation aims to segment the target modalities using a method designed in the source modality. Deep generative models can translate the target modality images into the source modality, thus enabling cross-modality segmentation. However, a vast body of existing cross-modality image translation methods relies on supervised learning. In this work, we aim to address the challenge of zero-shot learning-based image translation tasks (extreme scenarios in the target modality is unseen in the training phase). To leverage generative learning for zero-shot cross-modality image segmentation, we propose a novel unsupervised image translation method. The framework learns to translate the unseen source image to the target modality for image segmentation by leveraging the inherent statistical consistency between different modalities for diffusion guidance. Our framework captures identical cross-modality features in the statistical domain, offering diffusion guidance without relying on direct mappings between the source and target domains. This advantage allows our method to adapt to changing source domains without the need for retraining, making it highly practical when sufficient labeled source domain data is not available. The proposed framework is validated in zero-shot cross-modality image segmentation tasks through empirical comparisons with influential generative models, including adversarial-based and diffusion-based models.

eess.IV↗

arXiv2024

Manifolds for which Huber's Theorem holds

Extensions of Huber's Theorem to higher dimensions with $L^\frac{n}{2}$ bounded scalar curvature have been extensively studied over the years. In this paper, we delve into the properties of conformal metrics on a punctured ball with $\|R\|_{L^\frac{n}{2}}<+\infty$, aiming to identify necessary geometric constraints for Huber's theorem to be applicable. Unexpectedly, such metrics are more rigid than we initially anticipated. For instance, we found that the volume density at infinity is precisely one, and the blow-down of the metric is $\mathbb{R}^n$. Specifically, in four dimensions, we derive the $L^2$-integrability of the Ricci curvature, which directly leads to the conclusion that the Pfaffian 4-form is integrable and adheres to a Gauss-Bonnet-Chern formula. Additionally, we demonstrate that a Gauss-Bonnet-Chern formula, previously verified by Lu and Wang under the assumption that the second fundamental form belongs to $L^4$, remains valid for $R \in L^2$. Consequently, on an orientable 4-dimensional manifold conformal to a domain in a closed manifold, Huber's Theorem holds when $R \in L^2$, if and only if the negative part of the Pfaffian 4-form is integrable.

math.DG↗

arXiv2024

Traffic State Estimation and Uncertainty Quantification at Signalized Intersections with Low Penetration Rate Vehicle Trajectory Data

This paper studies the traffic state estimation problem at signalized intersections with low penetration rate vehicle trajectory data. While many existing studies have proposed different methods to estimate unknown traffic states and parameters (e.g., penetration rate, queue length) with this data, most of them only provide a point estimation without knowing the uncertainty of these estimated values. It is important to quantify the estimation uncertainty caused by limited available data since it can explicitly inform us whether the available data is sufficient to satisfy the desired estimation accuracy. To fill this gap, we formulate the partially observable system as a hidden Markov model (HMM) based on the recently developed probabilistic time-space (PTS) model. The PTS model is a stochastic traffic flow model that is designed for modeling traffic flow dynamics near signalized intersections. Based on the HMM formulation, a single recursive program is developed for the Bayesian estimation of both traffic states and parameters. As a Bayesian approach, the proposed method provides the distributional estimation outcomes and directly quantifies the estimation uncertainty. We validate the proposed method with simulation studies and showcase its applicability to real-world vehicle trajectory data.

eess.SY↗

arXiv2024

Meta Operator for Complex Query Answering on Knowledge Graphs

Knowledge graphs contain informative factual knowledge but are considered incomplete. To answer complex queries under incomplete knowledge, learning-based Complex Query Answering (CQA) models are proposed to directly learn from the query-answer samples to avoid the direct traversal of incomplete graph data. Existing works formulate the training of complex query answering models as multi-task learning and require a large number of training samples. In this work, we explore the compositional structure of complex queries and argue that the different logical operator types, rather than the different complex query types, are the key to improving generalizability. Accordingly, we propose a meta-learning algorithm to learn the meta-operators with limited data and adapt them to different instances of operators under various complex queries. Empirical results show that learning meta-operators is more effective than learning original CQA or meta-CQA models.

cs.LG↗

arXiv2024

RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generation

We explore how iterative revising a chain of thoughts with the help of information retrieval significantly improves large language models' reasoning and generation ability in long-horizon generation tasks, while hugely mitigating hallucination. In particular, the proposed method -- *retrieval-augmented thoughts* (RAT) -- revises each thought step one by one with retrieved information relevant to the task query, the current and the past thought steps, after the initial zero-shot CoT is generated. Applying RAT to GPT-3.5, GPT-4, and CodeLLaMA-7b substantially improves their performances on various long-horizon generation tasks; on average of relatively increasing rating scores by 13.63% on code generation, 16.96% on mathematical reasoning, 19.2% on creative writing, and 42.78% on embodied task planning. The demo page can be found at https://craftjarvis.github.io/RAT

cs.CL↗

arXiv2024

ZF Beamforming Tensor Compression for Massive MIMO Fronthaul

In the rapidly evolving landscape of 5G and beyond 5G (B5G) mobile cellular communications, efficient data compression and reconstruction strategies become paramount, especially in massive multiple-input multiple-output (MIMO) systems. A critical challenge in these systems is the capacity-limited fronthaul, particularly in the context of the Ethernet-based common public radio interface (eCPRI) connecting baseband units (BBUs) and remote radio units (RRUs). This capacity limitation hinders the effective handling of increased traffic and data flows. We propose a novel two-stage compression approach to address this bottleneck. The first stage employs sparse Tucker decomposition, targeting the weight tensor's low-rank components for compression. The second stage further compresses these components using complex givens decomposition and run-length encoding, substantially improving the compression ratio. Our approach specifically targets the Zero-Forcing (ZF) beamforming weights in BBUs. By reconstructing these weights in RRUs, we significantly alleviate the burden on eCPRI traffic, enabling a higher number of concurrent streams in the radio access network (RAN). Through comprehensive evaluations, we demonstrate the superior effectiveness of our method in Channel State Information (CSI) compression, paving the way for more efficient 5G/B5G fronthaul links.

cs.IT↗

arXiv2024

DPAdapter: Improving Differentially Private Deep Learning through Noise Tolerance Pre-training

Recent developments have underscored the critical role of \textit{differential privacy} (DP) in safeguarding individual data for training machine learning models. However, integrating DP oftentimes incurs significant model performance degradation due to the perturbation introduced into the training process, presenting a formidable challenge in the {differentially private machine learning} (DPML) field. To this end, several mitigative efforts have been proposed, typically revolving around formulating new DPML algorithms or relaxing DP definitions to harmonize with distinct contexts. In spite of these initiatives, the diminishment induced by DP on models, particularly large-scale models, remains substantial and thus, necessitates an innovative solution that adeptly circumnavigates the consequential impairment of model utility. In response, we introduce DPAdapter, a pioneering technique designed to amplify the model performance of DPML algorithms by enhancing parameter robustness. The fundamental intuition behind this strategy is that models with robust parameters are inherently more resistant to the noise introduced by DP, thereby retaining better performance despite the perturbations. DPAdapter modifies and enhances the sharpness-aware minimization (SAM) technique, utilizing a two-batch strategy to provide a more accurate perturbation estimate and an efficient gradient descent, thereby improving parameter robustness against noise. Notably, DPAdapter can act as a plug-and-play component and be combined with existing DPML algorithms to further improve their performance. Our experiments show that DPAdapter vastly enhances state-of-the-art DPML algorithms, increasing average accuracy from 72.92\% to 77.09\% with a privacy budget of $ε=4$.

cs.LG↗

arXiv2024

Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?

Recent progress in LLMs discussion suggests that multi-agent discussion improves the reasoning abilities of LLMs. In this work, we reevaluate this claim through systematic experiments, where we propose a novel group discussion framework to enrich the set of discussion mechanisms. Interestingly, our results show that a single-agent LLM with strong prompts can achieve almost the same performance as the best existing discussion approach on a wide range of reasoning tasks and backbone LLMs. We observe that the multi-agent discussion performs better than a single agent only when there is no demonstration in the prompt. Further study reveals the common interaction mechanisms of LLMs during the discussion.

cs.CL↗

arXiv2024

Exploring Emerging Trends in 5G Malicious Traffic Analysis and Incremental Learning Intrusion Detection Strategies

The popularity of 5G networks poses a huge challenge for malicious traffic detection technology. The reason for this is that as the use of 5G technology increases, so does the risk of malicious traffic activity on 5G networks. Malicious traffic activity in 5G networks not only has the potential to disrupt communication services, but also to compromise sensitive data. This can have serious consequences for individuals and organizations. In this paper, we first provide an in-depth study of 5G technology and 5G security. Next we analyze and discuss the latest malicious traffic detection under AI and their applicability to 5G networks, and compare the various traffic detection aspects addressed by SOTA. The SOTA in 5G traffic detection is also analyzed. Next, we propose seven criteria for traffic monitoring datasets to confirm their suitability for future traffic detection studies. Finally, we present three major issues that need to be addressed for traffic detection in 5G environment. The concept of incremental learning techniques is proposed and applied in the experiments, and the experimental results prove to be able to solve the three problems to some extent.

cs.CR↗

arXiv2024

Coexistence of Superconductivity and Antiferromagnetism in Topological Magnet MnBi2Te4 Films

The interface of two materials can harbor unexpected emergent phenomena. One example is interface-induced superconductivity. In this work, we employ molecular beam epitaxy to grow a series of heterostructures formed by stacking together two non-superconducting antiferromagnetic materials, an intrinsic antiferromagnetic topological insulator MnBi2Te4 and an antiferromagnetic iron chalcogenide FeTe. Our electrical transport measurements reveal interface-induced superconductivity in these heterostructures. By performing scanning tunneling microscopy and spectroscopy measurements, we observe a proximity-induced superconducting gap on the top surface of the MnBi2Te4 layer, confirming the interaction between superconductivity and antiferromagnetism in the MnBi2Te4 layer. Our findings will advance the fundamental inquiries into the topological superconducting phase in hybrid devices and provide a promising platform for the exploration of chiral Majorana physics in MnBi2Te4-based heterostructures.

cond-mat.supr-con↗

arXiv2024

Rhythmic soliton interactions for integrated dual-microcomb spectroscopy

Rotation symmetry of microresonators supports the generation of phase-locked counter-propagating (CP) solitons that can potentially miniaturize dual-comb systems. Realization of these dual-comb compatible solitons in photonic integrated circuits remains a challenge. Here, we synthesized such CP solitons in an integrated silicon nitride microresonator and observed forced soliton oscillation due to rhythmic, time-varying soliton interactions. The interactions result in seconds mutual-coherence passively. Temporal motion in the soliton streams is discerned by measuring a quadratic-scaling frequency noise peaks and an inverse quadratic-scaling microcomb sidebands. By generating a CP soliton trimer to have two synchronized solitons in one of the orbiting directions, we resolve the incapability of measuring two unsynchronized CP soliton dimer pulses by optical cross-correlation, and show CP solitons undergo complex motion trajectory. We further prove that precise dual-comb spectroscopy with an acquisition time as short as 0.6 $μ$s is feasible using these solitons, although the temporal motion limits the dynamic range. Besides revealing soliton interactions with different group velocities, our work propels the realization of photonic integrated dual-comb spectrometers with high passive coherence.

physics.optics↗