arXiv subjects

Hao Zhang

Publications and source records attributed to Hao Zhang.

At least 703 records · Page 39Linked to original sources

arXiv2024

DAE-Net: Deforming Auto-Encoder for fine-grained shape co-segmentation

We present an unsupervised 3D shape co-segmentation method which learns a set of deformable part templates from a shape collection. To accommodate structural variations in the collection, our network composes each shape by a selected subset of template parts which are affine-transformed. To maximize the expressive power of the part templates, we introduce a per-part deformation network to enable the modeling of diverse parts with substantial geometry variations, while imposing constraints on the deformation capacity to ensure fidelity to the originally represented parts. We also propose a training scheme to effectively overcome local minima. Architecturally, our network is a branched autoencoder, with a CNN encoder taking a voxel shape as input and producing per-part transformation matrices, latent codes, and part existence scores, and the decoder outputting point occupancies to define the reconstruction loss. Our network, coined DAE-Net for Deforming Auto-Encoder, can achieve unsupervised 3D shape co-segmentation that yields fine-grained, compact, and meaningful parts that are consistent across diverse shapes. We conduct extensive experiments on the ShapeNet Part dataset, DFAUST, and an animal subset of Objaverse to show superior performance over prior methods. Code and data are available at https://github.com/czq142857/DAE-Net.

cs.CV↗

arXiv2024

ConvBench: A Multi-Turn Conversation Evaluation Benchmark with Hierarchical Capability for Large Vision-Language Models

This paper presents ConvBench, a novel multi-turn conversation evaluation benchmark tailored for Large Vision-Language Models (LVLMs). Unlike existing benchmarks that assess individual capabilities in single-turn dialogues, ConvBench adopts a three-level multimodal capability hierarchy, mimicking human cognitive processes by stacking up perception, reasoning, and creativity. Each level focuses on a distinct capability, mirroring the cognitive progression from basic perception to logical reasoning and ultimately to advanced creativity. ConvBench comprises 577 meticulously curated multi-turn conversations encompassing 215 tasks reflective of real-world demands. Automatic evaluations quantify response performance at each turn and overall conversation level. Leveraging the capability hierarchy, ConvBench enables precise attribution of conversation mistakes to specific levels. Experimental results reveal a performance gap between multi-modal models, including GPT4-V, and human performance in multi-turn conversations. Additionally, weak fine-grained perception in multi-modal models contributes to reasoning and creation failures. ConvBench serves as a catalyst for further research aimed at enhancing visual dialogues.

cs.MM↗

arXiv2024

Quadratic Neuron-empowered Heterogeneous Autoencoder for Unsupervised Anomaly Detection

Inspired by the complexity and diversity of biological neurons, a quadratic neuron is proposed to replace the inner product in the current neuron with a simplified quadratic function. Employing such a novel type of neurons offers a new perspective on developing deep learning. When analyzing quadratic neurons, we find that there exists a function such that a heterogeneous network can approximate it well with a polynomial number of neurons but a purely conventional or quadratic network needs an exponential number of neurons to achieve the same level of error. Encouraged by this inspiring theoretical result on heterogeneous networks, we directly integrate conventional and quadratic neurons in an autoencoder to make a new type of heterogeneous autoencoders. To our best knowledge, it is the first heterogeneous autoencoder that is made of different types of neurons. Next, we apply the proposed heterogeneous autoencoder to unsupervised anomaly detection for tabular data and bearing fault signals. The anomaly detection faces difficulties such as data unknownness, anomaly feature heterogeneity, and feature unnoticeability, which is suitable for the proposed heterogeneous autoencoder. Its high feature representation ability can characterize a variety of anomaly data (heterogeneity), discriminate the anomaly from the normal (unnoticeability), and accurately learn the distribution of normal samples (unknownness). Experiments show that heterogeneous autoencoders perform competitively compared to other state-of-the-art models.

cs.NE↗

arXiv2024

MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI

Large Vision-Language Models (LVLMs) show significant strides in general-purpose multimodal applications such as visual dialogue and embodied navigation. However, existing multimodal evaluation benchmarks cover a limited number of multimodal tasks testing rudimentary capabilities, falling short in tracking LVLM development. In this study, we present MMT-Bench, a comprehensive benchmark designed to assess LVLMs across massive multimodal tasks requiring expert knowledge and deliberate visual recognition, localization, reasoning, and planning. MMT-Bench comprises $31,325$ meticulously curated multi-choice visual questions from various multimodal scenarios such as vehicle driving and embodied navigation, covering $32$ core meta-tasks and $162$ subtasks in multimodal understanding. Due to its extensive task coverage, MMT-Bench enables the evaluation of LVLMs using a task map, facilitating the discovery of in- and out-of-domain tasks. Evaluation results involving $30$ LVLMs such as the proprietary GPT-4V, GeminiProVision, and open-sourced InternVL-Chat, underscore the significant challenges posed by MMT-Bench. We anticipate that MMT-Bench will inspire the community to develop next-generation multimodal foundation models aimed at achieving general-purpose multimodal intelligence.

cs.CV↗

arXiv2024

Multi-view Content-aware Indexing for Long Document Retrieval

Long document question answering (DocQA) aims to answer questions from long documents over 10k words. They usually contain content structures such as sections, sub-sections, and paragraph demarcations. However, the indexing methods of long documents remain under-explored, while existing systems generally employ fixed-length chunking. As they do not consider content structures, the resultant chunks can exclude vital information or include irrelevant content. Motivated by this, we propose the Multi-view Content-aware indexing (MC-indexing) for more effective long DocQA via (i) segment structured document into content chunks, and (ii) represent each content chunk in raw-text, keywords, and summary views. We highlight that MC-indexing requires neither training nor fine-tuning. Having plug-and-play capability, it can be seamlessly integrated with any retrievers to boost their performance. Besides, we propose a long DocQA dataset that includes not only question-answer pair, but also document structure and answer scope. When compared to state-of-art chunking schemes, MC-indexing has significantly increased the recall by 42.8%, 30.0%, 23.9%, and 16.3% via top k= 1.5, 3, 5, and 10 respectively. These improved scores are the average of 8 widely used retrievers (2 sparse and 6 dense) via extensive experiments.

cs.CL↗

arXiv2024

Tunable atom-cavity interactions with configurable atomic chains

We investigate atomic chains with different spatial configurations coupled to a ring cavity comprising two counterpropagating traveling modes. We describe the collective atom-light scattering effect with a structure factor of the atomic chain and demonstrate that the interactions between the atoms and the cavity are controlled by the structure factor, resulting in distinctly different collective excitation modes and energy spectrum than for Fabry-Pérot cavities. Remarkably, we observe that a cavity dark mode emerges when the atomic spacings are integer multiples of the half-wavelength. The nodes of this standing-wave dark mode align precisely with the atomic positions, enabling intracavity field conversion without free-space scattering. By adjusting the configuration of the atomic chain, we realize tunable photon routing and a large optical phase shift with almost no photon loss, which can be used to implement versatile building blocks for optical quantum engineering.

quant-ph↗

arXiv2024

Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length

The quadratic complexity and weak length extrapolation of Transformers limits their ability to scale to long sequences, and while sub-quadratic solutions like linear attention and state space models exist, they empirically underperform Transformers in pretraining efficiency and downstream task accuracy. We introduce Megalodon, a neural architecture for efficient sequence modeling with unlimited context length. Megalodon inherits the architecture of Mega (exponential moving average with gated attention), and further introduces multiple technical components to improve its capability and stability, including complex exponential moving average (CEMA), timestep normalization layer, normalized attention mechanism and pre-norm with two-hop residual configuration. In a controlled head-to-head comparison with Llama2, Megalodon achieves better efficiency than Transformer in the scale of 7 billion parameters and 2 trillion training tokens. Megalodon reaches a training loss of 1.70, landing mid-way between Llama2-7B (1.75) and 13B (1.67). Code: https://github.com/XuezheMax/megalodon

cs.LG↗

arXiv2024

Reformulating Sequential Recommendation: Learning Dynamic User Interest with Content-enriched Language Modeling

Recommender systems are indispensable in the realm of online applications, and sequential recommendation has enjoyed considerable prevalence due to its capacity to encapsulate the dynamic shifts in user interests. However, previous sequential modeling methods still have limitations in capturing contextual information. The primary reason is the lack of understanding of domain-specific knowledge and item-related textual content. Fortunately, the emergence of powerful language models has unlocked the potential to incorporate extensive world knowledge into recommendation algorithms, enabling them to go beyond simple item attributes and truly understand the world surrounding user preferences. To achieve this, we propose LANCER, which leverages the semantic understanding capabilities of pre-trained language models to generate personalized recommendations. Our approach bridges the gap between language models and recommender systems, resulting in more human-like recommendations. We demonstrate the effectiveness of our approach through a series of experiments conducted on multiple benchmark datasets, showing promising results and providing valuable insights into the influence of our model on sequential recommendation tasks. Furthermore, our experimental codes are publicly available at https://github.com/Gnimixy/lancer.

cs.IR↗

arXiv2024

SQUID oscillations in PbTe nanowire networks

Network structures by semiconductor nanowires hold great promise for advanced quantum devices, especially for applications in topological quantum computing. In this study, we created networks of PbTe nanowires arranged in loop configurations. Using shadow-wall epitaxy, we defined superconducting quantum interference devices (SQUIDs) using the superconductor Pb. These SQUIDs exhibit oscillations in supercurrent upon the scanning of a magnetic field. Most of the oscillations can be fitted assuming a sinusoidal current-phase relation for each Josephson junction. Under certain conditions, the oscillations are found to be skewed, suggesting possible deviation from a sinusoidal behavior. Our results highlight the potential of PbTe nanowires for building complex quantum devices in the form of networks.

cond-mat.mes-hall↗

arXiv2024

SSwsrNet: A Semi-Supervised Few-Shot Learning Framework for Wireless Signal Recognition

Wireless signal recognition (WSR) is crucial in modern and future wireless communication networks since it aims to identify properties of the received signal. Although many deep learning-based WSR models have been developed, they still rely on a large amount of labeled training data. Thus, they cannot tackle the few-sample problem in the practically and dynamically changing wireless communication environment. To overcome this challenge, a novel SSwsrNet framework is proposed by using the deep residual shrinkage network (DRSN) and semi-supervised learning. The DRSN can learn discriminative features from noisy signals. Moreover, a modular semi-supervised learning method that combines labeled and unlabeled data using MixMatch is exploited to further improve the classification performance under few-sample conditions. Extensive simulation results on automatic modulation classification (AMC) and wireless technology classification (WTC) demonstrate that our proposed WSR scheme can achieve better performance than the benchmark schemes in terms of classification accuracy. This novel method enables more robust and adaptive signal recognition for next-generation wireless networks.

eess.SP↗

arXiv2024

Toward Inference-optimal Mixture-of-Expert Large Language Models

Mixture-of-Expert (MoE) based large language models (LLMs), such as the recent Mixtral and DeepSeek-MoE, have shown great promise in scaling model size without suffering from the quadratic growth of training cost of dense transformers. Like dense models, training MoEs requires answering the same question: given a training budget, what is the optimal allocation on the model size and number of tokens? We study the scaling law of MoE-based LLMs regarding the relations between the model performance, model size, dataset size, and the expert degree. Echoing previous research studying MoE in different contexts, we observe the diminishing return of increasing the number of experts, but this seems to suggest we should scale the number of experts until saturation, as the training cost would remain constant, which is problematic during inference time. We propose to amend the scaling law of MoE by introducing inference efficiency as another metric besides the validation loss. We find that MoEs with a few (4/8) experts are the most serving efficient solution under the same performance, but costs 2.5-3.5x more in training. On the other hand, training a (16/32) expert MoE much smaller (70-85%) than the loss-optimal solution, but with a larger training dataset is a promising setup under a training budget.

cs.LG↗

arXiv2024

Gate-tunable subband degeneracy in semiconductor nanowires

Degeneracy and symmetry have a profound relation in quantum systems. Here, we report gate-tunable subband degeneracy in PbTe nanowires with a nearly symmetric cross-sectional shape. The degeneracy is revealed in electron transport by the absence of a quantized plateau. Utilizing a dual gate design, we can apply an electric field to lift the degeneracy, reflected as emergence of the plateau. This degeneracy and its tunable lifting were challenging to observe in previous nanowire experiments, possibly due to disorder. Numerical simulations can qualitatively capture our observation, shedding light on device parameters for future applications.

cond-mat.mes-hall↗

arXiv2024

Selective-Area-Grown PbTe-Pb Planar Josephson Junctions for Quantum Devices

Planar Josephson junctions are predicted to host Majorana zero modes. The material platforms in previous studies are two dimensional electron gases (InAs, InSb, InAsSb and HgTe) coupled to a superconductor such as Al or Nb. Here, we introduce a new material platform for planar JJs, the PbTe-Pb hybrid. The semiconductor, PbTe, was grown as a thin film via selective area epitaxy. The Josephson junction was defined by a shadow wall during the deposition of the superconductor Pb. Scanning transmission electron microscopy reveals a sharp semiconductor-superconductor interface. Gate-tunable supercurrent and multiple Andreev reflections are observed. A perpendicular magnetic field causes interference patterns of the switching current, exhibiting Fraunhofer-like and SQUID-like behaviors. We further demonstrate a prototype device for Majorana detection, wherein phase bias and tunneling spectroscopy are applicable.

cond-mat.supr-con↗

arXiv2024

HAL3D: Hierarchical Active Learning for Fine-Grained 3D Part Labeling

We present the first active learning tool for fine-grained 3D part labeling, a problem which challenges even the most advanced deep learning (DL) methods due to the significant structural variations among the small and intricate parts. For the same reason, the necessary data annotation effort is tremendous, motivating approaches to minimize human involvement. Our labeling tool iteratively verifies or modifies part labels predicted by a deep neural network, with human feedback continually improving the network prediction. To effectively reduce human efforts, we develop two novel features in our tool, hierarchical and symmetry-aware active labeling. Our human-in-the-loop approach, coined HAL3D, achieves 100% accuracy (barring human errors) on any test set with pre-defined hierarchical part labels, with 80% time-saving over manual effort.

cs.CV↗

arXiv2024

DISTFLASHATTN: Distributed Memory-efficient Attention for Long-context LLMs Training

FlashAttention (Dao, 2023) effectively reduces the quadratic peak memory usage to linear in training transformer-based large language models (LLMs) on a single GPU. In this paper, we introduce DISTFLASHATTN, a distributed memory-efficient attention mechanism optimized for long-context LLMs training. We propose three key techniques: token-level workload balancing, overlapping key-value communication, and a rematerialization-aware gradient checkpointing algorithm. We evaluate DISTFLASHATTN on Llama-7B and variants with sequence lengths from 32K to 512K. DISTFLASHATTN achieves 8x longer sequences, 4.45 - 5.64x speedup compared to Ring Self-Attention, 2 - 8x longer sequences, 1.24 - 2.01x speedup compared to Megatron-LM with FlashAttention. It achieves 1.67x and 1.26 - 1.88x speedup compared to recent Ring Attention and DeepSpeed-Ulysses. Code is available at https://github.com/RulinShao/LightSeq.

cs.LG↗

arXiv2024

Conditional Denoising Diffusion Probabilistic Model for Ground-roll Attenuation

Ground-roll attenuation is a challenging seismic processing task in land seismic survey. The ground-roll coherent noise with low frequency and high amplitude seriously contaminate the valuable reflection events, corrupting the quality of seismic data. The transform-based filtering methods leverage the distinct characteristics of the ground roll and seismic reflections within the transform domain to attenuate the ground-roll noise. However, the ground roll and seismic reflections often share overlaps in the transform domain, making it challenging to remove ground-roll noise without also attenuating useful reflections. We propose to apply a conditional diffusion denoising probabilistic model (c-DDPM) to attenuate the ground-roll noise and recover the reflections efficiently. We prepare the training dataset by using the finite-difference modelling method and the convolution modelling method. After the training process, the c-DDPM can generate the clean data given the seismic data as condition. The ground roll obtained by subtracting the clean data from the seismic data might contain some residual reflection energy. Thus, we further improve the c-DDPM to allow for generating the clean data and ground roll simultaneously. We then demonstrate the feasibility and effectiveness of our proposed method by using the synthetic data and the field data. The methods based on the local time-frequency (LTF) transform and U-Net are also applied to these two examples for comparing with our proposed method. The test results show that the proposed method perform better in attenuating the ground-roll noise from the seismic data than the LTF and U-Net methods.

physics.geo-ph↗

arXiv2024

Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusion

Image fusion aims to combine information from different source images to create a comprehensively representative image. Existing fusion methods are typically helpless in dealing with degradations in low-quality source images and non-interactive to multiple subjective and objective needs. To solve them, we introduce a novel approach that leverages semantic text guidance image fusion model for degradation-aware and interactive image fusion task, termed as Text-IF. It innovatively extends the classical image fusion to the text guided image fusion along with the ability to harmoniously address the degradation and interaction issues during fusion. Through the text semantic encoder and semantic interaction fusion decoder, Text-IF is accessible to the all-in-one infrared and visible image degradation-aware processing and the interactive flexible fusion outcomes. In this way, Text-IF achieves not only multi-modal image fusion, but also multi-modal information fusion. Extensive experiments prove that our proposed text guided image fusion strategy has obvious advantages over SOTA methods in the image fusion performance and degradation treatment. The code is available at https://github.com/XunpengYi/Text-IF.

cs.CV↗

arXiv2024

On moment relaxations for linear state feedback controller synthesis with non-convex quadratic costs and constraints

We present a simple and effective way to account for non-convex costs and constraints~in~state feedback synthesis, and an interpretation for the variables in which state feedback synthesis is typically convex. We achieve this by deriving the controller design using moment matrices of state and input. It turns out that this approach allows the consideration of non-convex constraints by relaxing them as expectation constraints, and that the variables in which state feedback synthesis is typically convexified can be identified with blocks of these moment matrices.

math.OC↗