arXiv subjects

Chen Zhang

Publications and source records attributed to Chen Zhang.

At least 433 records · Page 24Linked to original sources

arXiv2023

Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation

Large diffusion models have been successful in text-to-audio (T2A) synthesis tasks, but they often suffer from common issues such as semantic misalignment and poor temporal consistency due to limited natural language understanding and data scarcity. Additionally, 2D spatial structures widely used in T2A works lead to unsatisfactory audio quality when generating variable-length audio samples since they do not adequately prioritize temporal information. To address these challenges, we propose Make-an-Audio 2, a latent diffusion-based T2A method that builds on the success of Make-an-Audio. Our approach includes several techniques to improve semantic alignment and temporal consistency: Firstly, we use pre-trained large language models (LLMs) to parse the text into structured pairs for better temporal information capture. We also introduce another structured-text encoder to aid in learning semantic alignment during the diffusion denoising process. To improve the performance of variable length generation and enhance the temporal information extraction, we design a feed-forward Transformer-based diffusion denoiser. Finally, we use LLMs to augment and transform a large amount of audio-label data into audio-text datasets to alleviate the problem of scarcity of temporal data. Extensive experiments show that our method outperforms baseline models in both objective and subjective metrics, and achieves significant gains in temporal information understanding, semantic consistency, and sound quality.

cs.SD↗

arXiv2023

Task-agnostic Distillation of Encoder-Decoder Language Models

Finetuning pretrained language models (LMs) have enabled appealing performance on a diverse array of tasks. The intriguing task-agnostic property has driven a shifted focus from task-specific to task-agnostic distillation of LMs. While task-agnostic, compute-efficient, performance-preserved LMs can be yielded by task-agnostic distillation, previous studies mainly sit in distillation of either encoder-only LMs (e.g., BERT) or decoder-only ones (e.g., GPT) yet largely neglect that distillation of encoder-decoder LMs (e.g., T5) can posit very distinguished behaviors. Frustratingly, we discover that existing task-agnostic distillation methods can fail to handle the distillation of encoder-decoder LMs. To the demand, we explore a few paths and uncover a path named as MiniEnD that successfully tackles the distillation of encoder-decoder LMs in a task-agnostic fashion. We examine MiniEnD on language understanding and abstractive summarization. The results showcase that MiniEnD is generally effective and is competitive compared to other alternatives. We further scale MiniEnD up to distillation of 3B encoder-decoder language models with interpolated distillation. The results imply the opportunities and challenges in distilling large language models (e.g., LLaMA).

cs.CL↗

arXiv2023

Instrumental Variable Learning for Chest X-ray Classification

The chest X-ray (CXR) is commonly employed to diagnose thoracic illnesses, but the challenge of achieving accurate automatic diagnosis through this method persists due to the complex relationship between pathology. In recent years, various deep learning-based approaches have been suggested to tackle this problem but confounding factors such as image resolution or noise problems often damage model performance. In this paper, we focus on the chest X-ray classification task and proposed an interpretable instrumental variable (IV) learning framework, to eliminate the spurious association and obtain accurate causal representation. Specifically, we first construct a structural causal model (SCM) for our task and learn the confounders and the preliminary representations of IV, we then leverage electronic health record (EHR) as auxiliary information and we fuse the above feature with our transformer-based semantic fusion module, so the IV has the medical semantic. Meanwhile, the reliability of IV is further guaranteed via the constraints of mutual information between related causal variables. Finally, our approach's performance is demonstrated using the MIMIC-CXR, NIH ChestX-ray 14, and CheXpert datasets, and we achieve competitive results.

eess.IV↗

arXiv2023

Chest X-ray Image Classification: A Causal Perspective

The chest X-ray (CXR) is one of the most common and easy-to-get medical tests used to diagnose common diseases of the chest. Recently, many deep learning-based methods have been proposed that are capable of effectively classifying CXRs. Even though these techniques have worked quite well, it is difficult to establish whether what these algorithms actually learn is the cause-and-effect link between diseases and their causes or just how to map labels to photos.In this paper, we propose a causal approach to address the CXR classification problem, which constructs a structural causal model (SCM) and uses the backdoor adjustment to select effective visual information for CXR classification. Specially, we design different probability optimization functions to eliminate the influence of confounders on the learning of real causality. Experimental results demonstrate that our proposed method outperforms the open-source NIH ChestX-ray14 in terms of classification performance.

eess.IV↗

arXiv2023

Lifting the Curse of Capacity Gap in Distilling Language Models

Pretrained language models (LMs) have shown compelling performance on various downstream tasks, but unfortunately they require a tremendous amount of inference compute. Knowledge distillation finds a path to compress LMs to small ones with a teacher-student paradigm. However, when the capacity gap between the teacher and the student is large, a curse of capacity gap appears, invoking a deficiency in distilling LMs. While a few studies have been carried out to fill the gap, the curse is not yet well tackled. In this paper, we aim at lifting the curse of capacity gap via enlarging the capacity of the student without notably increasing the inference compute. Largely motivated by sparse activation regime of mixture of experts (MoE), we propose a mixture of minimal experts (MiniMoE), which imposes extra parameters to the student but introduces almost no additional inference compute. Experimental results on GLUE and CoNLL demonstrate the curse of capacity gap is lifted by the magic of MiniMoE to a large extent. MiniMoE also achieves the state-of-the-art performance at small FLOPs compared with a range of competitive baselines. With a compression rate as much as $\sim$50$\times$, MiniMoE preserves $\sim$95\% GLUE score of the teacher.

cs.CL↗

arXiv2023

Chess-board Acoustic Crystals with Momentum-space Nonsymmorphic Symmetries

Spatial symmetries appearing in both real and momentum space are of fundamental significance to crystals. However, in the conventional framework, every space group in real space, either symmorphic or nonsymmorphic, corresponds to a symmorphic dual in momentum space. Our experiment breaks the framework by showing that in a 2D acoustic crystal with chess-board pattern of $π$ and 0 fluxes, mirror reflections are manifested nonsymmorphically as glide reflections in momentum space. These momentum-space nonsymmorphic symmetries stem from projective, rather than ordinary, representations of the real-space symmetries due to the peculiar flux pattern. Moreover, our experiment demonstrates that the glide reflection can reduce the topological type of the Brillouin zone from the torus to the Klein bottle, resulting in novel topological phases with new topological invariants. Since crystalline topologies are based on momentum-space symmetries, our work paves the way for utilizing engineerable gauge fluxes over artificial crystals to extend the current topological classifications into the broader regime of momentum-space nonsymmorphic symmetries.

cond-mat.mes-hall↗

arXiv2023

The constant term algebra of type $A$: the Structure

In this paper, we discover a new noncommutative algebra. We refer this algebra as the constant term algebra of type $A$, which is generated by certain constant term operators. We characterize a structural result of this algebra by establishing an explicit basis in terms of certain forests. This algebra arises when we apply the method of the iterated Laurent series to investigate Beck and Pixton's residue computation for the Ehrhart series of the Birkhoff polytope. This algebra seems to be the first structural result in the area of the constant term world since the discovery of the Dyson constant term identity in 1962.

math.CO↗

arXiv2023

Itinerant to relocalized transition of f electrons in the Kondo insulator CeRu4Sn6

The three-dimensional electronic structure and the nature of Ce 4f electrons of the Kondo insulator CeRu4Sn6 are investigated by angle-resolved photoemission spectroscopy, utilizing tunable photon energies. Our results reveal (i) the three-dimensional k-space nature of the Fermi surface, (ii) the localized-to-itinerant transition of f electrons occurs at a much high temperature than the hybridization gap opening temperature, and (iii) the "relocalization" of itinerant f electrons below 25 K, which could be the precursor to the establishment of magnetic order.

cond-mat.str-el↗

arXiv2023

Phoenix: Democratizing ChatGPT across Languages

This paper presents our efforts to democratize ChatGPT across language. We release a large language model "Phoenix", achieving competitive performance among open-source English and Chinese models while excelling in languages with limited resources (covering both Latin and non-Latin languages). We believe this work will be beneficial to make ChatGPT more accessible, especially in countries where people cannot use ChatGPT due to restrictions from OpenAI or local goverments. Our data, code, and models are available at https://github.com/FreedomIntelligence/LLMZoo.

cs.CL↗

arXiv2023

OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantization

Transformer-based large language models (LLMs) have achieved great success with the growing model size. LLMs' size grows by $240\times$ every two years, which outpaces the hardware progress and makes model inference increasingly costly. Model quantization is a promising approach to mitigate the widening gap between LLM size and hardware capacity. However, the existence of outliers, values with significant magnitudes, in LLMs makes existing quantization methods less effective. Prior outlier-aware quantization schemes adopt sparsity encoding techniques to separate outliers from normal values where the process requires global coordination (e.g., a global sparsity coordination list). This incurs complex encoding/decoding hardware logics and an extra orchestration controller for the computation between outlier and normal values. As such, it is not hardware-efficient and hence only achieves sub-optimal quantization benefits. We propose OliVe, an algorithm/architecture co-designed solution that adopts an outlier-victim pair (OVP) quantization and handles outlier values locally with low hardware overheads and high performance gains. The key insight of OliVe is that outliers are important while the normal values next to them are not. Thus those normal values (called victims) can be sacrificed to accommodate outliers. This enables a memory-aligned OVP encoding scheme, which can be efficiently integrated to the existing hardware accelerators like systolic array and tensor core. As a result, OliVe-based accelerator surpasses the existing outlier-aware accelerator, GOBO, by 4.5$\times$ speedup and 4.0$\times$ energy reduction, respectively, with a superior model accuracy.

cs.AR↗

arXiv2023

Nesting-driven antiferromagnetic order in Kondo lattice CePd5Al2

We investigated the electronic structure of the antiferromagnetic Kondo lattice CePd5Al2 using high-resolution angle-resolved photoemission spectroscopy. The experimentally determined band structure of the conduction electrons is predominated by the Pd 4d character. It contains multiple hole and electron Fermi pockets, in good agreement with density functional theory calculations. The Fermi surface is folded over Q0 = (0, 0, 1), manifested by Fermi surface reconstruction and band folding. Our results suggest that Fermi surface nesting drives the formation of antiferromagnetic order in CePd5Al2.

cond-mat.str-el↗

arXiv2023

Theoretical understanding of correlation between magnetic phase transition and the superconducting dome in high-Tc cuprates

Many issues concerning the origin of high-temperature superconductivity (HTS) are still under debate. For example, how the magnetic ordering varies with doping and its relationship with the superconducting temperature; and why the maximal Tc always occurs near the quantum critical point. In this paper, taking hole-doped La2CuO4 as a classical example, we employ the first-principles band structure and total energy calculations and Monte Carlo simulations to explore how the symmetry-breaking magnetic ground state evolves with hole doping and the origin of a dome-shaped superconductivity region in the phase diagram. We demonstrate that the local antiferromagnetic ordering and doping play key roles in determining the electron-phonon coupling, thus Tc. Initially, the La2CuO4 possesses a checkerboard local antiferromagnetic ground state. As the hole doping increases, Tc increases with the increase of the density of states at the Fermi surface. But as the doping increases further, the strength of the antiferromagnetic interaction weakens. At the critical doping level, a magnetic phase transition occurs that reduces the local antiferromagnetism-assisted electron-phonon coupling, thus diminishing the Tc. The superconductivity disappears in the heavily overdoped region when the antiferromagnetic ordering disappears. These observations could account for why cuprates have a dome-shaped superconductivity region in the phase diagram. Our study, thus, contributes to a fundamental understanding of the correlation between doping, local magnetic ordering, and superconductivity of HTS.

cond-mat.supr-con↗

arXiv2023

Modular Retrieval for Generalization and Interpretation

New retrieval tasks have always been emerging, thus urging the development of new retrieval models. However, instantiating a retrieval model for each new retrieval task is resource-intensive and time-consuming, especially for a retrieval model that employs a large-scale pre-trained language model. To address this issue, we shift to a novel retrieval paradigm called modular retrieval, which aims to solve new retrieval tasks by instead composing multiple existing retrieval modules. Built upon the paradigm, we propose a retrieval model with modular prompt tuning named REMOP. It constructs retrieval modules subject to task attributes with deep prompt tuning, and yields retrieval models subject to tasks with module composition. We validate that, REMOP inherently with modularity not only has appealing generalizability and interpretability in preliminary explorations, but also achieves comparable performance to state-of-the-art retrieval models on a zero-shot retrieval benchmark.\footnote{Our code is available at \url{https://github.com/FreedomIntelligence/REMOP}}

cs.IR↗

arXiv2023

Road Extraction with Satellite Images and Partial Road Maps

Road extraction is a process of automatically generating road maps mainly from satellite images. Existing models all target to generate roads from the scratch despite that a large quantity of road maps, though incomplete, are publicly available (e.g. those from OpenStreetMap) and can help with road extraction. In this paper, we propose to conduct road extraction based on satellite images and partial road maps, which is new. We then propose a two-branch Partial to Complete Network (P2CNet) for the task, which has two prominent components: Gated Self-Attention Module (GSAM) and Missing Part (MP) loss. GSAM leverages a channel-wise self-attention module and a gate module to capture long-range semantics, filter out useless information, and better fuse the features from two branches. MP loss is derived from the partial road maps, trying to give more attention to the road pixels that do not exist in partial road maps. Extensive experiments are conducted to demonstrate the effectiveness of our model, e.g. P2CNet achieves state-of-the-art performance with the IoU scores of 70.71% and 75.52%, respectively, on the SpaceNet and OSM datasets.

cs.CV↗

arXiv2023

Investigating the image lag of a scientific CMOS sensor in X-ray detection

In recent years, scientific CMOS (sCMOS) sensors have been vigorously developed and have outperformed CCDs in several aspects: higher readout frame rate, higher radiation tolerance, and higher working temperature. For silicon image sensors, image lag will occur when the charges of an event are not fully transferred inside pixels. It can degrade the image quality for optical imaging, and deteriorate the energy resolution for X-ray spectroscopy. In this work, the image lag of a sCMOS sensor is studied. To measure the image lag under low-light illumination, we constructed a new method to extract the image lag from X-ray photons. The image lag of a customized X-ray sCMOS sensor GSENSE1516BSI is measured, and its influence on X-ray performance is evaluated. The result shows that the image lag of this sensor exists only in the immediately subsequent frame and is always less than 0.05% for different incident photon energies and under different experimental conditions. The residual charge is smaller than 0.5 e- with the highest incident photon charge around 8 ke-. Compared to the readout noise level around 3 e-, the image lag of this sensor is too small to have a significant impact on the imaging quality and the energy resolution. The image lag shows a positive correlation with the incident photon energy and a negative correlation with the temperature. However, it has no dependence on the gain setting and the integration time. These relations can be explained qualitatively by the non-ideal potential structure inside the pixels. This method can also be applied to the study of image lag for other kinds of imaging sensors.

astro-ph.IM↗

arXiv2023

Improving the X-ray energy resolution of a scientific CMOS detector by pixel-level gain correction

Scientific Complementary Metal Oxide Semiconductor (sCMOS) sensors are finding increasingly more applications in astronomical observations, thanks to their advantages over charge-coupled devices (CCDs) such as a higher readout frame rate, higher radiation tolerance, and higher working temperature. In this work, we investigate the performance at the individual pixel level of a large-format sCMOS sensor, GSENSE1516BSI, which has 4096 * 4096 pixels, each of 15 μm in size. To achieve this, three areas on the sCMOS sensor, each consisting of 99 * 99 pixels, are chosen for the experiment. The readout noise, conversion gain and energy resolutions of the individual pixels in these areas are measured from a large number (more than 25,000) of X-ray events accumulated for each of the pixels through long time exposures. The energy resolution of these pixels can reach 140 eV at 6.4 keV at room temperature and shows a significant positive correlation with the readout noise. The accurate gain can also be derived individually for each of the pixels from its X-ray spectrum obtained. Variations of the gain values are found at a level of 0.56% statistically among the 30 thousand pixels in the areas studied. With the gain of each pixel determined accurately, a precise gain correction is performed pixel by pixel in these areas, in contrast to the standardized ensemble gain used in the conventional method. In this way, we could almost completely eliminate the degradation of energy resolutions caused by gain variations among pixels. As a result, the energy resolution at room temperature can be significantly improved to 124.6 eV at 4.5 keV and 140.7 eV at 6.4 keV. This pixel-by-pixel gain correction method can be applied to all kinds of CMOS sensors, and is expected to find interesting applications in X-ray spectroscopic observations in the future.

astro-ph.IM↗

arXiv2023

Cross-Lingual Question Answering over Knowledge Base as Reading Comprehension

Although many large-scale knowledge bases (KBs) claim to contain multilingual information, their support for many non-English languages is often incomplete. This incompleteness gives birth to the task of cross-lingual question answering over knowledge base (xKBQA), which aims to answer questions in languages different from that of the provided KB. One of the major challenges facing xKBQA is the high cost of data annotation, leading to limited resources available for further exploration. Another challenge is mapping KB schemas and natural language expressions in the questions under cross-lingual settings. In this paper, we propose a novel approach for xKBQA in a reading comprehension paradigm. We convert KB subgraphs into passages to narrow the gap between KB schemas and questions, which enables our model to benefit from recent advances in multilingual pre-trained language models (MPLMs) and cross-lingual machine reading comprehension (xMRC). Specifically, we use MPLMs, with considerable knowledge of cross-lingual mappings, for cross-lingual reading comprehension. Existing high-quality xMRC datasets can be further utilized to finetune our model, greatly alleviating the data scarcity issue in xKBQA. Extensive experiments on two xKBQA datasets in 12 languages show that our approach outperforms various baselines and achieves strong few-shot and zero-shot performance. Our dataset and code are released for further research.

cs.CL↗

arXiv2023

Spread Flows for Manifold Modelling

Flow-based models typically define a latent space with dimensionality identical to the observational space. In many problems, however, the data does not populate the full ambient data space that they natively reside in, rather inhabiting a lower-dimensional manifold. In such scenarios, flow-based models are unable to represent data structures exactly as their densities will always have support off the data manifold, potentially resulting in degradation of model performance. To address this issue, we propose to learn a manifold prior for flow models that leverage the recently proposed spread divergence towards fixing the crucial problem; the KL divergence and maximum likelihood estimation are ill-defined for manifold learning. In addition to improving both sample quality and representation quality, an auxiliary benefit enabled by our approach is the ability to identify the intrinsic dimension of the manifold distribution.

stat.ML↗