arXiv subjects

Xin Zhang

Publications and source records attributed to Xin Zhang.

At least 685 records · Page 38Linked to original sources

arXiv2024

SpeechGPT-Gen: Scaling Chain-of-Information Speech Generation

Benefiting from effective speech modeling, current Speech Large Language Models (SLLMs) have demonstrated exceptional capabilities in in-context speech generation and efficient generalization to unseen speakers. However, the prevailing information modeling process is encumbered by certain redundancies, leading to inefficiencies in speech generation. We propose Chain-of-Information Generation (CoIG), a method for decoupling semantic and perceptual information in large-scale speech generation. Building on this, we develop SpeechGPT-Gen, an 8-billion-parameter SLLM efficient in semantic and perceptual information modeling. It comprises an autoregressive model based on LLM for semantic information modeling and a non-autoregressive model employing flow matching for perceptual information modeling. Additionally, we introduce the novel approach of infusing semantic information into the prior distribution to enhance the efficiency of flow matching. Extensive experimental results demonstrate that SpeechGPT-Gen markedly excels in zero-shot text-to-speech, zero-shot voice conversion, and speech-to-speech dialogue, underscoring CoIG's remarkable proficiency in capturing and modeling speech's semantic and perceptual dimensions. Code and models are available at https://github.com/0nutation/SpeechGPT.

cs.CL↗

arXiv2024

SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models

Current speech large language models build upon discrete speech representations, which can be categorized into semantic tokens and acoustic tokens. However, existing speech tokens are not specifically designed for speech language modeling. To assess the suitability of speech tokens for building speech language models, we established the first benchmark, SLMTokBench. Our results indicate that neither semantic nor acoustic tokens are ideal for this purpose. Therefore, we propose SpeechTokenizer, a unified speech tokenizer for speech large language models. SpeechTokenizer adopts the Encoder-Decoder architecture with residual vector quantization (RVQ). Unifying semantic and acoustic tokens, SpeechTokenizer disentangles different aspects of speech information hierarchically across different RVQ layers. Furthermore, We construct a Unified Speech Language Model (USLM) leveraging SpeechTokenizer. Experiments show that SpeechTokenizer performs comparably to EnCodec in speech reconstruction and demonstrates strong performance on the SLMTokBench benchmark. Also, USLM outperforms VALL-E in zero-shot Text-to-Speech tasks. Code and models are available at https://github.com/ZhangXInFD/SpeechTokenizer/.

cs.CL↗

arXiv2024

CLIP Model for Images to Textual Prompts Based on Top-k Neighbors

Text-to-image synthesis, a subfield of multimodal generation, has gained significant attention in recent years. We propose a cost-effective approach for image-to-prompt generation that leverages generative models to generate textual prompts without the need for large amounts of annotated data. We divide our method into two stages: online stage and offline stage. We use a combination of the CLIP model and K-nearest neighbors (KNN) algorithm. The proposed system consists of two main parts: an offline task and an online task. Our method owns the highest metric 0.612 among these models, which is 0.013, 0.055, 0.011 higher than Clip, Clip + KNN(top 10) respectively.

cs.CV↗

arXiv2024

Stability and Sample Complexity of Divergence Regularized Optimal Transport

We study stability and sample complexity properties of divergence regularized optimal transport (DOT). First, we obtain quantitative stability results for optimizers of DOT measured in Wasserstein distance, which are applicable to a wide class of divergences and simultaneously improve known results for entropic optimal transport. Second, we study the case of sample complexity, where the DOT problem is approximated using empirical measures of the marginals. We show that divergence regularization can improve the corresponding convergence rate compared to unregularized optimal transport. To this end, we prove upper bounds which exploit both the regularity of cost function and divergence functional, as well as the intrinsic dimension of the marginals. Along the way, we establish regularity properties of dual optimizers of DOT, as well as general limit theorems for empirical measures with suitable classes of test functions.

math.OC↗

arXiv2024

Weiss-Weinstein bound of frequency estimation error for very weak GNSS signals

Tightness remains the center quest in all modern estimation bounds. For very weak signals, this is made possible with judicial choices of prior probability distribution and bound family. While current bounds in GNSS assess performance of carrier frequency estimators under Gaussian or uniform assumptions, the circular nature of frequency is overlooked. In addition, of all bounds in Bayesian framework, Weiss-Weinstein bound (WWB) stands out since it is free from regularity conditions or requirements on the prior distribution. Therefore, WWB is extended for the current frequency estimation problem. A divide-and-conquer type of hyperparameter tuning method is developed to level off the curse of computational complexity for the WWB family while enhancing tightness. Synthetic results show that with von Mises as prior probability distribution, WWB provides a bound up to 22.5% tighter than Ziv-Zakaï bound (ZZB) when SNR varies between -3.5 dB and -20 dB, where GNSS signal is deemed extremely weak.

eess.SP↗

arXiv2024

SpeechAgents: Human-Communication Simulation with Multi-Modal Multi-Agent Systems

Human communication is a complex and diverse process that not only involves multiple factors such as language, commonsense, and cultural backgrounds but also requires the participation of multimodal information, such as speech. Large Language Model (LLM)-based multi-agent systems have demonstrated promising performance in simulating human society. Can we leverage LLM-based multi-agent systems to simulate human communication? However, current LLM-based multi-agent systems mainly rely on text as the primary medium. In this paper, we propose SpeechAgents, a multi-modal LLM based multi-agent system designed for simulating human communication. SpeechAgents utilizes multi-modal LLM as the control center for individual agent and employes multi-modal signals as the medium for exchanged messages among agents. Additionally, we propose Multi-Agent Tuning to enhance the multi-agent capabilities of LLM without compromising general abilities. To strengthen and evaluate the effectiveness of human communication simulation, we build the Human-Communication Simulation Benchmark. Experimental results demonstrate that SpeechAgents can simulate human communication dialogues with consistent content, authentic rhythm, and rich emotions and demonstrate excellent scalability even with up to 25 agents, which can apply to tasks such as drama creation and audio novels generation. Code and models will be open-sourced at https://github. com/0nutation/SpeechAgents

cs.CL↗

arXiv2024

Understanding LLMs: A Comprehensive Overview from Training to Inference

The introduction of ChatGPT has led to a significant increase in the utilization of Large Language Models (LLMs) for addressing downstream tasks. There's an increasing focus on cost-efficient training and deployment within this context. Low-cost training and deployment of LLMs represent the future development trend. This paper reviews the evolution of large language model training techniques and inference deployment technologies aligned with this emerging trend. The discussion on training includes various aspects, including data preprocessing, training architecture, pre-training tasks, parallel training, and relevant content related to model fine-tuning. On the inference side, the paper covers topics such as model compression, parallel computation, memory scheduling, and structural optimization. It also explores LLMs' utilization and provides insights into their future development.

cs.CL↗

arXiv2024

TRLS: A Time Series Representation Learning Framework via Spectrogram for Medical Signal Processing

Representation learning frameworks in unlabeled time series have been proposed for medical signal processing. Despite the numerous excellent progresses have been made in previous works, we observe the representation extracted for the time series still does not generalize well. In this paper, we present a Time series (medical signal) Representation Learning framework via Spectrogram (TRLS) to get more informative representations. We transform the input time-domain medical signals into spectrograms and design a time-frequency encoder named Time Frequency RNN (TFRNN) to capture more robust multi-scale representations from the augmented spectrograms. Our TRLS takes spectrogram as input with two types of different data augmentations and maximizes the similarity between positive ones, which effectively circumvents the problem of designing negative samples. Our evaluation of four real-world medical signal datasets focusing on medical signal classification shows that TRLS is superior to the existing frameworks.

eess.SP↗

arXiv2024

HEAP: Unsupervised Object Discovery and Localization with Contrastive Grouping

Unsupervised object discovery and localization aims to detect or segment objects in an image without any supervision. Recent efforts have demonstrated a notable potential to identify salient foreground objects by utilizing self-supervised transformer features. However, their scopes only build upon patch-level features within an image, neglecting region/image-level and cross-image relationships at a broader scale. Moreover, these methods cannot differentiate various semantics from multiple instances. To address these problems, we introduce Hierarchical mErging framework via contrAstive grouPing (HEAP). Specifically, a novel lightweight head with cross-attention mechanism is designed to adaptively group intra-image patches into semantically coherent regions based on correlation among self-supervised features. Further, to ensure the distinguishability among various regions, we introduce a region-level contrastive clustering loss to pull closer similar regions across images. Also, an image-level contrastive loss is present to push foreground and background representations apart, with which foreground objects and background are accordingly discovered. HEAP facilitates efficient hierarchical image decomposition, which contributes to more accurate object discovery while also enabling differentiation among objects of various classes. Extensive experimental results on semantic segmentation retrieval, unsupervised object discovery, and saliency detection tasks demonstrate that HEAP achieves state-of-the-art performance.

cs.CV↗

arXiv2024

Efficient Sparse Least Absolute Deviation Regression with Differential Privacy

In recent years, privacy-preserving machine learning algorithms have attracted increasing attention because of their important applications in many scientific fields. However, in the literature, most privacy-preserving algorithms demand learning objectives to be strongly convex and Lipschitz smooth, which thus cannot cover a wide class of robust loss functions (e.g., quantile/least absolute loss). In this work, we aim to develop a fast privacy-preserving learning solution for a sparse robust regression problem. Our learning loss consists of a robust least absolute loss and an $\ell_1$ sparse penalty term. To fast solve the non-smooth loss under a given privacy budget, we develop a Fast Robust And Privacy-Preserving Estimation (FRAPPE) algorithm for least absolute deviation regression. Our algorithm achieves a fast estimation by reformulating the sparse LAD problem as a penalized least square estimation problem and adopts a three-stage noise injection to guarantee the $(ε,δ)$-differential privacy. We show that our algorithm can achieve better privacy and statistical accuracy trade-off compared with the state-of-the-art privacy-preserving regression algorithms. In the end, we conduct experiments to verify the efficiency of our proposed FRAPPE algorithm.

stat.ML↗

arXiv2024

The Taiji-TianQin-LISA network: Precisely measuring the Hubble constant using both bright and dark sirens

In the coming decades, the space-based gravitational-wave (GW) detectors such as Taiji, TianQin, and LISA are expected to form a network capable of detecting millihertz GWs emitted by the mergers of massive black hole binaries (MBHBs). In this work, we investigate the potential of GW standard sirens from the Taiji-TianQin-LISA network in constraining cosmological parameters. For the optimistic scenario in which electromagnetic (EM) counterparts can be detected, we predict the number of detectable bright sirens based on three different MBHB population models, i.e., pop III, Q3d, and Q3nod. Our results show that the Taiji-TianQin-LISA network alone could achieve a constraint precision of $0.9\%$ for the Hubble constant, meeting the standard of precision cosmology. Moreover, the Taiji-TianQin-LISA network could effectively break the cosmological parameter degeneracies generated by the CMB data, particularly in the dynamical dark energy models. When combined with the CMB data, the joint CMB+Taiji-TianQin-LISA data offer $σ(w)=0.036$ in the $w$CDM model, which is close to the latest constraint result obtained from the CMB+SN data. We also consider a conservative scenario in which EM counterparts are not available. Due to the precise sky localizations of MBHBs by the Taiji-TianQin-LISA network, the constraint precision of the Hubble constant is expected to reach $1.2\%$. In conclusion, the GW standard sirens from the Taiji-TianQin-LISA network will play a critical role in helping solve the Hubble tension and shedding light on the nature of dark energy.

astro-ph.CO↗

arXiv2023

Expanding the Pressure Frontier in Grüneisen Parameter Measurement: Study of Sodium Chloride

The Grüneisen parameter (γ) is crucial for determining many thermal properties, including the anharmonic effect, thermostatistics, and equation of state (EOS) of materials. However, the isentropic adiabatic compression conditions required to measure the Grüneisen parameter under high pressure are difficult to achieve. Thus, direct experimental Grüneisen parameter data in a wide range of pressures is sparse. In this work, we developed a new device that can apply pressure (up to tens of GPa) with an extremely short time about 0.5 ms, confidently achieving isentropic adiabatic compression. Then, we applied our new technique to sodium chloride and measured its Grüneisen parameter, which conforms to previous theoretical predictions. According to our obtained sodium chloride Grüneisen parameters, the calculated Hugoniot curve of the NaCl B1 phase appears up to 20 GPa and 960 K, which compares very well with the shock compression experiment data by Fritz et al. and other calculation works. Our results suggest that this new method can reliably measure the Grüneisen parameter of even more materials, which is significant for researching the equation of state in substances.

cond-mat.mtrl-sci↗

arXiv2023

Wireless, Customizable Coaxially-shielded Coils for Magnetic Resonance Imaging

Anatomy-specific RF receive coil arrays routinely adopted in magnetic resonance imaging (MRI) for signal acquisition, are commonly burdened by their bulky, fixed, and rigid configurations, which may impose patient discomfort, bothersome positioning, and suboptimal sensitivity in certain situations. Herein, leveraging coaxial cables' inherent flexibility and electric field confining property, for the first time, we present wireless, ultra-lightweight, coaxially-shielded MRI coils achieving a signal-to-noise ratio (SNR) comparable to or surpassing that of commercially available cutting-edge receive coil arrays with the potential for improved patient comfort, ease of implementation, and significantly reduced costs. The proposed coils demonstrate versatility by functioning both independently in form-fitting configurations, closely adapting to relatively small anatomical sites, and collectively by inductively coupling together as metamaterials, allowing for extension of the field-of-view of their coverage to encompass larger anatomical regions without compromising coil sensitivity. The wireless, coaxially-shielded MRI coils reported herein pave the way toward next generation MRI coils.

physics.med-ph↗

arXiv2023

Wearable Coaxially-shielded Metamaterial for Magnetic Resonance Imaging

Recent advancements in metamaterials have yielded the possibility of a wireless solution to improve signal-to-noise ratio (SNR) in magnetic resonance imaging (MRI). Unlike traditional closely packed local coil arrays with rigid designs and numerous components, these lightweight, cost-effective metamaterials eliminate the need for radio frequency (RF) cabling, baluns, adapters, and interfaces. However, their clinical adoption has been limited by their low sensitivity, bulky physical footprint, and limited, specific use cases. Herein, we introduce a wearable metamaterial developed using commercially available coaxial cable, designed for a 3.0 T MRI system. This metamaterial inherits the coaxially-shielded structure of its constituent coaxial cable, effectively containing the electric field within the cable, thereby mitigating the electric coupling to its loading while ensuring safer clinical adoption, lower signal loss, and resistance to frequency shifts. Weighing only 50g, the metamaterial maximizes its sensitivity by conforming to the anatomical region of interest. MRI images acquired using this metamaterial with various pulse sequences demonstrate an up to 2-fold SNR enhancement when compared to a state-of-the-art 16-channel knee coil. This work introduces a novel paradigm for constructing metamaterials in the MRI environment, paving the way for the development of next-generation wireless MRI technology.

physics.med-ph↗

arXiv2023

Stability of xenon-sodium compounds at moderately low pressures

A growing body of theoretical and experimental evidence suggests that inert gases (He, Ne, Ar, Kr, Xe, Rn) become less and less inert under increasing pressure. Here we use the ab initio evolutionary algorithm to predict stable compounds of Xe and Na at pressures below 100 GPa, and find three stable compounds, NaXe, NaXe$_3$ and NaXe$_4$. The NaXe belongs to a well-known cubic CsCl structure type. The NaXe$_4$'s structure is common in amphiboles, whereas the NaXe$_3$ has a unique structure, analogous to the "post-perovskite" orthorhombic CaIrO$_3$-type structure with Ir atoms removed. This is the first time that a cation-vacant version of the CaIrO$_3$ is found in any compound. NaXe, NaXe$_3$ and NaXe$_4$ are found to be metallic.

cond-mat.mtrl-sci↗

arXiv2023

TMID: A Comprehensive Real-world Dataset for Trademark Infringement Detection in E-Commerce

Annually, e-commerce platforms incur substantial financial losses due to trademark infringements, making it crucial to identify and mitigate potential legal risks tied to merchant information registered to the platforms. However, the absence of high-quality datasets hampers research in this area. To address this gap, our study introduces TMID, a novel dataset to detect trademark infringement in merchant registrations. This is a real-world dataset sourced directly from Alipay, one of the world's largest e-commerce and digital payment platforms. As infringement detection is a legal reasoning task requiring an understanding of the contexts and legal rules, we offer a thorough collection of legal rules and merchant and trademark-related contextual information with annotations from legal experts. We ensure the data quality by performing an extensive statistical analysis. Furthermore, we conduct an empirical study on this dataset to highlight its value and the key challenges. Through this study, we aim to contribute valuable resources to advance research into legal compliance related to trademark infringement within the e-commerce sphere. The dataset is available at https://github.com/emnlpTMID/emnlpTMID.github.io .

cs.CL↗

arXiv2023

GeV Variability Properties of TeV Blazars Detected by Fermi-LAT

Variability is a prominent observational feature of blazars. The high-energy radiation mechanism of jets has always been important but still unclear. In this work, we performed a detailed analysis using Fermi-LAT data across 15 years and obtained GeV light curve information for 78 TeV blazars detected by Fermi. We provided annual GeV fluxes and corresponding spectral indices for the 78 TeV blazars and thorough monthly GeV fluxes for a subsample of 41 bright blazars. Our results suggest a strong correlation between the $γ$-ray photon index and $\log L_{\rm γ}$ for the flat spectrum radio quasars (FSRQs) and high-energy-peaked BL Lacs (HBLs). 14 sources in our sample show significant GeV outbursts/flares above the relatively stable, low-flux light curve, with 6 of them showing a clear sharp peak profile in their 5-day binned light curves. We quantified the variability utilizing the fractional variability parameter $F_{\mathrm{var}}$, and found that the flux of the FSRQs showed significantly stronger variability than that of the BL Lacs. The 41 bright blazars in this work are best fit by a log-normal flux distribution. We checked the spectral behavior and found 11 out of the 14 sources show a 'bluer-when-brighter (BWB)' trend, suggesting this spectral behavior for these TeV blazars at the GeV band arises from the mechanism that the synchrotron-self Compton (SSC) process dominates the GeV emission. Our research offers a systematic analysis of the GeV variability properties of TeV blazars and serves as a helpful resource for further associated blazar studies.

astro-ph.HE↗

arXiv2023

Automated interpretation of congenital heart disease from multi-view echocardiograms

Congenital heart disease (CHD) is the most common birth defect and the leading cause of neonate death in China. Clinical diagnosis can be based on the selected 2D key-frames from five views. Limited by the availability of multi-view data, most methods have to rely on the insufficient single view analysis. This study proposes to automatically analyze the multi-view echocardiograms with a practical end-to-end framework. We collect the five-view echocardiograms video records of 1308 subjects (including normal controls, ventricular septal defect (VSD) patients and atrial septal defect (ASD) patients) with both disease labels and standard-view key-frame labels. Depthwise separable convolution-based multi-channel networks are adopted to largely reduce the network parameters. We also approach the imbalanced class problem by augmenting the positive training samples. Our 2D key-frame model can diagnose CHD or negative samples with an accuracy of 95.4\%, and in negative, VSD or ASD classification with an accuracy of 92.3\%. To further alleviate the work of key-frame selection in real-world implementation, we propose an adaptive soft attention scheme to directly explore the raw video data. Four kinds of neural aggregation methods are systematically investigated to fuse the information of an arbitrary number of frames in a video. Moreover, with a view detection module, the system can work without the view records. Our video-based model can diagnose with an accuracy of 93.9\% (binary classification), and 92.1\% (3-class classification) in a collected 2D video testing set, which does not need key-frame selection and view annotation in testing. The detailed ablation study and the interpretability analysis are provided.

eess.IV↗