arXiv subjects

Shuang Zhou

Publications and source records attributed to Shuang Zhou.

At least 55 records · Page 3Linked to original sources

arXiv2025

An evaluation of DeepSeek Models in Biomedical Natural Language Processing

The advancement of Large Language Models (LLMs) has significantly impacted biomedical Natural Language Processing (NLP), enhancing tasks such as named entity recognition, relation extraction, event extraction, and text classification. In this context, the DeepSeek series of models have shown promising potential in general NLP tasks, yet their capabilities in the biomedical domain remain underexplored. This study evaluates multiple DeepSeek models (Distilled-DeepSeek-R1 series and Deepseek-LLMs) across four key biomedical NLP tasks using 12 datasets, benchmarking them against state-of-the-art alternatives (Llama3-8B, Qwen2.5-7B, Mistral-7B, Phi-4-14B, Gemma-2-9B). Our results reveal that while DeepSeek models perform competitively in named entity recognition and text classification, challenges persist in event and relation extraction due to precision-recall trade-offs. We provide task-specific model recommendations and highlight future research directions. This evaluation underscores the strengths and limitations of DeepSeek models in biomedical NLP, guiding their future deployment and optimization.

cs.CL↗

arXiv2025

MMRAG: Multi-Mode Retrieval-Augmented Generation with Large Language Models for Biomedical In-Context Learning

Objective: To optimize in-context learning in biomedical natural language processing by improving example selection. Methods: We introduce a novel multi-mode retrieval-augmented generation (MMRAG) framework, which integrates four retrieval strategies: (1) Random Mode, selecting examples arbitrarily; (2) Top Mode, retrieving the most relevant examples based on similarity; (3) Diversity Mode, ensuring variation in selected examples; and (4) Class Mode, selecting category-representative examples. This study evaluates MMRAG on three core biomedical NLP tasks: Named Entity Recognition (NER), Relation Extraction (RE), and Text Classification (TC). The datasets used include BC2GM for gene and protein mention recognition (NER), DDI for drug-drug interaction extraction (RE), GIT for general biomedical information extraction (RE), and HealthAdvice for health-related text classification (TC). The framework is tested with two large language models (Llama2-7B, Llama3-8B) and three retrievers (Contriever, MedCPT, BGE-Large) to assess performance across different retrieval strategies. Results: The results from the Random mode indicate that providing more examples in the prompt improves the model's generation performance. Meanwhile, Top mode and Diversity mode significantly outperform Random mode on the RE (DDI) task, achieving an F1 score of 0.9669, a 26.4% improvement. Among the three retrievers tested, Contriever outperformed the other two in a greater number of experiments. Additionally, Llama 2 and Llama 3 demonstrated varying capabilities across different tasks, with Llama 3 showing a clear advantage in handling NER tasks. Conclusion: MMRAG effectively enhances biomedical in-context learning by refining example selection, mitigating data scarcity issues, and demonstrating superior adaptability for NLP-driven healthcare applications.

cs.CL↗

arXiv2025

DeepSeek-V3 Technical Report

We present DeepSeek-V3, a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token. To achieve efficient inference and cost-effective training, DeepSeek-V3 adopts Multi-head Latent Attention (MLA) and DeepSeekMoE architectures, which were thoroughly validated in DeepSeek-V2. Furthermore, DeepSeek-V3 pioneers an auxiliary-loss-free strategy for load balancing and sets a multi-token prediction training objective for stronger performance. We pre-train DeepSeek-V3 on 14.8 trillion diverse and high-quality tokens, followed by Supervised Fine-Tuning and Reinforcement Learning stages to fully harness its capabilities. Comprehensive evaluations reveal that DeepSeek-V3 outperforms other open-source models and achieves performance comparable to leading closed-source models. Despite its excellent performance, DeepSeek-V3 requires only 2.788M H800 GPU hours for its full training. In addition, its training process is remarkably stable. Throughout the entire training process, we did not experience any irrecoverable loss spikes or perform any rollbacks. The model checkpoints are available at https://github.com/deepseek-ai/DeepSeek-V3.

cs.CL↗

arXiv2025

Open-Set Cross-Network Node Classification via Unknown-Excluded Adversarial Graph Domain Alignment

Existing cross-network node classification methods are mainly proposed for closed-set setting, where the source network and the target network share exactly the same label space. Such a setting is restricted in real-world applications, since the target network might contain additional classes that are not present in the source. In this work, we study a more realistic open-set cross-network node classification (O-CNNC) problem, where the target network contains all the known classes in the source and further contains several target-private classes unseen in the source. Borrowing the concept from open-set domain adaptation, all target-private classes are defined as an additional unknown class. To address the challenging O-CNNC problem, we propose an unknown-excluded adversarial graph domain alignment (UAGA) model with a separate-adapt training strategy. Firstly, UAGA roughly separates known classes from unknown class, by training a graph neural network encoder and a neighborhood-aggregation node classifier in an adversarial framework. Then, unknown-excluded adversarial domain alignment is customized to align only target nodes from known classes with the source, while pushing target nodes from unknown class far away from the source, by assigning positive and negative domain adaptation coefficient to known class nodes and unknown class nodes. Extensive experiments on real-world datasets demonstrate significant outperformance of the proposed UAGA over state-of-the-art methods on O-CNNC.

cs.SI↗

arXiv2025

Electric field control of nonlinear Hall effect in Weyl semimetal TaIrTe4

The nonlinear Hall effect (NLHE), as an important probe to reveal the symmetry breaking in topological properties of materials, opens up a new dimension for exploring the energy band structure and electron transport mechanism of quantum materials. Current studies mainly focus on the observation of material intrinsic the NLHE or inducing the NLHE response by artificially constructing corrugated/twisted twodimensionalmaterial systems. Notably, the modulation of NLHE signal strength, a core parameter of device performance, has attracted much attention, while theoretical predictions suggest that an applied electric field can achieve the NLHE enhancement through modulation of the Berry curvature dipole (BCD). Here we report effective modulation the magnitude and sign of the NLHE by applying additional constant electric fields of different directions and magnitudes in the semimetal TaIrTe4. The NLHE response strength is enhanced by 168 times compared to the intrinsic one at 4 K when the additional constant electric field of -0.5 kV/cm is applied to the b-axis of TaIrTe4 and the through a.c. current is parallel to the TaIrTe4 a-axis. Scaling law analysis suggests that the enhancement may be the result of the combined effect of the electric field on the intrinsic BCD and disorder scattering effect of TaIrTe4. This work provides a means to study the properties of TaIrTe4, as well as a valuable reference for the study of novel electronic devices.

cond-mat.mtrl-sci↗

arXiv2025

Continually Evolved Multimodal Foundation Models for Cancer Prognosis

Cancer prognosis is a critical task that involves predicting patient outcomes and survival rates. To enhance prediction accuracy, previous studies have integrated diverse data modalities, such as clinical notes, medical images, and genomic data, leveraging their complementary information. However, existing approaches face two major limitations. First, they struggle to incorporate newly arrived data with varying distributions into training, such as patient records from different hospitals, thus rendering sub-optimal generalizability and limited utility in real-world applications. Second, most multimodal integration methods rely on simplistic concatenation or task-specific pipelines, which fail to capture the complex interdependencies across modalities. To address these, we propose a continually evolving multi-modal foundation model. Extensive experiments on the TCGA dataset demonstrate the effectiveness of our approach, highlighting its potential to advance cancer prognosis by enabling robust and adaptive multimodal integration.

cs.LG↗

arXiv2025

Field-free perpendicular magnetization switching of low critical current density at room temperature in TaIrTe4/ferromagnet heterostructures

Spin-orbit torque-induced perpendicular magnetization switching has attracted much attention due to the advantages of nonvolatility, high density, infinite read/write counts, and low power consumption in spintronic applications. To achieve field-free deterministic switching of perpendicular magnetization, additional magnetic field, magnetic layer assistance, or artificially designed structural symmetry breaking are usually required, which are not conducive to the high-density integration and application of low-power devices. However, 2D Weyl semimetals with low-symmetry structures have recently been found to generate z-spin-polarized currents, which may induce out-of-plane damping-like torques to their neighboring ferromagnetic layers, and realize deterministic perpendicular magnetization switching at zero magnetic field. In this Letter, we report that current-induced field-free magnetization switching at room temperature can be achieved in a perpendicularly magnetized TaIrTe4/Pt/Co/Pt device, and the critical switching current density can be lowered to be about 2.64*105 Acm-2. This study suggests that TaIrTe4 has great potential for the design of room-temperature efficient spintronic devices.

cond-mat.mtrl-sci↗

arXiv2024

RAMIE: Retrieval-Augmented Multi-task Information Extraction with Large Language Models on Dietary Supplements

\textbf{Objective:} We aimed to develop an advanced multi-task large language model (LLM) framework to extract multiple types of information about dietary supplements (DS) from clinical records. \textbf{Methods:} We used four core DS information extraction tasks - namely, named entity recognition (NER: 2,949 clinical sentences), relation extraction (RE: 4,892 sentences), triple extraction (TE: 2,949 sentences), and usage classification (UC: 2,460 sentences) as our multitasks. We introduced a novel Retrieval-Augmented Multi-task Information Extraction (RAMIE) Framework, including: 1) employed instruction fine-tuning techniques with task-specific prompts, 2) trained LLMs for multiple tasks with improved storage efficiency and lower training costs, and 3) incorporated retrieval augmentation generation (RAG) techniques by retrieving similar examples from the training set. We compared RAMIE's performance to LLMs with instruction fine-tuning alone and conducted an ablation study to assess the contributions of multi-task learning and RAG to improved multitasking performance. \textbf{Results:} With the aid of the RAMIE framework, Llama2-13B achieved an F1 score of 87.39 (3.51\% improvement) on the NER task and demonstrated outstanding performance on the RE task with an F1 score of 93.74 (1.15\% improvement). For the TE task, Llama2-7B scored 79.45 (14.26\% improvement), and MedAlpaca-7B achieved the highest F1 score of 93.45 (0.94\% improvement) on the UC task. The ablation study revealed that while MTL increased efficiency with a slight trade-off in performance, RAG significantly boosted overall accuracy. \textbf{Conclusion:} This study presents a novel RAMIE framework that demonstrates substantial improvements in multi-task information extraction for DS-related data from clinical records. Our framework can potentially be applied to other domains.

cs.CL↗

arXiv2024

Interpretable Differential Diagnosis with Dual-Inference Large Language Models

Automatic differential diagnosis (DDx) is an essential medical task that generates a list of potential diseases as differentials based on patient symptom descriptions. In practice, interpreting these differential diagnoses yields significant value but remains under-explored. Given the powerful capabilities of large language models (LLMs), we investigated using LLMs for interpretable DDx. Specifically, we curated the first DDx dataset with expert-derived interpretation on 570 clinical notes. Besides, we proposed Dual-Inf, a novel framework that enabled LLMs to conduct bidirectional inference (i.e., from symptoms to diagnoses and vice versa) for DDx interpretation. Both human and automated evaluation validated its efficacy in predicting and elucidating differentials across four base LLMs. In addition, Dual-Inf could reduce interpretation errors and hold promise for rare disease explanations. To the best of our knowledge, it is the first work that customizes LLMs for DDx explanation and comprehensively evaluates their interpretation performance. Overall, our study bridges a critical gap in DDx interpretation and enhances clinical decision-making.

cs.CL↗

arXiv2024

Deep Q-Exponential Processes

Motivated by deep neural networks, the deep Gaussian process (DGP) generalizes the standard GP by stacking multiple layers of GPs. Despite the enhanced expressiveness, GP, as an $L_2$ regularization prior, tends to be over-smooth and sub-optimal for inhomogeneous subjects, such as images with edges. Recently, Q-exponential process (Q-EP) has been proposed as an $L_q$ relaxation to GP and demonstrated with more desirable regularization properties through a parameter $q>0$ with $q=2$ corresponding to GP. Sharing the similar tractability of posterior and predictive distributions with GP, Q-EP can also be stacked to improve its modeling flexibility. In this paper, we generalize Q-EP to deep Q-EP to enjoy both proper regularization and improved expressiveness. The generalization is realized by introducing shallow Q-EP as a latent variable model and then building a hierarchy of the shallow Q-EP layers. Sparse approximation by inducing points and scalable variational strategy are applied to facilitate the inference. We demonstrate the numerical advantages of the proposed deep Q-EP model by comparing with multiple state-of-the-art deep probabilistic models.

stat.ML↗

arXiv2024

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

We present DeepSeek-V2, a strong Mixture-of-Experts (MoE) language model characterized by economical training and efficient inference. It comprises 236B total parameters, of which 21B are activated for each token, and supports a context length of 128K tokens. DeepSeek-V2 adopts innovative architectures including Multi-head Latent Attention (MLA) and DeepSeekMoE. MLA guarantees efficient inference through significantly compressing the Key-Value (KV) cache into a latent vector, while DeepSeekMoE enables training strong models at an economical cost through sparse computation. Compared with DeepSeek 67B, DeepSeek-V2 achieves significantly stronger performance, and meanwhile saves 42.5% of training costs, reduces the KV cache by 93.3%, and boosts the maximum generation throughput to 5.76 times. We pretrain DeepSeek-V2 on a high-quality and multi-source corpus consisting of 8.1T tokens, and further perform Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) to fully unlock its potential. Evaluation results show that, even with only 21B activated parameters, DeepSeek-V2 and its chat versions still achieve top-tier performance among open-source models.

cs.CL↗

arXiv2024

Denoising-Aware Contrastive Learning for Noisy Time Series

Time series self-supervised learning (SSL) aims to exploit unlabeled data for pre-training to mitigate the reliance on labels. Despite the great success in recent years, there is limited discussion on the potential noise in the time series, which can severely impair the performance of existing SSL methods. To mitigate the noise, the de facto strategy is to apply conventional denoising methods before model training. However, this pre-processing approach may not fully eliminate the effect of noise in SSL for two reasons: (i) the diverse types of noise in time series make it difficult to automatically determine suitable denoising methods; (ii) noise can be amplified after mapping raw data into latent space. In this paper, we propose denoising-aware contrastive learning (DECL), which uses contrastive learning objectives to mitigate the noise in the representation and automatically selects suitable denoising methods for every sample. Extensive experiments on various datasets verify the effectiveness of our method. The code is open-sourced.

cs.LG↗

arXiv2024

Fast Diffeomorphic Image Registration using Patch based Fully Convolutional Networks

Diffeomorphic image registration is a fundamental step in medical image analysis, owing to its capability to ensure the invertibility of transformations and preservation of topology. Currently, unsupervised learning-based registration techniques primarily extract features at the image level, potentially limiting their efficacy. This paper proposes a novel unsupervised learning-based fully convolutional network (FCN) framework for fast diffeomorphic image registration, emphasizing feature acquisition at the image patch level. Furthermore, a novel differential operator is introduced and integrated into the FCN architecture for parameter learning. Experiments are conducted on three distinct T1-weighted magnetic resonance imaging (T1w MRI) datasets. Comparative analyses with three state-of-the-art diffeomorphic image registration approaches including a typical conventional registration algorithm and two representative unsupervised learning-based methods, reveal that the proposed method exhibits superior performance in both registration accuracy and topology preservation.

cs.CV↗

arXiv2024

The separate effect of halo mass and stellar mass on the evolution of massive disk galaxies

We analyse a sample of massive disk galaxies selected from the SDSS-IV/MaNGA survey to investigate how the evolution of these galaxies depends on their stellar and halo masses. We applied a semi-analytic spectral fitting approach to the data from different regions in the galaxies to derive several of their key physical properties. From the best-fit model results, together with direct observables such as morphology, colour, and the Mgb/$\langle$Fe$\rangle$ index ratio measured within $1 R_{\rm e}$, we find that for central galaxies both their stellar and halo masses have a significant influence in their evolution. For a given halo mass, galaxies with higher stellar mass accumulate their stellar mass and become chemically enriched earlier than those with smaller stellar mass. Furthermore, at a given stellar mass, galaxies living in more massive halos have longer star-formation timescales and are delayed in becoming chemically enriched. In contrast, the evolution of massive satellite galaxies is mostly determined by their stellar mass. The results indicate that both the assembled halo mass and the halo assembly history impact the evolution of central galaxies. Our spatially resolved analysis indicates that only the galaxy properties in the central region ($0.0$--$0.5 R_{\rm e}$) show the dependencies described above. This fact supports a halo-driven formation scenario since the galaxies' central regions are more likely to contain old stars formed along with the halo itself, keeping a memory of the halo formation process.

astro-ph.GA↗

arXiv2024

SC-VAE: Sparse Coding-based Variational Autoencoder with Learned ISTA

Learning rich data representations from unlabeled data is a key challenge towards applying deep learning algorithms in downstream tasks. Several variants of variational autoencoders (VAEs) have been proposed to learn compact data representations by encoding high-dimensional data in a lower dimensional space. Two main classes of VAEs methods may be distinguished depending on the characteristics of the meta-priors that are enforced in the representation learning step. The first class of methods derives a continuous encoding by assuming a static prior distribution in the latent space. The second class of methods learns instead a discrete latent representation using vector quantization (VQ) along with a codebook. However, both classes of methods suffer from certain challenges, which may lead to suboptimal image reconstruction results. The first class suffers from posterior collapse, whereas the second class suffers from codebook collapse. To address these challenges, we introduce a new VAE variant, termed sparse coding-based VAE with learned ISTA (SC-VAE), which integrates sparse coding within variational autoencoder framework. The proposed method learns sparse data representations that consist of a linear combination of a small number of predetermined orthogonal atoms. The sparse coding problem is solved using a learnable version of the iterative shrinkage thresholding algorithm (ISTA). Experiments on two image datasets demonstrate that our model achieves improved image reconstruction results compared to state-of-the-art methods. Moreover, we demonstrate that the use of learned sparse code vectors allows us to perform downstream tasks like image generation and unsupervised image segmentation through clustering image patches.

cs.CV↗

arXiv2023

Where have all the low-metallicity galaxies gone? Tracing evolution in the mass--metallicity plane since a redshift of 0.7

Even over relatively recent epochs, galaxies have evolved significantly in their location in the mass-metallicity plane, which must be telling us something about the latter stages of galaxy evolution. In this paper, we analyse data from the LEGA-C survey using semi-analytic spectral and photometric fitting to determine these galaxies' evolution up to their observed epoch at $z \sim 0.7$. We confirm that, at $z \sim 0.7$, many objects already lie on the present-day mass-metallicity relation, but with a significant tail of high-mass low-metallicity galaxies that is not seen in the nearby Universe. Similar modelling of the evolution of galaxies in the nearby MaNGA survey allows us to reconstruct their properties at $z \sim 0.7$. Once selection criteria similar to those of LEGA-C are applied, we reassuringly find that the MaNGA galaxies populate the mass-metallicity plane in the same way at $z \sim 0.7$. Matching the LEGA-C sample to their mass-metallicity "twins" in MaNGA at this redshift, we can explore the likely subsequent evolution of individual LEGA-C galaxies. Galaxies already on the present-day mass--metallicity relation form few more stars and their disks fade, so they become smaller and more bulge-like. By contrast, the high-mass low-metallicity galaxies grow their disks through late star formation, and evolve rapidly to higher metallicities due to a cut-off in their wind-driven mass loss. There are significant indications that this late cut-off is associated with the belated end of strong AGN activity in these objects.

astro-ph.GA↗

arXiv2023

Mapping dust attenuation and the 2175 Å bump at kpc scales in nearby galaxies

We develop a novel approach to measure dust attenuation properties of galaxies, including the dust opacity, shape of the attenuation curve and the strength of the 2175Å absorption feature. From an observed spectrum, the method uses a model-independent approach to derive a relative attenuation curve, with absolute amplitude calibrated using NIR photometry. The dust-corrected spectrum is fitted with stellar population models to derive the dust-free model spectrum, which is compared with the observed SED/spectrum from NUV to NIR to determine dust attenuation properties. We apply this method to investigate dust attenuation on kpc scales, using a sample of 134 galaxies with integral field spectroscopy from MaNGA, NIR imaging from 2MASS, and NUV imaging from Swift/UVOT. We find the attenuation curve slope and the 2175Å bump in both optical and NUV span a wide range at kpc scales. The slope is shallower at higher optical opacity, regardless of the specific star formation rate (sSFR), minor-to-major axis ratio (b/a) of galaxies and the location of spaxels within individual galaxies. The 2175Å bump presents a strong negative correlation with the sSFR, while the correlations with the optical opacity, b/a and the location within individual galaxies are all weak. All these trends appear to be independent of the stellar mass of galaxies. Our results support the scenario that the variation of the 2175Å bump is driven predominantly by processes related to star formation, such as the destruction of small dust grains by UV radiation in star-forming regions.

astro-ph.GA↗

arXiv2023

The IRX-Beta Relation in kpc-sized Star Forming Regions in Nearby Galaxies

The effect of dust attenuation on a galaxy's light depends on a number of physical properties, such as geometry and dust composition, both of which can vary across the faces of galaxies. To investigate this variation, we continue analysis on star-forming regions in 29 galaxies studied previously. We analyse these regions using Swift/UVOT and WISE images, as well as SDSS/MaNGA emission line maps to constrain the relationship between the infrared excess (IRX) and the UV spectral index (beta) for each star forming region. This relationship can be used to constrain which dust attenuation law is appropriate for the region. We find that the value of Dn(4000) for a region is correlated with both IRX and beta, and that the gas-phase metallicity is strongly correlated with the IRX. This correlation between metallicity and IRX suggests that regardless of aperture, metal rich regions have steeper attenuation curves. We also find that integrated galactic light follows nearly the same IRX-beta relationship as that found for kiloparsec-sized star forming regions. This similarity may suggest that the attenuation law followed by the galaxy is essentially the same as that followed by the regions, although the relatively large size of our star forming regions complicates this interpretation because optical opacity and attenuation curves have been observed to vary within individual galaxies.

astro-ph.GA↗