arXiv subjects

Ruizhe Wang

Publications and source records attributed to Ruizhe Wang.

At least 19 recordsLinked to original sources

arXiv2026

DMT-Dens: Density-preserving manifold visualization for biological data

Motivation: Low-dimensional embeddings are widely used to explore cell-state heterogeneity in single-cell and other high-dimensional biological data. Although many methods preserve local neighborhoods, they may distort the apparent sampling density of processed observations, altering the visual contrast between dense and sparse regions and complicating the interpretation of rare, transitional, or continuous cell-state populations. Results: We present DMT-Dens, a parametric manifold-visualization method built on a latent-token Transformer encoder. The model integrates rank-based manifold alignment with hard-pair aggregation. To preserve density, it optimizes a loss based on the Pearson correlation between k-nearest-neighbor log-radius estimates in the processed input and two-dimensional embedding spaces. Benchmark evaluations demonstrate strong density preservation, particularly on biological datasets, while retaining competitive label separability. Availability: Source code, data-processing scripts, and resolved experiment configurations are available at https://github.com/Ruizhe-wang/DMT-Dens.

q-bio.QM↗

arXiv2026

Finding Vulnerabilities via LLM-Augmented Semantics-Aware Type-Checking

Vulnerability detection via static analysis traditionally relies on security experts encoding insecure coding patterns into algorithmic rules. However, this approach often focuses on syntactic patterns and overlooks deeper semantic information in the code, such as the meanings of variable and function names. As software systems grow more complex, modeling vulnerabilities using only syntactic rules becomes increasingly challenging. In this paper, we propose a semantics-aware approach to detecting software vulnerabilities. We present SETYPE, a semantics-aware type system that can be derived directly from source code based solely on the meanings of symbols and expressions in natural language. In the SETYPE type system, both type inference and checking are performed by Large Language Models (LLMs), and a failed type check indicates a potential vulnerability. We prototype PYSETYPE to demonstrate the feasibility of SETYPE for detecting vulnerabilities in Python web applications. Our evaluation on real-world applications achieves 87% detection precision and 88% detection accuracy. Using PYSETYPE, we identified 15 potential zero-day vulnerabilities, nine of which were confirmed by developers.

cs.CR↗

arXiv2026

AFDM-ISAC With Fractional Delay-Doppler Coupling

Affine frequency division multiplexing (AFDM) is a promising chirp-based multicarrier waveform for high-mobility integrated sensing and communication (ISAC). Accurate angle, delay, and Doppler estimation is essential for AFDM sensing. Since target delays and Doppler shifts are generally continuous-valued, representing them on a discrete delay--Doppler grid causes energy leakage and peak displacement in the discrete affine Fourier transform (DAFT) domain. The AFDM chirp also induces delay--Doppler coupling in the DAFT-domain response. The resulting DAFT-domain matching-score surface exhibits a local ridge that is not aligned with the normalized-delay and normalized-Doppler axes. To address these issues, this paper investigates joint estimation of angle and continuous-valued delay--Doppler parameters for a colocated AFDM-ISAC sensing architecture. A transform-domain sparse sensing model is formulated from the fractional DAFT-domain response. Based on this model, a coupled-coordinate Newtonized orthogonal matching pursuit (CC-NOMP) estimator is developed. CC-NOMP uses the AFDM-induced coupling coordinate to parameterize the dominant local ridge. It combines coupled-coordinate Newton refinement with safeguarded updates, coupling-aligned delay refinement, and cyclic multi-target refinement to estimate angle, continuous normalized delay, and normalized Doppler. A deterministic Cram\'er--Rao bound and a dominant-order complexity analysis are also derived. Simulation results with continuous-valued off-grid target parameters show that CC-NOMP achieves lower delay and Doppler error floors than the considered baselines while maintaining comparable angle-estimation accuracy.

cs.IT↗

arXiv2026

Diagnosing Urban Street Vitality via a Visual-Semantic and Spatiotemporal Framework for Street-Level Economics

Micro-scale street-level economic assessment is fundamental for precision spatial resource allocation. While Street View Imagery (SVI) advances urban sensing, existing approaches remain semantically superficial and overlook brand hierarchy heterogeneity and structural recession. To address this, we propose a visual-semantic and field-based spatiotemporal framework, operationalized via the Street Economic Vitality Index (SEVI). Our approach integrates physical and semantic streetscape parsing through instance segmentation of signboards, glass interfaces, and storefront closures. A dual-stage VLM-LLM pipeline standardizes signage into global hierarchies to quantify a spatially smoothed brand premium index. To overcome static SVI limitations, we introduce a temporal lag design using Location-Based Services (LBS) data to capture realized demand. Combined with a category-weighted Gaussian spillover model, we construct a three-dimensional diagnostic system covering Commercial Activity, Spatial Utilization, and Physical Environment. Experiments based on time-lagged geographically weighted regression across eight tidal periods in Nanjing reveal quasi-causal spatiotemporal heterogeneity. Street vibrancy arises from interactions between hierarchical brand clustering and mall-induced externalities. High-quality interfaces show peak attraction during midday and evening, while structural recession produces a lagged nighttime repulsion effect. The framework offers evidence-based support for precision spatial governance.

cs.CY↗

arXiv2026

Tensor Train Decomposition-based Channel Estimation for MIMO-AFDM Systems with Fractional Delay and Doppler

Affine Frequency Division Multiplexing (AFDM) has emerged as a promising chirp-based multicarrier technology for high-speed communication systems. To fully exploit the diversity gain offered by AFDM, accurate channel estimation is essential. However, existing studies have mainly focused on the integer-delay-tap scenario and single-symbol pilot-based estimation. Since delay taps in practice are generally fractional, approximating them as integers not only degrades delay estimation accuracy but also severely affects Doppler frequency estimation. To address this problem, in this paper, we investigate channel estimation for multiple-input multiple-output (MIMO)-AFDM systems. A time-affine frequency (T-AF) domain pilot structure is proposed to exploit time-domain phase variations. By leveraging the rotational invariance property in the spatial and temporal domains, a channel estimation algorithm based on Vandermonde-structured tensor-train (TT) decomposition is developed. The proposed algorithm demonstrates superior computational efficiency compared with state-of-the-art parameter estimation methods. Moreover, diverging from current studies, we derive the global Ziv-Zakai bound (ZZB) as an alternative parameter estimation error lower bound to the Cram\'{e}r-Rao bound (CRB). Numerical results show that the derived ZZB provides tighter global performance characterization and successfully captures the threshold phenomenon in mean square error (MSE) performance in the low-SNR regime. Furthermore, the proposed algorithm achieves superior communication performance relative to the existing schemes, while offering a computational speedup, reducing the execution time by an order of magnitude compared to the state-of-the-art iterative algorithms.

cs.IT↗

arXiv2025

Sigma-MoE-Tiny Technical Report

Mixture-of-Experts (MoE) has emerged as a promising paradigm for foundation models due to its efficient and powerful scalability. In this work, we present Sigma-MoE-Tiny, an MoE language model that achieves the highest sparsity compared to existing open-source models. Sigma-MoE-Tiny employs fine-grained expert segmentation with up to 96 experts per layer, while activating only one expert for each token, resulting in 20B total parameters with just 0.5B activated. The major challenge introduced by such extreme sparsity lies in expert load balancing. We find that the widely-used load balancing loss tends to become ineffective in the lower layers under this setting. To address this issue, we propose a progressive sparsification schedule aiming to balance expert utilization and training stability. Sigma-MoE-Tiny is pre-trained on a diverse and high-quality corpus, followed by post-training to further unlock its capabilities. The entire training process remains remarkably stable, with no occurrence of irrecoverable loss spikes. Comprehensive evaluations reveal that, despite activating only 0.5B parameters, Sigma-MoE-Tiny achieves top-tier performance among counterparts of comparable or significantly larger scale. In addition, we provide an in-depth discussion of load balancing in highly sparse MoE models, offering insights for advancing sparsity in future MoE architectures. Project page: https://qghuxmu.github.io/Sigma-MoE-Tiny Code: https://github.com/microsoft/ltp-megatron-lm

cs.CL↗

arXiv2025

SIGMA: An AI-Empowered Training Stack on Early-Life Hardware

An increasing variety of AI accelerators is being considered for large-scale training. However, enabling large-scale training on early-life AI accelerators faces three core challenges: frequent system disruptions and undefined failure modes that undermine reliability; numerical errors and training instabilities that threaten correctness and convergence; and the complexity of parallelism optimization combined with unpredictable local noise that degrades efficiency. To address these challenges, SIGMA is an open-source training stack designed to improve the reliability, stability, and efficiency of large-scale distributed training on early-life AI hardware. The core of this initiative is the LUCIA TRAINING PLATFORM (LTP), the system optimized for clusters with early-life AI accelerators. Since its launch in March 2025, LTP has significantly enhanced training reliability and operational productivity. Over the past five months, it has achieved an impressive 94.45% effective cluster accelerator utilization, while also substantially reducing node recycling and job-recovery times. Building on the foundation of LTP, the LUCIA TRAINING FRAMEWORK (LTF) successfully trained SIGMA-MOE, a 200B MoE model, using 2,048 AI accelerators. This effort delivered remarkable stability and efficiency outcomes, achieving 21.08% MFU, state-of-the-art downstream accuracy, and encountering only one stability incident over a 75-day period. Together, these advances establish SIGMA, which not only tackles the critical challenges of large-scale training but also establishes a new benchmark for AI infrastructure and platform innovation, offering a robust, cost-effective alternative to prevailing established accelerator stacks and significantly advancing AI capabilities and scalability. The source code of SIGMA is available at https://github.com/microsoft/LuciaTrainingPlatform.

cs.DC↗

arXiv2025

A Cyclic Shift Embedded Pilot based Channel Estimation for Multi-User MIMO-OTFS systems with fractional delay and Doppler

Orthogonal time frequency space (OTFS) modulation has been proposed to meet the demand for reliable communication in high-mobility scenarios for future wireless networks. However, in multi-user OTFS systems, conventional embedded pilot schemes require independent pilot allocation for each user, leading to linearly increasing pilot overhead. To address these issues, in this paper, we investigate the uplink channel estimation and pilot design for multi-user multiple-input multiple-output (MIMO)-OTFS systems. We propose a multi-dimensional decomposition-based channel estimation algorithm. Specifically, the proposed algorithm first estimates the angles of arrivals (AoAs) via subspace decomposition-based method. A spatial projection matrix, constructed from the estimated AOAs, decouples the received signal by propagation path subspace, effectively mitigating inter-path interference. The remaining fractional delay and Doppler can be obtained by a compressed sensing (CS)-based off-grid channel estimation method. Furthermore, to reduce the pilot overhead in multi-user OTFS systems, this paper proposes a novel cyclic shift embedded pilot (CSEP) structure, which can reuse users through cyclic shift-orthogonality of Zadoff-Chu (ZC) sequences. Compared with conventional embedded pilot structures, the CSEP structure can save over 30\% of pilot overhead. Finally, an imporved channel estimation method based on the CSEP structure is proposed. Simulation results demonstrate that it achieves superior performance in channel estimation. Moreover, the proposed CSEP structure and channel estimation algorithm achieve a favorable balance between computational complexity, estimation accuracy, and bit error rate (BER) performance.

cs.IT↗

arXiv2025

Beyond Sunk Costs: Boosting LLM Pre-training Efficiency via Orthogonal Growth of Mixture-of-Experts

As the computational demands for pre-training Large Language Models (LLMs) continue to surge, the need for efficient training paradigms becomes critical. Despite the vast resources already invested in existing pre-trained checkpoints, these assets often remain under-leveraged due to architectural limitations. We introduce an "orthogonal growth" strategy designed to "recycle" these checkpoints by strategically expanding their parameters prior to continued training. Our method focuses on optimizing converged Mixture-of-Experts (MoE) models through two dimensions: interpositional layer copying for increased depth and noisy expert duplication for expanded width. Through extensive scaling laws analysis, we demonstrate a strong positive correlation between the "sunk cost" (prior investment) and the final model accuracy. Empirical results on models up to 70B parameters and 1T tokens show that our recycling approach yields a 10.6% accuracy improvement compared to training from scratch under identical extra compute budgets. This work provides a cost-effective blueprint for sustainable large-scale LLM development.

cs.LG↗

arXiv2025

Training Matryoshka Mixture-of-Experts for Elastic Inference-Time Expert Utilization

Mixture-of-Experts (MoE) has emerged as a promising paradigm for efficiently scaling large language models without a proportional increase in computational cost. However, the standard training strategy of Top-K router prevents MoE models from realizing their full potential for elastic inference. When the number of activated experts is altered at inference time, these models exhibit precipitous performance degradation. In this work, we introduce Matryoshka MoE (M-MoE), a training framework that instills a coarse-to-fine structure directly into the expert ensemble. By systematically varying the number of activated experts during training, M-MoE compels the model to learn a meaningful ranking: top-ranked experts collaborate to provide essential, coarse-grained capabilities, while subsequent experts add progressively finer-grained detail. We explore this principle at multiple granularities, identifying a layer-wise randomization strategy as the most effective. Our experiments demonstrate that a single M-MoE model achieves remarkable elasticity, with its performance at various expert counts closely matching that of an entire suite of specialist models, but at only a fraction of the total training cost. This flexibility not only unlocks elastic inference but also enables optimizing performance by allocating different computational budgets to different model layers. Our work paves the way for more practical and adaptable deployments of large-scale MoE models.

cs.CL↗

arXiv2025

Controllable Generation of Large-Scale 3D Urban Layouts with Semantic and Structural Guidance

Urban modeling is essential for city planning, scene synthesis, and gaming. Existing image-based methods generate diverse layouts but often lack geometric continuity and scalability, while graph-based methods capture structural relations yet overlook parcel semantics. We present a controllable framework for large-scale 3D vector urban layout generation, conditioned on both geometry and semantics. By fusing geometric and semantic attributes, introducing edge weights, and embedding building height in the graph, our method extends 2D layouts to realistic 3D structures. It also enables users to directly control the output by modifying semantic attributes. Experiments show that it produces valid, large-scale urban models, offering an effective tool for data-driven planning and design.

cs.CV↗

arXiv2025

Open Source High-Resolution Exoplanet Atmosphere Retrievals with POSEIDON

High-resolution spectroscopy (R > 25,000) has opened new opportunities to characterize exoplanet atmospheres from the ground. By resolving individual lines in planetary emission and transmission spectra, one can sensitively probe the chemical inventory and temperature structure of exoplanets. However, a significant challenge to reliable and reproducible atmospheric inferences from high-resolution datasets has been the lack of open source codes for high-resolution retrievals. Here, we present a unified high-resolution retrieval framework, for both emission and transmission spectroscopy, made publicly available within the open source POSEIDON retrieval code. Our high-resolution retrieval framework is fast (typically < 12 hours), accessible (no GPUs required), and well-documented via Python notebooks. We validate our framework by reproducing previous emission retrievals of the hot Jupiter WASP-77Ab and transmission retrievals of the ultra-hot Jupiter WASP-121b. Our results are broadly consistent with those of published works when making the same data detrending assumptions, but we demonstrate that user choices can subtly propagate into retrieved chemical abundances.

astro-ph.EP↗

arXiv2025

Detecting Manipulated Contents Using Knowledge-Grounded Inference

The detection of manipulated content, a prevalent form of fake news, has been widely studied in recent years. While existing solutions have been proven effective in fact-checking and analyzing fake news based on historical events, the reliance on either intrinsic knowledge obtained during training or manually curated context hinders them from tackling zero-day manipulated content, which can only be recognized with real-time contextual information. In this work, we propose Manicod, a tool designed for detecting zero-day manipulated content. Manicod first sources contextual information about the input claim from mainstream search engines, and subsequently vectorizes the context for the large language model (LLM) through retrieval-augmented generation (RAG). The LLM-based inference can produce a "truthful" or "manipulated" decision and offer a textual explanation for the decision. To validate the effectiveness of Manicod, we also propose a dataset comprising 4270 pieces of manipulated fake news derived from 2500 recent real-world news headlines. Manicod achieves an overall F1 score of 0.856 on this dataset and outperforms existing methods by up to 1.9x in F1 score on their benchmarks on fact-checking and claim verification.

cs.CL↗

arXiv2025

GeoTexBuild: 3D Building Model Generation from Map Footprints

We introduce GeoTexBuild, a modular generative framework for creating 3D building models from footprints derived from site planning or map designs. The system is designed for architects and city planners, offering a seamless solution that directly converts map features into 3D buildings. The proposed framework employs a three-stage process comprising height map generation, geometry reconstruction, and appearance stylization, culminating in building models with detailed geometry and appearance attributes. By integrating customized ControlNet, Neural style field (NSF), and Multi-view diffusion model, we explore effective methods for controlling both geometric and visual attributes during the generation process. Our approach eliminates the problem of structural variations in a single facade image in existing 3D generation techniques for buildings. Experimental results at each stage validate the capability of GeoTexBuild to generate detailed and accurate building models from footprints.

cs.CV↗

arXiv2025

Optimizing Large Language Model Training Using FP4 Quantization

The growing computational demands of training large language models (LLMs) necessitate more efficient methods. Quantized training presents a promising solution by enabling low-bit arithmetic operations to reduce these costs. While FP8 precision has demonstrated feasibility, leveraging FP4 remains a challenge due to significant quantization errors and limited representational capacity. This work introduces the first FP4 training framework for LLMs, addressing these challenges with two key innovations: a differentiable quantization estimator for precise weight updates and an outlier clamping and compensation strategy to prevent activation collapse. To ensure stability, the framework integrates a mixed-precision training scheme and vector-wise quantization. Experimental results demonstrate that our FP4 framework achieves accuracy comparable to BF16 and FP8, with minimal degradation, scaling effectively to 13B-parameter LLMs trained on up to 100B tokens. With the emergence of next-generation hardware supporting FP4, our framework sets a foundation for efficient ultra-low precision training.

cs.LG↗

arXiv2024

Addressing the Mutual Interference in Uplink ISAC Receivers: A Projection Method

Dual function radar and communication (DFRC) is a promising research direction within integrated sensing and communication (ISAC), improving hardware and spectrum efficiency by merging sensing and communication (S&C) functionalities into a shared platform. However, the DFRC receiver (DFRC-R) is tasked with both uplink communication signal detection and simultaneously target-related parameter estimation from the echoes, leading to issues with mutual interference. In this paper, a projection-based scheme is proposed to equivalently transform the joint signal detection and target estimation problem into a joint signal detection process across multiple snapshots. Compared with conventional successive interference cancellation (SIC) schemes, our proposed approach achieves a higher signal-to-noise ratio (SNR), and a higher ergodic rate when the radar signal is non-negligible. Nonetheless, it introduces an ill-conditioned signal detection problem, which is addressed using a non-linear detector. By jointly processing an increased number of snapshots, the proposed scheme can achieve high S&C performance simultaneously.

cs.IT↗

arXiv2024

Channel Estimation for mmWave MIMO-OFDM Systems in High-Mobility Scenarios: Instantaneous Model or Statistical Model?

Classical linear statistical models, like the first-order auto-regressive (AR) model, are commonly used as channel model in high-mobility scenarios. However, compared to sub-6G, the effect of Doppler frequency shifts is more significant at millimeter wave (mmWave) frequencies, and the effectiveness of the statistical channel model in high-mobility mmWave scenarios should be reconsidered. In this paper, we investigate the channel estimation for mmWave multiple-input multiple-output-(MIMO) orthogonal frequency division multiplexing (OFDM) systems in high-mobility scenarios, with the focus on the comparison between the instantaneous channel model and the statistical channel model. For the instantaneous model, by leveraging the low-rank nature of mmWave channels and the multidimensional characteristics of MIMO-OFDM signals across space, time, and frequency, the received signals are structured as a fourth-order tensor fitting a low-rank CANDECOMP/PARAFAC (CP) model. Then, to solve the CP decomposition problem, an estimation of signal parameters via rotational invariance techniques (ESPRIT)-type decomposition based channel estimation method is proposed by exploring the Vandermonde structure of factor matrix, and the channel parameters are then estimated from the factor matrices. We analyze the uniqueness condition of the CP decomposition and develop a concise derivation of the Cramer-Rao bound (CRB) for channel parameters. Simulations show that our method outperforms the existing benchmarks. Furthermore, the results based on the wireless environment generated by Wireless InSite verify that the channel estimation based on the instantaneous channel model performs better than that based on the statistical channel model. Therefore, the instantaneous channel model is recommended for designing channel estimation algorithm for mmWave systems in high-mobility scenarios.

cs.IT↗

arXiv2024

Enhancement of 3D Gaussian Splatting using Raw Mesh for Photorealistic Recreation of Architectures

The photorealistic reconstruction and rendering of architectural scenes have extensive applications in industries such as film, games, and transportation. It also plays an important role in urban planning, architectural design, and the city's promotion, especially in protecting historical and cultural relics. The 3D Gaussian Splatting, due to better performance over NeRF, has become a mainstream technology in 3D reconstruction. Its only input is a set of images but it relies heavily on geometric parameters computed by the SfM process. At the same time, there is an existing abundance of raw 3D models, that could inform the structural perception of certain buildings but cannot be applied. In this paper, we propose a straightforward method to harness these raw 3D models to guide 3D Gaussians in capturing the basic shape of the building and improve the visual quality of textures and details when photos are captured non-systematically. This exploration opens up new possibilities for improving the effectiveness of 3D reconstruction techniques in the field of architectural design.

cs.CV↗