arXiv subjects

Bin Ma

Publications and source records attributed to Bin Ma.

At least 55 records · Page 3Linked to original sources

arXiv2025

Machine Learning-Guided Memory Optimization for DLRM Inference on Tiered Memory

Deep learning recommendation models (DLRMs) are widely used in industry, and their memory capacity requirements reach the terabyte scale. Tiered memory architectures provide a cost-effective solution but introduce challenges in embedding-vector placement due to complex embedding-access patterns. We propose RecMG, a machine learning (ML)-guided system for vector caching and prefetching on tiered memory. RecMG accurately predicts accesses to embedding vectors with long reuse distances or few reuses. The design of RecMG focuses on making ML feasible in the context of DLRM inference by addressing unique challenges in data labeling and navigating the search space for embedding-vector placement. By employing separate ML models for caching and prefetching, plus a novel differentiable loss function, RecMG narrows the prefetching search space and minimizes on-demand fetches. Compared to state-of-the-art temporal, spatial, and ML-based prefetchers, RecMG reduces on-demand fetches by 2.2x, 2.8x, and 1.5x, respectively. In industrial-scale DLRM inference scenarios, RecMG effectively reduces end-to-end DLRM inference time by up to 43%.

cs.PF↗

arXiv2025

Robust Identity Perceptual Watermark Against Deepfake Face Swapping

Notwithstanding offering convenience and entertainment to society, Deepfake face swapping has caused critical privacy issues with the rapid development of deep generative models. Due to imperceptible artifacts in high-quality synthetic images, passive detection models against face swapping in recent years usually suffer performance damping regarding the generalizability issue in cross-domain scenarios. Therefore, several studies have been attempted to proactively protect the original images against malicious manipulations by inserting invisible signals in advance. However, existing proactive defense approaches demonstrate unsatisfactory results with respect to visual quality, detection accuracy, and source tracing ability. In this study, to fulfill the research gap, we propose a robust identity perceptual watermarking framework that concurrently performs detection and source tracing against Deepfake face swapping proactively. We innovatively assign identity semantics regarding the image contents to the watermarks and devise an unpredictable and nonreversible chaotic encryption system to ensure watermark confidentiality. The watermarks are robustly encoded and recovered by jointly training an encoder-decoder framework along with adversarial image manipulations. For a suspect image, falsification is accomplished by justifying the consistency between the content-matched identity perceptual watermark and the recovered robust watermark, without requiring the ground-truth. Moreover, source tracing can be accomplished based on the identity semantics that the recovered watermark carries. Extensive experiments demonstrate state-of-the-art detection and source tracing performance against Deepfake face swapping with promising watermark robustness for both cross-dataset and cross-manipulation settings.

cs.CV↗

arXiv2025

Cosmic Ray Detection and Rejection for CSST

As a space telescope, the China Space Station Survey Telescope (CSST) will face significant challenges from cosmic ray (CR) contamination. These CRs will severely degrade image quality and further influence scientific analysis. Due to the CSST's sky survey strategy, traditional multi-frame stacking methods become invalid. The limited revisits prompted us to develop an effective single-image CR processing method for CSST. We retrained the DeepCR model based on CSST simulated images and achieved 97.90+-0.18% recall and 98.67+-0.05% precision on CR detection. Moreover, this paper puts forward an innovative morphology-sensitive inpainting method, which focuses more on areas with higher scientific value. We trained a UNet++ model especially on contaminated stellar/galactic areas, alongside adaptive median filtering for background regions. This method achieves effective for CRs with different intensities and different distances from centers of scientific targets. By this approach, the photometric errors of CR-corrected targets could be restricted to the level comparable to those of uncontaminated sources. Also, it increases the detection rate by 13.6% compared to CR masking. This method will provide a robust CR mitigation for next-generation space telescopes.

astro-ph.IM↗

arXiv2025

Optimizing Long-term Variability of AGN Light Curves. I. A Case Study with ZTF Observations in the EGS Field

Optical variability is a key observational probe for studying the accretion dynamics and central engine physics of Active Galactic Nuclei (AGNs). The quality and completeness of light curves have a direct impact on variability studies, particularly for faint AGNs and high-redshift AGNs. To improve the quality of long-term light curves for AGNs, we bin and stack multi-epoch images balancing the image depths and temporal resolution. As a case study, we apply this method to Zwicky Transient Facility (ZTF) observations in the Extended Groth Strip (EGS) field, where the overlapping region covers an area of about 370 arcmin$^2$ and includes $g$-band and $r$-band data taken from March 2018 to December 2024. The co-added images are approximately 2.0 to 2.5 magnitudes deeper than the ZTF single-epoch images. With co-added images, we construct light curves for 73 AGNs in the EGS field. Compared to the traditional ZTF light curves, our light curves maintain consistent long-term variability trends but with higher photometric precision. Furthermore, this method can help detect AGNs with weak variability which are missed from the traditional ZTF data due to the noisy light curves or below the detection limit in ZTF's single-epoch exposure. Among the 73 AGNs, the majority exhibit a bluer-when-brighter (BWB) trend on long-term timescales, which is consistent with previous studies. This work offers insights for optimizing AGN light curves in both current and upcoming all-sky time-domain surveys.

astro-ph.GA↗

arXiv2025

mLR: Scalable Laminography Reconstruction based on Memoization

ADMM-FFT is an iterative method with high reconstruction accuracy for laminography but suffers from excessive computation time and large memory consumption. We introduce mLR, which employs memoization to replace the time-consuming Fast Fourier Transform (FFT) operations based on an unique observation that similar FFT operations appear in iterations of ADMM-FFT. We introduce a series of techniques to make the application of memoization to ADMM-FFT performance-beneficial and scalable. We also introduce variable offloading to save CPU memory and scale ADMM-FFT across GPUs within and across nodes. Using mLR, we are able to scale ADMM-FFT on an input problem of 2Kx2Kx2K, which is the largest input problem laminography reconstruction has ever worked on with the ADMM-FFT solution on limited memory; mLR brings 52.8% performance improvement on average (up to 65.4%), compared to the original ADMM-FFT.

cs.DC↗

arXiv2025

Backdoor or Manipulation? Graph Mixture of Experts Can Defend Against Various Graph Adversarial Attacks

Extensive research has highlighted the vulnerability of graph neural networks (GNNs) to adversarial attacks, including manipulation, node injection, and the recently emerging threat of backdoor attacks. However, existing defenses typically focus on a single type of attack, lacking a unified approach to simultaneously defend against multiple threats. In this work, we leverage the flexibility of the Mixture of Experts (MoE) architecture to design a scalable and unified framework for defending against backdoor, edge manipulation, and node injection attacks. Specifically, we propose an MI-based logic diversity loss to encourage individual experts to focus on distinct neighborhood structures in their decision processes, thus ensuring a sufficient subset of experts remains unaffected under perturbations in local structures. Moreover, we introduce a robustness-aware router that identifies perturbation patterns and adaptively routes perturbed nodes to corresponding robust experts. Extensive experiments conducted under various adversarial settings demonstrate that our method consistently achieves superior robustness against multiple graph adversarial attacks.

cs.LG↗

arXiv2025

Antarctic Infrared Binocular Telescope. I. System Overview, Laboratory Testing, and On-Sky Performance Evaluation

Infrared time-domain surveys remain significantly underdeveloped compared with their optical counterparts. We have developed the Antarctic Infrared Binocular Telescope (AIRBT) to study the dynamic infrared sky at Dome A, Antarctica, taking advantage of the superb infrared observational conditions at this site. AIRBT consists of two identical 15 cm f/3 optical tube assemblies and two cost-effective indium gallium arsenide (InGaAs) cameras equipped with J and H filters, respectively. The cameras have 640 x 512 pixels with a size of 15 micrometers, providing a scale of 6.9 arcseconds per pixel and a field of view of 1.22 x 0.97 square degrees. We characterize the performance of the InGaAs cameras, including bias, readout noise, dark current, nonlinearity, and photon transfer curve. Our analysis highlights the distinct behaviors of InGaAs cameras compared with charge-coupled devices (CCDs). The bias and readout noise show temperature dependence, and the noise measured from the photon transfer curves has additional components that increase with exposure time. On-sky tests were conducted in October 2022 including system calibration, limiting depth, and photometric precision. For a single 3-second exposure, we achieved 5-sigma limiting magnitudes of 11.2 mag (Vega system) in J band and 9.7 mag in H band. The best photometric precision reached 20 millimagnitudes at the bright end, which could be further improved to sub-percent levels through image stacking. AIRBT was installed at Dome A in January 2023, and scientific observations began as soon as darkness set in.

astro-ph.IM↗

arXiv2025

Probing the cosmological 21~cm global signal from the Antarctic ice sheet

The redshifted 21 cm line, arising from neutral hydrogen, offers a unique probe into the intergalactic medium and the first stars and galaxies formed in the early universe. However, detecting this signal is a challenging task because of artificial radio-frequency interference (RFI) and systematic errors such as ground effects. The interior of the Antarctic continent provides an excellent location to make such observations, with minimal RFI and relatively stable foreground signals. Moreover, a flat plateau in central Antarctica, with an ice cap over 2000 m deep, will show less ground reflection of radio waves, reducing the signal complexity in the area around the probing antenna. It may be advantageous to perform cosmological 21 cm experiments in Antarctica, and a 21 cm Antarctic global spectrum experiment can potentially be deployed on the Antarctic ice cap. We have performed preliminary instrumental design, system calibration, and implementation of such an instrument optimized for extreme cold and capable of long-term autonomous operation. This system shows the ability to effectively detect the 21~cm signal, confirming Antarctica as an excellent observational site for radio cosmology.

astro-ph.IM↗

arXiv2025

SemanticShield: LLM-Powered Audits Expose Shilling Attacks in Recommender Systems

Recommender systems (RS) are widely used in e-commerce for personalized suggestions, yet their openness makes them susceptible to shilling attacks, where adversaries inject fake behaviors to manipulate recommendations. Most existing defenses emphasize user-side behaviors while overlooking item-side features such as titles and descriptions that can expose malicious intent. To address this gap, we propose a two-stage detection framework that integrates item-side semantics via large language models (LLMs). The first stage pre-screens suspicious users using low-cost behavioral criteria, and the second stage employs LLM-based auditing to evaluate semantic consistency. Furthermore, we enhance the auditing model through reinforcement fine-tuning on a lightweight LLM with carefully designed reward functions, yielding a specialized detector called SemanticShield. Experiments on six representative attack strategies demonstrate the effectiveness of SemanticShield against shilling attacks, and further evaluation on previously unseen attack methods shows its strong generalization capability. Code is available at https://github.com/FrankenstLee/SemanticShield.

cs.CL↗

arXiv2025

Insight Rumors: A Novel Textual Rumor Locating and Marking Model Leveraging Att_BiMamba2 Network

With the development of social media networks, rumor detection models have attracted more and more attention. Whereas, these models primarily focus on classifying contexts as rumors or not, lacking the capability to locate and mark specific rumor content. To address this limitation, this paper proposes a novel rumor detection model named Insight Rumors to locate and mark rumor content within textual data. Specifically, we propose the Bidirectional Mamba2 Network with Dot-Product Attention (Att_BiMamba2), a network that constructs a bidirectional Mamba2 model and applies dot-product attention to weight and combine the outputs from both directions, thereby enhancing the representation of high-dimensional rumor features. Simultaneously, a Rumor Locating and Marking module is designed to locate and mark rumors. The module constructs a skip-connection network to project high-dimensional rumor features onto low-dimensional label features. Moreover, Conditional Random Fields (CRF) is employed to impose strong constraints on the output label features, ensuring accurate rumor content location. Additionally, a labeled dataset for rumor locating and marking is constructed, with the effectiveness of the proposed model is evaluated through comprehensive experiments. Extensive experiments indicate that the proposed scheme not only detects rumors accurately but also locates and marks them in context precisely, outperforming state-of-the-art schemes that can only discriminate rumors roughly.

cs.SI↗

arXiv2025

CLMIR: A Textual Dataset for Rumor Identification and Marking

With the rise of social media, rumor detection has drawn increasing attention. Although numerous methods have been proposed with the development of rumor classification datasets, they focus on identifying whether a post is a rumor, lacking the ability to mark the specific rumor content. This limitation largely stems from the lack of fine-grained marks in existing datasets. Constructing a rumor dataset with rumor content information marking is of great importance for fine-grained rumor identification. Such a dataset can facilitate practical applications, including rumor tracing, content moderation, and emergency response. Beyond being utilized for overall performance evaluation, this dataset enables the training of rumor detection algorithms to learn content marking, and thus improves their interpretability and reasoning ability, enabling systems to effectively address specific rumor segments. This paper constructs a dataset for rumor detection with fine-grained markings, named CLMIR (Content-Level Marking Dataset for Identifying Rumors). In addition to determining whether a post is a rumor, this dataset further marks the specific content upon which the rumor is based.

cs.CY↗

arXiv2025

A Cross-Modal Rumor Detection Scheme via Contrastive Learning by Exploring Text and Image internal Correlations

Existing rumor detection methods often neglect the content within images as well as the inherent relationships between contexts and images across different visual scales, thereby resulting in the loss of critical information pertinent to rumor identification. To address these issues, this paper presents a novel cross-modal rumor detection scheme based on contrastive learning, namely the Multi-scale Image and Context Correlation exploration algorithm (MICC). Specifically, we design an SCLIP encoder to generate unified semantic embeddings for text and multi-scale image patches through contrastive pretraining, enabling their relevance to be measured via dot-product similarity. Building upon this, a Cross-Modal Multi-Scale Alignment module is introduced to identify image regions most relevant to the textual semantics, guided by mutual information maximization and the information bottleneck principle, through a Top-K selection strategy based on a cross-modal relevance matrix constructed between the text and multi-scale image patches. Moreover, a scale-aware fusion network is designed to integrate the highly correlated multi-scale image features with global text features by assigning adaptive weights to image regions based on their semantic importance and cross-modal relevance. The proposed methodology has been extensively evaluated on two real-world datasets. The experimental results demonstrate that it achieves a substantial performance improvement over existing state-of-the-art approaches in rumor detection, highlighting its effectiveness and potential for practical applications.

cs.CV↗

arXiv2025

FD-Bench: A Full-Duplex Benchmarking Pipeline Designed for Full Duplex Spoken Dialogue Systems

Full-duplex spoken dialogue systems (FDSDS) enable more natural human-machine interactions by allowing real-time user interruptions and backchanneling, compared to traditional SDS that rely on turn-taking. However, existing benchmarks lack metrics for FD scenes, e.g., evaluating model performance during user interruptions. In this paper, we present a comprehensive FD benchmarking pipeline utilizing LLMs, TTS, and ASR to address this gap. It assesses FDSDS's ability to handle user interruptions, manage delays, and maintain robustness in challenging scenarios with diverse novel metrics. We applied our benchmark to three open-source FDSDS (Moshi, Freeze-omni, and VITA-1.5) using over 40 hours of generated speech, with 293 simulated conversations and 1,200 interruptions. The results show that all models continue to face challenges, such as failing to respond to user interruptions, under frequent disruptions and noisy conditions. Demonstrations, data, and code will be released.

eess.AS↗

arXiv2025

ClearerVoice-Studio: Bridging Advanced Speech Processing Research and Practical Deployment

This paper introduces ClearerVoice-Studio, an open-source, AI-powered speech processing toolkit designed to bridge cutting-edge research and practical application. Unlike broad platforms like SpeechBrain and ESPnet, ClearerVoice-Studio focuses on interconnected speech tasks of speech enhancement, separation, super-resolution, and multimodal target speaker extraction. A key advantage is its state-of-the-art pretrained models, including FRCRN with 3 million uses and MossFormer with 2.5 million uses, optimized for real-world scenarios. It also offers model optimization tools, multi-format audio support, the SpeechScore evaluation toolkit, and user-friendly interfaces, catering to researchers, developers, and end-users. Its rapid adoption attracting 3000 GitHub stars and 239 forks highlights its academic and industrial impact. This paper details ClearerVoice-Studio's capabilities, architectures, training strategies, benchmarks, community impact, and future plan. Source code is available at https://github.com/modelscope/ClearerVoice-Studio.

cs.SD↗

arXiv2025

Online Audio-Visual Autoregressive Speaker Extraction

This paper proposes a novel online audio-visual speaker extraction model. In the streaming regime, most studies optimize the audio network only, leaving the visual frontend less explored. We first propose a lightweight visual frontend based on depth-wise separable convolution. Then, we propose a lightweight autoregressive acoustic encoder to serve as the second cue, to actively explore the information in the separated speech signal from past steps. Scenario-wise, for the first time, we study how the algorithm performs when there is a change in focus of attention, i.e., the target speaker. Experimental results on LRS3 datasets show that our visual frontend performs comparably to the previous state-of-the-art on both SkiM and ConvTasNet audio backbones with only 0.1 million network parameters and 2.1 MACs per second of processing. The autoregressive acoustic encoder provides an additional 0.9 dB gain in terms of SI-SNRi, and its momentum is robust against the change in attention.

eess.AS↗

arXiv2025

Plug-and-Play Co-Occurring Face Attention for Robust Audio-Visual Speaker Extraction

Audio-visual speaker extraction isolates a target speaker's speech from a mixture speech signal conditioned on a visual cue, typically using the target speaker's face recording. However, in real-world scenarios, other co-occurring faces are often present on-screen, providing valuable speaker activity cues in the scene. In this work, we introduce a plug-and-play inter-speaker attention module to process these flexible numbers of co-occurring faces, allowing for more accurate speaker extraction in complex multi-person environments. We integrate our module into two prominent models: the AV-DPRNN and the state-of-the-art AV-TFGridNet. Extensive experiments on diverse datasets, including the highly overlapped VoxCeleb2 and sparsely overlapped MISP, demonstrate that our approach consistently outperforms baselines. Furthermore, cross-dataset evaluations on LRS2 and LRS3 confirm the robustness and generalizability of our method.

eess.AS↗

arXiv2025

Multi-band Frequency Reconstruction for Neural Psychoacoustic Coding

Achieving high-fidelity audio compression while preserving perceptual quality across diverse content remains a key challenge in Neural Audio Coding (NAC). We introduce MUFFIN, a fully convolutional Neural Psychoacoustic Coding (NPC) framework that leverages psychoacoustically guided multi-band frequency reconstruction. At its core is a Multi-Band Spectral Residual Vector Quantization (MBS-RVQ) module that allocates bitrate across frequency bands based on perceptual salience. This design enables efficient compression while disentangling speaker identity from content using distinct codebooks. MUFFIN incorporates a transformer-inspired convolutional backbone and a modified snake activation to enhance resolution in fine-grained spectral regions. Experimental results on multiple benchmarks demonstrate that MUFFIN consistently outperforms existing approaches in reconstruction quality. A high-compression variant achieves a state-of-the-art 12.5 Hz rate with minimal loss. MUFFIN also proves effective in downstream generative tasks, highlighting its promise as a token representation for integration with language models. Audio samples and code are available.

cs.SD↗

arXiv2025

InspireMusic: Integrating Super Resolution and Large Language Model for High-Fidelity Long-Form Music Generation

We introduce InspireMusic, a framework integrated super resolution and large language model for high-fidelity long-form music generation. A unified framework generates high-fidelity music, songs, and audio, which incorporates an autoregressive transformer with a super-resolution flow-matching model. This framework enables the controllable generation of high-fidelity long-form music at a higher sampling rate from both text and audio prompts. Our model differs from previous approaches, as we utilize an audio tokenizer with one codebook that contains richer semantic information, thereby reducing training costs and enhancing efficiency. This combination enables us to achieve high-quality audio generation with long-form coherence of up to $8$ minutes. Then, an autoregressive transformer model based on Qwen 2.5 predicts audio tokens. Next, we employ a super-resolution flow-matching model to generate high-sampling rate audio with fine-grained details learned from an acoustic codec model. Comprehensive experiments show that the InspireMusic-1.5B-Long model has a comparable performance to recent top-tier open-source systems, including MusicGen and Stable Audio 2.0, on subjective and objective evaluations. The code and pre-trained models are released at https://github.com/FunAudioLLM/InspireMusic.

cs.SD↗