arXiv subjects

Yi Ren

Publications and source records attributed to Yi Ren.

At least 91 records · Page 5Linked to original sources

arXiv2025

Two-Player Zero-Sum Differential Games with One-Sided Information

Unlike Poker where the action space $\mathcal{A}$ is discrete, differential games in the physical world often have continuous action spaces not amenable to discrete abstraction, rendering no-regret algorithms with $\mathcal{O}(|\mathcal{A}|)$ complexity not scalable. To address this challenge within the scope of two-player zero-sum (2p0s) games with one-sided information, we show that (1) a computational complexity independent of $|\mathcal{A}|$ can be achieved by exploiting the convexification property of incomplete-information games and the Isaacs' condition that commonly holds for dynamical systems, and that (2) the computation of the two equilibrium strategies can be decoupled under one-sidedness of information. Leveraging these insights, we develop an algorithm that successfully approximates the optimal strategy in a homing game. Code available in https://github.com/ghimiremukesh/cams/tree/workshop

cs.GT↗

arXiv2025

A Unit-based System and Dataset for Expressive Direct Speech-to-Speech Translation

Current research in speech-to-speech translation (S2ST) primarily concentrates on translation accuracy and speech naturalness, often overlooking key elements like paralinguistic information, which is essential for conveying emotions and attitudes in communication. To address this, our research introduces a novel, carefully curated multilingual dataset from various movie audio tracks. Each dataset pair is precisely matched for paralinguistic information and duration. We enhance this by integrating multiple prosody transfer techniques, aiming for translations that are accurate, natural-sounding, and rich in paralinguistic details. Our experimental results confirm that our model retains more paralinguistic information from the source speech while maintaining high standards of translation accuracy and naturalness.

cs.CL↗

arXiv2025

When End-to-End is Overkill: Rethinking Cascaded Speech-to-Text Translation

Though end-to-end speech-to-text translation has been a great success, we argue that the cascaded speech-to-text translation model still has its place, which is usually criticized for the error propagation between automatic speech recognition (ASR) and machine translation (MT) models. In this paper, we explore the benefits of incorporating multiple candidates from ASR and self-supervised speech features into MT. Our analysis reveals that the primary cause of cascading errors stems from the increased divergence between similar samples in the speech domain when mapped to the text domain. By including multiple candidates and self-supervised speech features, our approach allows the machine translation model to choose the right words and ensure precise translation using various speech samples. This strategy minimizes error spread and takes advantage of large ASR and MT datasets, along with pre-trained ASR/MT models, while addressing associated issues.

cs.CL↗

arXiv2025

Acoustic Emission Sensor Network Optimization Based on Grid Loop Search and Particle Swarm Source Location

The layout of acoustic emission sensors plays a critical role in non-destructive structural testing. This study proposes a grid-based optimization method focused on multi-source location results, in contrast to traditional sensor layout optimization methods that construct a correlation matrix based on sensor layout and one source location. Based on the seismic source travel-time theory, the proposed method establishes a location objective function based on minimum travel-time differences, which is solved through the particle swarm optimization (PSO) algorithm. Furthermore, based on location accuracy across various configurations, the method systematically evaluates potential optimal sensor locations through grid search. Synthetic tests and laboratory pencil-lead break (PLB) experiments are conducted to compare the effectiveness of PSO, genetic algorithm, and simulated annealing, with the following conclusions: (1) In synthetic tests, the proposed method achieved an average location error of 1.78 mm, outperforming that based on the traditional layout, genetic algorithm (GA), and simulated annealing (SA). (2) For different noise cases, the location accuracy separately improved by 24.89% (σ=0.5μs), 12.59% (σ=2μs), and 15.06% (σ=5μs) compared with the traditional layout. (3) For the PLB experiments, the optimized layout achieved an average location error of 9.37 mm, which improved the location accuracy by 59.15% compared with the Traditional layout.

physics.geo-ph↗

arXiv2024

Searching for Galactic Red Supergiants with Gaia RVS Spectra

Red supergiants (RSGs) are essential to understanding the evolution and the contribution to the interstellar medium of massive stars. However, the number of identified RSGs within the Milky Way is still limited mainly due to the difficulty of measuring stellar extinction and distance. The release of approximately one million RVS spectra in Gaia DR3 presents new opportunity for identifying Galactic RSGs, because the equivalent width of the calcium triplet lines (EW(CaT)) in the spectra is an excellent indicator of stellar surface gravity. This work uses the RVS spectra with signal-to-noise ratio (SNR) greater than 100 to search for the Galactic RSGs. The dwarf stars and red giants are removed and the RSG candidates are selected by the location in the EW(CaT) vs. BP-RP diagram. The early-type RSG candidates (K0-M2) are then identified by BP-RP > 1.584 and EW(CaT) > 1.1 nm. To identify late-type RSG candidates (after M2), the criteria of the average equivalent widths of TiO in the XP spectra (EW(TiO)) > 10 nm, the color index K-W3 < 0.5 and the period-amplitude sequence from Gaia DR3 LPV catalog are further applied to reduce the contamination of late-type red giants and asymptotic giant branch stars. This method yields 30 early-type (K0-M2) and 6196 late-type (after M2) RSG candidates, which is a significant increase to the present Galactic RSG sample. The application of this approach to the spectra with SNR > 50 results in 48 early-type and 11,491 late-type RSG candidates. This preliminary analysis paves the way for more extensive research with Gaia DR4 when larger spectral datasets are expected to significantly enhance our understanding of Galactic RSG populations.

astro-ph.SR↗

arXiv2024

A New Approach to Identifying Red Supergiant Stars in Metal-poor Galaxies: A Case Study of NGC 6822

A complete sample of red supergiant stars (RSGs) is important for studying their properties. Identifying RSGs in extragalatic field first requires removing the Galactic foreground dwarfs. The color-color diagram (CCD) method, specifically using $r-z/z-H$ and $J-H/H-K$, has proven successful in several studies. However, in metal-poor galaxies, faint RSGs will mix into the dwarf branch in the CCD and would be removed, leading to an incomplete RSG sample. This work attempts to improve the CCD method in combination with the Gaia astrometric measurement to remove foreground contamination in order to construct a complete RSG sample in metal-poor galaxies. The empirical regions of RSGs in both CCDs are defined and modified by fitting the locations of RSGs in galaxies with a range of metallicity. The metal-poor galaxy NGC 6822 is taken as a case study for its low metallicity ([Fe/H] $\approx$ -1.0) and moderate distance (about 500 kpc). In the complete sample, we identify 1,184 RSG, 1,559 oxygen-rich AGB (O-AGBs), 1,075 carbon-rich AGB (C-AGBs), and 140 extreme AGB (x-AGBs) candidates, with a contamination rate of approximately 20.5%, 9.7%, 6.8%, and 5.0%, respectively. We also present a pure sample, containing only the sources away from the dwarf branch, which includes 843 RSG, 1,519 O-AGB, 1,059 C-AGB, and 140 x-AGB candidates, with a contamination rate of approximately 6.5%, 8.8%, 6.1%, and 5.0%, respectively. About 600 and 450 RSG candidates are newly identified in the complete and pure sample, respectively, compared to the previous RSG sample in NGC 6822.

astro-ph.GA↗

arXiv2024

MMDS: A Multimodal Medical Diagnosis System Integrating Image Analysis and Knowledge-based Departmental Consultation

We present MMDS, a system capable of recognizing medical images and patient facial details, and providing professional medical diagnoses. The system consists of two core components:The first component is the analysis of medical images and videos. We trained a specialized multimodal medical model capable of interpreting medical images and accurately analyzing patients' facial emotions and facial paralysis conditions. The model achieved an accuracy of 72.59% on the FER2013 facial emotion recognition dataset, with a 91.1% accuracy in recognizing the "happy" emotion. In facial paralysis recognition, the model reached an accuracy of 92%, which is 30% higher than that of GPT-4o. Based on this model, we developed a parser for analyzing facial movement videos of patients with facial paralysis, achieving precise grading of the paralysis severity. In tests on 30 videos of facial paralysis patients, the system demonstrated a grading accuracy of 83.3%.The second component is the generation of professional medical responses. We employed a large language model, integrated with a medical knowledge base, to generate professional diagnoses based on the analysis of medical images or videos. The core innovation lies in our development of a department-specific knowledge base routing management mechanism, in which the large language model categorizes data by medical departments and, during the retrieval process, determines the appropriate knowledge base to query. This significantly improves retrieval accuracy in the RAG (retrieval-augmented generation) process.

cs.CV↗

arXiv2024

Sharingan: Extract User Action Sequence from Desktop Recordings

Video recordings of user activities, particularly desktop recordings, offer a rich source of data for understanding user behaviors and automating processes. However, despite advancements in Vision-Language Models (VLMs) and their increasing use in video analysis, extracting user actions from desktop recordings remains an underexplored area. This paper addresses this gap by proposing two novel VLM-based methods for user action extraction: the Direct Frame-Based Approach (DF), which inputs sampled frames directly into VLMs, and the Differential Frame-Based Approach (DiffF), which incorporates explicit frame differences detected via computer vision techniques. We evaluate these methods using a basic self-curated dataset and an advanced benchmark adapted from prior work. Our results show that the DF approach achieves an accuracy of 70% to 80% in identifying user actions, with the extracted action sequences being re-playable though Robotic Process Automation. We find that while VLMs show potential, incorporating explicit UI changes can degrade performance, making the DF approach more reliable. This work represents the first application of VLMs for extracting user action sequences from desktop recordings, contributing new methods, benchmarks, and insights for future research.

cs.CV↗

arXiv2024

AMD: Autoregressive Motion Diffusion

Human motion generation aims to produce plausible human motion sequences according to various conditional inputs, such as text or audio. Despite the feasibility of existing methods in generating motion based on short prompts and simple motion patterns, they encounter difficulties when dealing with long prompts or complex motions. The challenges are two-fold: 1) the scarcity of human motion-captured data for long prompts and complex motions. 2) the high diversity of human motions in the temporal domain and the substantial divergence of distributions from conditional modalities, leading to a many-to-many mapping problem when generating motion with complex and long texts. In this work, we address these gaps by 1) elaborating the first dataset pairing long textual descriptions and 3D complex motions (HumanLong3D), and 2) proposing an autoregressive motion diffusion model (AMD). Specifically, AMD integrates the text prompt at the current timestep with the text prompt and action sequences at the previous timestep as conditional information to predict the current action sequences in an iterative manner. Furthermore, we present its generalization for X-to-Motion with "No Modality Left Behind", enabling the generation of high-definition and high-fidelity human motions based on user-defined modality input.

cs.MM↗

arXiv2024

MimicTalk: Mimicking a personalized and expressive 3D talking face in minutes

Talking face generation (TFG) aims to animate a target identity's face to create realistic talking videos. Personalized TFG is a variant that emphasizes the perceptual identity similarity of the synthesized result (from the perspective of appearance and talking style). While previous works typically solve this problem by learning an individual neural radiance field (NeRF) for each identity to implicitly store its static and dynamic information, we find it inefficient and non-generalized due to the per-identity-per-training framework and the limited training data. To this end, we propose MimicTalk, the first attempt that exploits the rich knowledge from a NeRF-based person-agnostic generic model for improving the efficiency and robustness of personalized TFG. To be specific, (1) we first come up with a person-agnostic 3D TFG model as the base model and propose to adapt it into a specific identity; (2) we propose a static-dynamic-hybrid adaptation pipeline to help the model learn the personalized static appearance and facial dynamic features; (3) To generate the facial motion of the personalized talking style, we propose an in-context stylized audio-to-motion model that mimics the implicit talking style provided in the reference video without information loss by an explicit style representation. The adaptation process to an unseen identity can be performed in 15 minutes, which is 47 times faster than previous person-dependent methods. Experiments show that our MimicTalk surpasses previous baselines regarding video quality, efficiency, and expressiveness. Source code and video samples are available at https://mimictalk.github.io .

cs.CV↗

arXiv2024

Bias Amplification in Language Model Evolution: An Iterated Learning Perspective

With the widespread adoption of Large Language Models (LLMs), the prevalence of iterative interactions among these models is anticipated to increase. Notably, recent advancements in multi-round self-improving methods allow LLMs to generate new examples for training subsequent models. At the same time, multi-agent LLM systems, involving automated interactions among agents, are also increasing in prominence. Thus, in both short and long terms, LLMs may actively engage in an evolutionary process. We draw parallels between the behavior of LLMs and the evolution of human culture, as the latter has been extensively studied by cognitive scientists for decades. Our approach involves leveraging Iterated Learning (IL), a Bayesian framework that elucidates how subtle biases are magnified during human cultural evolution, to explain some behaviors of LLMs. This paper outlines key characteristics of agents' behavior in the Bayesian-IL framework, including predictions that are supported by experimental verification with various LLMs. This theoretical framework could help to more effectively predict and guide the evolution of LLMs in desired directions.

cs.CL↗

arXiv2024

Research on Predicting Public Opinion Event Heat Levels Based on Large Language Models

In recent years, with the rapid development of large language models, serval models such as GPT-4o have demonstrated extraordinary capabilities, surpassing human performance in various language tasks. As a result, many researchers have begun exploring their potential applications in the field of public opinion analysis. This study proposes a novel large-language-models-based method for public opinion event heat level prediction. First, we preprocessed and classified 62,836 Chinese hot event data collected between July 2022 and December 2023. Then, based on each event's online dissemination heat index, we used the MiniBatchKMeans algorithm to automatically cluster the events and categorize them into four heat levels (ranging from low heat to very high heat). Next, we randomly selected 250 events from each heat level, totalling 1,000 events, to build the evaluation dataset. During the evaluation process, we employed various large language models to assess their accuracy in predicting event heat levels in two scenarios: without reference cases and with similar case references. The results showed that GPT-4o and DeepseekV2 performed the best in the latter case, achieving prediction accuracies of 41.4% and 41.5%, respectively. Although the overall prediction accuracy remains relatively low, it is worth noting that for low-heat (Level 1) events, the prediction accuracies of these two models reached 73.6% and 70.4%, respectively. Additionally, the prediction accuracy showed a downward trend from Level 1 to Level 4, which correlates with the uneven distribution of data across the heat levels in the actual dataset. This suggests that with the more robust dataset, public opinion event heat level prediction based on large language models will have significant research potential for the future.

cs.CL↗

arXiv2024

A Novel Adaptive Fine-Tuning Algorithm for Multimodal Models: Self-Optimizing Classification and Selection of High-Quality Datasets in Remote Sensing

We propose an adaptive fine-tuning algorithm for multimodal large models. The core steps of this algorithm involve two stages of truncation. First, the vast amount of data is projected into a semantic vector space, and the MiniBatchKMeans algorithm is used for automated clustering. This classification ensures that the data within each cluster exhibit high semantic similarity. Next, we process the data in each cluster, calculating the translational difference between the original and perturbed data in the multimodal large model's vector space. This difference serves as a generalization metric for the data. Based on this metric, we select the data with high generalization potential for training. We applied this algorithm to train the InternLM-XComposer2-VL-7B model on two 3090 GPUs using one-third of the GeoChat multimodal remote sensing dataset. The results demonstrate that our algorithm outperforms the state-of-the-art baselines. various baselines. The model trained on our optimally chosen one-third dataset, based on experimental validation, exhibited only 1% reduction in performance across various remote sensing metrics compared to the model trained on the full dataset. This approach significantly preserved general-purpose capabilities while reducing training time by 68.2%. Furthermore, the model achieved scores of 89.86 and 77.19 on the UCMerced and AID evaluation datasets, respectively, surpassing the GeoChat dataset by 5.43 and 5.16 points. It only showed a 0.91-point average decrease on the LRBEN evaluation dataset.

cs.CV↗

arXiv2024

Understanding Simplicity Bias towards Compositional Mappings via Learning Dynamics

Obtaining compositional mappings is important for the model to generalize well compositionally. To better understand when and how to encourage the model to learn such mappings, we study their uniqueness through different perspectives. Specifically, we first show that the compositional mappings are the simplest bijections through the lens of coding length (i.e., an upper bound of their Kolmogorov complexity). This property explains why models having such mappings can generalize well. We further show that the simplicity bias is usually an intrinsic property of neural network training via gradient descent. That partially explains why some models spontaneously generalize well when they are trained appropriately.

cs.LG↗

arXiv2024

Mass-loss Rate of Highly Evolved Stars in the Magellanic Clouds

Asymptotic giant branch stars (AGBs) and red supergiant stars (RSGs) exhibit significant mass loss phenomena and are considered important sources of interstellar dust. In this work, we employed an uniform method of spectral energy distribution fitting to analyze a large, and hence statistically significant, sample of approximately 40,000 RSGs and AGBs in the Magellanic Clouds (MCs), providing a new catalog of evolved stars that includes stellar parameters and dust properties. Our results reveal that the total dust-production rate (DPR) of the Large Magellanic Cloud is approximately $9.69\times10^{-6}\,\rm{M_{\odot }\, yr^{-1}}$, while it is around $1.75\times10^{-6}\,\rm{M_{\odot }\,yr^{-1}}$ for the Small Magellanic Cloud, with a few stars significantly contributing to the total DPR. No significant differences were observed in the contributions to DPR from carbon-rich and oxygen-rich (O-rich) evolved stars in the MCs. We explored the relations between stellar parameters (luminosity, infrared color, period, amplitude) and mass-loss rate (MLR) for evolved stars. A prominent turning point at $\log{(L/L_{\odot})} \approx 4.4$ appears in the luminosity-MLR diagram of RSGs, potentially related to the mass-loss mechanism of RSGs. The luminosity-MLR relation of AGBs is highly scattered. The DPR of AGBs shows a clear change with pulsation period and amplitude, with DPR exhibiting a drastic increase at pulsation periods of approximately 300 days and I-band amplitudes greater than 0.5 mag. Metallicity has some impact on the DPR of O-rich stars, with lower metallicity seeming to result in lower mean DPR and a higher proportion of optically thin stars.

astro-ph.SR↗

arXiv2024

A Centralized Discovery-Based Method for Integrating Data Distribution Service and Time-Sensitive Networking in In-Vehicle Networks

As the electronic and electrical architecture (E/EA) of intelligent and connected vehicles (ICVs) evolves, traditional distributed and signal-oriented architectures are being replaced by centralized, service-oriented architectures (SOA). This new generation of E/EA demands in-vehicle networks (IVNs) that offer high bandwidth, real-time, reliability, and service-oriented. data distribution service (DDS) and time-sensitive networking (TSN) are increasingly adopted to address these requirements. However, research on the integrated deployment of DDS and TSN in automotive applications is still in its infancy. This paper presents a DDS over TSN (DoT) communication architecture based on the centralized discovery architecture (CDA). First, a lightweight DDS implementation (FastDDS-lw) is developed for resource-constrained in-vehicle devices. Next, a DDS flow identification algorithm (DFIA) based on the CDA is introduced to identify potential DDS flows during the discovery phase automatically. Finally, the DoT communication architecture is designed, incorporating FastDDS-lw and DFIA. Experimental results show that the DoT architecture significantly reduces end-to-end latency and jitter for critical DDS flows compared to traditional Ethernet. Additionally, DoT provides an automated network configuration method that completes within a few tens of milliseconds.

cs.NI↗

arXiv2024

BiKC: Keypose-Conditioned Consistency Policy for Bimanual Robotic Manipulation

Bimanual manipulation tasks typically involve multiple stages which require efficient interactions between two arms, posing step-wise and stage-wise challenges for imitation learning systems. Specifically, failure and delay of one step will broadcast through time, hinder success and efficiency of each sub-stage task, and thereby overall task performance. Although recent works have made strides in addressing certain challenges, few approaches explicitly consider the multi-stage nature of bimanual tasks while simultaneously emphasizing the importance of inference speed. In this paper, we introduce a novel keypose-conditioned consistency policy tailored for bimanual manipulation. It is a hierarchical imitation learning framework that consists of a high-level keypose predictor and a low-level trajectory generator. The predicted keyposes provide guidance for trajectory generation and also mark the completion of one sub-stage task. The trajectory generator is designed as a consistency model trained from scratch without distillation, which generates action sequences conditioning on current observations and predicted keyposes with fast inference speed. Simulated and real-world experimental results demonstrate that the proposed approach surpasses baseline methods in terms of success rate and operational efficiency. Codes are available at https://github.com/ManUtdMoon/BiKC.

cs.RO↗

arXiv2024

MulliVC: Multi-lingual Voice Conversion With Cycle Consistency

Voice conversion aims to modify the source speaker's voice to resemble the target speaker while preserving the original speech content. Despite notable advancements in voice conversion these days, multi-lingual voice conversion (including both monolingual and cross-lingual scenarios) has yet to be extensively studied. It faces two main challenges: 1) the considerable variability in prosody and articulation habits across languages; and 2) the rarity of paired multi-lingual datasets from the same speaker. In this paper, we propose MulliVC, a novel voice conversion system that only converts timbre and keeps original content and source language prosody without multi-lingual paired data. Specifically, each training step of MulliVC contains three substeps: In step one the model is trained with monolingual speech data; then, steps two and three take inspiration from back translation, construct a cyclical process to disentangle the timbre and other information (content, prosody, and other language-related information) in the absence of multi-lingual data from the same speaker. Both objective and subjective results indicate that MulliVC significantly surpasses other methods in both monolingual and cross-lingual contexts, demonstrating the system's efficacy and the viability of the three-step approach with cycle consistency. Audio samples can be found on our demo page (mullivc.github.io).

cs.SD↗