arXiv subjects

Yudong Li

Publications and source records attributed to Yudong Li.

At least 37 records · Page 2Linked to original sources

arXiv2025

Asymptotic Boundedness of Distributed Set-Membership Filtering

Asymptotic boundedness is a crucial property of Distributed Set-Membership Filtering (DSMFing) that prevents the unbounded growth of the set estimates caused by the wrapping effect. However, this important property remains underinvestigated, compared to its noise-free and stochastic-noise counterparts, i.e., the convergence of Distributed Observers (DOs) and the bounded error covariance of Distributed Kalman Filters (DKFs). This paper studies the asymptotic boundedness of DSMFing for linear discrete-time systems. A novel concept, termed the Collective Observation-Information Tower (COIT), is introduced to characterize the fundamental relationship between the structure of graphs and the set estimates, which enables the boundedness analysis. Leveraging the COIT, an easily verifiable sufficient condition for the asymptotic boundedness of linear DSMFing is established. Surprisingly, the sufficient condition generalizes the well-known collective detectability condition for DOs and DKFs; it links DSMFs to existing distributed estimation methods and reveals the unique characteristic of DSMFs.

eess.SY↗

arXiv2025

FaceBench: A Multi-View Multi-Level Facial Attribute VQA Dataset for Benchmarking Face Perception MLLMs

Multimodal large language models (MLLMs) have demonstrated remarkable capabilities in various tasks. However, effectively evaluating these MLLMs on face perception remains largely unexplored. To address this gap, we introduce FaceBench, a dataset featuring hierarchical multi-view and multi-level attributes specifically designed to assess the comprehensive face perception abilities of MLLMs. Initially, we construct a hierarchical facial attribute structure, which encompasses five views with up to three levels of attributes, totaling over 210 attributes and 700 attribute values. Based on the structure, the proposed FaceBench consists of 49,919 visual question-answering (VQA) pairs for evaluation and 23,841 pairs for fine-tuning. Moreover, we further develop a robust face perception MLLM baseline, Face-LLaVA, by training with our proposed face VQA data. Extensive experiments on various mainstream MLLMs and Face-LLaVA are conducted to test their face perception ability, with results also compared against human performance. The results reveal that, the existing MLLMs are far from satisfactory in understanding the fine-grained facial attributes, while our Face-LLaVA significantly outperforms existing open-source models with a small amount of training data and is comparable to commercial ones like GPT-4o and Gemini. The dataset will be released at https://github.com/CVI-SZU/FaceBench.

cs.CV↗

arXiv2025

GTG: Generalizable Trajectory Generation Model for Urban Mobility

Trajectory data mining is crucial for smart city management. However, collecting large-scale trajectory datasets is challenging due to factors such as commercial conflicts and privacy regulations. Therefore, we urgently need trajectory generation techniques to address this issue. Existing trajectory generation methods rely on the global road network structure of cities. When the road network structure changes, these methods are often not transferable to other cities. In fact, there exist invariant mobility patterns between different cities: 1) People prefer paths with the minimal travel cost; 2) The travel cost of roads has an invariant relationship with the topological features of the road network. Based on the above insight, this paper proposes a Generalizable Trajectory Generation model (GTG). The model consists of three parts: 1) Extracting city-invariant road representation based on Space Syntax method; 2) Cross-city travel cost prediction through disentangled adversarial training; 3) Travel preference learning by shortest path search and preference update. By learning invariant movement patterns, the model is capable of generating trajectories in new cities. Experiments on three datasets demonstrates that our model significantly outperforms existing models in terms of generalization ability.

cs.LG↗

arXiv2024

Optimal Set-Membership Smoothing

This article studies the Set-Membership Smoothing (SMSing) problem for non-stochastic Hidden Markov Models. By adopting the mathematical concept of uncertain variables, an optimal SMSing framework is established for the first time. This optimal framework reveals the principles of SMSing and the relationship between set-membership filtering and smoothing. Based on the design principles, we put forward two SMSing algorithms: one for linear systems with zonotopic constrained uncertainties, where the solution is given in a closed form, and the other for a class of nonlinear systems. Numerical simulations corroborate the effectiveness of our theoretical results.

eess.SY↗

arXiv2024

Dynamic data sampler for cross-language transfer learning in large language models

Large Language Models (LLMs) have gained significant attention in the field of natural language processing (NLP) due to their wide range of applications. However, training LLMs for languages other than English poses significant challenges, due to the difficulty in acquiring large-scale corpus and the requisite computing resources. In this paper, we propose ChatFlow, a cross-language transfer-based LLM, to address these challenges and train large Chinese language models in a cost-effective manner. We employ a mix of Chinese, English, and parallel corpus to continuously train the LLaMA2 model, aiming to align cross-language representations and facilitate the knowledge transfer specifically to the Chinese language model. In addition, we use a dynamic data sampler to progressively transition the model from unsupervised pre-training to supervised fine-tuning. Experimental results demonstrate that our approach accelerates model convergence and achieves superior performance. We evaluate ChatFlow on popular Chinese and English benchmarks, the results indicate that it outperforms other Chinese models post-trained on LLaMA-2-7B.

cs.CL↗

arXiv2024

Freezing delay of a drop impacting on a monolayer of cold grains

We investigate a subfreezing droplet impact scenario in a low-humidity environment, where the target is a cold granular monolayer. When the undercooling degree of targets passes a threshold, such a granular layer effectively postpones the bulk freezing time of the droplet in comparison with the impact on the bare substrate underneath. In this case, the retraction of the droplet after impact reduces the contact area with the cold substrate, even though both the grains and the substrate are wettable to the liquid. We find that the significant changes in the dynamic behavior are triggered by freezing the liquid that wets the pores. Owing to the small dimension of the pores, the freezing process is rapid enough to match the dynamics over the droplet dimension. In certain circumstances, the rapid freezing may even stop liquid penetration and shed icing from the underneath surface.

physics.flu-dyn↗

arXiv2023

Optical Ranging Using Coherent Kerr Soliton Dual-microcombs with Extended Ambiguity Distance

Optical ranging is a key technology in metrology. Optical frequency combs are shown to provide several advantages in light ranging, offering high precision with high acquisition rate. However, performance of traditional ranging systems based on microcombs is limited by the short ambiguity distance and non-real-time processing. Here, we show that dual-comb ranging system using coherent Kerr soliton microcombs and optical switch realizes extended ambiguity distance and provides a route to real-time processing. The ambguity distance is extended to 3.28 m from about 1.5 mm and the uncertainty reaches about 1.05 times 10^-7, while the system is compatible with low-bandwidth detectors. Combining coherent microcomb ranging systems with special FPGA could enable comb-based real-time ranging systems for several applications such as industrial process monitoring.

physics.optics↗

arXiv2023

Cold granular targets slow the bulk freezing of an impacting droplet

When making contact with an undercooled target, a drop freezes. The colder the target is, the more rapid the freezing is supposed to be. In this research, we explore the impact of droplets on cold granular material. As the undercooling degree increases, the bulk freezing of the droplet is delayed by at least an order of magnitude. The postponement of the overall solidification is accompanied by substantial changes in dynamics, including the spreading-retraction process, satellite drop generation, and cratering in the target. The solidification of the wetted pores in the granular target primarily causes these effects. Owing to the small size of pores, solidification there is sufficiently fast to match the characteristic timescales of the impact dynamics at moderate undercooling degrees. As a result, the hydrophilic impact appears `hydrophobic', and the dimension of the solidified droplet shrinks. A monolayer of cold grains on a surface can reproduce these consequences. Our research presents a potential approach to regulate solidified morphology for subfreezing drop impacts. It additionally sheds light on the impact scenario of strong coupling between the dynamics and solidification.

physics.flu-dyn↗

arXiv2023

Rethinking Residual Connection in Training Large-Scale Spiking Neural Networks

Spiking Neural Network (SNN) is known as the most famous brain-inspired model, but the non-differentiable spiking mechanism makes it hard to train large-scale SNNs. To facilitate the training of large-scale SNNs, many training methods are borrowed from Artificial Neural Networks (ANNs), among which deep residual learning is the most commonly used. But the unique features of SNNs make prior intuition built upon ANNs not available for SNNs. Although there are a few studies that have made some pioneer attempts on the topology of Spiking ResNet, the advantages of different connections remain unclear. To tackle this issue, we analyze the merits and limitations of various residual connections and empirically demonstrate our ideas with extensive experiments. Then, based on our observations, we abstract the best-performing connections into densely additive (DA) connection, extend such a concept to other topologies, and propose four architectures for training large-scale SNNs, termed DANet, which brings up to 13.24% accuracy gain on ImageNet. Besides, in order to present a detailed methodology for designing the topology of large-scale SNNs, we further conduct in-depth discussions on their applicable scenarios in terms of their performance on various scales of datasets and demonstrate their advantages over prior architectures. At a low training expense, our best-performing ResNet-50/101/152 obtain 73.71%/76.13%/77.22% top-1 accuracy on ImageNet with 4 time steps. We believe that this work shall give more insights for future works to design the topology of their networks and promote the development of large-scale SNNs. The code will be publicly available.

cs.NE↗

arXiv2023

Existence and Completeness of Bounded Disturbance Observers: A Set-Membership Viewpoint

This paper investigates the boundedness of the Disturbance Observer (DO) for linear discrete-time systems. In contrast to previous studies that focus on analyzing and/or designing observer gains, our analysis and synthesis approach is based on a set-membership viewpoint. From this viewpoint, a necessary and sufficient existence condition of bounded DOs is first established, which can be easily verified. Furthermore, a set-membership filter-based DO is developed, and its completeness is proved; thus, our proposed DO is bounded if and only if bounded DOs exist. We also prove that the proposed DO has the capability to achieve the worst-case optimality, which can provide a benchmark for the design of DOs. Finally, numerical simulations are performed to corroborate the effectiveness of the theoretical results.

eess.SY↗

arXiv2023

Topological dissipative Kerr soliton combs in a valley photonic crystal resonator

Topological phases have become an enabling role in exploiting new applications of nonlinear optics in recent years. Here we theoretically propose a valley photonic crystal resonator emulating topologically protected dissipative Kerr soliton combs. It is shown that topological resonator modes can be observed in the resonator. Moreover, we also simulate the dynamic evolution of the topological resonator with the injection of a continuous-wave pump laser. We find that the topological optical frequency combs evolve from Turing rolls to chaotic states, and eventually into single soliton states. More importantly, such dissipative Kerr soliton combs generated in the resonator are inborn topologically protected, showing robustness against sharp bends and structural disorders. Our design supporting topologically protected dissipative Kerr soliton combs could be implemented experimentally in on-chip nanofabricated photonic devices.

physics.optics↗

arXiv2023

TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities

Recently, the success of pre-training in text domain has been fully extended to vision, audio, and cross-modal scenarios. The proposed pre-training models of different modalities are showing a rising trend of homogeneity in their model structures, which brings the opportunity to implement different pre-training models within a uniform framework. In this paper, we present TencentPretrain, a toolkit supporting pre-training models of different modalities. The core feature of TencentPretrain is the modular design. The toolkit uniformly divides pre-training models into 5 components: embedding, encoder, target embedding, decoder, and target. As almost all of common modules are provided in each component, users can choose the desired modules from different components to build a complete pre-training model. The modular design enables users to efficiently reproduce existing pre-training models or build brand-new one. We test the toolkit on text, vision, and audio benchmarks and show that it can match the performance of the original implementations.

cs.CL↗

arXiv2023

Tenrec: A Large-scale Multipurpose Benchmark Dataset for Recommender Systems

Existing benchmark datasets for recommender systems (RS) either are created at a small scale or involve very limited forms of user feedback. RS models evaluated on such datasets often lack practical values for large-scale real-world applications. In this paper, we describe Tenrec, a novel and publicly available data collection for RS that records various user feedback from four different recommendation scenarios. To be specific, Tenrec has the following five characteristics: (1) it is large-scale, containing around 5 million users and 140 million interactions; (2) it has not only positive user feedback, but also true negative feedback (vs. one-class recommendation); (3) it contains overlapped users and items across four different scenarios; (4) it contains various types of user positive feedback, in forms of clicks, likes, shares, and follows, etc; (5) it contains additional features beyond the user IDs and item IDs. We verify Tenrec on ten diverse recommendation tasks by running several classical baseline models per task. Tenrec has the potential to become a useful benchmark dataset for a majority of popular recommendation tasks.

cs.IR↗

arXiv2023

VisorGPT: Learning Visual Prior via Generative Pre-Training

Various stuff and things in visual data possess specific traits, which can be learned by deep neural networks and are implicitly represented as the visual prior, e.g., object location and shape, in the model. Such prior potentially impacts many vision tasks. For example, in conditional image synthesis, spatial conditions failing to adhere to the prior can result in visually inaccurate synthetic results. This work aims to explicitly learn the visual prior and enable the customization of sampling. Inspired by advances in language modeling, we propose to learn Visual prior via Generative Pre-Training, dubbed VisorGPT. By discretizing visual locations of objects, e.g., bounding boxes, human pose, and instance masks, into sequences, VisorGPT can model visual prior through likelihood maximization. Besides, prompt engineering is investigated to unify various visual locations and enable customized sampling of sequential outputs from the learned prior. Experimental results demonstrate that VisorGPT can effectively model the visual prior, which can be employed for many vision tasks, such as customizing accurate human pose for conditional image synthesis models like ControlNet. Code will be released at https://github.com/Sierkinhane/VisorGPT.

cs.CV↗

arXiv2022

Spikeformer: A Novel Architecture for Training High-Performance Low-Latency Spiking Neural Network

Spiking neural networks (SNNs) have made great progress on both performance and efficiency over the last few years,but their unique working pattern makes it hard to train a high-performance low-latency SNN.Thus the development of SNNs still lags behind traditional artificial neural networks (ANNs).To compensate this gap,many extraordinary works have been proposed.Nevertheless,these works are mainly based on the same kind of network structure (i.e.CNN) and their performance is worse than their ANN counterparts,which limits the applications of SNNs.To this end,we propose a novel Transformer-based SNN,termed "Spikeformer",which outperforms its ANN counterpart on both static dataset and neuromorphic dataset and may be an alternative architecture to CNN for training high-performance SNNs.First,to deal with the problem of "data hungry" and the unstable training period exhibited in the vanilla model,we design the Convolutional Tokenizer (CT) module,which improves the accuracy of the original model on DVS-Gesture by more than 16%.Besides,in order to better incorporate the attention mechanism inside Transformer and the spatio-temporal information inherent to SNN,we adopt spatio-temporal attention (STA) instead of spatial-wise or temporal-wise attention.With our proposed method,we achieve competitive or state-of-the-art (SOTA) SNN performance on DVS-CIFAR10,DVS-Gesture,and ImageNet datasets with the least simulation time steps (i.e.low latency).Remarkably,our Spikeformer outperforms other SNNs on ImageNet by a large margin (i.e.more than 5%) and even outperforms its ANN counterpart by 3.1% and 2.2% on DVS-Gesture and ImageNet respectively,indicating that Spikeformer is a promising architecture for training large-scale SNNs and may be more suitable for SNNs compared to CNN.We believe that this work shall keep the development of SNNs in step with ANNs as much as possible.Code will be available.

cs.NE↗

arXiv2022

CSL: A Large-scale Chinese Scientific Literature Dataset

Scientific literature serves as a high-quality corpus, supporting a lot of Natural Language Processing (NLP) research. However, existing datasets are centered around the English language, which restricts the development of Chinese scientific NLP. In this work, we present CSL, a large-scale Chinese Scientific Literature dataset, which contains the titles, abstracts, keywords and academic fields of 396k papers. To our knowledge, CSL is the first scientific document dataset in Chinese. The CSL can serve as a Chinese corpus. Also, this semi-structured data is a natural annotation that can constitute many supervised NLP tasks. Based on CSL, we present a benchmark to evaluate the performance of models across scientific domain tasks, i.e., summarization, keyword generation and text classification. We analyze the behavior of existing text-to-text models on the evaluation tasks and reveal the challenges for Chinese scientific NLP tasks, which provides a valuable reference for future research. Data and code are available at https://github.com/ydli-ai/CSL

cs.CL↗

arXiv2022

ET White Paper: To Find the First Earth 2.0

We propose to develop a wide-field and ultra-high-precision photometric survey mission, temporarily named "Earth 2.0 (ET)". This mission is designed to measure, for the first time, the occurrence rate and the orbital distributions of Earth-sized planets. ET consists of seven 30cm telescopes, to be launched to the Earth-Sun's L2 point. Six of these are transit telescopes with a field of view of 500 square degrees. Staring in the direction that encompasses the original Kepler field for four continuous years, this monitoring will return tens of thousands of transiting planets, including the elusive Earth twins orbiting solar-type stars. The seventh telescope is a 30cm microlensing telescope that will monitor an area of 4 square degrees toward the galactic bulge. This, combined with simultaneous ground-based KMTNet observations, will measure masses for hundreds of long-period and free-floating planets. Together, the transit and the microlensing telescopes will revolutionize our understandings of terrestrial planets across a large swath of orbital distances and free space. In addition, the survey data will also facilitate studies in the fields of asteroseismology, Galactic archeology, time-domain sciences, and black holes in binaries.

astro-ph.IM↗

arXiv2021

One Person, One Model, One World: Learning Continual User Representation without Forgetting

Learning user representations is a vital technique toward effective user modeling and personalized recommender systems. Existing approaches often derive an individual set of model parameters for each task by training on separate data. However, the representation of the same user potentially has some commonalities, such as preference and personality, even in different tasks. As such, these separately trained representations could be suboptimal in performance as well as inefficient in terms of parameter sharing. In this paper, we delve on research to continually learn user representations task by task, whereby new tasks are learned while using partial parameters from old ones. A new problem arises since when new tasks are trained, previously learned parameters are very likely to be modified, and as a result, an artificial neural network (ANN)-based model may lose its capacity to serve for well-trained previous tasks forever, this issue is termed catastrophic forgetting. To address this issue, we present \emph{Conure} the first \underline{con}tinual, or lifelong, \underline{u}ser \underline{re}presentation learner -- i.e., learning new tasks over time without forgetting old ones. Specifically, we propose iteratively removing less important weights of old tasks in a deep user representation model, motivated by the fact that neural network models are usually over-parameterized. In this way, we could learn many tasks with a single model by reusing the important weights, and modifying the less important weights to adapt to new tasks. We conduct extensive experiments on two real-world datasets with nine tasks and show that \emph{Conure} largely exceeds the standard model that does not purposely preserve such old "knowledge", and performs competitively or sometimes better than models which are trained either individually for each task or simultaneously by merging all task data.

cs.IR↗