Searcharxiv⌕ Search

arXiv subjects

Yong Li

Publications and source records attributed to Yong Li.

At least 559 records · Page 31Linked to original sources
arXiv2023

A Practical Large-Scale Roadside Multi-View Multi-Sensor Spatial Synchronization Framework for Intelligent Transportation Systems

Spatial synchronization in roadside scenarios is essential for integrating data from multiple sensors at different locations. Current methods using cascading spatial transformation (CST) often lead to cumulative errors in large-scale deployments. Manual camera calibration is insufficient and requires extensive manual work, and existing methods are limited to controlled or single-view scenarios. To address these challenges, our research introduces a parallel spatial transformation (PST)-based framework for large-scale, multi-view, multi-sensor scenarios. PST parallelizes sensor coordinate system transformation, reducing cumulative errors. We incorporate deep learning for precise roadside monocular global localization, reducing manual work. Additionally, we use geolocation cues and an optimization algorithm for improved synchronization accuracy. Our framework has been tested in real-world scenarios, outperforming CST-based methods. It significantly enhances large-scale roadside multi-perspective, multi-sensor spatial synchronization, reducing deployment costs.

eess.SP↗
arXiv2023

ROAM: memory-efficient large DNN training via optimized operator ordering and memory layout

As deep learning models continue to increase in size, the memory requirements for training have surged. While high-level techniques like offloading, recomputation, and compression can alleviate memory pressure, they also introduce overheads. However, a memory-efficient execution plan that includes a reasonable operator execution order and tensor memory layout can significantly increase the models' memory efficiency and reduce overheads from high-level techniques. In this paper, we propose ROAM which operates on computation graph level to derive memory-efficient execution plan with optimized operator order and tensor memory layout for models. We first propose sophisticated theories that carefully consider model structure and training memory load to support optimization for large complex graphs that have not been well supported in the past. An efficient tree-based algorithm is further proposed to search task divisions automatically, along with delivering high performance and effectiveness to solve the problem. Experiments show that ROAM achieves a substantial memory reduction of 35.7%, 13.3%, and 27.2% compared to Pytorch and two state-of-the-art methods and offers a remarkable 53.7x speedup. The evaluation conducted on the expansive GPT2-XL further validates ROAM's scalability.

cs.LG↗
arXiv2023

Kolmogorov's Theorem for Degenerate Hamiltonian Systems with Continuous Parameters

In this paper, we investigate Kolmogorov type theorems for small perturbations of degenerate Hamiltonian systems. These systems are index by a parameter $ξ$ as \( H(y,x,ξ) = \langleω(ξ),y\rangle + \varepsilon P(y,x,ξ,\varepsilon) \) where $\varepsilon>0$. We assume that the frequency map, $ω$, is continuous with respect to $ξ$. Additionally, the perturbation function, $P(y,x,\cdot, \varepsilon)$, maintains Hölder continuity about $ξ$. We prove that persistent invariant tori retain the same frequency as those of the unperturbed tori, under certain topological degree conditions and a weak convexity condition for the frequency mapping. Notably, this paper presents, to our understanding, pioneering results on the KAM theorem under such conditions-with only assumption of continuous dependence of frequency mapping $ω$ on the parameter.

math.DS↗
arXiv2023

Similarity Between Two Stochastic Differential Systems

The main focus of this paper is to explore how much similarity between two stochastic differential systems. Motivated by the conjugate theory of stochastic dynamic systems, we study the relationship between two systems by finding homeomorphic mappings $K$. Particularly, we use the minimizer $K^*$ to measure the degree of similarity. Under appropriate assumptions, we give sufficient and necessary conditions for the existence of the minimizer $K^*$. The former result can be regarded as a strong law of large numbers, while the latter is a stochastic maximum principle. Finally, we provide different examples of stochastic systems and an application to stochastic Hartman Grobman theorem. Thus, the results illustrate what is the similarity, extending the conjugacy in stochastic dynamical systems.

math.DS↗
arXiv2023

Relation-aware Ensemble Learning for Knowledge Graph Embedding

Knowledge graph (KG) embedding is a fundamental task in natural language processing, and various methods have been proposed to explore semantic patterns in distinctive ways. In this paper, we propose to learn an ensemble by leveraging existing methods in a relation-aware manner. However, exploring these semantics using relation-aware ensemble leads to a much larger search space than general ensemble methods. To address this issue, we propose a divide-search-combine algorithm RelEns-DSC that searches the relation-wise ensemble weights independently. This algorithm has the same computation cost as general ensemble methods but with much better performance. Experimental results on benchmark datasets demonstrate the effectiveness of the proposed method in efficiently searching relation-aware ensemble weights and achieving state-of-the-art embedding performance. The code is public at https://github.com/LARS-research/RelEns.

cs.LG↗
arXiv2023

Similarity Between Two Dynamical Systems

The main focus of this paper is to explore how much similarity between two dynamical systems. Analogous to the classical Hartman-Grobman theorem, the relationship between two systems can be linked by a homeomorphic map $K$, and the core is to study the minimizer $K^*$ to measure the degree of similarity. We prove the sufficient conditions and necessary conditions (the maximum principle) for the existence of the minimizer $K^*$. Further, we establish similarity theorem based on the Takens embedding theorem. As applications, Lorenz system, Chua's circuit system and Chen's system are simulated and tested. The results illustrate what is the similarity, which extends the conjugacy in dynamical systems.

math.DS↗
arXiv2023

Single-shot Non-destructive Quantum Sensing for Gaseous Samples with Hundreds of Chiral Molecules

Chiral discrimination that is efficient to tiny amounts of chiral substances, especially at the single-molecule level, is highly demanded. Here, we propose a single-shot nondestructive quantum sensing method addressing such an issue. Our scheme consists of two steps. In the first step, the two enantiomers are prepared in different rotational states via microwave enantio-specific state transfer. Then, the chiral discrimination is transferred to quantum hypothesis testing. In the second step, we for the first time introduce a non-destructive quantum-state detection technique assisted with a microwave resonator to chiral discrimination, through which the molecular chirality is determined by the sign of the output signals. Using a typical chiral molecule, 1,2-propanediol, and an experimentally feasible model based on spherical Fabry-Pérot cavity, we show that the molecular chirality of slowly moving enantiopure gaseous samples with $10^2 - 10^3$ molecules can be highly credibly distinguished in a single-shot detection. By further trapping chiral molecules, it is promising to achieve chiral discrimination at the single molecule level by using our approach.

quant-ph↗
arXiv2023

Dual-stage Flows-based Generative Modeling for Traceable Urban Planning

Urban planning, which aims to design feasible land-use configurations for target areas, has become increasingly essential due to the high-speed urbanization process in the modern era. However, the traditional urban planning conducted by human designers can be a complex and onerous task. Thanks to the advancement of deep learning algorithms, researchers have started to develop automated planning techniques. While these models have exhibited promising results, they still grapple with a couple of unresolved limitations: 1) Ignoring the relationship between urban functional zones and configurations and failing to capture the relationship among different functional zones. 2) Less interpretable and stable generation process. To overcome these limitations, we propose a novel generative framework based on normalizing flows, namely Dual-stage Urban Flows (DSUF) framework. Specifically, the first stage is to utilize zone-level urban planning flows to generate urban functional zones based on given surrounding contexts and human guidance. Then we employ an Information Fusion Module to capture the relationship among functional zones and fuse the information of different aspects. The second stage is to use configuration-level urban planning flows to obtain land-use configurations derived from fused information. We design several experiments to indicate that our framework can outperform compared to other generative models for the urban planning task.

cs.LG↗
arXiv2023

A Hierarchy-based Analysis Approach for Blended Learning: A Case Study with Chinese Students

Blended learning is generally defined as the combination of traditional face-to-face learning and online learning. This learning mode has been widely used in advanced education across the globe due to the COVID-19 pandemic's social distance restriction as well as the development of technology. Online learning plays an important role in blended learning, and as it requires more student autonomy, the quality of blended learning in advanced education has been a persistent concern. Existing literature offers several elements and frameworks regarding evaluating the quality of blended learning. However, most of them either have different favours for evaluation perspectives or simply offer general guidance for evaluation, reducing the completeness, objectivity and practicalness of related works. In order to carry out a more intuitive and comprehensive evaluation framework, this paper proposes a hierarchy-based analysis approach. Applying gradient boosting model and feature importance evaluation method, this approach mainly analyses student engagement and its three identified dimensions (behavioral engagement, emotional engagement, cognitive engagement) to eliminate some existing stubborn problems when it comes to blended learning evaluation. The results show that cognitive engagement and emotional engagement play a more important role in blended learning evaluation, implying that these two should be considered to improve for better learning as well as teaching quality.

cs.CY↗
arXiv2023

Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity

With the fast growth of parameter size, it becomes increasingly challenging to deploy large generative models as they typically require large GPU memory consumption and massive computation. Unstructured model pruning has been a common approach to reduce both GPU memory footprint and the overall computation while retaining good model accuracy. However, the existing solutions do not provide a highly-efficient support for handling unstructured sparsity on modern GPUs, especially on the highly-structured Tensor Core hardware. Therefore, we propose Flash-LLM for enabling low-cost and highly-efficient large generative model inference with the sophisticated support of unstructured sparsity on high-performance but highly restrictive Tensor Cores. Based on our key observation that the main bottleneck of generative model inference is the several skinny matrix multiplications for which Tensor Cores would be significantly under-utilized due to low computational intensity, we propose a general Load-as-Sparse and Compute-as-Dense methodology for unstructured sparse matrix multiplication. The basic insight is to address the significant memory bandwidth bottleneck while tolerating redundant computations that are not critical for end-to-end performance on Tensor Cores. Based on this, we design an effective software framework for Tensor Core based unstructured SpMM, leveraging on-chip resources for efficient sparse data extraction and computation/memory-access overlapping. At SpMM kernel level, Flash-LLM significantly outperforms the state-of-the-art library, i.e., Sputnik and SparTA by an average of 2.9x and 1.5x, respectively. At end-to-end framework level on OPT-30B/66B/175B models, for tokens per GPU-second, Flash-LLM achieves up to 3.8x and 3.6x improvement over DeepSpeed and FasterTransformer, respectively, with significantly lower inference cost.

cs.DC↗
arXiv2023

Towards Generative Modeling of Urban Flow through Knowledge-enhanced Denoising Diffusion

Although generative AI has been successful in many areas, its ability to model geospatial data is still underexplored. Urban flow, a typical kind of geospatial data, is critical for a wide range of urban applications. Existing studies mostly focus on predictive modeling of urban flow that predicts the future flow based on historical flow data, which may be unavailable in data-sparse areas or newly planned regions. Some other studies aim to predict OD flow among regions but they fail to model dynamic changes of urban flow over time. In this work, we study a new problem of urban flow generation that generates dynamic urban flow for regions without historical flow data. To capture the effect of multiple factors on urban flow, such as region features and urban environment, we employ diffusion model to generate urban flow for regions under different conditions. We first construct an urban knowledge graph (UKG) to model the urban environment and relationships between regions, based on which we design a knowledge-enhanced spatio-temporal diffusion model (KSTDiff) to generate urban flow for each region. Specifically, to accurately generate urban flow for regions with different flow volumes, we design a novel diffusion process guided by a volume estimator, which is learnable and customized for each region. Moreover, we propose a knowledge-enhanced denoising network to capture the spatio-temporal dependencies of urban flow as well as the impact of urban environment in the denoising process. Extensive experiments on four real-world datasets validate the superiority of our model over state-of-the-art baselines in urban flow generation. Further in-depth studies demonstrate the utility of generated urban flow data and the ability of our model for long-term flow generation and urban flow prediction. Our code is released at: https://github.com/tsinghua-fib-lab/KSTDiff-Urban-flow-generation.

cs.AI↗
arXiv2023

Mutual Harmony: Sequential Recommendation with Dual Contrastive Network

With the outbreak of today's streaming data, the sequential recommendation is a promising solution to achieve time-aware personalized modeling. It aims to infer the next interacted item of a given user based on the historical item sequence. Some recent works tend to improve the sequential recommendation via random masking on the historical item so as to generate self-supervised signals. But such approaches will indeed result in sparser item sequence and unreliable signals. Besides, the existing sequential recommendation models are only user-centric, i.e., based on the historical items by chronological order to predict the probability of candidate items, which ignores whether the items from a provider can be successfully recommended. Such user-centric recommendation will make it impossible for the provider to expose their new items, failing to consider the accordant interactions between user and item dimensions. In this paper, we propose a novel Dual Contrastive Network (DCN) to achieve mutual harmony between user and item provider, generating ground-truth self-supervised signals for sequential recommendation by auxiliary user-sequence from an item-centric dimension. Specifically, we propose dual representation contrastive learning to refine the representation learning by minimizing the Euclidean distance between the representations of a given user/item and historical items/users of them. Before the second contrastive learning module, we perform the next user prediction to capture the trends of items preferred by certain types of users and provide personalized exploration opportunities for item providers. Finally, we further propose dual interest contrastive learning to self-supervise the dynamic interest from the next item/user prediction and static interest of matching probability. Experiments on four benchmark datasets verify the effectiveness of our proposed method.

cs.IR↗
arXiv2023

Physics-Informed Neural Networks for Prognostics and Health Management of Lithium-Ion Batteries

For Prognostics and Health Management (PHM) of Lithium-ion (Li-ion) batteries, many models have been established to characterize their degradation process. The existing empirical or physical models can reveal important information regarding the degradation dynamics. However, there are no general and flexible methods to fuse the information represented by those models. Physics-Informed Neural Network (PINN) is an efficient tool to fuse empirical or physical dynamic models with data-driven models. To take full advantage of various information sources, we propose a model fusion scheme based on PINN. It is implemented by developing a semi-empirical semi-physical Partial Differential Equation (PDE) to model the degradation dynamics of Li-ion batteries. When there is little prior knowledge about the dynamics, we leverage the data-driven Deep Hidden Physics Model (DeepHPM) to discover the underlying governing dynamic models. The uncovered dynamics information is then fused with that mined by the surrogate neural network in the PINN framework. Moreover, an uncertainty-based adaptive weighting method is employed to balance the multiple learning tasks when training the PINN. The proposed methods are verified on a public dataset of Li-ion Phosphate (LFP)/graphite batteries.

eess.SP↗
arXiv2023

Seismic Data Strong Noise Attenuation Based on Diffusion Model and Principal Component Analysis

Seismic data noise processing is an important part of seismic exploration data processing, and the effect of noise elimination is directly related to the follow-up processing of data. In response to this problem, many authors have proposed methods based on rank reduction, sparse transformation, domain transformation, and deep learning. However, such methods are often not ideal when faced with strong noise. Therefore, we propose to use diffusion model theory for noise removal. The Bayesian equation is used to reverse the noise addition process, and the noise reduction work is divided into multiple steps to effectively deal with high-noise situations. Furthermore, we propose to evaluate the noise level of blind Gaussian seismic data using principal component analysis to determine the number of steps for noise reduction processing of seismic data. We train the model on synthetic data and validate it on field data through transfer learning. Experiments show that our proposed method can identify most of the noise with less signal leakage. This has positive significance for high-precision seismic exploration and future seismic data signal processing research.

physics.geo-ph↗
arXiv2023

Lower dimensional invariant tori for multi-scale Hamiltonian systems

The ``Fundamental Theorem" given by Arnold in [2] asserts the persistence of full dimensional invariant tori for 2-scale Hamiltonian systems. However, persistence in multi-scale systems is much more complicated and difficult. In this paper, we explore the persistence of lower dimensional invariant tori for multi-scale Hamiltonian systems, which play an important role in dynamics of resonant Hamiltonian systems. Moreover, using the corresponding results we give a quasi-periodic Poincaré Theorem for multi-scale Hamiltonian systems, i.e., at least $2^{m_0}$ families resonant tori survive small perturbations, where the integer $m_0$ is the multiplicity of resonance.

math.DS↗
arXiv2023

Alleviating Video-Length Effect for Micro-video Recommendation

Micro-videos platforms such as TikTok are extremely popular nowadays. One important feature is that users no longer select interested videos from a set, instead they either watch the recommended video or skip to the next one. As a result, the time length of users' watching behavior becomes the most important signal for identifying preferences. However, our empirical data analysis has shown a video-length effect that long videos are easier to receive a higher value of average view time, thus adopting such view-time labels for measuring user preferences can easily induce a biased model that favors the longer videos. In this paper, we propose a Video Length Debiasing Recommendation (VLDRec) method to alleviate such an effect for micro-video recommendation. VLDRec designs the data labeling approach and the sample generation module that better capture user preferences in a view-time oriented manner. It further leverages the multi-task learning technique to jointly optimize the above samples with original biased ones. Extensive experiments show that VLDRec can improve the users' view time by 1.81% and 11.32% on two real-world datasets, given a recommendation list of a fixed overall video length, compared with the best baseline method. Moreover, VLDRec is also more effective in matching users' interests in terms of the video content.

cs.IR↗
arXiv2023

Central Similarity Multi-View Hashing for Multimedia Retrieval

Hash representation learning of multi-view heterogeneous data is the key to improving the accuracy of multimedia retrieval. However, existing methods utilize local similarity and fall short of deeply fusing the multi-view features, resulting in poor retrieval accuracy. Current methods only use local similarity to train their model. These methods ignore global similarity. Furthermore, most recent works fuse the multi-view features via a weighted sum or concatenation. We contend that these fusion methods are insufficient for capturing the interaction between various views. We present a novel Central Similarity Multi-View Hashing (CSMVH) method to address the mentioned problems. Central similarity learning is used for solving the local similarity problem, which can utilize the global similarity between the hash center and samples. We present copious empirical data demonstrating the superiority of gate-based fusion over conventional approaches. On the MS COCO and NUS-WIDE, the proposed CSMVH performs better than the state-of-the-art methods by a large margin (up to 11.41% mean Average Precision (mAP) improvement).

cs.CV↗