Searcharxiv⌕ Search

arXiv subjects

Ling Li

Publications and source records attributed to Ling Li.

At least 91 records · Page 5Linked to original sources
arXiv2023

Contrastive Modules with Temporal Attention for Multi-Task Reinforcement Learning

In the field of multi-task reinforcement learning, the modular principle, which involves specializing functionalities into different modules and combining them appropriately, has been widely adopted as a promising approach to prevent the negative transfer problem that performance degradation due to conflicts between tasks. However, most of the existing multi-task RL methods only combine shared modules at the task level, ignoring that there may be conflicts within the task. In addition, these methods do not take into account that without constraints, some modules may learn similar functions, resulting in restricting the model's expressiveness and generalization capability of modular methods. In this paper, we propose the Contrastive Modules with Temporal Attention(CMTA) method to address these limitations. CMTA constrains the modules to be different from each other by contrastive learning and combining shared modules at a finer granularity than the task level with temporal attention, alleviating the negative transfer within the task and improving the generalization ability and the performance for multi-task RL. We conducted the experiment on Meta-World, a multi-task RL benchmark containing various robotics manipulation tasks. Experimental results show that CMTA outperforms learning each task individually for the first time and achieves substantial performance improvements over the baselines.

cs.LG↗
arXiv2023

Efficient Symbolic Policy Learning with Differentiable Symbolic Expression

Deep reinforcement learning (DRL) has led to a wide range of advances in sequential decision-making tasks. However, the complexity of neural network policies makes it difficult to understand and deploy with limited computational resources. Currently, employing compact symbolic expressions as symbolic policies is a promising strategy to obtain simple and interpretable policies. Previous symbolic policy methods usually involve complex training processes and pre-trained neural network policies, which are inefficient and limit the application of symbolic policies. In this paper, we propose an efficient gradient-based learning method named Efficient Symbolic Policy Learning (ESPL) that learns the symbolic policy from scratch in an end-to-end way. We introduce a symbolic network as the search space and employ a path selector to find the compact symbolic policy. By doing so we represent the policy with a differentiable symbolic expression and train it in an off-policy manner which further improves the efficiency. In addition, in contrast with previous symbolic policies which only work in single-task RL because of complexity, we expand ESPL on meta-RL to generate symbolic policies for unseen tasks. Experimentally, we show that our approach generates symbolic policies with higher performance and greatly improves data efficiency for single-task RL. In meta-RL, we demonstrate that compared with neural network policies the proposed symbolic policy achieves higher performance and efficiency and shows the potential to be interpretable.

cs.LG↗
arXiv2023

A Generalization of the Tang-Ding Binary Cyclic Codes

Cyclic codes are an interesting family of linear codes since they have efficient decoding algorithms and contain optimal codes as subfamilies. Constructing infinite families of cyclic codes with good parameters is important in both theory and practice. Recently, Tang and Ding [IEEE Trans. Inf. Theory, vol. 68, no. 12, pp. 7842--7849, 2022] proposed an infinite family of binary cyclic codes with good parameters. Shi et al. [arXiv:2309.12003v1, 2023] developed the binary Tang-Ding codes to the $4$-ary case. Inspired by these two works, we study $2^s$-ary Tang-Ding codes, where $s\geq 2$. Good lower bounds on the minimum distance of the $2^s$-ary Tang-Ding codes are presented. As a by-product, an infinite family of $2^s$-ary duadic codes with a square-root like lower bound is presented.

cs.IT↗
arXiv2023

Learning Domain-Aware Detection Head with Prompt Tuning

Domain adaptive object detection (DAOD) aims to generalize detectors trained on an annotated source domain to an unlabelled target domain. However, existing methods focus on reducing the domain bias of the detection backbone by inferring a discriminative visual encoder, while ignoring the domain bias in the detection head. Inspired by the high generalization of vision-language models (VLMs), applying a VLM as the robust detection backbone following a domain-aware detection head is a reasonable way to learn the discriminative detector for each domain, rather than reducing the domain bias in traditional methods. To achieve the above issue, we thus propose a novel DAOD framework named Domain-Aware detection head with Prompt tuning (DA-Pro), which applies the learnable domain-adaptive prompt to generate the dynamic detection head for each domain. Formally, the domain-adaptive prompt consists of the domain-invariant tokens, domain-specific tokens, and the domain-related textual description along with the class label. Furthermore, two constraints between the source and target domains are applied to ensure that the domain-adaptive prompt can capture the domains-shared and domain-specific knowledge. A prompt ensemble strategy is also proposed to reduce the effect of prompt disturbance. Comprehensive experiments over multiple cross-domain adaptation tasks demonstrate that using the domain-adaptive prompt can produce an effectively domain-related detection head for boosting domain-adaptive object detection. Our code is available at https://github.com/Therock90421/DA-Pro.

cs.CV↗
arXiv2023

Universal discontinuous percolation transition in the Earth's terrestrial topography

Based on the Topographic Wetness Index (TWI), we studied the percolation process of water on the Earth's land surface and discovered a universal discontinuous phase transition across scales, with a critical TWI threshold of 0.671 (0.054). The discontinuity is attributed to the long-range correlation and directionality of the percolation process. Furthermore, the criticality is shown to extend from the critical point to a region corresponding to the Griffiths phase, where natural lake systems are found to develop, indicating the governess of self-organized criticality within the Earth system.

physics.geo-ph↗
arXiv2023

Converging trend of global urban land expansion sheds new light on sustainable development

Urban land growth presents a major sustainability challenge, yet its growth patterns and dynamics remain unclear. We quantified urban land evolution by analyzing its statistical distribution in 14 regions and countries over 29 years. The results show a converging temporal trend in urban land expansion from sub-country to global scales, characterized by a coherent shift of urban area distributions from initial power law to exponential distributions, with the consequences of reduced system stability and resilience, and increased exposure of urban populations to extreme heat and air pollution. These changes are attributed to the increased influence from external economies of scale associated with globalization and are predicted to intensify in the future. The findings will advance urban science and direct current land urbanization practices toward sustainable development, especially in developing regions and medium-size cities.

physics.soc-ph↗
arXiv2023

Self-driven Grounding: Large Language Model Agents with Automatical Language-aligned Skill Learning

Large language models (LLMs) show their powerful automatic reasoning and planning capability with a wealth of semantic knowledge about the human world. However, the grounding problem still hinders the applications of LLMs in the real-world environment. Existing studies try to fine-tune the LLM or utilize pre-defined behavior APIs to bridge the LLMs and the environment, which not only costs huge human efforts to customize for every single task but also weakens the generality strengths of LLMs. To autonomously ground the LLM onto the environment, we proposed the Self-Driven Grounding (SDG) framework to automatically and progressively ground the LLM with self-driven skill learning. SDG first employs the LLM to propose the hypothesis of sub-goals to achieve tasks and then verify the feasibility of the hypothesis via interacting with the underlying environment. Once verified, SDG can then learn generalized skills with the guidance of these successfully grounded subgoals. These skills can be further utilized to accomplish more complex tasks which fail to pass the verification phase. Verified in the famous instruction following task set-BabyAI, SDG achieves comparable performance in the most challenging tasks compared with imitation learning methods that cost millions of demonstrations, proving the effectiveness of learned skills and showing the feasibility and efficiency of our framework.

cs.CL↗
arXiv2023

The Ultraviolet to Mid-infrared Extinction Law of the Taurus Molecular Cloud Based on the Gaia DR3, GALEX, APASS, Pan-STARRS1, 2MASS, and WISE Surveys

Interstellar dust extinction law is essential for interpreting observations. In this work, we investigate the ultraviolet (UV)--mid-infrared (IR) extinction law of the Taurus molecular cloud and its possible variations. We select 504,988 dwarf stars (4200 K < Teff < 8000 K) and 4,757 giant stars (4200 K < Teff < 5200 K) based on the stellar parameters of Gaia DR3 as tracers. We establish the Teff--intrinsic color relations and determine the intrinsic color indices and color excesses for different types of stars. In the determination of color excess ratios (CERs), we analyze and correct the curvature of CERs and derive the UV--mid-IR CERs of 16 bands. We consider different effective wavelengths for different types of stars when converting CERs to relative extinction, and obtain the extinction law with a better wavelength resolution. In addition, we analyze the possible regional variation of extinction law and derive the average extinction law of Rv=3.13+-0.32 for the Taurus molecular cloud. Only 0.9% of subregions have deviations >3sigma, indicating limited regional variation in the extinction law. We also discuss the effect of Gaia Teff overestimation on the determination of the Taurus extinction law and find that the effect is negligible.

astro-ph.GA↗
arXiv2023

Pushing the Limits of Machine Design: Automated CPU Design with AI

Design activity -- constructing an artifact description satisfying given goals and constraints -- distinguishes humanity from other animals and traditional machines, and endowing machines with design abilities at the human level or beyond has been a long-term pursuit. Though machines have already demonstrated their abilities in designing new materials, proteins, and computer programs with advanced artificial intelligence (AI) techniques, the search space for designing such objects is relatively small, and thus, "Can machines design like humans?" remains an open question. To explore the boundary of machine design, here we present a new AI approach to automatically design a central processing unit (CPU), the brain of a computer, and one of the world's most intricate devices humanity have ever designed. This approach generates the circuit logic, which is represented by a graph structure called Binary Speculation Diagram (BSD), of the CPU design from only external input-output observations instead of formal program code. During the generation of BSD, Monte Carlo-based expansion and the distance of Boolean functions are used to guarantee accuracy and efficiency, respectively. By efficiently exploring a search space of unprecedented size 10^{10^{540}}, which is the largest one of all machine-designed objects to our best knowledge, and thus pushing the limits of machine design, our approach generates an industrial-scale RISC-V CPU within only 5 hours. The taped-out CPU successfully runs the Linux operating system and performs comparably against the human-designed Intel 80486SX CPU. In addition to learning the world's first CPU only from input-output observations, which may reform the semiconductor industry by significantly reducing the design cycle, our approach even autonomously discovers human knowledge of the von Neumann architecture.

cs.AI↗
arXiv2023

Size distributions reveal regime transition of lake systems under different dominant driving forces

Power law size distribution is found to associate with fractal, self-organized behaviors and patterns of complex systems. Such distribution also emerges from natural lakes, with potentially important links to the dynamics of lake systems. But the driving mechanism that generates and shapes this feature in lake systems remains unclear. Moreover, the power law itself was found inadequate for fully describing the size distribution of lakes, due to deviations at the two ends of size range. Based on observed and simulated lakes in 11 hydro-climatic zones of China, we established a conceptual model for lake systems, which covers the whole size range of lake size distribution and reveals the underlying driving mechanism. The full lake size distribution is composed of three components, with three phases featured by exponential, stretched-exponential and power law distribution. The three phases represent system states with successively increasing degrees of heterogeneity and orderliness, and more importantly, indicate the dominance of exogenic and endogenic forces, respectively. As the dominant driving force changes from endogenic to exogenic, a phase transition occurs with lake size distribution shifted from power law to stretched-exponential and further to exponential distribution. Apart from compressing the power law phase, exogenic force also increases its scaling exponent, driving the corresponding lake size power spectrum into the regime of blue noise. During this process, the autocorrelation function of the lake system diverges with a possibility of going to infinity, indicating the loss of system resilience.

cond-mat.stat-mech↗
arXiv2023

ALL-E: Aesthetics-guided Low-light Image Enhancement

Evaluating the performance of low-light image enhancement (LLE) is highly subjective, thus making integrating human preferences into image enhancement a necessity. Existing methods fail to consider this and present a series of potentially valid heuristic criteria for training enhancement models. In this paper, we propose a new paradigm, i.e., aesthetics-guided low-light image enhancement (ALL-E), which introduces aesthetic preferences to LLE and motivates training in a reinforcement learning framework with an aesthetic reward. Each pixel, functioning as an agent, refines itself by recursive actions, i.e., its corresponding adjustment curve is estimated sequentially. Extensive experiments show that integrating aesthetic assessment improves both subjective experience and objective evaluation. Our results on various benchmarks demonstrate the superiority of ALL-E over state-of-the-art methods.

cs.CV↗
arXiv2023

Binary stochasticity enabled highly efficient neuromorphic deep learning achieves better-than-software accuracy

Deep learning needs high-precision handling of forwarding signals, backpropagating errors, and updating weights. This is inherently required by the learning algorithm since the gradient descent learning rule relies on the chain product of partial derivatives. However, it is challenging to implement deep learning in hardware systems that use noisy analog memristors as artificial synapses, as well as not being biologically plausible. Memristor-based implementations generally result in an excessive cost of neuronal circuits and stringent demands for idealized synaptic devices. Here, we demonstrate that the requirement for high precision is not necessary and that more efficient deep learning can be achieved when this requirement is lifted. We propose a binary stochastic learning algorithm that modifies all elementary neural network operations, by introducing (i) stochastic binarization of both the forwarding signals and the activation function derivatives, (ii) signed binarization of the backpropagating errors, and (iii) step-wised weight updates. Through an extensive hybrid approach of software simulation and hardware experiments, we find that binary stochastic deep learning systems can provide better performance than the software-based benchmarks using the high-precision learning algorithm. Also, the binary stochastic algorithm strongly simplifies the neural network operations in hardware, resulting in an improvement of the energy efficiency for the multiply-and-accumulate operations by more than three orders of magnitudes.

cs.NE↗
arXiv2023

Hyperbolic Summation for Fractional Sums

Let $f(n)$ be an arithmetic function with $f(n) \ll n^α$ for some $α\in[0,1)$ and let $\lfloor .\rfloor $ denote the integer part function. In this paper, we evaluate asymptotically the sums $$\sum_{n_{1}n_{2}\leq x}f \left( \left\lfloor \frac{x}{n_{1}n_{2}} \right\rfloor \right),$$ we use the estimation of three-dimensional exponential sums due to Robert and Sargos.

math.NT↗
arXiv2023

Conceptual Reinforcement Learning for Language-Conditioned Tasks

Despite the broad application of deep reinforcement learning (RL), transferring and adapting the policy to unseen but similar environments is still a significant challenge. Recently, the language-conditioned policy is proposed to facilitate policy transfer through learning the joint representation of observation and text that catches the compact and invariant information across environments. Existing studies of language-conditioned RL methods often learn the joint representation as a simple latent layer for the given instances (episode-specific observation and text), which inevitably includes noisy or irrelevant information and cause spurious correlations that are dependent on instances, thus hurting generalization performance and training efficiency. To address this issue, we propose a conceptual reinforcement learning (CRL) framework to learn the concept-like joint representation for language-conditioned policy. The key insight is that concepts are compact and invariant representations in human cognition through extracting similarities from numerous instances in real-world. In CRL, we propose a multi-level attention encoder and two mutual information constraints for learning compact and invariant concepts. Verified in two challenging environments, RTFM and Messenger, CRL significantly improves the training efficiency (up to 70%) and generalization ability (up to 30%) to the new environment dynamics.

cs.LG↗
arXiv2023

Ultra-low Precision Multiplication-free Training for Deep Neural Networks

The training for deep neural networks (DNNs) demands immense energy consumption, which restricts the development of deep learning as well as increases carbon emissions. Thus, the study of energy-efficient training for DNNs is essential. In training, the linear layers consume the most energy because of the intense use of energy-consuming full-precision (FP32) multiplication in multiply-accumulate (MAC). The energy-efficient works try to decrease the precision of multiplication or replace the multiplication with energy-efficient operations such as addition or bitwise shift, to reduce the energy consumption of FP32 multiplications. However, the existing energy-efficient works cannot replace all of the FP32 multiplications during both forward and backward propagation with low-precision energy-efficient operations. In this work, we propose an Adaptive Layer-wise Scaling PoT Quantization (ALS-POTQ) method and a Multiplication-Free MAC (MF-MAC) to replace all of the FP32 multiplications with the INT4 additions and 1-bit XOR operations. In addition, we propose Weight Bias Correction and Parameterized Ratio Clipping techniques for stable training and improving accuracy. In our training scheme, all of the above methods do not introduce extra multiplications, so we reduce up to 95.8% of the energy consumption in linear layers during training. Experimentally, we achieve an accuracy degradation of less than 1% for CNN models on ImageNet and Transformer model on the WMT En-De task. In summary, we significantly outperform the existing methods for both energy efficiency and accuracy.

cs.LG↗
arXiv2023

Evaluating the Efficacy of Skincare Product: A Realistic Short-Term Facial Pore Simulation

Simulating the effects of skincare products on face is a potential new way to communicate the efficacy of skincare products in skin diagnostics and product recommendations. Furthermore, such simulations enable one to anticipate his/her skin conditions and better manage skin health. However, there is a lack of effective simulations today. In this paper, we propose the first simulation model to reveal facial pore changes after using skincare products. Our simulation pipeline consists of 2 steps: training data establishment and facial pore simulation. To establish training data, we collect face images with various pore quality indexes from short-term (8-weeks) clinical studies. People often experience significant skin fluctuations (due to natural rhythms, external stressors, etc.,), which introduces large perturbations in clinical data. To address this problem, we propose a sliding window mechanism to clean data and select representative index(es) to represent facial pore changes. Facial pore simulation stage consists of 3 modules: UNet-based segmentation module to localize facial pores; regression module to predict time-dependent warping hyperparameters; and deformation module, taking warping hyperparameters and pore segmentation labels as inputs, to precisely deform pores accordingly. The proposed simulation is able to render realistic facial pore changes. And this work will pave the way for future research in facial skin simulation and skincare product developments.

cs.CV↗
arXiv2023

Towards High Performance One-Stage Human Pose Estimation

Making top-down human pose estimation method present both good performance and high efficiency is appealing. Mask RCNN can largely improve the efficiency by conducting person detection and pose estimation in a single framework, as the features provided by the backbone are able to be shared by the two tasks. However, the performance is not as good as traditional two-stage methods. In this paper, we aim to largely advance the human pose estimation results of Mask-RCNN and still keep the efficiency. Specifically, we make improvements on the whole process of pose estimation, which contains feature extraction and keypoint detection. The part of feature extraction is ensured to get enough and valuable information of pose. Then, we introduce a Global Context Module into the keypoints detection branch to enlarge the receptive field, as it is crucial to successful human pose estimation. On the COCO val2017 set, our model using the ResNet-50 backbone achieves an AP of 68.1, which is 2.6 higher than Mask RCNN (AP of 65.5). Compared to the classic two-stage top-down method SimpleBaseline, our model largely narrows the performance gap (68.1 AP vs. 68.9 AP) with a much faster inference speed (77 ms vs. 168 ms), demonstrating the effectiveness of the proposed method. Code is available at: https://github.com/lingl_space/maskrcnn_keypoint_refined.

cs.CV↗
arXiv2022

Causality-driven Hierarchical Structure Discovery for Reinforcement Learning

Hierarchical reinforcement learning (HRL) effectively improves agents' exploration efficiency on tasks with sparse reward, with the guide of high-quality hierarchical structures (e.g., subgoals or options). However, how to automatically discover high-quality hierarchical structures is still a great challenge. Previous HRL methods can hardly discover the hierarchical structures in complex environments due to the low exploration efficiency by exploiting the randomness-driven exploration paradigm. To address this issue, we propose CDHRL, a causality-driven hierarchical reinforcement learning framework, leveraging a causality-driven discovery instead of a randomness-driven exploration to effectively build high-quality hierarchical structures in complicated environments. The key insight is that the causalities among environment variables are naturally fit for modeling reachable subgoals and their dependencies and can perfectly guide to build high-quality hierarchical structures. The results in two complex environments, 2D-Minecraft and Eden, show that CDHRL significantly boosts exploration efficiency with the causality-driven paradigm.

cs.LG↗