arXiv subjects

Yang Li

Publications and source records attributed to Yang Li.

At least 811 records · Page 45Linked to original sources

arXiv2024

Electrical Load Forecasting Model Using Hybrid LSTM Neural Networks with Online Correction

Accurate electrical load forecasting is of great importance for the efficient operation and control of modern power systems. In this work, a hybrid long short-term memory (LSTM)-based model with online correction is developed for day-ahead electrical load forecasting. Firstly, four types of features are extracted from the original electrical load dataset, including the historical time series, time index features, historical statistical features, and similarity features. Then, a hybrid LSTM-based electrical load forecasting model is designed, where an LSTM neural network block and a fully-connected neural network block are integrated that can model both temporal features (historical time series) and non-temporal features (the rest features). A gradient regularization-based offline training algorithm and an output layer parameter fine-tuning-based online model correction method are developed to enhance the model's capabilities to defend against disturbance and adapt to the latest load data distribution, thus improving the forecasting accuracy. At last, extensive experiments are carried out to validate the effectiveness of the proposed electrical load forecasting strategy with superior accuracy compared with commonly used forecasting models.

eess.SY↗

arXiv2024

A Unified Model for Active Battery Equalization Systems

Lithium-ion battery packs demand effective active equalization systems to enhance their usable capacity and lifetime. Despite numerous topologies and control schemes proposed in the literature, conducting quantitative analyses, comprehensive comparisons, and systematic optimization of their performance remains challenging due to the absence of a unified mathematical model at the pack level. To address this gap, we introduce a novel, hypergraph-based approach to establish the first unified model for various active battery equalization systems. This model reveals the intrinsic relationship between battery cells and equalizers by representing them as the vertices and hyperedges of hypergraphs, respectively. With the developed model, we identify the necessary condition for all equalization systems to achieve balance through controllability analysis, offering valuable insights for selecting the number of equalizers. Moreover, we prove that the battery equalization time is inversely correlated with the second smallest eigenvalue of the hypergraph's Laplacian matrix of each equalization system. This significantly simplifies the selection and optimized design of equalization systems, obviating the need for extensive experiments or simulations to derive the equalization time. Illustrative results demonstrate the efficiency of the proposed model and validate our findings.

eess.SY↗

arXiv2024

High-Rate 16-node quantum access network based on passive optical network

Quantum key distribution can provide information-theoretical secure communication, which is now heading towards building the quantum secure network for real-world applications. In most built quantum secure networks, point-to-multipoint (PTMP) topology is one of the most popular schemes, especially for quantum access networks. However, due to the lack of custom protocols with high secret key rate and compatible with classical optical networks for PTMP scheme, there is still no efficient way for a high-performance quantum access network with a multitude of users. Here, we report an experimental demonstration of a high-rate 16-nodes quantum access network based on passive optical network, where a high-efficient coherent-state PTMP protocol is novelly designed to allow independent secret key generation between one transmitter and multiple receivers concurrently. Such accomplishment is attributed to a well-designed real-time shot-noise calibration method, a series of advanced digital signal processing algorithms and a flexible post-processing strategy with high success probability. Finally, the experimental results show that the average secret key rate is around 2.086 Mbps between the transmitter and each user, which is two orders of magnitude higher than previous demonstrations. With the advantages of low cost, excellent compatibility, and wide bandwidth, our work paves the way for building practical PTMP quantum access networks, thus constituting an important step towards scalable quantum secure networks.

quant-ph↗

arXiv2024

Pyramid Feature Attention Network for Monocular Depth Prediction

Deep convolutional neural networks (DCNNs) have achieved great success in monocular depth estimation (MDE). However, few existing works take the contributions for MDE of different levels feature maps into account, leading to inaccurate spatial layout, ambiguous boundaries and discontinuous object surface in the prediction. To better tackle these problems, we propose a Pyramid Feature Attention Network (PFANet) to improve the high-level context features and low-level spatial features. In the proposed PFANet, we design a Dual-scale Channel Attention Module (DCAM) to employ channel attention in different scales, which aggregate global context and local information from the high-level feature maps. To exploit the spatial relationship of visual features, we design a Spatial Pyramid Attention Module (SPAM) which can guide the network attention to multi-scale detailed information in the low-level feature maps. Finally, we introduce scale-invariant gradient loss to increase the penalty on errors in depth-wise discontinuous regions. Experimental results show that our method outperforms state-of-the-art methods on the KITTI dataset.

cs.CV↗

arXiv2024

Special Kähler geometry and holomorphic Lagrangian fibrations

Given a holomorphic Lagrangian fibration of a compact hyperkahler manifold, we use the differential geometry of the special Kahler metric that exists on the base away from the discriminant locus, and show that the pullback of the tangent bundle of the base to the total space of a family of minimal rational curves admits a parallel splitting. The splitting is nontrivial when the base is not half-dimensional projective space. Combining this with results of Voisin, Hwang and Bakker-Schnell, we deduce that the base must be projective space, a result first proved by Hwang.

math.AG↗

arXiv2024

Practical Transferability Estimation for Image Classification Tasks

Transferability estimation is an essential problem in transfer learning to predict how good the performance is when transferring a source model (or source task) to a target task. Recent analytical transferability metrics have been widely used for source model selection and multi-task learning. A major challenge is how to make transfereability estimation robust under the cross-domain cross-task settings. The recently proposed OTCE score solves this problem by considering both domain and task differences, with the help of transfer experiences on auxiliary tasks, which causes an efficiency overhead. In this work, we propose a practical transferability metric called JC-NCE score that dramatically improves the robustness of the task difference estimation in OTCE, thus removing the need for auxiliary tasks. Specifically, we build the joint correspondences between source and target data via solving an optimal transport problem with a ground cost considering both the sample distance and label distance, and then compute the transferability score as the negative conditional entropy of the matched labels. Extensive validations under the intra-dataset and inter-dataset transfer settings demonstrate that our JC-NCE score outperforms the auxiliary-task free version of OTCE for 7% and 12%, respectively, and is also more robust than other existing transferability metrics on average.

cs.CV↗

arXiv2024

Transferability Estimation for Semantic Segmentation Task

Transferability estimation is a fundamental problem in transfer learning to predict how good the performance is when transferring a source model (or source task) to a target task. With the guidance of transferability score, we can efficiently select the highly transferable source models without performing the real transfer in practice. Recent analytical transferability metrics are mainly designed for image classification problem, and currently there is no specific investigation for the transferability estimation of semantic segmentation task, which is an essential problem in autonomous driving, medical image analysis, etc. Consequently, we further extend the recent analytical transferability metric OTCE (Optimal Transport based Conditional Entropy) score to the semantic segmentation task. The challenge in applying the OTCE score is the high dimensional segmentation output, which is difficult to find the optimal coupling between so many pixels under an acceptable computation cost. Thus we propose to randomly sample N pixels for computing OTCE score and take the expectation over K repetitions as the final transferability score. Experimental evaluation on Cityscapes, BDD100K and GTA5 datasets demonstrates that the OTCE score highly correlates with the transfer performance.

cs.CV↗

arXiv2024

Transferability-Guided Cross-Domain Cross-Task Transfer Learning

We propose two novel transferability metrics F-OTCE (Fast Optimal Transport based Conditional Entropy) and JC-OTCE (Joint Correspondence OTCE) to evaluate how much the source model (task) can benefit the learning of the target task and to learn more transferable representations for cross-domain cross-task transfer learning. Unlike the existing metric that requires evaluating the empirical transferability on auxiliary tasks, our metrics are auxiliary-free such that they can be computed much more efficiently. Specifically, F-OTCE estimates transferability by first solving an Optimal Transport (OT) problem between source and target distributions, and then uses the optimal coupling to compute the Negative Conditional Entropy between source and target labels. It can also serve as a loss function to maximize the transferability of the source model before finetuning on the target task. Meanwhile, JC-OTCE improves the transferability robustness of F-OTCE by including label distances in the OT problem, though it may incur additional computation cost. Extensive experiments demonstrate that F-OTCE and JC-OTCE outperform state-of-the-art auxiliary-free metrics by 18.85% and 28.88%, respectively in correlation coefficient with the ground-truth transfer accuracy. By eliminating the training cost of auxiliary tasks, the two metrics reduces the total computation time of the previous method from 43 minutes to 9.32s and 10.78s, respectively, for a pair of tasks. When used as a loss function, F-OTCE shows consistent improvements on the transfer accuracy of the source model in few-shot classification experiments, with up to 4.41% accuracy gain.

cs.CV↗

arXiv2024

Stacking faults enabled second harmonic generation in centrosymmetric van der Waals RhI3

Second harmonic generation (SHG) in van der Waals (vdWs) materials has garnered significant attention due to its potential for integrated nonlinear optical and optoelectronic applications. Stacking faults in vdWs materials, a typical kind of planar defect, can introduce a new degree of freedom to modulate the crystal symmetry and resultant SHG response, however, the physical origin and tunability of stacking-fault-governed SHG in vdWs materials remain unclear. Here, taking the intrinsically centrosymmetric vdWs RhI3 as an example, we theoretically reveal the origin of stacking-fault-governed SHG response, where the SHG response comes from the energetically favorable AC- Cstacking fault of which the electrical transitions along the high symmetry paths Gamma-M and Gamma-K in the Brillion zone play the dominant role at 810 nm. Such stacking-fault-governed SHG response is further confirmed via structural characterizations and SHG measurements. Furthermore, by applying hydrostatic pressure on RhI3, the correlation between structural evolution and SHG response is revealed with SHG enhancement up to 6.9 times, where the decreased electronic transition energies and huger momentum matrix elements due to the stronger interlayer interactions upon compression magnify the SHG susceptibility. This study develops a promising foundation based on strategically designed stacking faults for pioneering new avenues in nonlinear nano-optics.

physics.optics↗

arXiv2024

Cooperative Open-ended Learning Framework for Zero-shot Coordination

Zero-shot coordination in cooperative artificial intelligence (AI) remains a significant challenge, which means effectively coordinating with a wide range of unseen partners. Previous algorithms have attempted to address this challenge by optimizing fixed objectives within a population to improve strategy or behaviour diversity. However, these approaches can result in a loss of learning and an inability to cooperate with certain strategies within the population, known as cooperative incompatibility. To address this issue, we propose the Cooperative Open-ended LEarning (COLE) framework, which constructs open-ended objectives in cooperative games with two players from the perspective of graph theory to assess and identify the cooperative ability of each strategy. We further specify the framework and propose a practical algorithm that leverages knowledge from game theory and graph theory. Furthermore, an analysis of the learning process of the algorithm shows that it can efficiently overcome cooperative incompatibility. The experimental results in the Overcooked game environment demonstrate that our method outperforms current state-of-the-art methods when coordinating with different-level partners. Our demo is available at https://sites.google.com/view/cole-2023.

cs.AI↗

arXiv2024

Rare events in a stochastic vegetation-water dynamical system based on machine learning

Stochastic vegetation-water dynamical systems play a pivotal role in ecological stability, biodiversity, water resource management, and adaptation to climate change. This research proposes a machine learning-based method for analyzing rare events in stochastic vegetation-water dynamical systems with multiplicative Gaussian noise. Utilizing the Freidlin-Wentzell large deviation theory, we derive the asymptotic expressions for the quasipotential and the mean first exit time. Based on the decomposition of vector field, we design a neural network architecture to compute the most probable transition paths and the mean first exit time for both non-characteristic and characteristic boundary scenarios. The results indicate that this method can effectively predict early warnings of vegetation degradation, providing new theoretical foundations and mathematical tools for ecological management and conservation. Moreover, the method offers new possibilities for exploring more complex and higher-dimensional stochastic dynamical systems.

math.DS↗

arXiv2024

HPE Transformer: Learning to Optimize Multi-Group Multicast Beamforming Under Nonconvex QoS Constraints

This paper studies the quality-of-service (QoS) constrained multi-group multicast beamforming design problem, where each multicast group is composed of a number of users requiring the same content. Due to the nonconvex QoS constraints, this problem is nonconvex and NP-hard. While existing optimization-based iterative algorithms can obtain a suboptimal solution, their iterative nature results in large computational complexity and delay. To facilitate real-time implementations, this paper proposes a deep learning-based approach, which consists of a beamforming structure assisted problem transformation and a customized neural network architecture named hierarchical permutation equivariance (HPE) transformer. The proposed HPE transformer is proved to be permutation equivariant with respect to the users within each multicast group, and also permutation equivariant with respect to different multicast groups. Simulation results demonstrate that the proposed HPE transformer outperforms state-of-the-art optimization-based and deep learning-based approaches for multi-group multicast beamforming design in terms of the total transmit power, the constraint violation, and the computational time. In addition, the proposed HPE transformer achieves pretty good generalization performance on different numbers of users, different numbers of multicast groups, and different signal-to-interference-plus-noise ratio targets.

cs.IT↗

arXiv2024

Unsupervised Text Style Transfer via LLMs and Attention Masking with Multi-way Interactions

Unsupervised Text Style Transfer (UTST) has emerged as a critical task within the domain of Natural Language Processing (NLP), aiming to transfer one stylistic aspect of a sentence into another style without changing its semantics, syntax, or other attributes. This task is especially challenging given the intrinsic lack of parallel text pairings. Among existing methods for UTST tasks, attention masking approach and Large Language Models (LLMs) are deemed as two pioneering methods. However, they have shortcomings in generating unsmooth sentences and changing the original contents, respectively. In this paper, we investigate if we can combine these two methods effectively. We propose four ways of interactions, that are pipeline framework with tuned orders; knowledge distillation from LLMs to attention masking model; in-context learning with constructed parallel examples. We empirically show these multi-way interactions can improve the baselines in certain perspective of style strength, content preservation and text fluency. Experiments also demonstrate that simply conducting prompting followed by attention masking-based revision can consistently surpass the other systems, including supervised text style transfer systems. On Yelp-clean and Amazon-clean datasets, it improves the previously best mean metric by 0.5 and 3.0 absolute percentages respectively, and achieves new SOTA results.

cs.CL↗

arXiv2024

Continuous-variable quantum key distribution over 28.6 km fiber with an integrated silicon photonic receiver chip

Quantum key distribution, which ensures information-theoretically secret key generation, is currently advancing through photonic integration to achieve high performance, cost reduction and compact size, thereby facilitating the large-scale deployment. Continuous-variable quantum key distribution is an attractive approach for photonic integrations due to its compatibility with off-the-shelf optical communication devices. However, its chip-based systems have encountered significant limitations primarily related to the shot-noise-limited receiver design, which demands low noise, wide bandwidth, high clearance and well stability. Here, we report the implementation of a real local oscillator continuous-variable quantum key distribution system with an integrated silicon photonic receiver chip. Thanks to the well-designed chip-based homodyne detectors with a bandwidth up to 1.5 GHz and a clearance up to 7.42 dB, the transmission distance of the system has been extended to 28.6 km, achieving a secret key generation rate of Mbps level. This technological advancement enables the quantum key distribution systems with photonic integrated receivers to achieve the coverage in both access network scenarios and short-distance metropolitan interconnections, paving the way for the development of the next-generation quantum key distribution networks on a large scale.

quant-ph↗

arXiv2024

FormalGeo: An Extensible Formalized Framework for Olympiad Geometric Problem Solving

This is the first paper in a series of work we have accomplished over the past three years. In this paper, we have constructed a consistent formal plane geometry system. This will serve as a crucial bridge between IMO-level plane geometry challenges and readable AI automated reasoning. Within this formal framework, we have been able to seamlessly integrate modern AI models with our formal system. AI is now capable of providing deductive reasoning solutions to IMO-level plane geometry problems, just like handling other natural languages, and these proofs are readable, traceable, and verifiable. We propose the geometry formalization theory (GFT) to guide the development of the geometry formal system. Based on the GFT, we have established the FormalGeo, which consists of 88 geometric predicates and 196 theorems. It can represent, validate, and solve IMO-level geometry problems. we also have crafted the FGPS (formal geometry problem solver) in Python. It serves as both an interactive assistant for verifying problem-solving processes and an automated problem solver. We've annotated the formalgeo7k and formalgeo-imo datasets. The former contains 6,981 (expand to 133,818 through data augmentation) geometry problems, while the latter includes 18 (expand to 2,627 and continuously increasing) IMO-level challenging geometry problems. All annotated problems include detailed formal language descriptions and solutions. Implementation of the formal system and experiments validate the correctness and utility of the GFT. The backward depth-first search method only yields a 2.42% problem-solving failure rate, and we can incorporate deep learning techniques to achieve lower one. The source code of FGPS and datasets are available at https://github.com/BitSecret/FGPS.

cs.AI↗

arXiv2024

BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data

We introduce a text-to-speech (TTS) model called BASE TTS, which stands for $\textbf{B}$ig $\textbf{A}$daptive $\textbf{S}$treamable TTS with $\textbf{E}$mergent abilities. BASE TTS is the largest TTS model to-date, trained on 100K hours of public domain speech data, achieving a new state-of-the-art in speech naturalness. It deploys a 1-billion-parameter autoregressive Transformer that converts raw texts into discrete codes ("speechcodes") followed by a convolution-based decoder which converts these speechcodes into waveforms in an incremental, streamable manner. Further, our speechcodes are built using a novel speech tokenization technique that features speaker ID disentanglement and compression with byte-pair encoding. Echoing the widely-reported "emergent abilities" of large language models when trained on increasing volume of data, we show that BASE TTS variants built with 10K+ hours and 500M+ parameters begin to demonstrate natural prosody on textually complex sentences. We design and share a specialized dataset to measure these emergent abilities for text-to-speech. We showcase state-of-the-art naturalness of BASE TTS by evaluating against baselines that include publicly available large-scale text-to-speech systems: YourTTS, Bark and TortoiseTTS. Audio samples generated by the model can be heard at https://amazon-ltts-paper.com/.

cs.LG↗

arXiv2024

Instantaneous Quasiphoton in Strong-Field Ionization

Photon is a concept that does not apply at the instantaneous level when light is described by classical electromagnetic fields. Exploiting the dynamical rotational symmetry of circularly or elliptically polarized classical light pulses, however, we demonstrate the existence of instantaneous quasiphotons down to the subcycle level. We illustrate the concept of quasiphotons in strong-field ionization through the correlated spectrum of angular momentum and energy of photoelectrons, both at the tunnel exit and in the asymptotic region. Moreover, we propose a protocol based on electron vortices to directly visualize the existence of quasiphotons. Our work paves the pathway towards a deeper understanding of fundamental light-matter interactions with photonic characteristics on the subcycle scale.

physics.atom-ph↗

arXiv2024

CCPrefix: Counterfactual Contrastive Prefix-Tuning for Many-Class Classification

Recently, prefix-tuning was proposed to efficiently adapt pre-trained language models to a broad spectrum of natural language classification tasks. It leverages soft prefix as task-specific indicators and language verbalizers as categorical-label mentions to narrow the formulation gap from pre-training language models. However, when the label space increases considerably (i.e., many-class classification), such a tuning technique suffers from a verbalizer ambiguity problem since the many-class labels are represented by semantic-similar verbalizers in short language phrases. To overcome this, inspired by the human-decision process that the most ambiguous classes would be mulled over for each instance, we propose a brand-new prefix-tuning method, Counterfactual Contrastive Prefix-tuning (CCPrefix), for many-class classification. Basically, an instance-dependent soft prefix, derived from fact-counterfactual pairs in the label space, is leveraged to complement the language verbalizers in many-class classification. We conduct experiments on many-class benchmark datasets in both the fully supervised setting and the few-shot setting, which indicates that our model outperforms former baselines.

cs.CL↗