arXiv subjects

Xin Liu

Publications and source records attributed to Xin Liu.

At least 559 records · Page 31Linked to original sources

arXiv2024

Adam-family Methods for Nonsmooth Optimization with Convergence Guarantees

In this paper, we present a comprehensive study on the convergence properties of Adam-family methods for nonsmooth optimization, especially in the training of nonsmooth neural networks. We introduce a novel two-timescale framework that adopts a two-timescale updating scheme, and prove its convergence properties under mild assumptions. Our proposed framework encompasses various popular Adam-family methods, providing convergence guarantees for these methods in training nonsmooth neural networks. Furthermore, we develop stochastic subgradient methods that incorporate gradient clipping techniques for training nonsmooth neural networks with heavy-tailed noise. Through our framework, we show that our proposed methods converge even when the evaluation noises are only assumed to be integrable. Extensive numerical experiments demonstrate the high efficiency and robustness of our proposed methods.

math.OC↗

arXiv2024

Gait Event Detection and Travel Distance Using Waist-Worn Accelerometers across a Range of Speeds: Automated Approach

Estimation of temporospatial clinical features of gait (CFs), such as step count and length, step duration, step frequency, gait speed, and distance traveled, is an important component of community-based mobility evaluation using wearable accelerometers. However, accurate unsupervised computerized measurement of CFs of individuals with Duchenne muscular dystrophy (DMD) who have progressive loss of ambulatory mobility is difficult due to differences in patterns and magnitudes of acceleration across their range of attainable gait velocities. This paper proposes a novel calibration method. It aims to detect steps, estimate stride lengths, and determine travel distance. The approach involves a combination of clinical observation, machine-learning-based step detection, and regression-based stride length prediction. The method demonstrates high accuracy in children with DMD and typically developing controls (TDs) regardless of the participant's level of ability. Fifteen children with DMD and fifteen TDs underwent supervised clinical testing across a range of gait speeds using 10 m or 25 m run/walk (10 MRW, 25 MRW), 100 m run/walk (100 MRW), 6-min walk (6 MWT), and free-walk (FW) evaluations while wearing a mobile-phone-based accelerometer at the waist near the body's center of mass. Following calibration by a trained clinical evaluator, CFs were extracted from the accelerometer data using a multi-step machine-learning-based process and the results were compared to ground-truth observation data. Model predictions vs. observed values for step counts, distance traveled, and step length showed a strong correlation. Our study findings indicate that a single waist-worn accelerometer calibrated to an individual's stride characteristics using our methods accurately measures CFs and estimates travel distances across a common range of gait speeds in both DMD and TD peers.

eess.SP↗

arXiv2024

Spatio-Temporal Contrastive Learning Enhanced GNNs for Session-based Recommendation

Session-based recommendation (SBR) systems aim to utilize the user's short-term behavior sequence to predict the next item without the detailed user profile. Most recent works try to model the user preference by treating the sessions as between-item transition graphs and utilize various graph neural networks (GNNs) to encode the representations of pair-wise relations among items and their neighbors. Some of the existing GNN-based models mainly focus on aggregating information from the view of spatial graph structure, which ignores the temporal relations within neighbors of an item during message passing and the information loss results in a sub-optimal problem. Other works embrace this challenge by incorporating additional temporal information but lack sufficient interaction between the spatial and temporal patterns. To address this issue, inspired by the uniformity and alignment properties of contrastive learning techniques, we propose a novel framework called Session-based Recommendation with Spatio-Temporal Contrastive Learning Enhanced GNNs (RESTC). The idea is to supplement the GNN-based main supervised recommendation task with the temporal representation via an auxiliary cross-view contrastive learning mechanism. Furthermore, a novel global collaborative filtering graph (CFG) embedding is leveraged to enhance the spatial view in the main task. Extensive experiments demonstrate the significant performance of RESTC compared with the state-of-the-art baselines e.g., with an improvement as much as 27.08% gain on HR@20 and 20.10% gain on MRR@20.

cs.IR↗

arXiv2024

Detecting Quadratically Coupled Ultra-light Dark Matter with Stimulated Annihilation

Ultra-light Dark Matter (ULDM) is one of the most promising DM candidates. Due to the Bose enhancement, we find the annihilation rate of the ULDM in the presence of background photon radiation can be greatly enhanced and produce a distinctive reflected electromagnetic wave with an angular frequency equal to the ULDM mass. We propose to utilize such stimulated annihilation to probe the ULDM with the electromagnetic quadratic coupling by emitting a beam of radio into space. With a power of 50 MW emitter, we forecast the sensitivity of quadratic coupling in different local halo models for low-frequency radio telescopes, such as LOFAR, UTR-2 and ngLOBO.

hep-ph↗

arXiv2024

High-quality Extragalactic Legacy-field Monitoring (HELM) with DECam

High-quality Extragalactic Legacy-field Monitoring (HELM) is a long-term observing program that photometrically monitors several well-studied extragalactic legacy fields with the Dark Energy Camera (DECam) imager on the CTIO 4m Blanco telescope. Since Feb 2019, HELM has been monitoring regions within COSMOS, XMM-LSS, CDF-S, S-CVZ, ELAIS-S1, and SDSS Stripe 82 with few-day cadences in the $(u)gri(z)$ bands, over a collective sky area of $\sim 38$ deg${\rm ^2}$. The main science goal of HELM is to provide high-quality optical light curves for a large sample of active galactic nuclei (AGNs), and to build decades-long time baselines when combining past and future optical light curves in these legacy fields. These optical images and light curves will facilitate the measurements of AGN reverberation mapping lags, as well as studies of AGN variability and its dependences on accretion properties. In addition, the time-resolved and coadded DECam photometry will enable a broad range of science applications from galaxy evolution to time-domain science. We describe the design and implementation of the program and present the first data release that includes source catalogs and the first $\sim 3.5$ years of light curves during 2019A--2022A.

astro-ph.GA↗

arXiv2024

ChatGPT Evaluation on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse Relations

This paper aims to quantitatively evaluate the performance of ChatGPT, an interactive large language model, on inter-sentential relations such as temporal relations, causal relations, and discourse relations. Given ChatGPT's promising performance across various tasks, we proceed to carry out thorough evaluations on the whole test sets of 11 datasets, including temporal and causal relations, PDTB2.0-based, and dialogue-based discourse relations. To ensure the reliability of our findings, we employ three tailored prompt templates for each task, including the zero-shot prompt template, zero-shot prompt engineering (PE) template, and in-context learning (ICL) prompt template, to establish the initial baseline scores for all popular sentence-pair relation classification tasks for the first time. Through our study, we discover that ChatGPT exhibits exceptional proficiency in detecting and reasoning about causal relations, albeit it may not possess the same level of expertise in identifying the temporal order between two events. While it is capable of identifying the majority of discourse relations with existing explicit discourse connectives, the implicit discourse relation remains a formidable challenge. Concurrently, ChatGPT demonstrates subpar performance in the dialogue discourse parsing task that requires structural understanding in a dialogue before being aware of the discourse relation.

cs.CL↗

arXiv2024

Spatiotemporal optical vortices with controllable radial and azimuthal quantum numbers

Optical spatiotemporal vortices with transverse photon orbital angular momentum (OAM) have recently become a focal point of research. In this work we theoretically and experimentally investigate optical spatiotemporal vortices with radial and azimuthal quantum numbers, known as spatiotemporal Laguerre-Gaussian (STLG) wavepackets. These 3D wavepackets exhibit phase singularities and cylinder-shaped edge dislocations, resulting in a multi-ring topology in its spatiotemporal profile. Unlike conventional ST optical vortices, STLG wavepackets with non-zero p and l values carry a composite transverse OAM consisting of two directionally opposite components. We further demonstrate mode conversion between an STLG wavepacket and an ST Hermite-Gaussian wavepacket through the application of strong spatiotemporal astigmatism. The converted STHG wavepacket is de-coupled in intensity in space-time domain that can be utilized to implement the efficient and accurate recognition of ultrafast STLG wavepackets carried various p and l. This study may offer new insights into high-dimensional quantum information, photonic topology, and nonlinear optics, while promising potential applications in other wave phenomena such as acoustics and electron waves.

physics.optics↗

arXiv2024

Fast Adversarial Training against Textual Adversarial Attacks

Many adversarial defense methods have been proposed to enhance the adversarial robustness of natural language processing models. However, most of them introduce additional pre-set linguistic knowledge and assume that the synonym candidates used by attackers are accessible, which is an ideal assumption. We delve into adversarial training in the embedding space and propose a Fast Adversarial Training (FAT) method to improve the model robustness in the synonym-unaware scenario from the perspective of single-step perturbation generation and perturbation initialization. Based on the observation that the adversarial perturbations crafted by single-step and multi-step gradient ascent are similar, FAT uses single-step gradient ascent to craft adversarial examples in the embedding space to expedite the training process. Based on the observation that the perturbations generated on the identical training sample in successive epochs are similar, FAT fully utilizes historical information when initializing the perturbation. Extensive experiments demonstrate that FAT significantly boosts the robustness of BERT models in the synonym-unaware scenario, and outperforms the defense baselines under various attacks with character-level and word-level modifications.

cs.CL↗

arXiv2024

Experience-Learning Inspired Two-Step Reward Method for Efficient Legged Locomotion Learning Towards Natural and Robust Gaits

Multi-legged robots offer enhanced stability in complex terrains, yet autonomously learning natural and robust motions in such environments remains challenging. Drawing inspiration from animals' progressive learning patterns, from simple to complex tasks, we introduce a universal two-stage learning framework with two-step reward setting based on self-acquired experience, which efficiently enables legged robots to incrementally learn natural and robust movements. In the first stage, robots learn through gait-related rewards to track velocity on flat terrain, acquiring natural, robust movements and generating effective motion experience data. In the second stage, mirroring animal learning from existing experiences, robots learn to navigate challenging terrains with natural and robust movements using adversarial imitation learning. To demonstrate our method's efficacy, we trained both quadruped robots and a hexapod robot, and the policy were successfully transferred to a physical quadruped robot GO1, which exhibited natural gait patterns and remarkable robustness in various terrains.

cs.RO↗

arXiv2024

A Decentralized Proximal Gradient Tracking Algorithm for Composite Optimization on Riemannian Manifolds

This paper focuses on minimizing a smooth function combined with a nonsmooth regularization term on a compact Riemannian submanifold embedded in the Euclidean space under a decentralized setting. Typically, there are two types of approaches at present for tackling such composite optimization problems. The first, subgradient-based approaches, rely on subgradient information of the objective function to update variables, achieving an iteration complexity of $\mathcal{O}(ε^{-4}\log^2(ε^{-2}))$. The second, smoothing approaches, involve constructing a smooth approximation of the nonsmooth regularization term, resulting in an iteration complexity of $\mathcal{O}(ε^{-4})$. This paper proposes a proximal gradient type algorithm that fully exploits the composite structure. The global convergence to a stationary point is established with a significantly improved iteration complexity of $\mathcal{O}(ε^{-2})$. To validate the effectiveness and efficiency of our proposed method, we present numerical results in real-world applications, showcasing its superior performance.

math.OC↗

arXiv2024

Deep learning and random light structuring ensure robust free-space communications

Having shown early promise, free-space optical communications (FSO) face formidable challenges in the age of information explosion. The ever-growing demand for greater channel communication capacity is one of the challenges. The inter-channel crosstalk, which severely degrades the quality of transmitted information, creates another roadblock in the way of efficient FSO implementation. Here we advance theoretically and realize experimentally a potentially high-capacity FSO protocol that enables high-fidelity transfer of an image, or set of images through a complex environment. In our protocol, we complement random light structuring at the transmitter with a deep learning image classification platform at the receiver. Multiplexing novel, independent, mutually orthogonal degrees of freedom available to structured random light can potentially significantly boost the channel communication capacity of our protocol without introducing any deleterious crosstalk. Specifically, we show how one can multiplex the degrees of freedom associated with the source coherence radius and a spatial position of a beamlet within an array of structured random beams to greatly enhance the capacity of our communication link. The superb resilience of structured random light to environmental noise, as well as extreme efficiency of deep learning networks at classifying images guarantees high-fidelity image transfer within the framework of our protocol.

physics.optics↗

arXiv2024

Validating quantum-supremacy experiments with exact and fast tensor network contraction

The quantum supremacy experiment, such as Google Sycamore [Nature \textbf{574}, 505 (2019)], poses great challenge for classical verification due to the exponentially-increasing compute cost. Using a new-generation Sunway supercomputer within $8.5$ days, we provide a direct verification by computing three million exact amplitudes for the experimentally generated bitstrings, obtaining an XEB fidelity of $0.191\%$ (the estimated value is $0.224\%$). The leap of simulation capability is built on a multiple-amplitude tensor network contraction algorithm which systematically exploits the ``classical advantage" (the inherent ``store-and-compute" operation mode of von Neumann machines) of current supercomputers, and a fused tensor network contraction algorithm which drastically increases the compute efficiency on heterogeneous architectures. Our method has a far-reaching impact in solving quantum many-body problems, statistical problems as well as combinatorial optimization problems.

quant-ph↗

arXiv2024

Time-varying k-domain modulation around a point sink in time reversal cavity

This paper derives and investigates a time-varying k-domain modulation in vector form using a time-reversal (TR) field decomposition theory proposed for the first time. First, the proposed theory illustrates that the TR field can exhibit super-resolution property from the perspective of spatial focusing pattern if a point sink is set at the initial source point. Afterward, the instantaneous TR fields with and without the point sink, as well as their k-domain patterns are compared to illustrate the time-varying k-domain modulation, which accounts for the superresolution property. The phenomenon observed and derived in this paper shows great potential in the applications empowered by super-resolution focusing, such as wireless communication carrying the subwavelength information and high spatial resolution wireless power transfer.

physics.optics↗

arXiv2024

Local well-posedness and global stability of one-dimensional shallow water equations with surface tension and constant contact angle

We consider the one-dimensional shallow water problem with capillary surfaces and moving contact {lines}. An energy-based model is derived from the two-dimensional water wave equations, where we explicitly discuss the case of a stationary force balance at a moving contact line and highlight necessary changes to consider dynamic contact angles. The moving contact line becomes our free boundary at the level of shallow water equations, and the depth of the shallow water degenerates near the free boundary, which causes singularities for the derivatives and degeneracy for the viscosity. This is similar to the physical vacuum of compressible flows in the literature. The equilibrium, the global stability of the equilibrium, and the local well-posedness theory are established in this paper.

math.AP↗

arXiv2024

Flexomagnetoelectric effect in Sr2IrO4 thin films

Symmetry engineering is explicitly effective to manipulate and even create phases and orderings in strongly correlated materials. Flexural stress is universally practical to break the space-inversion or time-reversal symmetry. Here, by introducing strain gradient in a centrosymmetric antiferromagnet Sr2IrO4, the space-inversion symmetry is broken accompanying a non-equivalent O p-Ir d orbital hybridization along z axis. Thus, emergent polar phase and out-of-plane magnetic moment have been simultaneously observed in these asymmetric Sr2IrO4 thin films, which both are absent in its ground state. Furthermore, upon the application of magnetic field, such polarization can be controlled by modifying the occupied d orbitals through spin-orbit interaction, giving rise to a flexomagnetoelectric effect. This work provides a general strategy to artificially design multiple symmetries and ferroic orderings in strongly correlated systems.

cond-mat.str-el↗

arXiv2024

Enhancing RBF-FD Efficiency for Highly Non-Uniform Node Distributions via Adaptivity

Radial basis function generated finite-difference (RBF-FD) methods have recently gained popularity due to their flexibility with irregular node distributions. However, the convergence theories in the literature, when applied to nonuniform node distributions, require shrinking fill distance and do not take advantage of areas with high data density. Non-adaptive approach using same stencil size and degree of appended polynomial will have higher local accuracy at high density region, but has no effect on the overall order of convergence and could be a waste of computational power. This work proposes an adaptive RBF-FD method that utilizes the local data density to achieve a desirable order accuracy. By performing polynomial refinement and using adaptive stencil size based on data density, the adaptive RBF-FD method yields differentiation matrices with higher sparsity while achieving the same user-specified convergence order for nonuniform point distributions. This allows the method to better leverage regions with higher node density, maintaining both accuracy and efficiency compared to standard non-adaptive RBF-FD methods.

math.NA↗

arXiv2024

An Inexact Preconditioned Zeroth-order Proximal Method for Composite Optimization

In this paper, we consider the composite optimization problem, where the objective function integrates a continuously differentiable loss function with a nonsmooth regularization term. Moreover, only the function values for the differentiable part of the objective function are available. To efficiently solve this composite optimization problem, we propose a preconditioned zeroth-order proximal gradient method in which the gradients and preconditioners are estimated by finite-difference schemes based on the function values at the same trial points. We establish the global convergence and worst-case complexity for our proposed method. Numerical experiments exhibit the superiority of our developed method.

math.OC↗

arXiv2024

DeepSeek LLM: Scaling Open-Source Language Models with Longtermism

The rapid development of open-source large language models (LLMs) has been truly remarkable. However, the scaling law described in previous literature presents varying conclusions, which casts a dark cloud over scaling LLMs. We delve into the study of scaling laws and present our distinctive findings that facilitate scaling of large scale models in two commonly used open-source configurations, 7B and 67B. Guided by the scaling laws, we introduce DeepSeek LLM, a project dedicated to advancing open-source language models with a long-term perspective. To support the pre-training phase, we have developed a dataset that currently consists of 2 trillion tokens and is continuously expanding. We further conduct supervised fine-tuning (SFT) and Direct Preference Optimization (DPO) on DeepSeek LLM Base models, resulting in the creation of DeepSeek Chat models. Our evaluation results demonstrate that DeepSeek LLM 67B surpasses LLaMA-2 70B on various benchmarks, particularly in the domains of code, mathematics, and reasoning. Furthermore, open-ended evaluations reveal that DeepSeek LLM 67B Chat exhibits superior performance compared to GPT-3.5.

cs.CL↗