arXiv subjects

Liang Li

Publications and source records attributed to Liang Li.

At least 181 records · Page 10Linked to original sources

arXiv2024

Thermodynamic origin of the phonon Hall effect in a honeycomb antiferromagnet

The underlying mechanism of the thermal Hall effect (THE) generated by phonons in a variety of insulators is yet to be identified. Here, we report on a sizeable thermal Hall conductivity in NiPS$_3$, a van der Waals stack of honeycomb layers with a zigzag antiferromagnetic order below $T_N$ = 155 K. The longitudinal ($κ_{aa}$) and the transverse ($κ_{ab}$) thermal conductivities peak at the same temperature and the thermal Hall angle, at this peak, respects a previously identified bound. The amplitude of $κ_{ab}$ is extremely sensitive to the amplitude of magnetization along the $b$-axis, in contrast to the phonon mean free path, which is not at all. We show that the magnon and acoustic phonon bands cross each other along the $b^\ast$ orientation in the momentum space. The relevance of a thermodynamic property, combined with the irrelevance of the mean free path, points to an intrinsic origin.

cond-mat.str-el↗

arXiv2024

Temporal Correlation Meets Embedding: Towards a 2nd Generation of JDE-based Real-Time Multi-Object Tracking

Joint Detection and Embedding (JDE) trackers have demonstrated excellent performance in Multi-Object Tracking (MOT) tasks by incorporating the extraction of appearance features as auxiliary tasks through embedding Re-Identification task (ReID) into the detector, achieving a balance between inference speed and tracking performance. However, solving the competition between the detector and the feature extractor has always been a challenge. Meanwhile, the issue of directly embedding the ReID task into MOT has remained unresolved. The lack of high discriminability in appearance features results in their limited utility. In this paper, a new learning approach using cross-correlation to capture temporal information of objects is proposed. The feature extraction network is no longer trained solely on appearance features from each frame but learns richer motion features by utilizing feature heatmaps from consecutive frames, which addresses the challenge of inter-class feature similarity. Furthermore, our learning approach is applied to a more lightweight feature extraction network, and treat the feature matching scores as strong cues rather than auxiliary cues, with an appropriate weight calculation to reflect the compatibility between our obtained features and the MOT task. Our tracker, named TCBTrack, achieves state-of-the-art performance on multiple public benchmarks, i.e., MOT17, MOT20, and DanceTrack datasets. Specifically, on the DanceTrack test set, we achieve 56.8 HOTA, 58.1 IDF1 and 92.5 MOTA, making it the best online tracker capable of achieving real-time performance. Comparative evaluations with other trackers prove that our tracker achieves the best balance between speed, robustness and accuracy. Code is available at https://github.com/yfzhang1214/TCBTrack.

cs.CV↗

arXiv2024

Generating High-quality Symbolic Music Using Fine-grained Discriminators

Existing symbolic music generation methods usually utilize discriminator to improve the quality of generated music via global perception of music. However, considering the complexity of information in music, such as rhythm and melody, a single discriminator cannot fully reflect the differences in these two primary dimensions of music. In this work, we propose to decouple the melody and rhythm from music, and design corresponding fine-grained discriminators to tackle the aforementioned issues. Specifically, equipped with a pitch augmentation strategy, the melody discriminator discerns the melody variations presented by the generated samples. By contrast, the rhythm discriminator, enhanced with bar-level relative positional encoding, focuses on the velocity of generated notes. Such a design allows the generator to be more explicitly aware of which aspects should be adjusted in the generated music, making it easier to mimic human-composed music. Experimental results on the POP909 benchmark demonstrate the favorable performance of the proposed method compared to several state-of-the-art methods in terms of both objective and subjective metrics.

cs.SD↗

arXiv2024

Multipolar Electromagnetic Emission of Newborn Magnetar

It is generally recognized that the electromagnetic multipolar emission from magnetars can be used to explain radiation from Soft Gamma Repeaters (SGRs) or Anomalous X-ray Pulsars (AXPs), but they have little impact on the spindown of magnetars. We here present a comprehensive analytical solution for the neutron star multipolar electromagnetic fields and their associated expected luminosities. We find that for newborn millisecond magnetars, the spin-down luminosity from higher multipolar components can match or even exceed that from the dipole component. Such high-intensity radiation will undoubtedly affect related astrophysical phenomena at the birth of a magnetar. We show that the spin-down luminosity from multipoles can well explain the majority of Gamma-Ray Bursts (GRBs) afterglows, from the plateau starting at several hundred seconds till the normal decay phase lasting for many years. The fitted magnetar parameters for GRB afterglows are all typical values, with spins in the millisecond range and magnetic field strengths in the order of $10^{14} - 10^{15}$ Gauss. Our results in turn, provide support for the hypothesis that GRBs originate from the birth of magnetars with a few millisecond period, thus deepening our understanding of the complex magnetic field structure and the equation of state of magnetars.

astro-ph.HE↗

arXiv2024

Downstream-Pretext Domain Knowledge Traceback for Active Learning

Active learning (AL) is designed to construct a high-quality labeled dataset by iteratively selecting the most informative samples. Such sampling heavily relies on data representation, while recently pre-training is popular for robust feature learning. However, as pre-training utilizes low-level pretext tasks that lack annotation, directly using pre-trained representation in AL is inadequate for determining the sampling score. To address this problem, we propose a downstream-pretext domain knowledge traceback (DOKT) method that traces the data interactions of downstream knowledge and pre-training guidance for selecting diverse and instructive samples near the decision boundary. DOKT consists of a traceback diversity indicator and a domain-based uncertainty estimator. The diversity indicator constructs two feature spaces based on the pre-training pretext model and the downstream knowledge from annotation, by which it locates the neighbors of unlabeled data from the downstream space in the pretext space to explore the interaction of samples. With this mechanism, DOKT unifies the data relations of low-level and high-level representations to estimate traceback diversity. Next, in the uncertainty estimator, domain mixing is designed to enforce perceptual perturbing to unlabeled samples with similar visual patches in the pretext space. Then the divergence of perturbed samples is measured to estimate the domain uncertainty. As a result, DOKT selects the most diverse and important samples based on these two modules. The experiments conducted on ten datasets show that our model outperforms other state-of-the-art methods and generalizes well to various application scenarios such as semantic segmentation and image captioning.

cs.LG↗

arXiv2024

Distractors-Immune Representation Learning with Cross-modal Contrastive Regularization for Change Captioning

Change captioning aims to succinctly describe the semantic change between a pair of similar images, while being immune to distractors (illumination and viewpoint changes). Under these distractors, unchanged objects often appear pseudo changes about location and scale, and certain objects might overlap others, resulting in perturbational and discrimination-degraded features between two images. However, most existing methods directly capture the difference between them, which risk obtaining error-prone difference features. In this paper, we propose a distractors-immune representation learning network that correlates the corresponding channels of two image representations and decorrelates different ones in a self-supervised manner, thus attaining a pair of stable image representations under distractors. Then, the model can better interact them to capture the reliable difference features for caption generation. To yield words based on the most related difference features, we further design a cross-modal contrastive regularization, which regularizes the cross-modal alignment by maximizing the contrastive alignment between the attended difference features and generated words. Extensive experiments show that our method outperforms the state-of-the-art methods on four public datasets. The code is available at https://github.com/tuyunbin/DIRL.

cs.CV↗

arXiv2024

Technique Report of CVPR 2024 PBDL Challenges

The intersection of physics-based vision and deep learning presents an exciting frontier for advancing computer vision technologies. By leveraging the principles of physics to inform and enhance deep learning models, we can develop more robust and accurate vision systems. Physics-based vision aims to invert the processes to recover scene properties such as shape, reflectance, light distribution, and medium properties from images. In recent years, deep learning has shown promising improvements for various vision tasks, and when combined with physics-based vision, these approaches can enhance the robustness and accuracy of vision systems. This technical report summarizes the outcomes of the Physics-Based Vision Meets Deep Learning (PBDL) 2024 challenge, held in CVPR 2024 workshop. The challenge consisted of eight tracks, focusing on Low-Light Enhancement and Detection as well as High Dynamic Range (HDR) Imaging. This report details the objectives, methodologies, and results of each track, highlighting the top-performing solutions and their innovative approaches.

cs.CV↗

arXiv2024

StyleDubber: Towards Multi-Scale Style Learning for Movie Dubbing

Given a script, the challenge in Movie Dubbing (Visual Voice Cloning, V2C) is to generate speech that aligns well with the video in both time and emotion, based on the tone of a reference audio track. Existing state-of-the-art V2C models break the phonemes in the script according to the divisions between video frames, which solves the temporal alignment problem but leads to incomplete phoneme pronunciation and poor identity stability. To address this problem, we propose StyleDubber, which switches dubbing learning from the frame level to phoneme level. It contains three main components: (1) A multimodal style adaptor operating at the phoneme level to learn pronunciation style from the reference audio, and generate intermediate representations informed by the facial emotion presented in the video; (2) An utterance-level style learning module, which guides both the mel-spectrogram decoding and the refining processes from the intermediate embeddings to improve the overall style expression; And (3) a phoneme-guided lip aligner to maintain lip sync. Extensive experiments on two of the primary benchmarks, V2C and Grid, demonstrate the favorable performance of the proposed method as compared to the current stateof-the-art. The code will be made available at https://github.com/GalaxyCong/StyleDubber.

cs.CL↗

arXiv2024

Multi-Level Sequence Denoising with Cross-Signal Contrastive Learning for Sequential Recommendation

Sequential recommender systems (SRSs) aim to suggest next item for a user based on her historical interaction sequences. Recently, many research efforts have been devoted to attenuate the influence of noisy items in sequences by either assigning them with lower attention weights or discarding them directly. The major limitation of these methods is that the former would still prone to overfit noisy items while the latter may overlook informative items. To the end, in this paper, we propose a novel model named Multi-level Sequence Denoising with Cross-signal Contrastive Learning (MSDCCL) for sequential recommendation. To be specific, we first introduce a target-aware user interest extractor to simultaneously capture users' long and short term interest with the guidance of target items. Then, we develop a multi-level sequence denoising module to alleviate the impact of noisy items by employing both soft and hard signal denoising strategies. Additionally, we extend existing curriculum learning by simulating the learning pattern of human beings. It is worth noting that our proposed model can be seamlessly integrated with a majority of existing recommendation models and significantly boost their effectiveness. Experimental studies on five public datasets are conducted and the results demonstrate that the proposed MSDCCL is superior to the state-of-the-art baselines. The source code is publicly available at https://github.com/lalunex/MSDCCL/tree/main.

cs.IR↗

arXiv2024

Failure-Resilient Distributed Inference with Model Compression over Heterogeneous Edge Devices

The distributed inference paradigm enables the computation workload to be distributed across multiple devices, facilitating the implementations of deep learning based intelligent services on extremely resource-constrained Internet of Things (IoT) scenarios. Yet it raises great challenges to perform complicated inference tasks relying on a cluster of IoT devices that are heterogeneous in their computing/communication capacity and prone to crash or timeout failures. In this paper, we present RoCoIn, a robust cooperative inference mechanism for locally distributed execution of deep neural network-based inference tasks over heterogeneous edge devices. It creates a set of independent and compact student models that are learned from a large model using knowledge distillation for distributed deployment. In particular, the devices are strategically grouped to redundantly deploy and execute the same student model such that the inference process is resilient to any local failures, while a joint knowledge partition and student model assignment scheme are designed to minimize the response latency of the distributed inference system in the presence of devices with diverse capacities. Extensive simulations are conducted to corroborate the superior performance of our RoCoIn for distributed inference compared to several baselines, and the results demonstrate its efficacy in timely inference and failure resiliency.

cs.DC↗

arXiv2024

Probing Thermal Electrons in GRB Afterglows

Particle-in-cell simulations have unveiled that shock-accelerated electrons do not follow a pure power-law distribution, but have an additional low-energy "thermal" part, which owns a considerable portion of the total energy of electrons. Investigating the effects of these thermal electrons on gamma-ray burst (GRB) afterglows may provide valuable insights into the particle acceleration mechanisms. We solve the continuity equation of electrons in the energy space, from which multi-wavelength afterglows are derived by incorporating processes including synchrotron radiation, synchrotron self-absorption, synchrotron self-Compton scattering, and gamma-gamma annihilation. First, there is an underlying positive correlation between temporal and spectral indices due to the cooling of electrons. Moreover, thermal electrons would result in the simultaneous non-monotonic variation in both spectral and temporal indices at multi-wavelength, which could be individually recorded by the 2.5-meter Wide Field Survey Telescope and Vera Rubin Observatory Legacy Survey of Space and Time (LSST). The thermal electrons could also be diagnosed from afterglow spectra by synergy observation in the optical (with LSST) and X-ray bands (with the Microchannel X-ray Telescope on board the Space Variable Objects Monitor). Finally, we use Monte Carlo simulations to obtain the distribution of peak flux ratio ($R_{\rm X}$) between soft and hard X-rays, and of the time delay ($Δt$) between peak times of soft X-ray and optical light curves. The thermal electrons significantly raise the upper limits of both $R_{\rm X}$ and $Δt$. Thus the distribution of GRB afterglows with thermal electrons is more dispersive in the $R_{\rm X} - Δt$ plane.

astro-ph.HE↗

arXiv2024

Context-aware Difference Distilling for Multi-change Captioning

Multi-change captioning aims to describe complex and coupled changes within an image pair in natural language. Compared with single-change captioning, this task requires the model to have higher-level cognition ability to reason an arbitrary number of changes. In this paper, we propose a novel context-aware difference distilling (CARD) network to capture all genuine changes for yielding sentences. Given an image pair, CARD first decouples context features that aggregate all similar/dissimilar semantics, termed common/difference context features. Then, the consistency and independence constraints are designed to guarantee the alignment/discrepancy of common/difference context features. Further, the common context features guide the model to mine locally unchanged features, which are subtracted from the pair to distill locally difference features. Next, the difference context features augment the locally difference features to ensure that all changes are distilled. In this way, we obtain an omni-representation of all changes, which is translated into linguistic sentences by a transformer decoder. Extensive experiments on three public datasets show CARD performs favourably against state-of-the-art methods.The code is available at https://github.com/tuyunbin/CARD.

cs.CV↗

arXiv2024

Ten Supernova-rise in Binary Driven Gamma-ray Bursts

The observation of a gamma-ray burst (GRB) associated with a supernova (SN) coincides remarkably with the energy output from a binary system comprising a very massive carbon-oxygen (CO) core and an associated binary neutron star (NS) by the Binary-Driven Hypernova (BdHN) model. The dragging effect in the late evolution of such systems leads to co-rotation, with binary periods on the order of minutes, resulting in a very fast rotating core and a binary NS companion at a distance of $\sim 10^5$ km. Such a fast-rotating CO core, stripped of its hydrogen and helium, undergoes gravitational collapse and, within a fraction of seconds, leads to a supernova (SN) and a newly born, fast-spinning neutron star ($ν$NS), we name the emergence of the SN and the $ν$NS as the SN-rise and $ν$NS-rise. Typically, the SN energies range from $10^{51}$ to $10^{53}$ erg. We address this issue by examining 10 cases of Type-I BdHNe, the most energetic ones, in which SN accretion onto the companion NS leads to the formation of a black hole (BH). In all ten cases, the energetics of the SN events are estimated, ranging between $0.18$ and $12 \times 10^{52}$ erg. Additionally, in all 8 sources at redshift $z$ closer than $4.61$, a clear thermal blackbody component has been identified, with temperatures between $6.2$ and $39.99$ keV, as a possible signature of pair-driven SN. The triggering of the X-ray afterglow induced by the $ν$NS-rise are identified in three cases at high redshift where early X-ray observations are achievable, benefits from the interplay of cosmological effects.

astro-ph.HE↗

arXiv2024

An Experimental Study on Exploring Strong Lightweight Vision Transformers via Masked Image Modeling Pre-Training

Masked image modeling (MIM) pre-training for large-scale vision transformers (ViTs) has enabled promising downstream performance on top of the learned self-supervised ViT features. In this paper, we question if the \textit{extremely simple} lightweight ViTs' fine-tuning performance can also benefit from this pre-training paradigm, which is considerably less studied yet in contrast to the well-established lightweight architecture design methodology. We use an observation-analysis-solution flow for our study. We first systematically observe different behaviors among the evaluated pre-training methods with respect to the downstream fine-tuning data scales. Furthermore, we analyze the layer representation similarities and attention maps across the obtained models, which clearly show the inferior learning of MIM pre-training on higher layers, leading to unsatisfactory transfer performance on data-insufficient downstream tasks. This finding is naturally a guide to designing our distillation strategies during pre-training to solve the above deterioration problem. Extensive experiments have demonstrated the effectiveness of our approach. Our pre-training with distillation on pure lightweight ViTs with vanilla/hierarchical design ($5.7M$/$6.5M$) can achieve $79.4\%$/$78.9\%$ top-1 accuracy on ImageNet-1K. It also enables SOTA performance on the ADE20K segmentation task ($42.8\%$ mIoU) and LaSOT tracking task ($66.1\%$ AUC) in the lightweight regime. The latter even surpasses all the current SOTA lightweight CPU-realtime trackers.

cs.CV↗

arXiv2024

Global existence and Rayleigh-Taylor instability for the semi-dissipative Boussinesq system with Naiver boundary conditions

Considered herein is the global existence of weak, strong solutions and Rayleigh-Taylor (RT) instability for 2D semi-dissipative Boussinesq equations in an infinite strip domain $Ω_{\infty}$ subject to Navier boundary conditions with non-positive slip coefficients. We first prove the global existence of weak and strong solutions on bounded domain $Ω_{R}$ via the Galerkin method, characteristic analyzing technique and Stokes estimates etc. Based on above results, we further derive the uniform estimates, independent of the length of horizontal direction of $Ω_{R}$, ensuring the global existence of weak and strong solutions in unbounded case $Ω_{\infty}$ by utilizing the domain expansion method. Moreover, when the steady temperature is higher with decreasing height (i.e., RT steady-state) on certain region, we demonstrate that the steady-state is linear unstable through the construction of energy functional and the settlement of a family of modified variational problems. Furthermore, with the help of unstable solutions constructed in linear instability and global existence theorems, we confirm the instability of nonlinear problem in a Lipschitz structural sense. Finally, we give a series of rigorous verification (see Appendix) including the spectra of Stokes equations with Navier boundary conditions, Sobolev embedding inequalities, trace inequalities, and Stokes estimates under Navier boundary conditions etc, used in the proof of main conclusions.

math.DS↗

arXiv2024

Progressive Depth Decoupling and Modulating for Flexible Depth Completion

Image-guided depth completion aims at generating a dense depth map from sparse LiDAR data and RGB image. Recent methods have shown promising performance by reformulating it as a classification problem with two sub-tasks: depth discretization and probability prediction. They divide the depth range into several discrete depth values as depth categories, serving as priors for scene depth distributions. However, previous depth discretization methods are easy to be impacted by depth distribution variations across different scenes, resulting in suboptimal scene depth distribution priors. To address the above problem, we propose a progressive depth decoupling and modulating network, which incrementally decouples the depth range into bins and adaptively generates multi-scale dense depth maps in multiple stages. Specifically, we first design a Bins Initializing Module (BIM) to construct the seed bins by exploring the depth distribution information within a sparse depth map, adapting variations of depth distribution. Then, we devise an incremental depth decoupling branch to progressively refine the depth distribution information from global to local. Meanwhile, an adaptive depth modulating branch is developed to progressively improve the probability representation from coarse-grained to fine-grained. And the bi-directional information interactions are proposed to strengthen the information interaction between those two branches (sub-tasks) for promoting information complementation in each branch. Further, we introduce a multi-scale supervision mechanism to learn the depth distribution information in latent features and enhance the adaptation capability across different scenes. Experimental results on public datasets demonstrate that our method outperforms the state-of-the-art methods. The code will be open-sourced at [this https URL](https://github.com/Cisse-away/PDDM).

cs.CV↗

arXiv2024

Propensity Score Analysis with Guaranteed Subgroup Balance

Estimating the causal treatment effects by subgroups is important in observational studies when the treatment effect heterogeneity may be present. Existing propensity score methods rely on a correctly specified propensity score model. Model misspecification results in biased treatment effect estimation and covariate imbalance. We proposed a new algorithm, the propensity score analysis with guaranteed subgroup balance (G-SBPS), to achieve covariate mean balance in all subgroups. We further incorporated nonparametric kernel regression for the propensity scores and developed a kernelized G-SBPS (kG-SBPS) to improve the subgroup mean balance of covariate transformations in a rich functional class. This extension is more robust to propensity score model misspecification. Extensive numerical studies showed that G-SBPS and kG-SBPS improve both subgroup covariate balance and subgroup treatment effect estimation, compared to existing approaches. We applied G-SBPS and kG-SBPS to a dataset on right heart catheterization to estimate the subgroup average treatment effects on the hospital length of stay and a dataset on diabetes self-management training to estimate the subgroup average treatment effects for the treated on the hospitalization rate.

stat.ME↗

arXiv2024

Oxygen vacancies modulated VO2 for neurons and Spiking Neural Network construction

Artificial neuronal devices are the basic building blocks for neuromorphic computing systems, which have been motivated by realistic brain emulation. Aiming for these applications, various device concepts have been proposed to mimic the neuronal dynamics and functions. While till now, the artificial neuron devices with high efficiency, high stability and low power consumption are still far from practical application. Due to the special insulator-metal phase transition, Vanadium Dioxide (VO2) has been considered as an idea candidate for neuronal device fabrication. However, its intrinsic insulating state requires the VO2 neuronal device to be driven under large bias voltage, resulting in high power consumption and low frequency. Thus in the current study, we have addressed this challenge by preparing oxygen vacancies modulated VO2 film(VO2-x) and fabricating the VO2-x neuronal devices for Spiking Neural Networks (SNNs) construction. Results indicate the neuron devices can be operated under lower voltage with improved processing speed. The proposed VO2-x based back-propagation SNNs (BP-SNNs) system, trained with the MNIST dataset, demonstrates excellent accuracy in image recognition. Our study not only demonstrates the VO2-x based neurons and SNN system for practical application, but also offers an effective way to optimize the future neuromorphic computing systems by defect engineering strategy.

cs.NE↗