arXiv subjects

Jie Liu

Publications and source records attributed to Jie Liu.

At least 379 records · Page 21Linked to original sources

arXiv2024

MS-RNN: A Flexible Multi-Scale Framework for Spatiotemporal Predictive Learning

Spatiotemporal predictive learning, which predicts future frames through historical prior knowledge with the aid of deep learning, is widely used in many fields. Previous work essentially improves the model performance by widening or deepening the network, but it also brings surging memory overhead, which seriously hinders the development and application of this technology. In order to improve the performance without increasing memory consumption, we focus on scale, which is another dimension to improve model performance but with low memory requirement. The effectiveness has been widely demonstrated in many CNN-based tasks such as image classification and semantic segmentation, but it has not been fully explored in recent RNN models. In this paper, learning from the benefit of multi-scale, we propose a general framework named Multi-Scale RNN (MS-RNN) to boost recent RNN models for spatiotemporal predictive learning. We verify the MS-RNN framework by thorough theoretical analyses and exhaustive experiments, where the theory focuses on memory reduction and performance improvement while the experiments employ eight RNN models (ConvLSTM, TrajGRU, PredRNN, PredRNN++, MIM, MotionRNN, PredRNN-V2, and PrecipLSTM) and four datasets (Moving MNIST, TaxiBJ, KTH, and Germany). The results show the efficiency that RNN models incorporating our framework have much lower memory cost but better performance than before. Our code is released at \url{https://github.com/mazhf/MS-RNN}.

cs.CV↗

arXiv2024

AlignMiF: Geometry-Aligned Multimodal Implicit Field for LiDAR-Camera Joint Synthesis

Neural implicit fields have been a de facto standard in novel view synthesis. Recently, there exist some methods exploring fusing multiple modalities within a single field, aiming to share implicit features from different modalities to enhance reconstruction performance. However, these modalities often exhibit misaligned behaviors: optimizing for one modality, such as LiDAR, can adversely affect another, like camera performance, and vice versa. In this work, we conduct comprehensive analyses on the multimodal implicit field of LiDAR-camera joint synthesis, revealing the underlying issue lies in the misalignment of different sensors. Furthermore, we introduce AlignMiF, a geometrically aligned multimodal implicit field with two proposed modules: Geometry-Aware Alignment (GAA) and Shared Geometry Initialization (SGI). These modules effectively align the coarse geometry across different modalities, significantly enhancing the fusion process between LiDAR and camera data. Through extensive experiments across various datasets and scenes, we demonstrate the effectiveness of our approach in facilitating better interaction between LiDAR and camera modalities within a unified neural field. Specifically, our proposed AlignMiF, achieves remarkable improvement over recent implicit fusion methods (+2.01 and +3.11 image PSNR on the KITTI-360 and Waymo datasets) and consistently surpasses single modality performance (13.8% and 14.2% reduction in LiDAR Chamfer Distance on the respective datasets).

cs.CV↗

arXiv2024

MaskMA: Towards Zero-Shot Multi-Agent Decision Making with Mask-Based Collaborative Learning

Building a single generalist agent with strong zero-shot capability has recently sparked significant advancements. However, extending this capability to multi-agent decision making scenarios presents challenges. Most current works struggle with zero-shot transfer, due to two challenges particular to the multi-agent settings: (a) a mismatch between centralized training and decentralized execution; and (b) difficulties in creating generalizable representations across diverse tasks due to varying agent numbers and action spaces. To overcome these challenges, we propose a Mask-Based collaborative learning framework for Multi-Agent decision making (MaskMA). Firstly, we propose to randomly mask part of the units and collaboratively learn the policies of unmasked units to handle the mismatch. In addition, MaskMA integrates a generalizable action representation by dividing the action space into intrinsic actions solely related to the unit itself and interactive actions involving interactions with other units. This flexibility allows MaskMA to tackle tasks with varying agent numbers and thus different action spaces. Extensive experiments in SMAC reveal MaskMA, with a single model trained on 11 training maps, can achieve an impressive 77.8% average zero-shot win rate on 60 unseen test maps by decentralized execution, while also performing effectively on other types of downstream tasks (e.g., varied policies collaboration, ally malfunction, and ad hoc team play).

cs.AI↗

arXiv2024

ConceptMath: A Bilingual Concept-wise Benchmark for Measuring Mathematical Reasoning of Large Language Models

This paper introduces ConceptMath, a bilingual (English and Chinese), fine-grained benchmark that evaluates concept-wise mathematical reasoning of Large Language Models (LLMs). Unlike traditional benchmarks that evaluate general mathematical reasoning with an average accuracy, ConceptMath systematically organizes math problems under a hierarchy of math concepts, so that mathematical reasoning can be evaluated at different granularity with concept-wise accuracies. Based on our ConcepthMath, we evaluate a broad range of LLMs, and we observe existing LLMs, though achieving high average accuracies on traditional benchmarks, exhibit significant performance variations across different math concepts and may even fail catastrophically on the most basic ones. Besides, we also introduce an efficient fine-tuning strategy to enhance the weaknesses of existing LLMs. Finally, we hope ConceptMath could guide the developers to understand the fine-grained mathematical abilities of their models and facilitate the growth of foundation models.

cs.CL↗

arXiv2024

GCOF: Self-iterative Text Generation for Copywriting Using Large Language Model

Large language models(LLM) such as ChatGPT have substantially simplified the generation of marketing copy, yet producing content satisfying domain specific requirements, such as effectively engaging customers, remains a significant challenge. In this work, we introduce the Genetic Copy Optimization Framework (GCOF) designed to enhance both efficiency and engagememnt of marketing copy creation. We conduct explicit feature engineering within the prompts of LLM. Additionally, we modify the crossover operator in Genetic Algorithm (GA), integrating it into the GCOF to enable automatic feature engineering. This integration facilitates a self-iterative refinement of the marketing copy. Compared to human curated copy, Online results indicate that copy produced by our framework achieves an average increase in click-through rate (CTR) of over $50\%$.

cs.CL↗

arXiv2024

MS-LSTM: Exploring Spatiotemporal Multiscale Representations in Video Prediction Domain

The drastic variation of motion in spatial and temporal dimensions makes the video prediction task extremely challenging. Existing RNN models obtain higher performance by deepening or widening the model. They obtain the multi-scale features of the video only by stacking layers, which is inefficient and brings unbearable training costs (such as memory, FLOPs, and training time). Different from them, this paper proposes a spatiotemporal multi-scale model called MS-LSTM wholly from a multi-scale perspective. On the basis of stacked layers, MS-LSTM incorporates two additional efficient multi-scale designs to fully capture spatiotemporal context information. Concretely, we employ LSTMs with mirrored pyramid structures to construct spatial multi-scale representations and LSTMs with different convolution kernels to construct temporal multi-scale representations. We theoretically analyze the training cost and performance of MS-LSTM and its components. Detailed comparison experiments with twelve baseline models on four video datasets show that MS-LSTM has better performance but lower training costs.

cs.CV↗

arXiv2024

NMR evidence of spinon localization in kagome antiferromagnet YCu$_3$(OH)$_6$Br$_2$[Br$_{1-x}$(OH)$_x$]

We performed nuclear magnetic resonance studies on a kagome antiferromagnet YCu$_3$(OH)$_6$Br$_2$[Br$_{1-x}$(OH)$_{x}$]. No significant NMR spectral broadening is found in the Br center peak from 1 K down to 0.05 K, indicating absence of static antiferromagnetic ordering. In contrast to signatures of dominant 2D kagome antiferromagnetic fluctuations at temperature above 30 K, both the Knight shift $K_{\rm{n}}$ and the spin-lattice relaxation rate $1/T_{1}$ increase when the sample is cooled from 30 K to 8 K, which can be attributed to the scattering of spin excitations by strong non-magnetic impurities. Unusually, a hump is observed in $K_{\rm{n}}$ and $1/T_{2}$ close to 2 K (far below the exchange energy), which indicates the existence of excitations with a large density of states close to zero energy. These phenomena are reproduced by a mean-field simulation of Heisenberg model with bond-dependent exchange interactions, where the sign fluctuations in the spinon kinetic terms caused by impurities result in localization of spinons and an almost flat band close to the Fermi energy.

cond-mat.str-el↗

arXiv2024

The Spectral base and quotients of bounded symmetric domains

In this article, we explore Higgs bundles on a projective manifold $X$, focusing on their spectral bases, a concept introduced by T.Chen and B.Ngô. The spectral base is a specific closed subscheme within the space of symmetric differentials. We observe that if the spectral base vanishes, then any reductive representation $ρ: π_1(X) \to \text{GL}_r(\mathbb{C})$ is both rigid and integral. Additionally, we prove that for $X=Ω/Γ$, a quotient of a bounded symmetric domain $Ω$ of rank at least $2$ by a torsion-free cocompact irreducible lattice $Γ$, the spectral base indeed vanishes, which generalizes a result of B.Klingler.

math.AG↗

arXiv2024

Dynamic Prototype Adaptation with Distillation for Few-shot Point Cloud Segmentation

Few-shot point cloud segmentation seeks to generate per-point masks for previously unseen categories, using only a minimal set of annotated point clouds as reference. Existing prototype-based methods rely on support prototypes to guide the segmentation of query point clouds, but they encounter challenges when significant object variations exist between the support prototypes and query features. In this work, we present dynamic prototype adaptation (DPA), which explicitly learns task-specific prototypes for each query point cloud to tackle the object variation problem. DPA achieves the adaptation through prototype rectification, aligning vanilla prototypes from support with the query feature distribution, and prototype-to-query attention, extracting task-specific context from query point clouds. Furthermore, we introduce a prototype distillation regularization term, enabling knowledge transfer between early-stage prototypes and their deeper counterparts during adaption. By iteratively applying these adaptations, we generate task-specific prototypes for accurate mask predictions on query point clouds. Extensive experiments on two popular benchmarks show that DPA surpasses state-of-the-art methods by a significant margin, e.g., 7.43\% and 6.39\% under the 2-way 1-shot setting on S3DIS and ScanNet, respectively. Code is available at https://github.com/jliu4ai/DPA.

cs.CV↗

arXiv2024

Sketch and Refine: Towards Fast and Accurate Lane Detection

Lane detection is to determine the precise location and shape of lanes on the road. Despite efforts made by current methods, it remains a challenging task due to the complexity of real-world scenarios. Existing approaches, whether proposal-based or keypoint-based, suffer from depicting lanes effectively and efficiently. Proposal-based methods detect lanes by distinguishing and regressing a collection of proposals in a streamlined top-down way, yet lack sufficient flexibility in lane representation. Keypoint-based methods, on the other hand, construct lanes flexibly from local descriptors, which typically entail complicated post-processing. In this paper, we present a "Sketch-and-Refine" paradigm that utilizes the merits of both keypoint-based and proposal-based methods. The motivation is that local directions of lanes are semantically simple and clear. At the "Sketch" stage, local directions of keypoints can be easily estimated by fast convolutional layers. Then we can build a set of lane proposals accordingly with moderate accuracy. At the "Refine" stage, we further optimize these proposals via a novel Lane Segment Association Module (LSAM), which allows adaptive lane segment adjustment. Last but not least, we propose multi-level feature integration to enrich lane feature representations more efficiently. Based on the proposed "Sketch and Refine" paradigm, we propose a fast yet effective lane detector dubbed "SRLane". Experiments show that our SRLane can run at a fast speed (i.e., 278 FPS) while yielding an F1 score of 78.9\%. The source code is available at: https://github.com/passerer/SRLane.

cs.CV↗

arXiv2024

Animal3D: A Comprehensive Dataset of 3D Animal Pose and Shape

Accurately estimating the 3D pose and shape is an essential step towards understanding animal behavior, and can potentially benefit many downstream applications, such as wildlife conservation. However, research in this area is held back by the lack of a comprehensive and diverse dataset with high-quality 3D pose and shape annotations. In this paper, we propose Animal3D, the first comprehensive dataset for mammal animal 3D pose and shape estimation. Animal3D consists of 3379 images collected from 40 mammal species, high-quality annotations of 26 keypoints, and importantly the pose and shape parameters of the SMAL model. All annotations were labeled and checked manually in a multi-stage process to ensure highest quality results. Based on the Animal3D dataset, we benchmark representative shape and pose estimation models at: (1) supervised learning from only the Animal3D data, (2) synthetic to real transfer from synthetically generated images, and (3) fine-tuning human pose and shape estimation models. Our experimental results demonstrate that predicting the 3D shape and pose of animals across species remains a very challenging task, despite significant advances in human pose estimation. Our results further demonstrate that synthetic pre-training is a viable strategy to boost the model performance. Overall, Animal3D opens new directions for facilitating future research in animal 3D pose and shape estimation, and is publicly available.

cs.CV↗

arXiv2024

Perturbative variational quantum algorithms for material simulations

Reducing circuit depth is essential for implementing quantum simulations of electronic structure on near-term quantum devices. In this work, we propose a variational quantum eigensolver (VQE) based perturbation theory algorithm to accurately simulate electron correlation of periodic materials with shallow ansatz circuits, which are generated from Adaptive Derivative-Assembled Pseudo-Trotter or Qubit-Excitation-based VQE calculations using a loose convergence criteria. Here, the major part of the electron correlation is described using the VQE ansatz circuit and the remaining correlation energy is described by either multireference or similarity transformation-based perturbation theory. Numerical results demonstrate that the new algorithms are able to accurately describe electron correlation of the LiH crystal with only one circuit parameter, in contrast with ~30 parameters required in the adaptive VQE to achieve the same accuracy. Meanwhile, for fixed-depth Ansatze, e.g. unitary coupled cluster, we demonstrate that the VQE-base perturbation theory provides an appealing scheme to improve their accuracy.

quant-ph↗

arXiv2024

LF-ViT: Reducing Spatial Redundancy in Vision Transformer for Efficient Image Recognition

The Vision Transformer (ViT) excels in accuracy when handling high-resolution images, yet it confronts the challenge of significant spatial redundancy, leading to increased computational and memory requirements. To address this, we present the Localization and Focus Vision Transformer (LF-ViT). This model operates by strategically curtailing computational demands without impinging on performance. In the Localization phase, a reduced-resolution image is processed; if a definitive prediction remains elusive, our pioneering Neighborhood Global Class Attention (NGCA) mechanism is triggered, effectively identifying and spotlighting class-discriminative regions based on initial findings. Subsequently, in the Focus phase, this designated region is used from the original image to enhance recognition. Uniquely, LF-ViT employs consistent parameters across both phases, ensuring seamless end-to-end optimization. Our empirical tests affirm LF-ViT's prowess: it remarkably decreases Deit-S's FLOPs by 63\% and concurrently amplifies throughput twofold. Code of this project is at https://github.com/edgeai1/LF-ViT.git.

cs.CV↗

arXiv2024

Hierarchical Aligned Multimodal Learning for NER on Tweet Posts

Mining structured knowledge from tweets using named entity recognition (NER) can be beneficial for many down stream applications such as recommendation and intention understanding. With tweet posts tending to be multimodal, multimodal named entity recognition (MNER) has attracted more attention. In this paper, we propose a novel approach, which can dynamically align the image and text sequence and achieve the multi-level cross-modal learning to augment textual word representation for MNER improvement. To be specific, our framework can be split into three main stages: the first stage focuses on intra-modality representation learning to derive the implicit global and local knowledge of each modality, the second evaluates the relevance between the text and its accompanying image and integrates different grained visual information based on the relevance, the third enforces semantic refinement via iterative cross-modal interactions and co-attention. We conduct experiments on two open datasets, and the results and detailed analysis demonstrate the advantage of our model.

cs.CL↗

arXiv2023

Multiphoton fusion of light nuclei in intense laser fields

We investigate the fusion cross sections of light nuclei in the presence of linearly polarized intense laser fields. By combining the Coulomb-Volkov solutions with the complex spherical square-well nuclear potential, we derive an explicit formulation of the multiphoton cross section in a self-consistent manner. Our analysis is specifically focused on deuteron-triton (DT) and proton-boron (p$^{11}{\rm B}$) fusion reactions, both of which have garnered widespread attention. The theoretical results reveal that, under conditions of longer laser wavelengths and lower incident particle kinetic energies, a few thousands of photons can participate in the fusion reactions, resulting in a substantial enhancement of fusion cross sections by almost ten orders of magnitude. We elucidate the multiphoton mechanism underlying these findings and discuss their implications.

nucl-th↗

arXiv2023

Information-Theoretic Generalization Analysis for Topology-aware Heterogeneous Federated Edge Learning over Noisy Channels

With the rapid growth of edge intelligence, the deployment of federated learning (FL) over wireless networks has garnered increasing attention, which is called Federated Edge Learning (FEEL). In FEEL, both mobile devices transmitting model parameters over noisy channels and collecting data in diverse environments pose challenges to the generalization of trained models. Moreover, devices can engage in decentralized FL via Device-to-Device communication while the communication topology of connected devices also impacts the generalization of models. Most recent theoretical studies overlook the incorporation of all these effects into FEEL when developing generalization analyses. In contrast, our work presents an information-theoretic generalization analysis for topology-aware FEEL in the presence of data heterogeneity and noisy channels. Additionally, we propose a novel regularization method called Federated Global Mutual Information Reduction (FedGMIR) to enhance the performance of models based on our analysis. Numerical results validate our theoretical findings and provide evidence for the effectiveness of the proposed method.

cs.IT↗

arXiv2023

Short-Term Multi-Horizon Line Loss Rate Forecasting of a Distribution Network Using Attention-GCN-LSTM

Accurately predicting line loss rates is vital for effective line loss management in distribution networks, especially over short-term multi-horizons ranging from one hour to one week. In this study, we propose Attention-GCN-LSTM, a novel method that combines Graph Convolutional Networks (GCN), Long Short-Term Memory (LSTM), and a three-level attention mechanism to address this challenge. By capturing spatial and temporal dependencies, our model enables accurate forecasting of line loss rates across multiple horizons. Through comprehensive evaluation using real-world data from 10KV feeders, our Attention-GCN-LSTM model consistently outperforms existing algorithms, exhibiting superior performance in terms of prediction accuracy and multi-horizon forecasting. This model holds significant promise for enhancing line loss management in distribution networks.

cs.LG↗

arXiv2023

Phase modulation of directed transport, energy diffusion and quantum scrambling in a Floquet non-Hermitian system

We investigate both theoretically and numerically the wavepacket's dynamics in momentum space for a Floquet non-Hermitian system with a periodically-kicked driven potential. We have deduced the exact expression of a time-evolving wavepacket under the condition of quantum resonance. With this analytical expression, we can investigate thoroughly the temporal behaviors of the directed transport, energy diffusion and quantum scrambling. We find interestingly that, by tuning the relative phase between the real part and imaginary part of the kicking potential, one can manipulate the directed propagation, energy diffusion and quantum scrambling efficiently: when the phase equals to $π/2$, we observe a maximum directed current and energy diffusion, while a minimum scrambling phenomenon protected by the $\mathcal{PT}$-symmetry; when the phase is $π$, both the directed transport and the energy diffusion are suppressed, in contrast, the quantum scrambling is enhanced by the non-Hermiticity. Possible applications of our findings are discussed.

quant-ph↗