arXiv subjects

Peng Han

Publications and source records attributed to Peng Han.

At least 37 records · Page 2Linked to original sources

arXiv2024

Not All Layers of LLMs Are Necessary During Inference

Due to the large number of parameters, the inference phase of Large Language Models (LLMs) is resource-intensive. However, not all requests posed to LLMs are equally difficult to handle. Through analysis, we show that for some tasks, LLMs can achieve results comparable to the final output at some intermediate layers. That is, not all layers of LLMs are necessary during inference. If we can predict at which layer the inferred results match the final results (produced by evaluating all layers), we could significantly reduce the inference cost. To this end, we propose a simple yet effective algorithm named AdaInfer to adaptively terminate the inference process for an input instance. AdaInfer relies on easily obtainable statistical features and classic classifiers like SVM. Experiments on well-known LLMs like the Llama2 series and OPT, show that AdaInfer can achieve an average of 17.8% pruning ratio, and up to 43% on sentiment tasks, with nearly no performance drop (<1%). Because AdaInfer does not alter LLM parameters, the LLMs incorporated with AdaInfer maintain generalizability across tasks.

cs.CL↗

arXiv2024

What Affects the Stability of Tool Learning? An Empirical Study on the Robustness of Tool Learning Frameworks

Tool learning methods have enhanced the ability of large language models (LLMs) to interact with real-world applications. Many existing works fine-tune LLMs or design prompts to enable LLMs to select appropriate tools and correctly invoke them to meet user requirements. However, it is observed in previous works that the performance of tool learning varies from tasks, datasets, training settings, and algorithms. Without understanding the impact of these factors, it can lead to inconsistent results, inefficient model deployment, and suboptimal tool utilization, ultimately hindering the practical integration and scalability of LLMs in real-world scenarios. Therefore, in this paper, we explore the impact of both internal and external factors on the performance of tool learning frameworks. Through extensive experiments on two benchmark datasets, we find several insightful conclusions for future work, including the observation that LLMs can benefit significantly from increased trial and exploration. We believe our empirical study provides a new perspective for future tool learning research.

cs.CL↗

arXiv2024

The characterization of hyper-bent function with multiple trace terms in the extension field

Bent functions are maximally nonlinear Boolean functions with an even number of variables, which include a subclass of functions, the so-called hyper-bent functions whose properties are stronger than bent functions and a complete classification of hyper-bent functions is elusive and inavailable.~In this paper,~we solve an open problem of Mesnager that describes hyper-bentness of hyper-bent functions with multiple trace terms via Dillon-like exponents with coefficients in the extension field~$\mathbb{F}_{2^{2m}}$~of this field~$\mathbb{F}_{2^{m}}$. By applying Möbius transformation and the theorems of hyperelliptic curves, hyper-bentness of these functions are successfully characterized in this field~$\mathbb{F}_{2^{2m}}$ with~$m$~odd integer.

cs.IT↗

arXiv2024

MotionGS : Compact Gaussian Splatting SLAM by Motion Filter

With their high-fidelity scene representation capability, the attention of SLAM field is deeply attracted by the Neural Radiation Field (NeRF) and 3D Gaussian Splatting (3DGS). Recently, there has been a surge in NeRF-based SLAM, while 3DGS-based SLAM is sparse. A novel 3DGS-based SLAM approach with a fusion of deep visual feature, dual keyframe selection and 3DGS is presented in this paper. Compared with the existing methods, the proposed tracking is achieved by feature extraction and motion filter on each frame. The joint optimization of poses and 3D Gaussians runs through the entire mapping process. Additionally, the coarse-to-fine pose estimation and compact Gaussian scene representation are implemented by dual keyframe selection and novel loss functions. Experimental results demonstrate that the proposed algorithm not only outperforms the existing methods in tracking and mapping, but also has less memory usage.

cs.CV↗

arXiv2024

DRE: Generating Recommendation Explanations by Aligning Large Language Models at Data-level

Recommendation systems play a crucial role in various domains, suggesting items based on user behavior.However, the lack of transparency in presenting recommendations can lead to user confusion. In this paper, we introduce Data-level Recommendation Explanation (DRE), a non-intrusive explanation framework for black-box recommendation models.Different from existing methods, DRE does not require any intermediary representations of the recommendation model or latent alignment training, mitigating potential performance issues.We propose a data-level alignment method, leveraging large language models to reason relationships between user data and recommended items.Additionally, we address the challenge of enriching the details of the explanation by introducing target-aware user preference distillation, utilizing item reviews. Experimental results on benchmark datasets demonstrate the effectiveness of the DRE in providing accurate and user-centric explanations, enhancing user engagement with recommended item.

cs.IR↗

arXiv2024

nanoLM: an Affordable LLM Pre-training Benchmark via Accurate Loss Prediction across Scales

As language models scale up, it becomes increasingly expensive to verify research ideas because conclusions on small models do not trivially transfer to large ones. A possible solution is to establish a generic system that accurately predicts certain metrics for large models without training them. Existing scaling laws require hyperparameter search on the largest models, limiting their predicative capability. In this paper, we present an approach (namely μScaling) to predict the pre-training loss, based on our observations that Maximal Update Parametrization (μP) enables accurate fitting of scaling laws close to common loss basins in hyperparameter space. With μScaling, different model designs can be compared on large scales by training only their smaller counterparts. Further, we introduce nanoLM: an affordable LLM pre-training benchmark that facilitates this new research paradigm. With around 14% of the one-time pre-training cost, we can accurately forecast the loss for models up to 52B. Our goal with nanoLM is to empower researchers with limited resources to reach meaningful conclusions on large models. We also aspire for our benchmark to serve as a bridge between the academic community and the industry. Code for μScaling is available at https://github.com/cofe-ai/Mu-scaling. Code for nanoLLM will be available later.

cs.CL↗

arXiv2024

V2VSSC: A 3D Semantic Scene Completion Benchmark for Perception with Vehicle to Vehicle Communication

Semantic scene completion (SSC) has recently gained popularity because it can provide both semantic and geometric information that can be used directly for autonomous vehicle navigation. However, there are still challenges to overcome. SSC is often hampered by occlusion and short-range perception due to sensor limitations, which can pose safety risks. This paper proposes a fundamental solution to this problem by leveraging vehicle-to-vehicle (V2V) communication. We propose the first generalized collaborative SSC framework that allows autonomous vehicles to share sensing information from different sensor views to jointly perform SSC tasks. To validate the proposed framework, we further build V2VSSC, the first V2V SSC benchmark, on top of the large-scale V2V perception dataset OPV2V. Extensive experiments demonstrate that by leveraging V2V communication, the SSC performance can be increased by 8.3% on geometric metric IoU and 6.0% mIOU.

cs.CV↗

arXiv2023

Learning Time-aware Graph Structures for Spatially Correlated Time Series Forecasting

Spatio-temporal forecasting of future values of spatially correlated time series is important across many cyber-physical systems (CPS). Recent studies offer evidence that the use of graph neural networks to capture latent correlations between time series holds a potential for enhanced forecasting. However, most existing methods rely on pre-defined or self-learning graphs, which are either static or unintentionally dynamic, and thus cannot model the time-varying correlations that exhibit trends and periodicities caused by the regularity of the underlying processes in CPS. To tackle such limitation, we propose Time-aware Graph Structure Learning (TagSL), which extracts time-aware correlations among time series by measuring the interaction of node and time representations in high-dimensional spaces. Notably, we introduce time discrepancy learning that utilizes contrastive learning with distance-based regularization terms to constrain learned spatial correlations to a trend sequence. Additionally, we propose a periodic discriminant function to enable the capture of periodic changes from the state of nodes. Next, we present a Graph Convolution-based Gated Recurrent Unit (GCGRU) that jointly captures spatial and temporal dependencies while learning time-aware and node-specific patterns. Finally, we introduce a unified framework named Time-aware Graph Convolutional Recurrent Network (TGCRN), combining TagSL, and GCGRU in an encoder-decoder architecture for multi-step spatio-temporal forecasting. We report on experiments with TGCRN and popular existing approaches on five real-world datasets, thus providing evidence that TGCRN is capable of advancing the state-of-the-art. We also cover a detailed ablation study and visualization analysis, offering detailed insight into the effectiveness of time-aware structure learning.

cs.LG↗

arXiv2023

Determination of the phonon sidebands in the photoluminescence spectrum of semiconductor nanoclusters from ab initio calculations

We propose a theoretical approach based on (constrained) density functional theory and the Franck-Condon approximation for the calculation of the temperature dependent photoluminescence of nanostructures. The method is computationally advantageous and only slightly more demanding than a standard density functional theory calculation and includes transitions into multiphonon final states (higher class transitions). We use the approach for Si and CdSe colloidal nanoclusters (NCs) with up to 693 atoms and obtain very good agreement with experiment which allows us to identify specific peaks and explain their origin. Generally, breathing type modes are shown to dominate the phonon replicas, while optical modes have significant contributions for CdSe NCs and play a lesser role in Si NCs. We obtain significant anti-Stokes peak starting at 140K for Si NC explaining the broadening observed in the corresponding experiment. We also apply the method to small molecular-like carbon structures (diamondoids), where electron-phonon coupling is typically large, and find that multiphonon processes (up to class 4) are very relevant and necessary to compare favorably with experiment. While it is crucial to include these multiphonon states in the small diamondoids with few tens of atoms, neglecting them in only marginally larger Si$_{87}$H$_{76}$ and Cd$_{43}$Se$_{44}$H$^*_{76}$ (and larger) quantum dots represents a good approximation.

cond-mat.mes-hall↗

arXiv2023

Optimal energy harvesting efficiency from vortex-induced vibration of a circular cylinder under flow

This work applies a combined approach a reduced-order model (ROM) together with experiments and direct numerical simulations to investigate the optimal efficiency of fluid-flow energy harvesting from transverse vortex-induced vibration (VIV) of a circular cylinder. High resolution efficiency maps were predicted over wide ranges of flow reduced velocities and structural damping ratios, and the maximum efficiency and optimal settings of damping ratio and reduced velocity were then examined for different mass ratios and Reynolds numbers. Efficiencies predicted by the ROM were also validated against either experiments or direct simulations. The present work indicates that: (i) the maximum efficiency is controlled by both the incoming reduced velocity and the product of mass ratio and structural damping ratio, which is similar to the maximum amplitude of VIV; (ii) the maximum efficiency at a relatively high Reynolds number ($Re \approx 6 \times 10^3$) in subcritical regime is higher than that of a low Reynolds number ($Re = 150$) in laminar regime; (iii) the energy harvesting efficiency from VIV of a circular cylinder with a low mass ratio is more robust than that with a high mass ratio. This finding suggests that the VIV harvester performs better in water than in air.

physics.flu-dyn↗

arXiv2022

RetroGraph: Retrosynthetic Planning with Graph Search

Retrosynthetic planning, which aims to find a reaction pathway to synthesize a target molecule, plays an important role in chemistry and drug discovery. This task is usually modeled as a search problem. Recently, data-driven methods have attracted many research interests and shown promising results for retrosynthetic planning. We observe that the same intermediate molecules are visited many times in the searching process, and they are usually independently treated in previous tree-based methods (e.g., AND-OR tree search, Monte Carlo tree search). Such redundancies make the search process inefficient. We propose a graph-based search policy that eliminates the redundant explorations of any intermediate molecules. As searching over a graph is more complicated than over a tree, we further adopt a graph neural network to guide the search over graphs. Meanwhile, our method can search a batch of targets together in the graph and remove the inter-target duplication in the tree-based search methods. Experimental results on two datasets demonstrate the effectiveness of our method. Especially on the widely used USPTO benchmark, we improve the search success rate to 99.47%, advancing previous state-of-the-art performance for 2.6 points.

cs.AI↗

arXiv2022

Experimental identification of the second-order non-Hermitian skin effect with physics-graph-informed machine learning

Topological phases of matter are conventionally characterized by the bulk-boundary correspondence in Hermitian systems: The topological invariant of the bulk in $d$ dimensions corresponds to the number of $(d-1)$-dimensional boundary states. By extension, higher-order topological insulators reveal a bulk-edge-corner correspondence, such that $n$-th order topological phases feature $(d-n)$-dimensional boundary states. The advent of non-Hermitian topological systems sheds new light on the emergence of the non-Hermitian skin effect (NHSE) with an extensive number of boundary modes under open boundary conditions. Still, the higher-order NHSE remains largely unexplored, particularly in the experiment. We introduce an unsupervised approach -- physics-graph-informed machine learning (PGIML) -- to enhance the data mining ability of machine learning with limited domain knowledge. Through PGIML, we experimentally demonstrate the second-order NHSE in a two-dimensional non-Hermitian topolectrical circuit. The admittance spectra of the circuit exhibit an extensive number of corner skin modes and extreme sensitivity of the spectral flow to the boundary conditions. The violation of the conventional bulk-boundary correspondence in the second-order NHSE implies that modification of the topological band theory is inevitable in higher dimensional non-Hermitian systems.

cond-mat.mes-hall↗

arXiv2021

GEO satellites on-orbit repairing mission planning with mission deadline constraint using a large neighborhood search-genetic algorithm

This paper proposed a novel large neighborhood search-adaptive genetic algorithm (LNS-AGA) for many-to-many on-orbit repairing mission planning of geosynchronous orbit (GEO) satellites with mission deadline constraint. In the many-to-many on-orbit repairing scenario, several servicing spacecrafts and target satellites are located in GEO orbits which have different inclination, RAAN and true anomaly. Each servicing spacecraft need to rendezvous with target satellites to perform repairing missions under limited fuel. The mission objective is to find the optimal servicing sequence and orbit rendezvous time of every servicing spacecraft to minimize total cost of all servicing spacecrafts with all target satellites repaired. Firstly, a time-dependent orbital rendezvous strategy is proposed, which can handle the mission deadline constraint. Besides, it is also cost-effective compared with the existing strategy. Based on this strategy, the many-to-many on-orbit repairing mission planning model can be simplified to an integer programming problem, which is established based on the vehicle routing problem with time windows (VRPTW) model. In order to efficiently find a feasible optimal solution under complicated constraints, a hybrid adaptive genetic algorithm combining the large neighborhood search procedure is designed. The operations of "destroy" and "repair" are used on the elite individuals in each generation of the genetic algorithm to enhance local search capabilities. Finally, the simulations under different scenarios are carried out to verify the effectiveness of the presented algorithm and orbital rendezvous strategy, which performs better than the traditional genetic algorithm.

eess.SY↗

arXiv2021

An Open Problem on the Bentness of Mesnager's Functions

Let $n=2m$. In the present paper, we study the binomial Boolean functions of the form $$f_{a,b}(x) = \mathrm{Tr}_1^{n}(a x^{2^m-1 }) +\mathrm{Tr}_1^{2}(bx^{\frac{2^n-1}{3} }), $$ where $m$ is an even positive integer, $a\in \mathbb{F}_{2^n}^*$ and $b\in \mathbb{F}_4^*$. We show that $ f_{a,b}$ is a bent function if the Kloosterman sum $$K_{m}\left(a^{2^m+1}\right)=1+ \sum_{x\in \mathbb{F}_{2^m}^*} (-1)^{\mathrm{Tr}_1^{m}(a^{2^m+1} x+ \frac{1}{x})}$$ equals $4$, thus settling an open problem of Mesnager. The proof employs tools including computing Walsh coefficients of Boolean functions via multiplicative characters, divisibility properties of Gauss sums, and graph theory.

cs.IT↗

arXiv2020

Exceptional surface-enhanced Rotation Sensing with Robustness in an optical microcavity

Exceptional points (EPs) are special singularities of non-Hermitian Hamiltonians. At an EP, two or more eigenvalues and the corresponding eigenstates coalesce. Recently, EP-based optical gyroscope near an EP was extensively investigated to improve the response to rotation. However, the highly resultant dimensionality in the eigenstate space causing from the EP-based system is sensitivity to more external perturbations, that is, the great response at EPs requires strict implementation conditions. To solve this problem, a new non-Hermitian structure was proposed that realizes an exceptional surface (EP surface) constructed of numerous EPs embedded in a high-dimensional parameter space. With respect to the isolated EPs, non-Hermitian EP surface provides additional degrees of freedom to permit the undesired perturbations (such as fabrication errors) shifts along the EP surface which corresponds to the robustness. On the other hand, the rotation will force the system away from these EPs obtaining a sensitivity enhancement of three orders of magnitude, compared to the traditional Sagnac effect rotation detecting. The EP surface system has a potential to combine robustness with enhanced sensitivity of the rotation.

physics.optics↗

arXiv2020

High-sensitivity bio-sensor based on the real-splitting indirectly coupled Anti-Parity time symmetric WGMs

Detecting the size of single nanoparticle with high precision is crucial to understanding the characteristic of the nanoparticle. In this paper, we research the single particle detection based on the Anti-parity time symmetric (APT) indirectly coupled WGMs. The results show that the Anti-parity time symmetric WGM nanoparticle sensor exhibits giant enhancement in frequency splitting compared with single WGM sensor, when the system operating at exceptional point (EP). With respect to the parity-time symmetric nanoparticle sensor, our research exhibits a real eigenfrequency splitting, which can be directly detected.

physics.optics↗

arXiv2019

PT-ResNet: Perspective Transformation-Based Residual Network for Semantic Road Image Segmentation

Semantic road region segmentation is a high-level task, which paves the way towards road scene understanding. This paper presents a residual network trained for semantic road segmentation. Firstly, we represent the projections of road disparities in the v-disparity map as a linear model, which can be estimated by optimizing the v-disparity map using dynamic programming. This linear model is then utilized to reduce the redundant information in the left and right road images. The right image is also transformed into the left perspective view, which greatly enhances the road surface similarity between the two images. Finally, the processed stereo images and their disparity maps are concatenated to create a set of 3D images, which are then utilized to train our neural network. The experimental results illustrate that our network achieves a maximum F1-measure of approximately 91.19% when analyzing the images from the KITTI road dataset.

cs.CV↗

arXiv2019

Autonomous UAV Landing System Based on Visual Navigation

In this paper, we present an autonomous unmanned aerial vehicle (UAV) landing system based on visual navigation. We design the landmark as a topological pattern in order to enable the UAV to distinguish the landmark from the environment easily. In addition, a dynamic thresholding method is developed for image binarization to improve detection efficiency. The relative distance in the horizontal plane is calculated according to effective image information, and the relative height is obtained using a linear interpolation method. The landing experiments are performed on a static and a moving platform, respectively. The experimental results illustrate that our proposed landing system performs robustly and accurately.

cs.RO↗