arXiv subjects

Jun Wang

Publications and source records attributed to Jun Wang.

At least 631 records · Page 35Linked to original sources

arXiv2024

Attosecond Delays in X-ray Molecular Ionization

The photoelectric effect is not truly instantaneous, but exhibits attosecond delays that can reveal complex molecular dynamics. Sub-femtosecond duration light pulses provide the requisite tools to resolve the dynamics of photoionization. Accordingly, the past decade has produced a large volume of work on photoionization delays following single photon absorption of an extreme ultraviolet (XUV) photon. However, the measurement of time-resolved core-level photoionization remained out of reach. The required x-ray photon energies needed for core-level photoionization were not available with attosecond tabletop sources. We have now measured the x-ray photoemission delay of core-level electrons, and here report unexpectedly large delays, ranging up to 700 attoseconds in NO near the oxygen K-shell threshold. These measurements exploit attosecond soft x-ray pulses from a free-electron laser (XFEL) to scan across the entire region near the K-shell threshold. Furthermore, we find the delay spectrum is richly modulated, suggesting several contributions including transient trapping of the photoelectron due to shape resonances, collisions with the Auger-Meitner electron that is emitted in the rapid non-radiative relaxation of the molecule, and multi-electron scattering effects. The results demonstrate how x-ray attosecond experiments, supported by comprehensive theoretical modelling, can unravel the complex correlated dynamics of core-level photoionization.

physics.atom-ph↗

arXiv2024

User Behavior Simulation with Large Language Model based Agents

Simulating high quality user behavior data has always been a fundamental problem in human-centered applications, where the major difficulty originates from the intricate mechanism of human decision process. Recently, substantial evidences have suggested that by learning huge amounts of web knowledge, large language models (LLMs) can achieve human-like intelligence. We believe these models can provide significant opportunities to more believable user behavior simulation. To inspire such direction, we propose an LLM-based agent framework and design a sandbox environment to simulate real user behaviors. Based on extensive experiments, we find that the simulated behaviors of our method are very close to the ones of real humans. Concerning potential applications, we simulate and study two social phenomenons including (1) information cocoons and (2) user conformity behaviors. This research provides novel simulation paradigms for human-centered applications.

cs.IR↗

arXiv2024

Case Study: Testing Model Capabilities in Some Reasoning Tasks

Large Language Models (LLMs) excel in generating personalized content and facilitating interactive dialogues, showcasing their remarkable aptitude for a myriad of applications. However, their capabilities in reasoning and providing explainable outputs, especially within the context of reasoning abilities, remain areas for improvement. In this study, we delve into the reasoning abilities of LLMs, highlighting the current challenges and limitations that hinder their effectiveness in complex reasoning scenarios.

cs.CL↗

arXiv2024

Natural Language Reinforcement Learning

Reinforcement Learning (RL) has shown remarkable abilities in learning policies for decision-making tasks. However, RL is often hindered by issues such as low sample efficiency, lack of interpretability, and sparse supervision signals. To tackle these limitations, we take inspiration from the human learning process and introduce Natural Language Reinforcement Learning (NLRL), which innovatively combines RL principles with natural language representation. Specifically, NLRL redefines RL concepts like task objectives, policy, value function, Bellman equation, and policy iteration in natural language space. We present how NLRL can be practically implemented with the latest advancements in large language models (LLMs) like GPT-4. Initial experiments over tabular MDPs demonstrate the effectiveness, efficiency, and also interpretability of the NLRL framework.

cs.CL↗

arXiv2024

Intelligent Agricultural Management Considering N$_2$O Emission and Climate Variability with Uncertainties

This study examines how artificial intelligence (AI), especially Reinforcement Learning (RL), can be used in farming to boost crop yields, fine-tune nitrogen use and watering, and reduce nitrate runoff and greenhouse gases, focusing on Nitrous Oxide (N$_2$O) emissions from soil. Facing climate change and limited agricultural knowledge, we use Partially Observable Markov Decision Processes (POMDPs) with a crop simulator to model AI agents' interactions with farming environments. We apply deep Q-learning with Recurrent Neural Network (RNN)-based Q networks for training agents on optimal actions. Also, we develop Machine Learning (ML) models to predict N$_2$O emissions, integrating these predictions into the simulator. Our research tackles uncertainties in N$_2$O emission estimates with a probabilistic ML approach and climate variability through a stochastic weather model, offering a range of emission outcomes to improve forecast reliability and decision-making. By incorporating climate change effects, we enhance agents' climate adaptability, aiming for resilient agricultural practices. Results show these agents can align crop productivity with environmental concerns by penalizing N$_2$O emissions, adapting effectively to climate shifts like warmer temperatures and less rain. This strategy improves farm management under climate change, highlighting AI's role in sustainable agriculture.

cs.LG↗

arXiv2024

Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training

Recent works like Tree-of-Thought (ToT) and Reasoning via Planning (RAP) aim to augment the reasoning capabilities of LLMs by using tree-search algorithms to guide multi-step reasoning. These methods rely on prompting a pre-trained model to serve as a value function and focus on problems with low search depth. As a result, these methods will not work in domains where the pre-trained LLM does not have enough knowledge to serve as an effective value function or in domains that require long-horizon planning. To address these limitations, we present an AlphaZero-like tree-search learning framework for LLMs (termed TS-LLM), systematically illustrating how tree-search with a learned value function can guide LLM decoding. TS-LLM distinguishes itself in two key ways. (1) Leveraging a learned value function and AlphaZero-like algorithms, our approach can be generally adaptable to a wide range of tasks, language models of any size, and tasks of varying search depths. (2) Our approach can guide LLMs during both inference and training, iteratively improving the LLM. Empirical results across reasoning, planning, alignment, and decision-making tasks show that TS-LLM outperforms existing approaches and can handle trees with a depth of 64.

cs.LG↗

arXiv2024

A Statistical Model of Bursty Mixed Gaussian-impulsive Noise: Model and Parameter Estimation

Non-Gaussian impulsive noise (IN) with memory exists in many practical applications. When it is mixed with white Gaussian noise (WGN), the resultant mixed noise will be bursty. The performance of communication systems will degrade significantly under bursty mixed noise if the bursty characteristic is ignored. A proper model for the bursty mixed noise and corresponding algorithms needs to be designed to obtain desirable performance but there is no such model reported to the best of our knowledge. The important problem is addressed in the two-part paper. In the first part, we propose a closed-form heavy-tailed multivariate probability density function (PDF) that to model the bursty mixed noise. This model is the weighted addition of gaussian distribution and student distribution. Then, we present the parameter estimation method based on the empirical characteristic function of the proposed model and analyze the performance of the parameter estimation. Numerical results show that our proposed bursty mixed noise model matches the measured bursty noise well. Meanwhile, the parameters of the proposed noise model can be accurately estimated in terms of mean square error (MSE).

eess.SP↗

arXiv2024

CounterCLR: Counterfactual Contrastive Learning with Non-random Missing Data in Recommendation

Recommender systems are designed to learn user preferences from observed feedback and comprise many fundamental tasks, such as rating prediction and post-click conversion rate (pCVR) prediction. However, the observed feedback usually suffer from two issues: selection bias and data sparsity, where biased and insufficient feedback seriously degrade the performance of recommender systems in terms of accuracy and ranking. Existing solutions for handling the issues, such as data imputation and inverse propensity score, are highly susceptible to additional trained imputation or propensity models. In this work, we propose a novel counterfactual contrastive learning framework for recommendation, named CounterCLR, to tackle the problem of non-random missing data by exploiting the advances in contrast learning. Specifically, the proposed CounterCLR employs a deep representation network, called CauNet, to infer non-random missing data in recommendations and perform user preference modeling by further introducing a self-supervised contrastive learning task. Our CounterCLR mitigates the selection bias problem without the need for additional models or estimators, while also enhancing the generalization ability in cases of sparse data. Experiments on real-world datasets demonstrate the effectiveness and superiority of our method.

cs.IR↗

arXiv2024

Understanding Adversarial Robustness from Feature Maps of Convolutional Layers

The adversarial robustness of a neural network mainly relies on two factors: model capacity and anti-perturbation ability. In this paper, we study the anti-perturbation ability of the network from the feature maps of convolutional layers. Our theoretical analysis discovers that larger convolutional feature maps before average pooling can contribute to better resistance to perturbations, but the conclusion is not true for max pooling. It brings new inspiration to the design of robust neural networks and urges us to apply these findings to improve existing architectures. The proposed modifications are very simple and only require upsampling the inputs or slightly modifying the stride configurations of downsampling operators. We verify our approaches on several benchmark neural network architectures, including AlexNet, VGG, RestNet18, and PreActResNet18. Non-trivial improvements in terms of both natural accuracy and adversarial robustness can be achieved under various attack and defense mechanisms. The code is available at \url{https://github.com/MTandHJ/rcm}.

cs.CV↗

arXiv2024

Experimental Demonstration of Attosecond Pump-Probe Spectroscopy with an X-ray Free-Electron Laser

Pump-probe experiments with sub-femtosecond resolution are the key to understanding electronic dynamics in quantum systems. Here we demonstrate the generation and control of sub-femtosecond pulse pairs from a two-colour X-ray free-electron laser (XFEL). By measuring the delay between the two pulses with an angular streaking diagnostic, we characterise the group velocity of the XFEL and demonstrate control of the pulse delay down to 270 as. We demonstrate the application of this technique to a pump-probe measurement in core-excited para-aminophenol. These results demonstrate the ability to perform pump-probe experiments with sub-femtosecond resolution and atomic site specificity.

physics.acc-ph↗

arXiv2024

InfoRank: Unbiased Learning-to-Rank via Conditional Mutual Information Minimization

Ranking items regarding individual user interests is a core technique of multiple downstream tasks such as recommender systems. Learning such a personalized ranker typically relies on the implicit feedback from users' past click-through behaviors. However, collected feedback is biased toward previously highly-ranked items and directly learning from it would result in a "rich-get-richer" phenomenon. In this paper, we propose a simple yet sufficient unbiased learning-to-rank paradigm named InfoRank that aims to simultaneously address both position and popularity biases. We begin by consolidating the impacts of those biases into a single observation factor, thereby providing a unified approach to addressing bias-related issues. Subsequently, we minimize the mutual information between the observation estimation and the relevance estimation conditioned on the input features. By doing so, our relevance estimation can be proved to be free of bias. To implement InfoRank, we first incorporate an attention mechanism to capture latent correlations within user-item features, thereby generating estimations of observation and relevance. We then introduce a regularization term, grounded in conditional mutual information, to promote conditional independence between relevance estimation and observation estimation. Experimental evaluations conducted across three extensive recommendation and search datasets reveal that InfoRank learns more precise and unbiased ranking strategies.

cs.IR↗

arXiv2024

Systematic Performance Evaluation Framework for LEO Mega-Constellation Satellite Networks

Low Earth orbit (LEO) mega-constellation satellite networks have shown great potential to extend the coverage capability of conventional terrestrial networks. How to systematically define, quantify, and assess the technical performance of LEO mega-constellation satellite networks remains an open issue. In this paper, we propose a comprehensive key performance indicator (KPI) framework for mega-constellation based LEO satellite networks. An efficient LEO constellation oriented performance evaluation methodology is then carefully designed by resorting to the concept of interfering area and spherical geographic cell. We have carried out rigorous system-level simulations and provided numerical results to assess the KPI framework. It can be observed that the achieved area traffic capacity of the reference LEO constellation is around 4 Kbps/km2, with service availability ranging from 0.36 to 0.39. Besides, the average access success probability and handover failure rate is approximate to 96% and 10%, respectively, in the nearest satellite association scheme.

cs.PF↗

arXiv2024

A Framework to Implement 1+N Multi-task Fine-tuning Pattern in LLMs Using the CGC-LORA Algorithm

With the productive evolution of large language models (LLMs) in the field of natural language processing (NLP), tons of effort has been made to effectively fine-tune common pre-trained LLMs to fulfill a variety of tasks in one or multiple specific domain. In practice, there are two prevailing ways, in which the adaptation can be achieved: (i) Multiple Independent Models: Pre-trained LLMs are fine-tuned a few times independently using the corresponding training samples from each task. (ii) An Integrated Model: Samples from all tasks are employed to fine-tune a pre-trianed LLM unitedly. To address the high computing cost and seesawing issue simultaneously, we propose a unified framework that implements a 1 + N mutli-task fine-tuning pattern in LLMs using a novel Customized Gate Control (CGC) Low-rank Adaptation (LoRA) algorithm. Our work aims to take an advantage of both MTL (i.e., CGC) and PEFT (i.e., LoRA) scheme. For a given cluster of tasks, we design an innovative layer that contains two types of experts as additional trainable parameters to make LoRA be compatible with MTL. To comprehensively evaluate the proposed framework, we conduct well-designed experiments on two public datasets. The experimental results demonstrate that the unified framework with CGC-LoRA modules achieves higher evaluation scores than all benchmarks on both two datasets.

cs.CL↗

arXiv2024

Achieving the Fundamental Limit of Lossless Analog Compression via Polarization

In this paper, we study the lossless analog compression for i.i.d. nonsingular signals via the polarization-based framework. We prove that for nonsingular source, the error probability of maximum a posteriori (MAP) estimation polarizes under the Hadamard transform, which extends the polarization phenomenon to analog domain. Building on this insight, we propose partial Hadamard compression and develop the corresponding analog successive cancellation (SC) decoder. The proposed scheme consists of deterministic measurement matrices and non-iterative reconstruction algorithm, providing benefits in both space and computational complexity. Using the polarization of error probability, we prove that our approach achieves the information-theoretical limit for lossless analog compression developed by Wu and Verdu.

cs.IT↗

arXiv2024

Large Language Models Are Neurosymbolic Reasoners

A wide range of real-world applications is characterized by their symbolic nature, necessitating a strong capability for symbolic reasoning. This paper investigates the potential application of Large Language Models (LLMs) as symbolic reasoners. We focus on text-based games, significant benchmarks for agents with natural language capabilities, particularly in symbolic tasks like math, map reading, sorting, and applying common sense in text-based worlds. To facilitate these agents, we propose an LLM agent designed to tackle symbolic challenges and achieve in-game objectives. We begin by initializing the LLM agent and informing it of its role. The agent then receives observations and a set of valid actions from the text-based games, along with a specific symbolic module. With these inputs, the LLM agent chooses an action and interacts with the game environments. Our experimental results demonstrate that our method significantly enhances the capability of LLMs as automated agents for symbolic reasoning, and our LLM agent is effective in text-based games involving symbolic tasks, achieving an average performance of 88% across all tasks.

cs.CL↗

arXiv2024

Adversarial Masking Contrastive Learning for vein recognition

Vein recognition has received increasing attention due to its high security and privacy. Recently, deep neural networks such as Convolutional neural networks (CNN) and Transformers have been introduced for vein recognition and achieved state-of-the-art performance. Despite the recent advances, however, existing solutions for finger-vein feature extraction are still not optimal due to scarce training image samples. To overcome this problem, in this paper, we propose an adversarial masking contrastive learning (AMCL) approach, that generates challenging samples to train a more robust contrastive learning model for the downstream palm-vein recognition task, by alternatively optimizing the encoder in the contrastive learning model and a set of latent variables. First, a huge number of masks are generated to train a robust generative adversarial network (GAN). The trained generator transforms a latent variable from the latent variable space into a mask space. Then, we combine the trained generator with a contrastive learning model to obtain our AMCL, where the generator produces challenging masking images to increase the contrastive loss and the contrastive learning model is trained based on the harder images to learn a more robust feature representation. After training, the trained encoder in the contrastive learning model is combined with a classification layer to build a classifier, which is further fine-tuned on labeled training data for vein recognition. The experimental results on three databases demonstrate that our approach outperforms existing contrastive learning approaches in terms of improving identification accuracy of vein classifiers and achieves state-of-the-art recognition results.

cs.CV↗

arXiv2024

Existence and nonexistence of minimizer for Thomas-Fermi-Dirac-Von Weizsäcker model on lattice graph

The focus of our paper is to investigate the possibility of a minimizer for the Thomas-Fermi-Dirac-von Weizsäcker model on the lattice graph $\mathbb{Z}^{3}$. The model is described by the following functional: \begin{equation*} E(φ)=\sum_{y\in\mathbb{Z}^{3}}\left(|\nablaφ(y)|^2+ (φ(y))^{\frac{10}{3}}-(φ(y))^{\frac{8}{3}}\right)+ \sum_{x,y\in\mathbb{Z}^{3}\atop ~\ y\neq x\hfill}\frac{φ^2(x)φ^2(y)}{|x-y|}, \end{equation*} with the additional constraint that $\sum\limits_{y\in\mathbb{Z}^{3}} φ^2(y)=m$ is sufficiently small. We also prove the nonexistence of a minimizer provided the mass $m$ is adequately large. Furthermore, we extend our analysis to a subset $Ω\subset \mathbb{Z}^{3}$ and prove the nonexistence of a minimizer for the following functional: \begin{equation*} E(Ω)=|\partialΩ|+\sum_{x,y\inΩ\atop ~y\neq x\hfill}\frac{1}{|x-y|}, \end{equation*} under the constraint that $|Ω|=V$ is sufficiently large.

math.AP↗

arXiv2024

Multi-bump solutions for the nonlinear magnetic Schrödinger equation with logarithmic nonlinearity

In this paper, we study the following nonlinear magnetic Schrödinger equation with logarithmic nonlinearity \begin{equation*} -(\nabla+iA(x))^2u+λV(x)u =|u|^{q-2}u+u\log |u|^2,\ u\in H^1(\mathbb{R}^N,\mathbb{C}), \end{equation*} where the magnetic potential $A \in L_{l o c}^2\left(\mathbb{R}^N, \mathbb{R}^N\right)$, $2 0$ is a parameter and the nonnegative continuous function $V: \mathbb{R}^N \rightarrow \mathbb{R}$ has the deepening potential well. Using the variational methods, we obtain that the equation has at least $2^k-1$ multi-bump solutions when $λ>0$ is large enough.

math.AP↗