arXiv subjects

Ning Wu

Publications and source records attributed to Ning Wu.

At least 37 records · Page 2Linked to original sources

arXiv2024

HGCH: A Hyperbolic Graph Convolution Network Model for Heterogeneous Collaborative Graph Recommendation

User-item interaction data in collaborative filtering and graph modeling tasks often exhibit power-law characteristics, which suggest the suitability of hyperbolic space modeling. Hyperbolic Graph Convolution Neural Networks (HGCNs) are a novel technique that leverages the advantages of GCN and hyperbolic space, and then achieves remarkable results. However, existing HGCN methods have several drawbacks: they fail to fully leverage hyperbolic space properties due to arbitrary embedding initialization and imprecise tangent space aggregation; they overlook auxiliary information that could enrich the collaborative graph; and their training convergence is slow due to margin ranking loss and random negative sampling. To overcome these challenges, we propose Hyperbolic Graph Collaborative for Heterogeneous Recommendation (HGCH), an enhanced HGCN-based model for collaborative filtering that integrates diverse side information into a heterogeneous collaborative graph and improves training convergence speed. HGCH first preserves the long-tailed nature of the graph by initializing node embeddings with power law prior; then it aggregates neighbors in hyperbolic space using the gyromidpoint method for accurate computation; finally, it fuses multiple embeddings from different hyperbolic spaces by the gate fusion with prior. Moreover, HGCH employs a hyperbolic user-specific negative sampling to speed up convergence. We evaluate HGCH on four real datasets, and the results show that HGCH achieves competitive results and outperforms leading baselines, including HGCNs. Extensive ablation studies further confirm its effectiveness.

cs.IR↗

arXiv2024

Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations

Existing research predominantly focuses on developing powerful language learning models (LLMs) for mathematical reasoning within monolingual languages, with few explorations in preserving efficacy in a multilingual context. To bridge this gap, this paper pioneers exploring and training powerful Multilingual Math Reasoning (xMR) LLMs. Firstly, by utilizing translation, we construct the first multilingual math reasoning instruction dataset, MGSM8KInstruct, encompassing ten distinct languages, thus addressing the issue of training data scarcity in xMR tasks. Based on the collected dataset, we propose different training strategies to build powerful xMR LLMs, named MathOctopus, notably outperform conventional open-source LLMs and exhibit superiority over ChatGPT in few-shot scenarios. Notably, MathOctopus-13B reaches 47.6% accuracy which exceeds ChatGPT 46.3% on MGSM testset. Beyond remarkable results, we unearth several pivotal observations and insights from extensive experiments: (1) When extending the rejection sampling strategy to the multilingual context, it proves effective for model performances, albeit limited. (2) Employing parallel corpora for math Supervised Fine-Tuning (SFT) across multiple languages not only significantly enhances model performance multilingually but also elevates their monolingual performance. This indicates that crafting multilingual corpora can be regarded as a vital strategy for enhancing model performance in a specific language, especially in mathematical reasoning tasks. For instance, MathOctopus-7B improves its counterparts that trained on English from 42.2% to 50.8% on GSM8K testset. Codes are available at https://github.com/microsoft/MathOctopus.

cs.CL↗

arXiv2024

ControlMath: Controllable Data Generation Promotes Math Generalist Models

Utilizing large language models (LLMs) for data augmentation has yielded encouraging results in mathematical reasoning. However, these approaches face constraints in problem diversity, potentially restricting them to in-domain/distribution data generation. To this end, we propose ControlMath, an iterative method involving an equation-generator module and two LLM-based agents. The module creates diverse equations, which the Problem-Crafter agent then transforms into math word problems. The Reverse-Agent filters and selects high-quality data, adhering to the "less is more" principle, achieving better results with fewer data points. This approach enables the generation of diverse math problems, not limited to specific domains or distributions. As a result, we collect ControlMathQA, which involves 190k math word problems. Extensive results prove that combining our dataset with in-domain datasets like GSM8K can help improve the model's mathematical ability to generalize, leading to improved performances both within and beyond specific domains.

cs.LG↗

arXiv2024

Evolution of two-magnon bound states in a higher-spin ferromagnetic chain with single-ion anisotropy: A complete solution

Few-magnon bound states in quantum spin chains have been long studied and attracted much recent attentions. For a higher-spin ferromagnetic XXZ chain with single-ion anisotropy, several features regarding the evolution of the low-lying two-magnon bound states with varying wave number were observed in the literature. However, most of these observations are only qualitatively understood due to the lack of analytical tools. By combining a set of exact two-magnon Bloch states and a plane-wave ansatz, we achieve a complete solution of the two-magnon problem in such a system. We identify parameter regions that support different types of two-magnon bound states, with the boundaries defined by algebraic equations. We discover for the first time a narrow region in which two single-ion bound states coexist. We show that the phase diagrams for distinct wave numbers are similar to each other, which enables us to map the evolution of the bound states to the rectilinear movement of a representative point for given parameters in a rescaled phase diagram. This dynamic picture provides quantitative interpretations of the observed features.

cond-mat.str-el↗

arXiv2024

Relaxing towards generalized one-body Boltzmann states

Isolated quantum systems follow the reversible unitary evolution; if we focus on the dynamics of local states and observables, they exhibit the irreversible relaxation behaviors. Here we study the local relaxation process in an isolated chain consisting of \emph{N} three level systems. Though the entropy of the full many body state keeps a constant, it turns out the total correlation of this system approximately exhibits a monotonically increasing behavior. More importantly, a variation analysis shows that, the total correlation entropy would achieve its theoretical maximum when each site stays in a generalized one-body Boltzmann state, which is not solely determined by the energy but also depends on the spin value of each onsite level. It turns out such a theoretical correlation maximum is highly coincident with the result obtained from the exact time dependent evolution. In this sense, the total correlation entropy well serves as an indicator for the dynamical irreversibility of the nonequilibrium relaxation in this isolated system.

quant-ph↗

arXiv2024

Finetuning Large Language Model for Personalized Ranking

Large Language Models (LLMs) have demonstrated remarkable performance across various domains, motivating researchers to investigate their potential use in recommendation systems. However, directly applying LLMs to recommendation tasks has proven challenging due to the significant disparity between the data used for pre-training LLMs and the specific requirements of recommendation tasks. In this study, we introduce Direct Multi-Preference Optimization (DMPO), a streamlined framework designed to bridge the gap and enhance the alignment of LLMs for recommendation tasks. DMPO enhances the performance of LLM-based recommenders by simultaneously maximizing the probability of positive samples and minimizing the probability of multiple negative samples. We conducted experimental evaluations to compare DMPO against traditional recommendation methods and other LLM-based recommendation approaches. The results demonstrate that DMPO significantly improves the recommendation capabilities of LLMs across three real-world public datasets in few-shot scenarios. Additionally, the experiments indicate that DMPO exhibits superior generalization ability in cross-domain recommendations. A case study elucidates the reasons behind these consistent improvements and also underscores DMPO's potential as an explainable recommendation system.

cs.IR↗

arXiv2024

Few-magnon excitations in a frustrated spin-$S$ ferromagnetic chain with single-ion anisotropy

We study few-magnon excitations in a finite-size spin-$S$ chain with ferromagnetic nearest-neighbor (NN) interaction $J>0$ and antiferromagnetic next-nearest-neighbor (NNN) interaction $J'<0$, in the presence of the single-ion (SI) anisotropy $D$. We first reveal the condition for the emergence of zero-excitation-energy states. In the isotropic case with $Δ=Δ'=1$ ($Δ$ and $Δ'$ are the corresponding anisotropy parameters), a threshold of $J/|J'|$ above which the ground state is ferromagnetic is determined by exact diagonalization for short chains up to $12$ sites. Using a set of exact two-magnon Bloch states, we then map the two-magnon problem to a single-particle one on an effective open chain with both NN and NNN hoppings. The whole two-magnon excitation spectrum is calculated for large systems and the commensurate-incommensurate transition in the lowest-lying mode is found to exhibit different behaviors between $S=1/2$ and higher spins due to the interplay of the SI anisotropy and the NNN interaction. For the commensurate momentum $k=-π$, the effective lattice is decoupled into two NN open chains that can be exactly solved via a plane-wave ansatz. Based on this, we analytically identify in the $Δ'-D/|J'|$ plane the regions supporting the SI or NNN exchange two-magnon bound states near the edge of the band. In particular, we prove that there always exists a lower-lying NN exchange two-magnon bound state near the band edge for arbitrary $S\geq 1/2$. Finally, we numerically calculate the $n$-magnon spectra for $S=1/2$ with $n\leq5$ by using a spin-operator matrix element method. The corresponding $n$-magnon commensurate instability regions are determined for finite chains and consistent results with prior literature are observed.

cond-mat.str-el↗

arXiv2024

Machine-learning-inspired quantum control in many-body dynamics

Achieving precise preparation of quantum many-body states is crucial for the practical implementation of quantum computation and quantum simulation. However, the inherent challenges posed by unavoidable excitations at critical points during quench processes necessitate careful design of control fields. In this work, we introduce a promising and versatile dynamic control neural network tailored to optimize control fields. We address the problem of suppressing defect density and enhancing cat-state fidelity during the passage across the critical point in the quantum Ising model. Our method facilitates seamless transitions between different objective functions by adjusting the {optimization strategy}. In comparison to gradient-based power-law quench methods, our approach demonstrates significant advantages for both small system sizes and long-term evolutions. We provide a detailed analysis of the specific forms of control fields and summarize common features for experimental implementation. Furthermore, numerical simulations demonstrate the robustness of our proposal against random noise and spin number fluctuations. The optimized defect density and cat-state fidelity exhibit a transition at a critical ratio of the quench duration to the system size, coinciding with the quantum speed limit for quantum evolution.

quant-ph↗

arXiv2024

Ploutos: Towards interpretable stock movement prediction with financial large language model

Recent advancements in large language models (LLMs) have opened new pathways for many domains. However, the full potential of LLMs in financial investments remains largely untapped. There are two main challenges for typical deep learning-based methods for quantitative finance. First, they struggle to fuse textual and numerical information flexibly for stock movement prediction. Second, traditional methods lack clarity and interpretability, which impedes their application in scenarios where the justification for predictions is essential. To solve the above challenges, we propose Ploutos, a novel financial LLM framework that consists of PloutosGen and PloutosGPT. The PloutosGen contains multiple primary experts that can analyze different modal data, such as text and numbers, and provide quantitative strategies from different perspectives. Then PloutosGPT combines their insights and predictions and generates interpretable rationales. To generate accurate and faithful rationales, the training strategy of PloutosGPT leverage rearview-mirror prompting mechanism to guide GPT-4 to generate rationales, and a dynamic token weighting mechanism to finetune LLM by increasing key tokens weight. Extensive experiments show our framework outperforms the state-of-the-art methods on both prediction accuracy and interpretability.

q-fin.ST↗

arXiv2024

Is Bigger and Deeper Always Better? Probing LLaMA Across Scales and Layers

This paper presents an in-depth analysis of Large Language Models (LLMs), focusing on LLaMA, a prominent open-source foundational model in natural language processing. Instead of assessing LLaMA through its generative output, we design multiple-choice tasks to probe its intrinsic understanding in high-order tasks such as reasoning and computation. We examine the model horizontally, comparing different sizes, and vertically, assessing different layers. We unveil several key and uncommon findings based on the designed probing tasks: (1) Horizontally, enlarging model sizes almost could not automatically impart additional knowledge or computational prowess. Instead, it can enhance reasoning abilities, especially in math problem solving, and helps reduce hallucinations, but only beyond certain size thresholds; (2) In vertical analysis, the lower layers of LLaMA lack substantial arithmetic and factual knowledge, showcasing logical thinking, multilingual and recognitive abilities, with top layers housing most computational power and real-world knowledge.

cs.CL↗

arXiv2023

Large Language Models are Diverse Role-Players for Summarization Evaluation

Text summarization has a wide range of applications in many scenarios. The evaluation of the quality of the generated text is a complex problem. A big challenge to language evaluation is that there is a clear divergence between existing metrics and human evaluation. A document summary's quality can be assessed by human annotators on various criteria, both objective ones like grammar and correctness, and subjective ones like informativeness, succinctness, and appeal. Most of the automatic evaluation methods like BLUE/ROUGE may be not able to adequately capture the above dimensions. In this paper, we propose a new evaluation framework based on LLMs, which provides a comprehensive evaluation framework by comparing generated text and reference text from both objective and subjective aspects. First, we propose to model objective and subjective dimensions of generated text based on roleplayers prompting mechanism. Furthermore, we introduce a context-based prompting mechanism that is able to generate dynamic roleplayer profiles based on input context. Finally, we design a multi-roleplayer prompting technology based on batch prompting and integrate multiple outputs into the final evaluation results. Experimental results on three real datasets for summarization show that our model is highly competitive and has a very high consistency with human annotators.

cs.CL↗

arXiv2023

RUEL: Retrieval-Augmented User Representation with Edge Browser Logs for Sequential Recommendation

Online recommender systems (RS) aim to match user needs with the vast amount of resources available on various platforms. A key challenge is to model user preferences accurately under the condition of data sparsity. To address this challenge, some methods have leveraged external user behavior data from multiple platforms to enrich user representation. However, all of these methods require a consistent user ID across platforms and ignore the information from similar users. In this study, we propose RUEL, a novel retrieval-based sequential recommender that can effectively incorporate external anonymous user behavior data from Edge browser logs to enhance recommendation. We first collect and preprocess a large volume of Edge browser logs over a one-year period and link them to target entities that correspond to candidate items in recommendation datasets. We then design a contrastive learning framework with a momentum encoder and a memory bank to retrieve the most relevant and diverse browsing sequences from the full browsing log based on the semantic similarity between user representations. After retrieval, we apply an item-level attentive selector to filter out noisy items and generate refined sequence embeddings for the final predictor. RUEL is the first method that connects user browsing data with typical recommendation datasets and can be generalized to various recommendation scenarios and datasets. We conduct extensive experiments on four real datasets for sequential recommendation tasks and demonstrate that RUEL significantly outperforms state-of-the-art baselines. We also conduct ablation studies and qualitative analysis to validate the effectiveness of each component of RUEL and provide additional insights into our method.

cs.IR↗

arXiv2023

Quantum many-body scars in spin-1 Kitaev chain with uniaxial single-ion anisotropy

To establish a solid-state-based framework for the coexistence of quantum many-body scars and quantum criticality, we investigate the spin-1 Kitaev chain with uniaxial single-ion anisotropy (SIA). In the subspace with uniform $\mathbb{Z}_2$ gauge fields, this model can be exactly mapped to the spin-1/2 effective detuned PXP Hamiltonian, where the SIA plays a role of the static detuning term. The quench dynamics starting from the product states is symmetric between positive and negative values of the SIA, while a quantum phase transition from the Kitaev spin liquid to the dimer phase only occurs at the critical point with a negative $D_c$, implying the spontaneous breaking of the translational symmetry. We find that the coherent oscillations of quantum fidelity and certain local observables are sustained against small SIA perturbations in a quantum quench from special initial states. While the oscillation amplitudes of these observables decay with time as the SIA strength is increased, the system completely thermalizes upon approaching the critical point. In contrast, the initial polarized state, which shows an absence of revivals of quantum fidelity, will exhibit long revivals for $D<D_c$. Finally, we investigate the evolution of phase boundaries of the Kitaev spin liquid and dimer phase by introducing Heisenberg interactions, which spoil the $\mathbb{Z}_2$ gauge fields. A complete phase diagram is given by the infinite time-evolving block decimation method and the ground state properties of each phase are accurately captured by various spin correlations. Our work opens the door to understanding exotic connections between many-body scars and quantum criticality in systems with higher spins.

cond-mat.str-el↗

arXiv2023

Lexicon-Enhanced Self-Supervised Training for Multilingual Dense Retrieval

Recent multilingual pre-trained models have shown better performance in various multilingual tasks. However, these models perform poorly on multilingual retrieval tasks due to lacking multilingual training data. In this paper, we propose to mine and generate self-supervised training data based on a large-scale unlabeled corpus. We carefully design a mining method which combines the sparse and dense models to mine the relevance of unlabeled queries and passages. And we introduce a query generator to generate more queries in target languages for unlabeled passages. Through extensive experiments on Mr. TYDI dataset and an industrial dataset from a commercial search engine, we demonstrate that our method performs better than baselines based on various pre-trained multilingual models. Our method even achieves on-par performance with the supervised method on the latter dataset.

cs.IR↗

arXiv2023

Empowering Dual-Encoder with Query Generator for Cross-Lingual Dense Retrieval

In monolingual dense retrieval, lots of works focus on how to distill knowledge from cross-encoder re-ranker to dual-encoder retriever and these methods achieve better performance due to the effectiveness of cross-encoder re-ranker. However, we find that the performance of the cross-encoder re-ranker is heavily influenced by the number of training samples and the quality of negative samples, which is hard to obtain in the cross-lingual setting. In this paper, we propose to use a query generator as the teacher in the cross-lingual setting, which is less dependent on enough training samples and high-quality negative samples. In addition to traditional knowledge distillation, we further propose a novel enhancement method, which uses the query generator to help the dual-encoder align queries from different languages, but does not need any additional parallel sentences. The experimental results show that our method outperforms the state-of-the-art methods on two benchmark datasets.

cs.IR↗

arXiv2022

Relaxation of antiferromagnetic order and growth of Rényi entropy in a generalized Heisenberg star

Interacting central spin systems, in which a central spin is coupled to a strongly correlated spin bath with intrabath interaction, consist of an important class of spin systems beyond the usual Gaudin magnet. These systems are relevant to several realistic setups and serve as an interesting platform to study interaction controlled decoherence and frustration induced instability of magnetic order. Using an equations-of-motion method based on analytical representations of spin-operator matrix elements in the XX chain, we obtain exact long-time dynamics of a generalized Heisenberg star consisting of a spin-$S$ central spin and an inhomogeneously coupled XXZ chain of $N\leq 16$ bath spins. In contrast to previous studies where the central spin dynamics is mainly concerned, we focus on the influence of the central spin on the dynamics of magnetic orders within the spin bath. By preparing the XXZ bath in a Néel state, we find that in the gapless phase of the bath even weak system-bath coupling could lead to nearly perfect relaxation of the antiferromagnetic order. In the gapped phase, the staggered magnetization decays rapidly and approaches a steady value that increases with increasing anisotropy parameter. These findings suggest the possibility of controlling internal dynamics of strongly correlated many-spin systems by certain coupled auxiliary systems of even few degrees of freedom. We also study the dynamics of the Rényi entanglement entropy of the central spin when the bath is prepared in the ground state. Both the overall profile and initial growth rate of the Rényi entropy are found to exhibit minima at the critical point of the XXZ bath.

cond-mat.mes-hall↗

arXiv2022

Ground-state and dynamical properties of a spin-$S$ Heisenberg star

We generalize the Heisenberg star consisting of a spin-1/2 central spin and a homogeneously coupled spin bath modeled by the XXX ring [Richter J and Voigt A 1994 \emph{J. Phys. A: Math. Gen.} \textbf{27} 1139-1149] to the case of arbitrary central-spin size $S 1/2$ exhibits collapse-revival behaviors with fine structures. However, the collapse-revival phenomena is found to be fragile with respect to anisotropy of the intrabath coupling.

quant-ph↗

arXiv2022

Unsupervised Context Aware Sentence Representation Pretraining for Multi-lingual Dense Retrieval

Recent research demonstrates the effectiveness of using pretrained language models (PLM) to improve dense retrieval and multilingual dense retrieval. In this work, we present a simple but effective monolingual pretraining task called contrastive context prediction~(CCP) to learn sentence representation by modeling sentence level contextual relation. By pushing the embedding of sentences in a local context closer and pushing random negative samples away, different languages could form isomorphic structure, then sentence pairs in two different languages will be automatically aligned. Our experiments show that model collapse and information leakage are very easy to happen during contrastive training of language model, but language-specific memory bank and asymmetric batch normalization operation play an essential role in preventing collapsing and information leakage, respectively. Besides, a post-processing for sentence embedding is also very effective to achieve better retrieval performance. On the multilingual sentence retrieval task Tatoeba, our model achieves new SOTA results among methods without using bilingual data. Our model also shows larger gain on Tatoeba when transferring between non-English pairs. On two multi-lingual query-passage retrieval tasks, XOR Retrieve and Mr.TYDI, our model even achieves two SOTA results in both zero-shot and supervised setting among all pretraining models using bilingual data.

cs.IR↗