arXiv subjects

Wei Lin

Publications and source records attributed to Wei Lin.

At least 163 records · Page 9Linked to original sources

arXiv2024

Outage Probability Analysis of Uplink Heterogeneous Non-terrestrial Networks: A Novel Stochastic Geometry Model

In harsh environments such as mountainous terrain, dense vegetation areas, or urban landscapes, a single type of unmanned aerial vehicles (UAVs) may encounter challenges like flight restrictions, difficulty in task execution, or increased risk. Therefore, employing multiple types of UAVs, along with satellite assistance, to collaborate becomes essential in such scenarios. In this context, we present a stochastic geometry based approach for modeling the heterogeneous non-terrestrial networks (NTNs) by using the classical binomial point process and introducing a novel point process, called Mat{é}rn hard-core cluster process (MHCCP). Our MHCCP possesses both the exclusivity and the clustering properties, thus it can better model the aircraft group composed of multiple clusters. Then, we derive closed-form expressions of the outage probability (OP) for the uplink (aerial-to-satellite) of heterogeneous NTNs. Unlike existing studies, our analysis relies on a more advanced system configuration, where the integration of beamforming and frequency division multiple access, and the shadowed-Rician (SR) fading model for interference power, are considered. The accuracy of our theoretical derivation is confirmed by Monte Carlo simulations. Our research offers fundamental insights into the system-level performance optimization of NTNs.

cs.NI↗

arXiv2024

BladeDISC++: Memory Optimizations Based On Symbolic Shape

Recent deep learning workloads exhibit dynamic characteristics, leading to the rising adoption of dynamic shape compilers. These compilers can generate efficient kernels for dynamic shape graphs characterized by a fixed graph topology and uncertain tensor shapes. However, memory optimization, although particularly crucial in this large model era, remains relatively underexplored for dynamic shape graphs. The fundamental challenge lies in the lack of precise tensor shapes which are essential in conventional methods such as operation scheduling(op scheduling) and rematerialization. To address this challenge, we propose op scheduling and rematerialization approaches based on symbolic shapes and developed BladeDISC++. Besides, since rematerialization decisions cannot be made solely at compile time when tensor shapes are unknown, BladeDISC++ employs a compilation-runtime combined strategy to optimally address shape dynamics. Evaluations indicate that BladeDISC++ effectively reduces memory usage for dynamic shape graphs, achieving memory consumption comparable to optimizations using precise shapes, thereby promoting the broader adoption of dynamic shape compilers.

cs.DC↗

arXiv2024

Utilizing Causal Network Markers to Identify Tipping Points ahead of Critical Transition

Early-warning signals of delicate design are always used to predict critical transitions in complex systems, which makes it possible to render the systems far away from the catastrophic state by introducing timely interventions. Traditional signals including the dynamical network biomarker (DNB), based on statistical properties such as variance and autocorrelation of nodal dynamics, overlook directional interactions and thus have limitations in capturing underlying mechanisms and simultaneously sustaining robustness against noise perturbations. This paper therefore introduces a framework of causal network markers (CNMs) by incorporating causality indicators, which reflect the directional influence between variables. Actually, to detect and identify the tipping points ahead of critical transition, two markers are designed: CNM-GC for linear causality and CNM-TE for non-linear causality, as well as a functional representation of different causality indicators and a clustering technique to verify the system's dominant group. Through demonstrations using benchmark models and real-world datasets of epileptic seizure, the framework of CNMs shows higher predictive power and accuracy than the traditional DNB indicator. It is believed that, due to the versatility and scalability, the CNMs are suitable for comprehensively evaluating the systems. The most possible direction for application includes the identification of tipping points in clinical disease.

cs.LG↗

arXiv2024

Semantic Convergence: Harmonizing Recommender Systems via Two-Stage Alignment and Behavioral Semantic Tokenization

Large language models (LLMs), endowed with exceptional reasoning capabilities, are adept at discerning profound user interests from historical behaviors, thereby presenting a promising avenue for the advancement of recommendation systems. However, a notable discrepancy persists between the sparse collaborative semantics typically found in recommendation systems and the dense token representations within LLMs. In our study, we propose a novel framework that harmoniously merges traditional recommendation models with the prowess of LLMs. We initiate this integration by transforming ItemIDs into sequences that align semantically with the LLMs space, through the proposed Alignment Tokenization module. Additionally, we design a series of specialized supervised learning tasks aimed at aligning collaborative signals with the subtleties of natural language semantics. To ensure practical applicability, we optimize online inference by pre-caching the top-K results for each user, reducing latency and improving effciency. Extensive experimental evidence indicates that our model markedly improves recall metrics and displays remarkable scalability of recommendation systems.

cs.IR↗

arXiv2024

ConMe: Rethinking Evaluation of Compositional Reasoning for Modern VLMs

Compositional Reasoning (CR) entails grasping the significance of attributes, relations, and word order. Recent Vision-Language Models (VLMs), comprising a visual encoder and a Large Language Model (LLM) decoder, have demonstrated remarkable proficiency in such reasoning tasks. This prompts a crucial question: have VLMs effectively tackled the CR challenge? We conjecture that existing CR benchmarks may not adequately push the boundaries of modern VLMs due to the reliance on an LLM-only negative text generation pipeline. Consequently, the negatives produced either appear as outliers from the natural language distribution learned by VLMs' LLM decoders or as improbable within the corresponding image context. To address these limitations, we introduce ConMe -- a compositional reasoning benchmark and a novel data generation pipeline leveraging VLMs to produce `hard CR Q&A'. Through a new concept of VLMs conversing with each other to collaboratively expose their weaknesses, our pipeline autonomously generates, evaluates, and selects challenging compositional reasoning questions, establishing a robust CR benchmark, also subsequently validated manually. Our benchmark provokes a noteworthy, up to 33%, decrease in CR performance compared to preceding benchmarks, reinstating the CR challenge even for state-of-the-art VLMs.

cs.CV↗

arXiv2024

Wafer-scale Semiconductor Grafting: Enabling High-Performance, Lattice-Mismatched Heterojunctions

Semiconductor heterojunctions are foundational to many advanced electronic and optoelectronic devices. However, achieving high-quality, lattice-mismatched interfaces remains challenging, limiting both scalability and device performance. Semiconductor grafting offers a promising solution by directly forming electrically active, lattice-mismatched heterojunctions between dissimilar materials. However, its scalability and uniformity at the wafer level have yet to be demonstrated. This work demonstrates the achievement of highly uniform, reproducible results across silicon, sapphire, and gallium nitride (GaN) substrates using wafer-scale semiconductor grafting. To illustrate this scalability, we conducted an in-depth study of a grafted Si/GaN heterojunction, examining band alignment through X-ray photoelectron spectroscopy and confirming crystallinity and interfacial integrity with scanning transmission electron microscopy. The resulting p-n diodes exhibit significantly enhanced electrical performance and wafer-scale uniformity compared to conventional approaches. This work establishes wafer-scale semiconductor grafting as a versatile and scalable technology, bridging the gap between laboratory-scale research and industrial manufacturing for heterogeneous semiconductor integration, and paving the way for novel, high-performance electronic and optoelectronic devices.

physics.app-ph↗

arXiv2024

High-throughput Screening of Ferrimagnetic Semiconductors With Ultrahigh N$\acute{e}$el Temperature

Ferrimagnetic semiconductors, integrated with net magnetization, antiferromagnetic coupling and semi-conductivity, have constructed an ideal platform for spintronics. For practical applications, achieving high N$\acute{e}$el temperatures ($T_{\mathrm{N}}$) is very desirable, but remains a significant challenge. Here, via high-throughput density-functional-theory calculations, we identify 19 intrinsic ferrimagnetic semiconductor candidates from nearly 44,000 structures in the Materials Project database, including 10 ferrimagnetic bipolar magnetic semiconductors (BMS) and 9 ferrimagnetic half semiconductors (HSC). Notably, the BMS \ce{NaFe5O8} possesses a high $T_{\mathrm{N}}$ of 768 K. By element substitutions, we obtain an HSC \ce{NaFe5S8} with a $T_{\mathrm{N}}$ of 957 K and a BMS \ce{LiFe5O8} with a $T_{\mathrm{N}}$ reaching 1059 K. Our results pave a promising avenue toward the development of ferrimagnetic spintronics at ambient temperature.

cond-mat.mtrl-sci↗

arXiv2024

Text2NKG: Fine-Grained N-ary Relation Extraction for N-ary relational Knowledge Graph Construction

Beyond traditional binary relational facts, n-ary relational knowledge graphs (NKGs) are comprised of n-ary relational facts containing more than two entities, which are closer to real-world facts with broader applications. However, the construction of NKGs remains at a coarse-grained level, which is always in a single schema, ignoring the order and variable arity of entities. To address these restrictions, we propose Text2NKG, a novel fine-grained n-ary relation extraction framework for n-ary relational knowledge graph construction. We introduce a span-tuple classification approach with hetero-ordered merging and output merging to accomplish fine-grained n-ary relation extraction in different arity. Furthermore, Text2NKG supports four typical NKG schemas: hyper-relational schema, event-based schema, role-based schema, and hypergraph-based schema, with high flexibility and practicality. The experimental results demonstrate that Text2NKG achieves state-of-the-art performance in F1 scores on the fine-grained n-ary relation extraction benchmark. Our code and datasets are publicly available.

cs.AI↗

arXiv2024

ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models

Knowledge Base Question Answering (KBQA) aims to answer natural language questions over large-scale knowledge bases (KBs), which can be summarized into two crucial steps: knowledge retrieval and semantic parsing. However, three core challenges remain: inefficient knowledge retrieval, mistakes of retrieval adversely impacting semantic parsing, and the complexity of previous KBQA methods. To tackle these challenges, we introduce ChatKBQA, a novel and simple generate-then-retrieve KBQA framework, which proposes first generating the logical form with fine-tuned LLMs, then retrieving and replacing entities and relations with an unsupervised retrieval method, to improve both generation and retrieval more directly. Experimental results show that ChatKBQA achieves new state-of-the-art performance on standard KBQA datasets, WebQSP, and CWQ. This work can also be regarded as a new paradigm for combining LLMs with knowledge graphs (KGs) for interpretable and knowledge-required question answering. Our code is publicly available.

cs.CL↗

arXiv2024

Governing equation discovery of a complex system from snapshots

Complex systems in physics, chemistry, and biology that evolve over time with inherent randomness are typically described by stochastic differential equations (SDEs). A fundamental challenge in science and engineering is to determine the governing equations of a complex system from snapshot data. Traditional equation discovery methods often rely on stringent assumptions, such as the availability of the trajectory information or time-series data, and the presumption that the underlying system is deterministic. In this work, we introduce a data-driven, simulation-free framework, called Sparse Identification of Differential Equations from Snapshots (SpIDES), that discovers the governing equations of a complex system from snapshots by utilizing the advanced machine learning techniques to perform three essential steps: probability flow reconstruction, probability density estimation, and Bayesian sparse identification. We validate the effectiveness and robustness of SpIDES by successfully identifying the governing equation of an over-damped Langevin system confined within two potential wells. By extracting interpretable drift and diffusion terms from the SDEs, our framework provides deeper insights into system dynamics, enhances predictive accuracy, and facilitates more effective strategies for managing and simulating stochastic systems.

cs.LG↗

arXiv2024

PertEval: Unveiling Real Knowledge Capacity of LLMs with Knowledge-Invariant Perturbations

Expert-designed close-ended benchmarks are indispensable in assessing the knowledge capacity of large language models (LLMs). Despite their widespread use, concerns have mounted regarding their reliability due to limited test scenarios and an unavoidable risk of data contamination. To rectify this, we present PertEval, a toolkit devised for in-depth probing of LLMs' knowledge capacity through \textbf{knowledge-invariant perturbations}. These perturbations employ human-like restatement techniques to generate on-the-fly test samples from static benchmarks, meticulously retaining knowledge-critical content while altering irrelevant details. Our toolkit further includes a suite of \textbf{response consistency analyses} that compare performance on raw vs. perturbed test sets to precisely assess LLMs' genuine knowledge capacity. Six representative LLMs are re-evaluated using PertEval. Results reveal significantly inflated performance of the LLMs on raw benchmarks, including an absolute 25.8% overestimation for GPT-4. Additionally, through a nuanced response pattern analysis, we discover that PertEval retains LLMs' uncertainty to specious knowledge, and reveals their potential rote memorization to correct options which leads to overestimated performance. We also find that the detailed response consistency analyses by PertEval could illuminate various weaknesses in existing LLMs' knowledge mastery and guide the development of refinement. Our findings provide insights for advancing more robust and genuinely knowledgeable LLMs. Our code is available at \url{https://github.com/aigc-apps/PertEval}.

cs.CL↗

arXiv2024

CTA-Net: A CNN-Transformer Aggregation Network for Improving Multi-Scale Feature Extraction

Convolutional neural networks (CNNs) and vision transformers (ViTs) have become essential in computer vision for local and global feature extraction. However, aggregating these architectures in existing methods often results in inefficiencies. To address this, the CNN-Transformer Aggregation Network (CTA-Net) was developed. CTA-Net combines CNNs and ViTs, with transformers capturing long-range dependencies and CNNs extracting localized features. This integration enables efficient processing of detailed local and broader contextual information. CTA-Net introduces the Light Weight Multi-Scale Feature Fusion Multi-Head Self-Attention (LMF-MHSA) module for effective multi-scale feature integration with reduced parameters. Additionally, the Reverse Reconstruction CNN-Variants (RRCV) module enhances the embedding of CNNs within the transformer architecture. Extensive experiments on small-scale datasets with fewer than 100,000 samples show that CTA-Net achieves superior performance (TOP-1 Acc 86.76\%), fewer parameters (20.32M), and greater efficiency (FLOPs 2.83B), making it a highly efficient and lightweight solution for visual tasks on small-scale datasets (fewer than 100,000).

cs.CV↗

arXiv2024

Meso-inflationary Peccei-Quinn symmetry breaking with non-minimal coupling

We study a realization of the inflationary scenario where the Peccei-Quinn (PQ) symmetry is spontaneously broken during inflation, facilitated by its non-minimal coupling to gravity. This results in effectively two-field inflation: the early stage is driven by an inflaton field with the PQ symmetry intact, and the later stage is driven by the PQ scalar after its effective mass becomes tachyonic, causing destabilization from the origin. The non-minimal coupling serves the dual purpose of restoring the PQ symmetry during early inflation and flattening the PQ potential post-tachyonic shift, allowing for continued slow roll. We analyze the inflationary background solutions and scalar perturbations, which are amplified at small scales via significant isocurvature perturbations generated near the symmetry-breaking epoch. These perturbations lead to second-order gravitational waves, detectable by next-generation space-based experiments.

hep-th↗

arXiv2024

Exact first passage time distribution for second-order reactions in chemical networks

The first passage time (FPT) is a generic measure that quantifies when a random quantity reaches a specific state. We consider the FTP distribution in nonlinear stochastic biochemical networks, where obtaining exact solutions of the distribution is a challenging problem. Even simple two-particle collisions cause strong nonlinearities that hinder the theoretical determination of the full FPT distribution. Previous research has either focused on analyzing the mean FPT, which provides limited information about a system, or has considered time-consuming stochastic simulations that do not clearly expose causal relationships between parameters and the system's dynamics. This paper presents the first exact theoretical solution of the full FPT distribution in a broad class of chemical reaction networks involving $A + B \rightarrow C$ type of second-order reactions. Our exact theoretical method outperforms stochastic simulations, in terms of computational efficiency, and deviates from approximate analytical solutions. Given the prevalence of bimolecular reactions in biochemical systems, our approach has the potential to enhance the understanding of real-world biochemical processes.

q-bio.MN↗

arXiv2024

Network Structure Governs Drosophila Brain Functionality

How intelligence emerges from living beings has been a fundamental question in neuroscience. However, it remains largely unanswered due to the complex neuronal dynamics and intricate connections between neurons in real neural systems. To address this challenge, we leveraged the largest available adult Drosophila connectome data set, and constructed a comprehensive computational framework based on simplified neuronal activation mechanisms to simulate the observed activation behavior within the connectome. The results revealed that even with rudimentary neuronal activation mechanisms, models grounded in real neural network structures can generate activation patterns strikingly similar to those observed in the actual brain. A significant discovery was the consistency of activation patterns across various neuronal dynamic models. This consistency, achieved with the same network structure, underscores the pivotal role of network topology in neural information processing. These results challenge the prevailing view that solely relies on neuron count or complex individual neuron dynamics. Further analysis demonstrated a near-complete separation of the visual and olfactory systems at the network level. Moreover, we found that the network distance, rather than spatial distance, is the primary determinant of activation patterns. Additionally, our experiments revealed that a reconnect rate of at least 0.1% was sufficient to disrupt the previously observed activation patterns. We also observed synergistic effects between the brain hemispheres: Even with unilateral input stimuli, visual-related neurons in both hemispheres were activated, highlighting the importance of interhemispheric communication. These findings emphasize the crucial role of network structure in neural activation and offer novel insights into the fundamental principles governing brain functionality.

q-bio.NC↗

arXiv2024

First Activations Matter: Training-Free Methods for Dynamic Activation in Large Language Models

Dynamic activation (DA) techniques, such as DejaVu and MoEfication, have demonstrated their potential to significantly enhance the inference efficiency of large language models (LLMs). However, these techniques often rely on ReLU activation functions or require additional parameters and training to maintain performance. This paper introduces a training-free Threshold-based Dynamic Activation(TDA) method that leverage sequence information to exploit the inherent sparsity of models across various architectures. This method is designed to accelerate generation speed by 18-25\% without significantly compromising task performance, thereby addressing the limitations of existing DA techniques. Moreover, we delve into the root causes of LLM sparsity and theoretically analyze two of its critical features: history-related activation uncertainty and semantic-irrelevant activation inertia. Our comprehensive analyses not only provide a robust theoretical foundation for DA methods but also offer valuable insights to guide future research in optimizing LLMs for greater efficiency and effectiveness.

cs.CL↗

arXiv2024

LARR: Large Language Model Aided Real-time Scene Recommendation with Semantic Understanding

Click-Through Rate (CTR) prediction is crucial for Recommendation System(RS), aiming to provide personalized recommendation services for users in many aspects such as food delivery, e-commerce and so on. However, traditional RS relies on collaborative signals, which lacks semantic understanding to real-time scenes. We also noticed that a major challenge in utilizing Large Language Models (LLMs) for practical recommendation purposes is their efficiency in dealing with long text input. To break through the problems above, we propose Large Language Model Aided Real-time Scene Recommendation(LARR), adopt LLMs for semantic understanding, utilizing real-time scene information in RS without requiring LLM to process the entire real-time scene text directly, thereby enhancing the efficiency of LLM-based CTR modeling. Specifically, recommendation domain-specific knowledge is injected into LLM and then RS employs an aggregation encoder to build real-time scene information from separate LLM's outputs. Firstly, a LLM is continual pretrained on corpus built from recommendation data with the aid of special tokens. Subsequently, the LLM is fine-tuned via contrastive learning on three kinds of sample construction strategies. Through this step, LLM is transformed into a text embedding model. Finally, LLM's separate outputs for different scene features are aggregated by an encoder, aligning to collaborative signals in RS, enhancing the performance of recommendation model.

cs.IR↗

arXiv2024

Aligning Explanations for Recommendation with Rating and Feature via Maximizing Mutual Information

Providing natural language-based explanations to justify recommendations helps to improve users' satisfaction and gain users' trust. However, as current explanation generation methods are commonly trained with an objective to mimic existing user reviews, the generated explanations are often not aligned with the predicted ratings or some important features of the recommended items, and thus, are suboptimal in helping users make informed decision on the recommendation platform. To tackle this problem, we propose a flexible model-agnostic method named MMI (Maximizing Mutual Information) framework to enhance the alignment between the generated natural language explanations and the predicted rating/important item features. Specifically, we propose to use mutual information (MI) as a measure for the alignment and train a neural MI estimator. Then, we treat a well-trained explanation generation model as the backbone model and further fine-tune it through reinforcement learning with guidance from the MI estimator, which rewards a generated explanation that is more aligned with the predicted rating or a pre-defined feature of the recommended item. Experiments on three datasets demonstrate that our MMI framework can boost different backbone models, enabling them to outperform existing baselines in terms of alignment with predicted ratings and item features. Additionally, user studies verify that MI-enhanced explanations indeed facilitate users' decisions and are favorable compared with other baselines due to their better alignment properties.

cs.IR↗