arXiv subjects

Xin Liu

Publications and source records attributed to Xin Liu.

At least 469 records · Page 26Linked to original sources

arXiv2024

MIND: Multimodal Shopping Intention Distillation from Large Vision-language Models for E-commerce Purchase Understanding

Improving user experience and providing personalized search results in E-commerce platforms heavily rely on understanding purchase intention. However, existing methods for acquiring large-scale intentions bank on distilling large language models with human annotation for verification. Such an approach tends to generate product-centric intentions, overlook valuable visual information from product images, and incurs high costs for scalability. To address these issues, we introduce MIND, a multimodal framework that allows Large Vision-Language Models (LVLMs) to infer purchase intentions from multimodal product metadata and prioritize human-centric ones. Using Amazon Review data, we apply MIND and create a multimodal intention knowledge base, which contains 1,264,441 million intentions derived from 126,142 co-buy shopping records across 107,215 products. Extensive human evaluations demonstrate the high plausibility and typicality of our obtained intentions and validate the effectiveness of our distillation framework and filtering mechanism. Additional experiments reveal that our obtained intentions significantly enhance large language models in two intention comprehension tasks.

cs.CL↗

arXiv2024

Benchmarking Counterfactual Interpretability in Deep Learning Models for Time Series Classification

The popularity of deep learning methods in the time series domain boosts interest in interpretability studies, including counterfactual (CF) methods. CF methods identify minimal changes in instances to alter the model predictions. Despite extensive research, no existing work benchmarks CF methods in the time series domain. Additionally, the results reported in the literature are inconclusive due to the limited number of datasets and inadequate metrics. In this work, we redesign quantitative metrics to accurately capture desirable characteristics in CFs. We specifically redesign the metrics for sparsity and plausibility and introduce a new metric for consistency. Combined with validity, generation time, and proximity, we form a comprehensive metric set. We systematically benchmark 6 different CF methods on 20 univariate datasets and 10 multivariate datasets with 3 different classifiers. Results indicate that the performance of CF methods varies across metrics and among different models. Finally, we provide case studies and a guideline for practical usage.

cs.LG↗

arXiv2024

The Cognitive Capabilities of Generative AI: A Comparative Analysis with Human Benchmarks

There is increasing interest in tracking the capabilities of general intelligence foundation models. This study benchmarks leading large language models and vision language models against human performance on the Wechsler Adult Intelligence Scale (WAIS-IV), a comprehensive, population-normed assessment of underlying human cognition and intellectual abilities, with a focus on the domains of VerbalComprehension (VCI), Working Memory (WMI), and Perceptual Reasoning (PRI). Most models demonstrated exceptional capabilities in the storage, retrieval, and manipulation of tokens such as arbitrary sequences of letters and numbers, with performance on the Working Memory Index (WMI) greater or equal to the 99.5th percentile when compared to human population normative ability. Performance on the Verbal Comprehension Index (VCI) which measures retrieval of acquired information, and linguistic understanding about the meaning of words and their relationships to each other, also demonstrated consistent performance at or above the 98th percentile. Despite these broad strengths, we observed consistently poor performance on the Perceptual Reasoning Index (PRI; range 0.1-10th percentile) from multimodal models indicating profound inability to interpret and reason on visual information. Smaller and older model versions consistently performed worse, indicating that training data, parameter count and advances in tuning are resulting in significant advances in cognitive ability.

cs.AI↗

arXiv2024

$ϕ_0$-junction and Josephson diode effect in high-temperature superconductor

Motivated by recent progress in both the Josephson diode effect (JDE) and the high-temperature Josephson junction, we propose to realize the JDE in an s-wave/d-wave/s-wave (s-d-s) superconductor junction and investigate the high-temperature superconducting order parameters. The interlayer coupling between s-wave and d-wave superconductors can induce an effective $d+is$ superconducting state, spontaneously breaking time-reversal symmetry. The asymmetric s-d interlayer couplings break the inversion symmetry. Remarkably, the breaking of these two symmetries leads to a $ϕ_0$-junction but does not generate JDE. We find that the emergence of the JDE in this junction depends on the $C_4$ rotational symmetry of the system. Although breaking $C_4$ rotational symmetry does not affect time-reversal and inversion symmetries, it can control the magnitude and polarity of diode efficiency. Furthermore, we propose observing C$_{4}$ symmetry breaking controlled JDE through asymmetric Shapiro steps. Our work suggests a JDE mechanism that relies on high-temperature d-wave pairing, which could inversely contribute to a potential experimental method for detecting the unconventional pairing symmetry in superconductors.

cond-mat.supr-con↗

arXiv2024

Blind Image Quality Assessment Using Multi-Stream Architecture with Spatial and Channel Attention

BIQA (Blind Image Quality Assessment) is an important field of study that evaluates images automatically. Although significant progress has been made, blind image quality assessment remains a difficult task since images vary in content and distortions. Most algorithms generate quality without emphasizing the important region of interest. In order to solve this, a multi-stream spatial and channel attention-based algorithm is being proposed. This algorithm generates more accurate predictions with a high correlation to human perceptual assessment by combining hybrid features from two different backbones, followed by spatial and channel attention to provide high weights to the region of interest. Four legacy image quality assessment datasets are used to validate the effectiveness of our proposed approach. Authentic and synthetic distortion image databases are used to demonstrate the effectiveness of the proposed method, and we show that it has excellent generalization properties with a particular focus on the perceptual foreground information.

cs.CV↗

arXiv2024

NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surrounding

Large Language Models (LLMs) have sparked substantial interest and debate concerning their potential emergence of Theory of Mind (ToM) ability. Theory of mind evaluations currently focuses on testing models using machine-generated data or game settings prone to shortcuts and spurious correlations, which lacks evaluation of machine ToM ability in real-world human interaction scenarios. This poses a pressing demand to develop new real-world scenario benchmarks. We introduce NegotiationToM, a new benchmark designed to stress-test machine ToM in real-world negotiation surrounding covered multi-dimensional mental states (i.e., desires, beliefs, and intentions). Our benchmark builds upon the Belief-Desire-Intention (BDI) agent modeling theory and conducts the necessary empirical experiments to evaluate large language models. Our findings demonstrate that NegotiationToM is challenging for state-of-the-art LLMs, as they consistently perform significantly worse than humans, even when employing the chain-of-thought (CoT) method.

cs.CL↗

arXiv2024

Persona Knowledge-Aligned Prompt Tuning Method for Online Debate

Debate is the process of exchanging viewpoints or convincing others on a particular issue. Recent research has provided empirical evidence that the persuasiveness of an argument is determined not only by language usage but also by communicator characteristics. Researchers have paid much attention to aspects of languages, such as linguistic features and discourse structures, but combining argument persuasiveness and impact with the social personae of the audience has not been explored due to the difficulty and complexity. We have observed the impressive simulation and personification capability of ChatGPT, indicating a giant pre-trained language model may function as an individual to provide personae and exert unique influences based on diverse background knowledge. Therefore, we propose a persona knowledge-aligned framework for argument quality assessment tasks from the audience side. This is the first work that leverages the emergence of ChatGPT and injects such audience personae knowledge into smaller language models via prompt tuning. The performance of our pipeline demonstrates significant and consistent improvement compared to competitive architectures.

cs.CL↗

arXiv2024

IntentionQA: A Benchmark for Evaluating Purchase Intention Comprehension Abilities of Language Models in E-commerce

Enhancing Language Models' (LMs) ability to understand purchase intentions in E-commerce scenarios is crucial for their effective assistance in various downstream tasks. However, previous approaches that distill intentions from LMs often fail to generate meaningful and human-centric intentions applicable in real-world E-commerce contexts. This raises concerns about the true comprehension and utilization of purchase intentions by LMs. In this paper, we present IntentionQA, a double-task multiple-choice question answering benchmark to evaluate LMs' comprehension of purchase intentions in E-commerce. Specifically, LMs are tasked to infer intentions based on purchased products and utilize them to predict additional purchases. IntentionQA consists of 4,360 carefully curated problems across three difficulty levels, constructed using an automated pipeline to ensure scalability on large E-commerce platforms. Human evaluations demonstrate the high quality and low false-negative rate of our benchmark. Extensive experiments across 19 language models show that they still struggle with certain scenarios, such as understanding products and intentions accurately, jointly reasoning with products and intentions, and more, in which they fall far behind human performances. Our code and data are publicly available at https://github.com/HKUST-KnowComp/IntentionQA.

cs.CL↗

arXiv2024

What Are the Odds? Language Models Are Capable of Probabilistic Reasoning

Language models (LM) are capable of remarkably complex linguistic tasks; however, numerical reasoning is an area in which they frequently struggle. An important but rarely evaluated form of reasoning is understanding probability distributions. In this paper, we focus on evaluating the probabilistic reasoning capabilities of LMs using idealized and real-world statistical distributions. We perform a systematic evaluation of state-of-the-art LMs on three tasks: estimating percentiles, drawing samples, and calculating probabilities. We evaluate three ways to provide context to LMs 1) anchoring examples from within a distribution or family of distributions, 2) real-world context, 3) summary statistics on which to base a Normal approximation. Models can make inferences about distributions, and can be further aided by the incorporation of real-world context, example shots and simplified assumptions, even if these assumptions are incorrect or misspecified. To conduct this work, we developed a comprehensive benchmark distribution dataset with associated question-answer pairs that we have released publicly.

cs.CL↗

arXiv2024

A penalized online sequential test of heterogeneous treatment effects for generalized linear models

Identification of heterogeneous treatment effects (HTEs) has been increasingly popular and critical in various penalized strategy decisions using the A/B testing approach, especially in the scenario of a consecutive online collection of samples. However, in high-dimensional settings, such an identification remains challenging in the sense of lack of detection power of HTEs with insufficient sample instances for each batch sequentially collected online. In this article, a novel high-dimensional test is proposed, named as the penalized online sequential test (POST), to identify HTEs and select useful covariates simultaneously under continuous monitoring in generalized linear models (GLMs), which achieves high detection power and controls the Type I error. A penalized score test statistic is developed along with an extended p-value process for the online collection of samples, and the proposed POST method is further extended to multiple online testing scenarios, where both high true positive rates and under-controlled false discovery rates are achieved simultaneously. Asymptotic results are established and justified to guarantee properties of the POST, and its performance is evaluated through simulations and analysis of real data, compared with the state-of-the-art online test methods. Our findings indicate that the POST method exhibits selection consistency and superb detection power of HTEs as well as excellent control over the Type I error, which endows our method with the capability for timely and efficient inference for online A/B testing in high-dimensional GLMs framework.

stat.ME↗

arXiv2024

Varstrometry for Off-nucleus and Dual sub-Kpc AGN (VODKA): A Mix of Singles, Lenses, and True Duals at Cosmic Noon

Dual Active Galactic Nuclei (dual AGNs), a phase in some galaxy mergers during which both central supermassive black holes (SMBHs) are active, are expected to be a key observable stage leading up to SMBH mergers. Constraining the population of dual AGNs in both the nearby and high-z universe has proven to be elusive until very recently. We present a multi-wavelength follow-up campaign to confirm the nature of a sample of 20 candidate dual AGNs at cosmic noon (z~2) from the VODKA sample. Through a combination of Hubble Space Telescope (HST) and Very Large Array (VLA) imaging, we refute the possibility of gravitational lensing in all but one target. We find evidence of dual AGNs in four systems, while seven exhibit single AGN in galaxy pairs, either through strong radio emission or ancillary emission line data. The remaining systems are either confirmed as quasar-star superpositions (six) or non-lensed pairs (two) that require further investigations to establish AGN activity. Among the systems with radio detections, we find a variety of radio spectral slopes and UV/optical colors suggesting that our sample contains a range of AGN properties, from obscured radio-quiet objects to those with powerful synchrotron-emitting jets. This study presents one of the largest dedicated multi-wavelength follow-up campaigns to date searching for dual AGNs at high redshift. We confirm several of the highest-z systems at small physical separations, thus representing some of the most evolved dual AGN systems at the epoch of peak quasar activity known to date.

astro-ph.GA↗

arXiv2024

Constrained Reasoning Chains for Enhancing Theory-of-Mind in Large Language Models

Theory-of-Mind (ToM) ability possessed by Large Language Models (LLMs) has been shown to be limited. Most existing methods for improving ToM in LLMs adopt zero-shot prompting, and they face challenges including poor performance in complex ToM reasoning tasks and an inability to handle non-narrative contexts. We propose a zero-shot prompting method named Constrained Chain-of-ToM (CCoToM) that leverages domain knowledge and the causal relations between ToM dimensions to address these limitations. Specifically, CCoToM guides LLMs to construct explicit reasoning chains by first prompting LLMs to infer related ToM dimensions (e.g., belief). Afterward, CCoToM prompts LLMs to infer the queried ToM dimension based on the generated related ToM dimensions and corresponding causal relations. Additionally, CCoToM adaptively imposes constraints on prompts to introduce inductive biases and improve consistency between ToM dimensions. Besides narratives, CCoToM can also handle non-narrative contexts like conversations. Extensive experiments show that CCoToM consistently outperforms previous state-of-the-art methods by large margins across all LLMs and datasets used. We also conduct in-depth analyses to gain deeper insights into CCoToM. We have made our code publicly available.

cs.CL↗

arXiv2024

Self-Updating Vehicle Monitoring Framework Employing Distributed Acoustic Sensing towards Real-World Settings

The recent emergence of Distributed Acoustic Sensing (DAS) technology has facilitated the effective capture of traffic-induced seismic data. The traffic-induced seismic wave is a prominent contributor to urban vibrations and contain crucial information to advance urban exploration and governance. However, identifying vehicular movements within massive noisy data poses a significant challenge. In this study, we introduce a real-time semi-supervised vehicle monitoring framework tailored to urban settings. It requires only a small fraction of manual labels for initial training and exploits unlabeled data for model improvement. Additionally, the framework can autonomously adapt to newly collected unlabeled data. Before DAS data undergo object detection as two-dimensional images to preserve spatial information, we leveraged comprehensive one-dimensional signal preprocessing to mitigate noise. Furthermore, we propose a novel prior loss that incorporates the shapes of vehicular traces to track a single vehicle with varying speeds. To evaluate our model, we conducted experiments with seismic data from the Stanford 2 DAS Array. The results showed that our model outperformed the baseline model Efficient Teacher and its supervised counterpart, YOLO (You Only Look Once), in both accuracy and robustness. With only 35 labeled images, our model surpassed YOLO's mAP 0.5:0.95 criterion by 18% and showed a 7% increase over Efficient Teacher. We conducted comparative experiments with multiple update strategies for self-updating and identified an optimal approach. This approach surpasses the performance of non-overfitting training conducted with all data in a single pass.

physics.geo-ph↗

arXiv2024

Infrared and Visible Image Fusion with Hierarchical Human Perception

Image fusion combines images from multiple domains into one image, containing complementary information from source domains. Existing methods take pixel intensity, texture and high-level vision task information as the standards to determine preservation of information, lacking enhancement for human perception. We introduce an image fusion method, Hierarchical Perception Fusion (HPFusion), which leverages Large Vision-Language Model to incorporate hierarchical human semantic priors, preserving complementary information that satisfies human visual system. We propose multiple questions that humans focus on when viewing an image pair, and answers are generated via the Large Vision-Language Model according to images. The texts of answers are encoded into the fusion network, and the optimization also aims to guide the human semantic distribution of the fused image more similarly to source images, exploring complementary information within the human perception domain. Extensive experiments demonstrate our HPFusoin can achieve high-quality fusion results both for information preservation and human visual enhancement.

cs.CV↗

arXiv2024

DiffFAS: Face Anti-Spoofing via Generative Diffusion Models

Face anti-spoofing (FAS) plays a vital role in preventing face recognition (FR) systems from presentation attacks. Nowadays, FAS systems face the challenge of domain shift, impacting the generalization performance of existing FAS methods. In this paper, we rethink about the inherence of domain shift and deconstruct it into two factors: image style and image quality. Quality influences the purity of the presentation of spoof information, while style affects the manner in which spoof information is presented. Based on our analysis, we propose DiffFAS framework, which quantifies quality as prior information input into the network to counter image quality shift, and performs diffusion-based high-fidelity cross-domain and cross-attack types generation to counter image style shift. DiffFAS transforms easily collectible live faces into high-fidelity attack faces with precise labels while maintaining consistency between live and spoof face identities, which can also alleviate the scarcity of labeled data with novel type attacks faced by nowadays FAS system. We demonstrate the effectiveness of our framework on challenging cross-domain and cross-attack FAS datasets, achieving the state-of-the-art performance. Available at https://github.com/murphytju/DiffFAS.

cs.CV↗

arXiv2024

A Double Tracking Method for Optimization with Decentralized Generalized Orthogonality Constraints

In this paper, we consider the decentralized optimization problems with generalized orthogonality constraints, where both the objective function and the constraint exhibit a distributed structure. Such optimization problems, albeit ubiquitous in practical applications, remain unsolvable by existing algorithms in the presence of distributed constraints. To address this issue, we convert the original problem into an unconstrained penalty model by resorting to the recently proposed constraint-dissolving operator. However, this transformation compromises the essential property of separability in the resulting penalty function, rendering it impossible to employ existing algorithms to solve. We overcome this difficulty by introducing a novel algorithm that tracks the gradient of the objective function and the Jacobian of the constraint mapping simultaneously. The global convergence guarantee is rigorously established with an iteration complexity. To substantiate the effectiveness and efficiency of our proposed algorithm, we present numerical results on both synthetic and real-world datasets.

math.OC↗

arXiv2024

Fire-Flyer AI-HPC: A Cost-Effective Software-Hardware Co-Design for Deep Learning

The rapid progress in Deep Learning (DL) and Large Language Models (LLMs) has exponentially increased demands of computational power and bandwidth. This, combined with the high costs of faster computing chips and interconnects, has significantly inflated High Performance Computing (HPC) construction costs. To address these challenges, we introduce the Fire-Flyer AI-HPC architecture, a synergistic hardware-software co-design framework and its best practices. For DL training, we deployed the Fire-Flyer 2 with 10,000 PCIe A100 GPUs, achieved performance approximating the DGX-A100 while reducing costs by half and energy consumption by 40%. We specifically engineered HFReduce to accelerate allreduce communication and implemented numerous measures to keep our Computation-Storage Integrated Network congestion-free. Through our software stack, including HaiScale, 3FS, and HAI-Platform, we achieved substantial scalability by overlapping computation and communication. Our system-oriented experience from DL training provides valuable insights to drive future advancements in AI-HPC.

cs.DC↗

arXiv2024

Local well-posedness of a system coupling compressible atmospheric dynamics and a micro-physics model of moisture in air

We establish here the local-in-time well-posedness of strong solutions with large initial data to a system coupling compressible atmospheric dynamics with a refined moisture model introduced in Hittmeir and Klein (Theor. Comput. Fluid Dyn. 32:137--164, 2018). This micro-physics moisture model is a refinement of the one communicated by Klein and Majda (Theor. Comput. Fluid Dyn. 20:525--551, 2006), which serves as a foundation of multi-scale asymptotic expansions for moist deep convection in the atmosphere. Specifically, we show, in this paper, that the system of compressible Navier--Stokes equations coupled with the moisture dynamics is well-posed. In particular, the micro-physics of moisture model incorporates the phase changes of water in moist air.

math.AP↗