arXiv subjects

Qing Li

Publications and source records attributed to Qing Li.

At least 433 records · Page 24Linked to original sources

arXiv2023

Counterfactual Explanation for Fairness in Recommendation

Fairness-aware recommendation eliminates discrimination issues to build trustworthy recommendation systems.Explaining the causes of unfair recommendations is critical, as it promotes fairness diagnostics, and thus secures users' trust in recommendation models. Existing fairness explanation methods suffer high computation burdens due to the large-scale search space and the greedy nature of the explanation search process. Besides, they perform score-based optimizations with continuous values, which are not applicable to discrete attributes such as gender and race. In this work, we adopt the novel paradigm of counterfactual explanation from causal inference to explore how minimal alterations in explanations change model fairness, to abandon the greedy search for explanations. We use real-world attributes from Heterogeneous Information Networks (HINs) to empower counterfactual reasoning on discrete attributes. We propose a novel Counterfactual Explanation for Fairness (CFairER) that generates attribute-level counterfactual explanations from HINs for recommendation fairness. Our CFairER conducts off-policy reinforcement learning to seek high-quality counterfactual explanations, with an attentive action pruning reducing the search space of candidate counterfactuals. The counterfactual explanations help to provide rational and proximate explanations for model fairness, while the attentive action pruning narrows the search space of attributes. Extensive experiments demonstrate our proposed model can generate faithful explanations while maintaining favorable recommendation performance.

cs.IR↗

arXiv2023

Identification of Causal Relationship between Amyloid-beta Accumulation and Alzheimer's Disease Progression via Counterfactual Inference

Alzheimer's disease (AD) is a neurodegenerative disorder that is beginning with amyloidosis, followed by neuronal loss and deterioration in structure, function, and cognition. The accumulation of amyloid-beta in the brain, measured through 18F-florbetapir (AV45) positron emission tomography (PET) imaging, has been widely used for early diagnosis of AD. However, the relationship between amyloid-beta accumulation and AD pathophysiology remains unclear, and causal inference approaches are needed to uncover how amyloid-beta levels can impact AD development. In this paper, we propose a graph varying coefficient neural network (GVCNet) for estimating the individual treatment effect with continuous treatment levels using a graph convolutional neural network. We highlight the potential of causal inference approaches, including GVCNet, for measuring the regional causal connections between amyloid-beta accumulation and AD pathophysiology, which may serve as a robust tool for early diagnosis and tailored care.

cs.LG↗

arXiv2023

Farthest Streamline Sampling for the Uniform Distribution of Forearm Muscle Fiber Tracts from Diffusion Tensor Imaging

Background: Diffusion tensor imaging (DTI) has been used to characterize forearm muscle architecture. Since only uniform sampling is performed for seed points rather than fiber tracts, the tracts may be unevenly distributed in the muscle volume. Purpose: To reconstruct uniformly distributed fiber tracts in human forearm by filtering the tracts from DTI. Assessment: Farthest streamline sampling (FSS) was proposed for filtering and compared with two conventional methods, i.e., two-dimensional sampling and three-dimensional sampling. The uniform coverage performance of the methods was evaluated by streamline coverage (SC) and the coefficient of variation of streamline density (SDCV). Architectural parameters were calculated for 17 forearm muscles. Anatomical correctness was verified by 1. visually assessing the fiber orientation, 2. checking whether the architectural parameters were within physiological ranges, and 3. classifying the architectural types. Results: FSS had the highest SC (0.93+0.04) and the lowest SDCV (0.34+0.06) among the three methods (P<0.05). FSS reduced the sampling of long tracts (10% reduction in fiber length, P<0.05), and the architectural parameters were within physiological ranges (two parameters with P<0.05). The fiber orientation of the tractography was visually consistent with that of the cadaveric specimen. The architectural types of 16 muscles were correctly classified, except for the palmaris longus, which had a linear arrangement of fiber endpoints (R2=0.95+0.02, P<0.001). Data Conclusion: FSS reconstructed more muscle regions and uniformly distributed fiber tracts. The tracts were anatomically correct, indicating the validity of fiber tracts. Key Words: diffusion tensor imaging; forearm muscles; architectural properties

eess.IV↗

arXiv2023

Recurrent Attention Networks for Long-text Modeling

Self-attention-based models have achieved remarkable progress in short-text mining. However, the quadratic computational complexities restrict their application in long text processing. Prior works have adopted the chunking strategy to divide long documents into chunks and stack a self-attention backbone with the recurrent structure to extract semantic representation. Such an approach disables parallelization of the attention mechanism, significantly increasing the training cost and raising hardware requirements. Revisiting the self-attention mechanism and the recurrent structure, this paper proposes a novel long-document encoding model, Recurrent Attention Network (RAN), to enable the recurrent operation of self-attention. Combining the advantages from both sides, the well-designed RAN is capable of extracting global semantics in both token-level and document-level representations, making it inherently compatible with both sequential and classification tasks, respectively. Furthermore, RAN is computationally scalable as it supports parallelization on long document processing. Extensive experiments demonstrate the long-text encoding ability of the proposed RAN model on both classification and sequential tasks, showing its potential for a wide range of applications.

cs.CL↗

arXiv2023

Absence of near-ambient superconductivity in LuH$_{2\pm\text{x}}$N$_y$

Recently near-ambient superconductivity was claimed in N-doped lutetium hydride (ref. 1). This induces a worldwide fanaticism about the dream of room temperature superconductivity under low pressures. By using a high pressure and high temperature synthesis technique, we have successfully obtained the nitrogen doped lutetium hydride (LuH$_{2\pm\text{x}}$N$_y$) with a dark-bluish color and a structure with the space group of $Fm\bar{3}m$ evidenced by x-ray diffraction. This structure is the same as that reported in ref. 1. The energy dispersive X-ray spectroscopy (EDS) confirmed the existence of nitrogen in some areas of the samples. At ambient pressure, we witness a kink of resistivity and magnetization at about 300 K, which may correspond to a rearrangement of hydrogen/nitrogen atoms, namely a structural transition. However, by applying a pressure from 1 GPa to 6 GPa, we have seen a progressively optimized metallic behavior without showing superconductivity down to 10 K. Temperature dependence of magnetization shows a roughly flat feature between 100 and 320 K, and the magnetization increases with magnetic field at 100 K, all these are not expected for superconductivity at 100 K. Thus, we conclude the absence of near-ambient superconductivity in this nitrogen-doped lutetium hydride under pressures below 6 GPa.

cond-mat.supr-con↗

arXiv2023

FERN: Leveraging Graph Attention Networks for Failure Evaluation and Robust Network Design

Robust network design, which aims to guarantee network availability under various failure scenarios while optimizing performance/cost objectives, has received significant attention. Existing approaches often rely on model-based mixed-integer optimization that is hard to scale or employ deep learning to solve specific engineering problems yet with limited generalizability. In this paper, we show that failure evaluation provides a common kernel to improve the tractability and scalability of existing solutions. By providing a neural network function approximation of this common kernel using graph attention networks, we develop a unified learning-based framework, FERN, for scalable Failure Evaluation and Robust Network design. FERN represents rich problem inputs as a graph and captures both local and global views by attentively performing feature extraction from the graph. It enables a broad range of robust network design problems, including robust network validation, network upgrade optimization, and fault-tolerant traffic engineering that are discussed in this paper, to be recasted with respect to the common kernel and thus computed efficiently using neural networks and over a small set of critical failure scenarios. Extensive experiments on real-world network topologies show that FERN can efficiently and accurately identify key failure scenarios for both OSPF and optimal routing scheme, and generalizes well to different topologies and input traffic patterns. It can speed up multiple robust network design problems by more than 80x, 200x, 10x, respectively with negligible performance gap.

cs.NI↗

arXiv2023

A Comprehensive Study on Dataset Distillation: Performance, Privacy, Robustness and Fairness

The aim of dataset distillation is to encode the rich features of an original dataset into a tiny dataset. It is a promising approach to accelerate neural network training and related studies. Different approaches have been proposed to improve the informativeness and generalization performance of distilled images. However, no work has comprehensively analyzed this technique from a security perspective and there is a lack of systematic understanding of potential risks. In this work, we conduct extensive experiments to evaluate current state-of-the-art dataset distillation methods. We successfully use membership inference attacks to show that privacy risks still remain. Our work also demonstrates that dataset distillation can cause varying degrees of impact on model robustness and amplify model unfairness across classes when making predictions. This work offers a large-scale benchmarking framework for dataset distillation evaluation.

cs.LG↗

arXiv2023

Inferring Attracting Basins of Power System with Machine Learning

Power systems dominated by renewable energy encounter frequently large, random disturbances, and a critical challenge faced in power-system management is how to anticipate accurately whether the perturbed systems will return to the functional state after the transient or collapse. Whereas model-based studies show that the key to addressing the challenge lies in the attracting basins of the functional and dysfunctional states in the phase space, the finding of the attracting basins for realistic power systems remains a challenge, as accurate models describing the system dynamics are generally unavailable. Here we propose a new machine learning technique, namely balanced reservoir computing, to infer the attracting basins of a typical power system based on measured data. Specifically, trained by the time series of a handful of perturbation events, we demonstrate that the trained machine can predict accurately whether the system will return to the functional state in response to a large, random perturbation, thereby reconstructing the attracting basin of the functional state. The working mechanism of the new machine is analyzed, and it is revealed that the success of the new machine is attributed to the good balance between the echo and fading properties of the reservoir network; the effect of noisy signals on the prediction performance is also investigated, and a stochastic-resonance-like phenomenon is observed. Finally, we demonstrate that the new technique can be also utilized to infer the attracting basins of coexisting attractors in typical chaotic systems.

cs.LG↗

arXiv2023

MD-Manifold: A Medical-Distance-Based Representation Learning Approach for Medical Concept and Patient Representation

Effectively representing medical concepts and patients is important for healthcare analytical applications. Representing medical concepts for healthcare analytical tasks requires incorporating medical domain knowledge and prior information from patient description data. Current methods, such as feature engineering and mapping medical concepts to standardized terminologies, have limitations in capturing the dynamic patterns from patient description data. Other embedding-based methods have difficulties in incorporating important medical domain knowledge and often require a large amount of training data, which may not be feasible for most healthcare systems. Our proposed framework, MD-Manifold, introduces a novel approach to medical concept and patient representation. It includes a new data augmentation approach, concept distance metric, and patient-patient network to incorporate crucial medical domain knowledge and prior data information. It then adapts manifold learning methods to generate medical concept-level representations that accurately reflect medical knowledge and patient-level representations that clearly identify heterogeneous patient cohorts. MD-Manifold also outperforms other state-of-the-art techniques in various downstream healthcare analytical tasks. Our work has significant implications in information systems research in representation learning, knowledge-driven machine learning, and using design science as middle-ground frameworks for downstream explorative and predictive analyses. Practically, MD-Manifold has the potential to create effective and generalizable representations of medical concepts and patients by incorporating medical domain knowledge and prior data information. It enables deeper insights into medical data and facilitates the development of new analytical applications for better healthcare outcomes.

cs.LG↗

arXiv2023

A Minimalist Dataset for Systematic Generalization of Perception, Syntax, and Semantics

Inspired by humans' exceptional ability to master arithmetic and generalize to new problems, we present a new dataset, Handwritten arithmetic with INTegers (HINT), to examine machines' capability of learning generalizable concepts at three levels: perception, syntax, and semantics. In HINT, machines are tasked with learning how concepts are perceived from raw signals such as images (i.e., perception), how multiple concepts are structurally combined to form a valid expression (i.e., syntax), and how concepts are realized to afford various reasoning tasks (i.e., semantics), all in a weakly supervised manner. Focusing on systematic generalization, we carefully design a five-fold test set to evaluate both the interpolation and the extrapolation of learned concepts w.r.t. the three levels. Further, we design a few-shot learning split to determine whether or not models can rapidly learn new concepts and generalize them to more complex scenarios. To comprehend existing models' limitations, we undertake extensive experiments with various sequence-to-sequence models, including RNNs, Transformers, and GPT-3 (with the chain of thought prompting). The results indicate that current models struggle to extrapolate to long-range syntactic dependency and semantics. Models exhibit a considerable gap toward human-level generalization when evaluated with new concepts in a few-shot setting. Moreover, we discover that it is infeasible to solve HINT by merely scaling up the dataset and the model size; this strategy contributes little to the extrapolation of syntax and semantics. Finally, in zero-shot GPT-3 experiments, the chain of thought prompting exhibits impressive results and significantly boosts the test accuracy. We believe the HINT dataset and the experimental findings are of great interest to the learning community on systematic generalization.

cs.LG↗

arXiv2023

AttMEMO : Accelerating Transformers with Memoization on Big Memory Systems

Transformer models gain popularity because of their superior inference accuracy and inference throughput. However, the transformer is computation-intensive, causing a long inference time. The existing works on transformer inference acceleration have limitations caused by either the modification of transformer architectures or the need of specialized hardware. In this paper, we identify the opportunities of using memoization to accelerate the self-attention mechanism in transformers without the above limitations. Built upon a unique observation that there is rich similarity in attention computation across inference sequences, we build a memoization database that leverages the emerging big memory system. We introduce a novel embedding technique to find semantically similar inputs to identify computation similarity. We also introduce a series of techniques such as memory mapping and selective memoization to avoid memory copy and unnecessary overhead. We enable 22% inference-latency reduction on average (up to 68%) with negligible loss in inference accuracy.

cs.PF↗

arXiv2023

SQA3D: Situated Question Answering in 3D Scenes

We propose a new task to benchmark scene understanding of embodied agents: Situated Question Answering in 3D Scenes (SQA3D). Given a scene context (e.g., 3D scan), SQA3D requires the tested agent to first understand its situation (position, orientation, etc.) in the 3D scene as described by text, then reason about its surrounding environment and answer a question under that situation. Based upon 650 scenes from ScanNet, we provide a dataset centered around 6.8k unique situations, along with 20.4k descriptions and 33.4k diverse reasoning questions for these situations. These questions examine a wide spectrum of reasoning capabilities for an intelligent agent, ranging from spatial relation comprehension to commonsense understanding, navigation, and multi-hop reasoning. SQA3D imposes a significant challenge to current multi-modal especially 3D reasoning models. We evaluate various state-of-the-art approaches and find that the best one only achieves an overall score of 47.20%, while amateur human participants can reach 90.06%. We believe SQA3D could facilitate future embodied AI research with stronger situation understanding and reasoning capability.

cs.CV↗

arXiv2023

Pressure induced color change and evolution of metallic behavior in nitrogen-doped lutetium hydride

By applying pressures up to 42 GPa on the nitrogen-doped lutetium hydride (LuH$_{2\pm\text{x}}$N$_y$), we have found a gradual change of color from dark-blue to pink-violet in the pressure region of about 12 GPa to 21 GPa. The temperature dependence of resistivity under pressures up to 50.5 GPa shows progressively optimized metallic behavior with pressure. Interestingly, in the pressure region for the color change, a clear decrease of resistivity is observed with the increase of pressure, which is accompanied by a clear increase of the residual resistivity ratio (RRR). Fitting to the low temperature resistivity gives exponents of about 2, suggesting a Fermi liquid behavior in low temperature region. The general behavior in wide temperature region suggests that the electron-phonon scattering is still the dominant one. The magnetoresistance up to 9 tesla in the state under a pressure of 50.5 GPa shows an almost negligible effect, which suggests that the electric conduction in the pink-violet state is dominated by a single band. It is highly desired to have theoretical efforts in understanding the evolution of color and resistivity in this interesting system.

cond-mat.supr-con↗

arXiv2023

Self-Ensemling for 3D Point Cloud Domain Adaption

Recently 3D point cloud learning has been a hot topic in computer vision and autonomous driving. Due to the fact that it is difficult to manually annotate a qualitative large-scale 3D point cloud dataset, unsupervised domain adaptation (UDA) is popular in 3D point cloud learning which aims to transfer the learned knowledge from the labeled source domain to the unlabeled target domain. However, the generalization and reconstruction errors caused by domain shift with simply-learned model are inevitable which substantially hinder the model's capability from learning good representations. To address these issues, we propose an end-to-end self-ensembling network (SEN) for 3D point cloud domain adaption tasks. Generally, our SEN resorts to the advantages of Mean Teacher and semi-supervised learning, and introduces a soft classification loss and a consistency loss, aiming to achieve consistent generalization and accurate reconstruction. In SEN, a student network is kept in a collaborative manner with supervised learning and self-supervised learning, and a teacher network conducts temporal consistency to learn useful representations and ensure the quality of point clouds reconstruction. Extensive experiments on several 3D point cloud UDA benchmarks show that our SEN outperforms the state-of-the-art methods on both classification and segmentation tasks. Moreover, further analysis demonstrates that our SEN also achieves better reconstruction results.

cs.CV↗

arXiv2023

Accurate and Efficient Trajectory-based Contact Tracing with Secure Computation and Geo-Indistinguishability

Contact tracing has been considered as an effective measure to limit the transmission of infectious disease such as COVID-19. Trajectory-based contact tracing compares the trajectories of users with the patients, and allows the tracing of both direct contacts and indirect contacts. Although trajectory data is widely considered as sensitive and personal data, there is limited research on how to securely compare trajectories of users and patients to conduct contact tracing with excellent accuracy, high efficiency, and strong privacy guarantee. Traditional Secure Multiparty Computation (MPC) techniques suffer from prohibitive running time, which prevents their adoption in large cities with millions of users. In this work, we propose a technical framework called ContactGuard to achieve accurate, efficient, and privacy-preserving trajectory-based contact tracing. It improves the efficiency of the MPC-based baseline by selecting only a small subset of locations of users to compare against the locations of the patients, with the assist of Geo-Indistinguishability, a differential privacy notion for Location-based services (LBS) systems. Extensive experiments demonstrate that ContactGuard runs up to 2.6$\times$ faster than the MPC baseline, with no sacrifice in terms of the accuracy of contact tracing.

cs.DB↗

arXiv2023

Effective Stabilized Self-Training on Few-Labeled Graph Data

Graph neural networks (GNNs) are designed for semi-supervised node classification on graphs where only a subset of nodes have class labels. However, under extreme cases when very few labels are available (e.g., 1 labeled node per class), GNNs suffer from severe performance degradation. Specifically, we observe that existing GNNs suffer from unstable training process on few-labeled graphs, resulting to inferior performance on node classification. Therefore, we propose an effective framework, Stabilized Self-Training (SST), which is applicable to existing GNNs to handle the scarcity of labeled data, and consequently, boost classification accuracy. We conduct thorough empirical and theoretical analysis to support our findings and motivate the algorithmic designs in SST. We apply SST to two popular GNN models GCN and DAGNN, to get SSTGCN and SSTDA methods respectively, and evaluate the two methods against 10 competitors over 5 benchmarking datasets. Extensive experiments show that the proposed SST framework is highly effective, especially when few labeled data are available. Our methods achieve superior performance under almost all settings over all datasets. For instance, on a Cora dataset with only 1 labeled node per class, the accuracy of SSTGCN is 62.5%, 17.9% higher than GCN, and the accuracy of SSTDA is 66.4%, which outperforms DAGNN by 6.6%.

cs.LG↗

arXiv2023

Attention-based Spatial-Temporal Graph Convolutional Recurrent Networks for Traffic Forecasting

Traffic forecasting is one of the most fundamental problems in transportation science and artificial intelligence. The key challenge is to effectively model complex spatial-temporal dependencies and correlations in modern traffic data. Existing methods, however, cannot accurately model both long-term and short-term temporal correlations simultaneously, limiting their expressive power on complex spatial-temporal patterns. In this paper, we propose a novel spatial-temporal neural network framework: Attention-based Spatial-Temporal Graph Convolutional Recurrent Network (ASTGCRN), which consists of a graph convolutional recurrent module (GCRN) and a global attention module. In particular, GCRN integrates gated recurrent units and adaptive graph convolutional networks for dynamically learning graph structures and capturing spatial dependencies and local temporal relationships. To effectively extract global temporal dependencies, we design a temporal attention layer and implement it as three independent modules based on multi-head self-attention, transformer, and informer respectively. Extensive experiments on five real traffic datasets have demonstrated the excellent predictive performance of all our three models with all their average MAE, RMSE and MAPE across the test datasets lower than the baseline methods.

cs.LG↗

arXiv2023

Joint Multimodal Entity-Relation Extraction Based on Edge-enhanced Graph Alignment Network and Word-pair Relation Tagging

Multimodal named entity recognition (MNER) and multimodal relation extraction (MRE) are two fundamental subtasks in the multimodal knowledge graph construction task. However, the existing methods usually handle two tasks independently, which ignores the bidirectional interaction between them. This paper is the first to propose jointly performing MNER and MRE as a joint multimodal entity-relation extraction task (JMERE). Besides, the current MNER and MRE models only consider aligning the visual objects with textual entities in visual and textual graphs but ignore the entity-entity relationships and object-object relationships. To address the above challenges, we propose an edge-enhanced graph alignment network and a word-pair relation tagging (EEGA) for JMERE task. Specifically, we first design a word-pair relation tagging to exploit the bidirectional interaction between MNER and MRE and avoid the error propagation. Then, we propose an edge-enhanced graph alignment network to enhance the JMERE task by aligning nodes and edges in the cross-graph. Compared with previous methods, the proposed method can leverage the edge information to auxiliary alignment between objects and entities and find the correlations between entity-entity relationships and object-object relationships. Experiments are conducted to show the effectiveness of our model.

cs.CL↗