arXiv subjects

Ning Liu

Publications and source records attributed to Ning Liu.

At least 163 records · Page 9Linked to original sources

arXiv2023

The application of Nano-silica gel in sealing well micro-annuli and cement channeling

The possibility for hydrocarbon fluids to migrate through debonded micro-annuli wells is a major concern in the petroleum industry. With effective permeability of 0.1-1.0 mD, the existence of channels in a cement annulus with apertures of 10-300 micrometer constitutes a major threat. Squeeze cement is typically difficult to repair channels-leakage with small apertures; hence, a low-viscosity sealer that can be inserted into these channels while producing a long-term resilient seal is sought. A novel application using nano-silica sealants could be the key to seal these channels. In the construction and sealing of hydrocarbon wells, cementing is a critical phase. Cement is prone to cracking during the life cycle of a well because of the changes in downhole conditions. The usage of micro-sized cross-linked nano-silica gel as a sealant material to minimize damaged cement sheaths is investigated in this study. Fluid leakage through channels in the cement was investigated using an experimental system. With a diameter of 0.05 inches, the impact of the cement channel size was explored. The sealing efficiency increased from 86 percent to 95 percent when the nano-silica concentration of the sealing gel increased from 13 percent to 25 percent. This demonstrates that the concentration of nano-silica in the sealing gel affects the gel's ability to seal against fluid flow. This research proposes a new way for improving cement zonal isolation and thereby lowering the impact of cement failure in the oil and gas industry.

physics.flu-dyn↗

arXiv2023

Well Cement Degradation and Wellbore Integrity in Geological CO2 Storages: A Literature Review

Carbon capture and storage (CCS) has emerged as the most effective method to curb the CO2 concentration in the atmosphere. It can store up to 5 billion tons of CO2 per year. To guarantee a safe and economical geological storage, the well cement degradation and wellbore integrity need to be studied thoroughly. This review paper is designed to provide a fundamental background of well cement degradation and wellbore integrity in geological CO2 storages to support the researchers in further investigation. The review mainly focuses on mechanical, thermal, chemical property changes and corrosion time for cement in experiments and simulation during geological CO2 storage. However, the debonding interface between casing/cement or cement/formation has not been addressed profoundly. A further investigation should inspect how pressure, temperature, and chemical reaction affect the micro-annuli of casing/cement or cement/formation. Also, a mathe-matical model should be established to predict the corrosion rate in geological CO2 storage.

physics.geo-ph↗

arXiv2022

Alignment-Enriched Tuning for Patch-Level Pre-trained Document Image Models

Alignment between image and text has shown promising improvements on patch-level pre-trained document image models. However, investigating more effective or finer-grained alignment techniques during pre-training requires a large amount of computation cost and time. Thus, a question naturally arises: Could we fine-tune the pre-trained models adaptive to downstream tasks with alignment objectives and achieve comparable or better performance? In this paper, we propose a new model architecture with alignment-enriched tuning (dubbed AETNet) upon pre-trained document image models, to adapt downstream tasks with the joint task-specific supervised and alignment-aware contrastive objective. Specifically, we introduce an extra visual transformer as the alignment-ware image encoder and an extra text transformer as the alignment-ware text encoder before multimodal fusion. We consider alignment in the following three aspects: 1) document-level alignment by leveraging the cross-modal and intra-modal contrastive loss; 2) global-local alignment for modeling localized and structural information in document images; and 3) local-level alignment for more accurate patch-level information. Experiments on various downstream tasks show that AETNet can achieve state-of-the-art performance on various downstream tasks. Notably, AETNet consistently outperforms state-of-the-art pre-trained models, such as LayoutLMv3 with fine-tuning techniques, on three different downstream tasks.

cs.CV↗

arXiv2022

Probing compressed higgsinos with forward protons at the LHC

Supersymmetric models with nearly-degenerate light higgsinos provide a consistent solution to the naturalness problem under rigorous constraints from current experimental searches for sparticles. However, it is challenging to probe the compressed scenarios at collider experiments due to the hard-to-detect soft final states. To overcome this issue, strategies have been proposed to take advantage of the photon fusion along the ultraperipheral collision at the Large Hadron Collider, which are feasible with the forward detectors installed at the ATLAS and CMS detectors. In this paper, we demonstrate a search strategy for the chargino pair production via photon fusion at the 13 TeV LHC, realizing a good sensitivity that can exclude $m_{\tildeχ^{\pm}_{1}}$ up to 270 GeV (308 GeV) for a mass splitting $Δm(\tildeχ^{\pm}_{1},\tildeχ^{0}_{1})\in[1,15]$ GeV with an integrated luminosity of 100 fb$^{-1}$ (3 ab$^{-1}$) at 95% C.L.

hep-ph↗

arXiv2022

A No-reference Quality Assessment Metric for Point Cloud Based on Captured Video Sequences

Point cloud is one of the most widely used digital formats of 3D models, the visual quality of which is quite sensitive to distortions such as downsampling, noise, and compression. To tackle the challenge of point cloud quality assessment (PCQA) in scenarios where reference is not available, we propose a no-reference quality assessment metric for colored point cloud based on captured video sequences. Specifically, three video sequences are obtained by rotating the camera around the point cloud through three specific orbits. The video sequences not only contain the static views but also include the multi-frame temporal information, which greatly helps understand the human perception of the point clouds. Then we modify the ResNet3D as the feature extraction model to learn the correlation between the capture videos and corresponding subjective quality scores. The experimental results show that our method outperforms most of the state-of-the-art full-reference and no-reference PCQA metrics, which validates the effectiveness of the proposed method.

eess.IV↗

arXiv2022

CAEN: A Hierarchically Attentive Evolution Network for Item-Attribute-Change-Aware Recommendation in the Growing E-commerce Environment

Traditional recommendation systems mainly focus on modeling user interests. However, the dynamics of recommended items caused by attribute modifications (e.g. changes in prices) are also of great importance in real systems, especially in the fast-growing e-commerce environment, which may cause the users' demands to emerge, shift and disappear. Recent studies that make efforts on dynamic item representations treat the item attributes as side information but ignore its temporal dependency, or model the item evolution with a sequence of related users but do not consider item attributes. In this paper, we propose Core Attribute Evolution Network (CAEN), which partitions the user sequence according to the attribute value and thus models the item evolution over attribute dynamics with these users. Under this framework, we further devise a hierarchical attention mechanism that applies attribute-aware attention for user aggregation under each attribute, as well as personalized attention for activating similar users in assessing the matching degree between target user and item. Results from the extensive experiments over actual e-commerce datasets show that our approach outperforms the state-of-art methods and achieves significant improvements on the items with rapid changes over attributes, therefore helping the item recommendation to adapt to the growth of the e-commerce platform.

cs.IR↗

arXiv2022

FuncFooler: A Practical Black-box Attack Against Learning-based Binary Code Similarity Detection Methods

The binary code similarity detection (BCSD) method measures the similarity of two binary executable codes. Recently, the learning-based BCSD methods have achieved great success, outperforming traditional BCSD in detection accuracy and efficiency. However, the existing studies are rather sparse on the adversarial vulnerability of the learning-based BCSD methods, which cause hazards in security-related applications. To evaluate the adversarial robustness, this paper designs an efficient and black-box adversarial code generation algorithm, namely, FuncFooler. FuncFooler constrains the adversarial codes 1) to keep unchanged the program's control flow graph (CFG), and 2) to preserve the same semantic meaning. Specifically, FuncFooler consecutively 1) determines vulnerable candidates in the malicious code, 2) chooses and inserts the adversarial instructions from the benign code, and 3) corrects the semantic side effect of the adversarial code to meet the constraints. Empirically, our FuncFooler can successfully attack the three learning-based BCSD models, including SAFE, Asm2Vec, and jTrans, which calls into question whether the learning-based BCSD is desirable.

cs.CR↗

arXiv2022

Effective Few-Shot Named Entity Linking by Meta-Learning

Entity linking aims to link ambiguous mentions to their corresponding entities in a knowledge base, which is significant and fundamental for various downstream applications, e.g., knowledge base completion, question answering, and information extraction. While great efforts have been devoted to this task, most of these studies follow the assumption that large-scale labeled data is available. However, when the labeled data is insufficient for specific domains due to labor-intensive annotation work, the performance of existing algorithms will suffer an intolerable decline. In this paper, we endeavor to solve the problem of few-shot entity linking, which only requires a minimal amount of in-domain labeled data and is more practical in real situations. Specifically, we firstly propose a novel weak supervision strategy to generate non-trivial synthetic entity-mention pairs based on mention rewriting. Since the quality of the synthetic data has a critical impact on effective model training, we further design a meta-learning mechanism to assign different weights to each synthetic entity-mention pair automatically. Through this way, we can profoundly exploit rich and precious semantic information to derive a well-trained entity linking model under the few-shot setting. The experiments on real-world datasets show that the proposed method can extensively improve the state-of-the-art few-shot entity linking model and achieve impressive performance when only a small amount of labeled data is available. Moreover, we also demonstrate the outstanding ability of the model's transferability.

cs.CL↗

arXiv2022

IPAPRec: A promising tool for learning high-performance mapless navigation skills with deep reinforcement learning

This paper studies how to improve the generalization performance and learning speed of the navigation agents trained with deep reinforcement learning (DRL). Although DRL exhibits huge potential in robot mapless navigation, DRL agents performing well in training scenarios are often found to perform poorly in unfamiliar scenarios. In this work, we propose that the representation of LiDAR readings is a key factor behind the degradation of agents' performance and present a powerful input pre-processing (IP) approach to address this issue. As this approach uses adaptively parametric reciprocal functions to pre-process LiDAR readings, we refer to this approach as IPAPRec and its normalized version as IPAPRecN. IPAPRec/IPAPRecN can highlight important short-distance values and compress the range of less-important long-distance values in laser scans, which well address the issues induced by conventional representations of laser scans. Their high performance was validated by extensive simulation and real-world experiments. The results show that our methods can substantially improve navigation agents' generalization performance and greatly reduce the training time compared to conventional methods.

cs.RO↗

arXiv2022

An Automatic and Efficient BERT Pruning for Edge AI Systems

With the yearning for deep learning democratization, there are increasing demands to implement Transformer-based natural language processing (NLP) models on resource-constrained devices for low-latency and high accuracy. Existing BERT pruning methods require domain experts to heuristically handcraft hyperparameters to strike a balance among model size, latency, and accuracy. In this work, we propose AE-BERT, an automatic and efficient BERT pruning framework with efficient evaluation to select a "good" sub-network candidate (with high accuracy) given the overall pruning ratio constraints. Our proposed method requires no human experts experience and achieves a better accuracy performance on many NLP tasks. Our experimental results on General Language Understanding Evaluation (GLUE) benchmark show that AE-BERT outperforms the state-of-the-art (SOTA) hand-crafted pruning methods on BERT$_{\mathrm{BASE}}$. On QNLI and RTE, we obtain 75\% and 42.8\% more overall pruning ratio while achieving higher accuracy. On MRPC, we obtain a 4.6 higher score than the SOTA at the same overall pruning ratio of 0.5. On STS-B, we can achieve a 40\% higher pruning ratio with a very small loss in Spearman correlation compared to SOTA hand-crafted pruning methods. Experimental results also show that after model compression, the inference time of a single BERT$_{\mathrm{BASE}}$ encoder on Xilinx Alveo U200 FPGA board has a 1.83$\times$ speedup compared to Intel(R) Xeon(R) Gold 5218 (2.30GHz) CPU, which shows the reasonableness of deploying the proposed method generated subnets of BERT$_{\mathrm{BASE}}$ model on computation restricted devices.

cs.CL↗

arXiv2022

Differentiable Multi-Agent Actor-Critic for Multi-Step Radiology Report Summarization

The IMPRESSIONS section of a radiology report about an imaging study is a summary of the radiologist's reasoning and conclusions, and it also aids the referring physician in confirming or excluding certain diagnoses. A cascade of tasks are required to automatically generate an abstractive summary of the typical information-rich radiology report. These tasks include acquisition of salient content from the report and generation of a concise, easily consumable IMPRESSIONS section. Prior research on radiology report summarization has focused on single-step end-to-end models -- which subsume the task of salient content acquisition. To fully explore the cascade structure and explainability of radiology report summarization, we introduce two innovations. First, we design a two-step approach: extractive summarization followed by abstractive summarization. Second, we additionally break down the extractive part into two independent tasks: extraction of salient (1) sentences and (2) keywords. Experiments on English radiology reports from two clinical sites show our novel approach leads to a more precise summary compared to single-step and to two-step-with-single-extractive-process baselines with an overall improvement in F1 score Of 3-4%.

cs.CL↗

arXiv2022

DAIS: Automatic Channel Pruning via Differentiable Annealing Indicator Search

The convolutional neural network has achieved great success in fulfilling computer vision tasks despite large computation overhead against efficient deployment. Structured (channel) pruning is usually applied to reduce the model redundancy while preserving the network structure, such that the pruned network can be easily deployed in practice. However, existing structured pruning methods require hand-crafted rules which may lead to tremendous pruning space. In this paper, we introduce Differentiable Annealing Indicator Search (DAIS) that leverages the strength of neural architecture search in the channel pruning and automatically searches for the effective pruned model with given constraints on computation overhead. Specifically, DAIS relaxes the binarized channel indicators to be continuous and then jointly learns both indicators and model parameters via bi-level optimization. To bridge the non-negligible discrepancy between the continuous model and the target binarized model, DAIS proposes an annealing-based procedure to steer the indicator convergence towards binarized states. Moreover, DAIS designs various regularizations based on a priori structural knowledge to control the pruning sparsity and to improve model performance. Experimental results show that DAIS outperforms state-of-the-art pruning methods on CIFAR-10, CIFAR-100, and ImageNet.

cs.CV↗

arXiv2022

The Green polynomials via vertex operators

An iterative formula for the Green polynomial is given using the vertex operator realization of the Hall-Littlewood function. Based on this, (1) a general combinatorial formula of the Green polynomial is given; (2) several compact formulas are given for Green's polynomials associated with upper partitions of length $\leq 3$ and the diagonal lengths $\leq 3$; (3) a Murnaghan-Nakayama type formula for the Green polynomial is obtained; and (4) an iterative formula is derived for the bitrace of the finite general linear group $G$ and the Iwahori-Hecke algebra of type $A$ on the permutation module of $G$ by its Borel subgroup.

math.QA↗

arXiv2021

A Computational Efficient Maximum Likelihood Direct Position Determination Approach for Multiple Emitters Using Angle and Doppler Measurements

Emitter localization is widely applied in the military and civilian _elds. In this paper, we tackle the problem of position estimation for multiple stationary emitters using Doppler frequency shifts and angles by moving receivers. The computational load for the exhaustive maximum likelihood (ML) direct position determination (DPD) search is insu_erable. Based on the Pincus' theorem and importance sampling (IS) concept, we propose a novel non-iterative ML DPD method. The proposed method transforms the original multidimensional grid search into random variables generation with multiple low-dimensional pseudo-probability density functions (PDF), and the circular mean is used for superior position estimation performance. The computational complexity of the proposed method is modest, and the o_-grid problem that most existing DPD techniques face is signi_cantly alleviated. Moreover, it can be implemented in parallel separately. Simulation results demonstrate that the proposed ML DPD estimator can achieve better estimation accuracy than state-of-the-art DPD techniques. With a reasonable parameter choice, the estimation performance of the proposed technique is very close to the Cram_er-Rao lower bound (CRLB), even in the adverse conditions of low signal-to-noise ratios (SNR) levels.

eess.SP↗

arXiv2021

Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?

There have been long-standing controversies and inconsistencies over the experiment setup and criteria for identifying the "winning ticket" in literature. To reconcile such, we revisit the definition of lottery ticket hypothesis, with comprehensive and more rigorous conditions. Under our new definition, we show concrete evidence to clarify whether the winning ticket exists across the major DNN architectures and/or applications. Through extensive experiments, we perform quantitative analysis on the correlations between winning tickets and various experimental factors, and empirically study the patterns of our observations. We find that the key training hyperparameters, such as learning rate and training epochs, as well as the architecture characteristics such as capacities and residual connections, are all highly correlated with whether and when the winning tickets can be identified. Based on our analysis, we summarize a guideline for parameter settings in regards of specific architecture characteristics, which we hope to catalyze the research progress on the topic of lottery ticket hypothesis. Our codes are publicly available at: https://github.com/boone891214/sanity-check-LTH.

cs.LG↗

arXiv2021

MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge

Recently, a new trend of exploring sparsity for accelerating neural network training has emerged, embracing the paradigm of training on the edge. This paper proposes a novel Memory-Economic Sparse Training (MEST) framework targeting for accurate and fast execution on edge devices. The proposed MEST framework consists of enhancements by Elastic Mutation (EM) and Soft Memory Bound (&S) that ensure superior accuracy at high sparsity ratios. Different from the existing works for sparse training, this current work reveals the importance of sparsity schemes on the performance of sparse training in terms of accuracy as well as training speed on real edge devices. On top of that, the paper proposes to employ data efficiency for further acceleration of sparse training. Our results suggest that unforgettable examples can be identified in-situ even during the dynamic exploration of sparsity masks in the sparse training process, and therefore can be removed for further training speedup on edge devices. Comparing with state-of-the-art (SOTA) works on accuracy, our MEST increases Top-1 accuracy significantly on ImageNet when using the same unstructured sparsity scheme. Systematical evaluation on accuracy, training speed, and memory footprint are conducted, where the proposed MEST framework consistently outperforms representative SOTA works. A reviewer strongly against our work based on his false assumptions and misunderstandings. On top of the previous submission, we employ data efficiency for further acceleration of sparse training. And we explore the impact of model sparsity, sparsity schemes, and sparse training algorithms on the number of removable training examples. Our codes are publicly available at: https://github.com/boone891214/MEST.

cs.LG↗

arXiv2021

Scalable Rule-Based Representation Learning for Interpretable Classification

Rule-based models, e.g., decision trees, are widely used in scenarios demanding high model interpretability for their transparent inner structures and good model expressivity. However, rule-based models are hard to optimize, especially on large data sets, due to their discrete parameters and structures. Ensemble methods and fuzzy/soft rules are commonly used to improve performance, but they sacrifice the model interpretability. To obtain both good scalability and interpretability, we propose a new classifier, named Rule-based Representation Learner (RRL), that automatically learns interpretable non-fuzzy rules for data representation and classification. To train the non-differentiable RRL effectively, we project it to a continuous space and propose a novel training method, called Gradient Grafting, that can directly optimize the discrete model using gradient descent. An improved design of logical activation functions is also devised to increase the scalability of RRL and enable it to discretize the continuous features end-to-end. Exhaustive experiments on nine small and four large data sets show that RRL outperforms the competitive interpretable approaches and can be easily adjusted to obtain a trade-off between classification accuracy and model complexity for different scenarios. Our code is available at: https://github.com/12wang3/rrl.

cs.LG↗

arXiv2021

Lottery Ticket Preserves Weight Correlation: Is It Desirable or Not?

In deep model compression, the recent finding "Lottery Ticket Hypothesis" (LTH) (Frankle & Carbin, 2018) pointed out that there could exist a winning ticket (i.e., a properly pruned sub-network together with original weight initialization) that can achieve competitive performance than the original dense network. However, it is not easy to observe such winning property in many scenarios, where for example, a relatively large learning rate is used even if it benefits training the original dense model. In this work, we investigate the underlying condition and rationale behind the winning property, and find that the underlying reason is largely attributed to the correlation between initialized weights and final-trained weights when the learning rate is not sufficiently large. Thus, the existence of winning property is correlated with an insufficient DNN pretraining, and is unlikely to occur for a well-trained DNN. To overcome this limitation, we propose the "pruning & fine-tuning" method that consistently outperforms lottery ticket sparse training under the same pruning algorithm and the same total training epochs. Extensive experiments over multiple deep models (VGG, ResNet, MobileNet-v2) on different datasets have been conducted to justify our proposals.

cs.LG↗