arXiv subjects

Bin Liu

Publications and source records attributed to Bin Liu.

At least 307 records · Page 17Linked to original sources

arXiv2022

Ternary Weight Networks

We present a memory and computation efficient ternary weight networks (TWNs) - with weights constrained to +1, 0 and -1. The Euclidian distance between full (float or double) precision weights and the ternary weights along with a scaling factor is minimized in training stage. Besides, a threshold-based ternary function is optimized to get an approximated solution which can be fast and easily computed. TWNs have shown better expressive abilities than binary precision counterparts. Meanwhile, TWNs achieve up to 16$\times$ model compression rate and need fewer multiplications compared with the float32 precision counterparts. Extensive experiments on MNIST, CIFAR-10, and ImageNet datasets show that the TWNs achieve much better result than the Binary-Weight-Networks (BWNs) and the classification performance on MNIST and CIFAR-10 is very close to the full precision networks. We also verify our method on object detection task and show that TWNs significantly outperforms BWN by more than 10\% mAP on PASCAL VOC dataset. The pytorch version of source code is available at: https://github.com/Thinklab-SJTU/twns.

cs.CV↗

arXiv2022

Physics-driven Deep Learning Inversion for Direct Current Resistivity Survey Data

The direct-current (DC) resistivity method is a commonly used geophysical technique for surveying adverse geological conditions. Inversion can reconstruct the resistivity model from data, which is an important step in the geophysical survey. However, the inverse problem is a serious ill-posed problem that makes it easy to obtain incorrect inversion results. Deep learning (DL) provides new avenues for solving inverse problems, and has been widely studied. Currently, most DL inversion methods for resistivity are purely data-driven and depend heavily on labels (real resistivity models). However, real resistivity models are difficult to obtain through field surveys. An inversion network may not be effectively trained without labels. In this study, we built an unsupervised learning resistivity inversion scheme based on the physical law of electric field propagation. First, a forward modeling process was embedded into the network training, which converted the predicted model to predicted data and formed a data misfit to the observation data. Unsupervised training independent of the real model was realized using the data misfit as a loss function. Moreover, a dynamic smoothing constraint was imposed on the loss function to alleviate the ill-posed inverse problem. Finally, a transfer learning scheme was applied to adapt the trained network with simulated data to field data. Numerical simulations and field tests showed that the proposed method can accurately locate and depict geological targets.

physics.geo-ph↗

arXiv2022

SML:Enhance the Network Smoothness with Skip Meta Logit for CTR Prediction

In light of the smoothness property brought by skip connections in ResNet, this paper proposed the Skip Logit to introduce the skip connection mechanism that fits arbitrary DNN dimensions and embraces similar properties to ResNet. Meta Tanh Normalization (MTN) is designed to learn variance information and stabilize the training process. With these delicate designs, our Skip Meta Logit (SML) brought incremental boosts to the performance of extensive SOTA ctr prediction models on two real-world datasets. In the meantime, we prove that the optimization landscape of arbitrarily deep skip logit networks has no spurious local optima. Finally, SML can be easily added to building blocks and has delivered offline accuracy and online business metrics gains on app ads learning to rank systems at TikTok.

cs.IR↗

arXiv2022

Vortex gap solitons in spin-orbit-coupled Bose-Einstein condensates with competing nonlinearities

The formation and dynamics of full vortex gap solitons (FVGSs) is investigated in two-component Bose-Einstein condensates with spin-orbit coupling (SOC), Zeeman splitting (ZS), and competing cubic and quintic nonlinear terms, while the usual kinetic energy is neglected, assuming that it is much smaller than the SOC and ZS terms. Unlike previous SOC system with the cubic-only attractive nonlinearity, in which solely semi-vortices may be stable, with the vorticity carried by a single component, the present system supports stable FVGS states, with the vorticity present in both components (such states are called here full vortex solitons, to stress the difference from the half-vortices). They populate the bandgap in the system's linear spectrum. In the case of the cubic self-attraction and quintic repulsion, stable FVGSs with a positive effective mass exist near the top of the bandgap. On the contrary, the system with cubic self-repulsion and quintic attraction produces stable FVGSs with a negative mass near the bottom of the bandgap. Mobility and collisions of FVGSs with different topological charges are investigated too.

cond-mat.quant-gas↗

arXiv2022

Baseline correction for FAST radio recombination lines: a modified penalized least squares smoothing technique

A pilot project has been proceeded to map 1 deg$^2$ on the Galactic plane for radio recombination lines (RRLs) using the Five hundred meter Aperture Spherical Telescope (FAST). The motivation is to verify the techniques and reliabilities for a large-scale Galactic plane RRL survey with FAST aiming to investigate the ionized environment in the Galaxy. The data shows that the bandpass of the FAST 19 beam L-band is severely affected by radio frequency interferences (RFIs) and standing wave ripples, which can hardly be corrected by traditional low order polynomials. In this paper, we investigate a series of penalized least square (PLS) based baseline correction methods for radio astronomical spectra that usually contain weak signals with high level of noise. Three promising penalized least squares based methods, AsLS, arPLS, and asPLS are evaluated. Adopting their advantages, a modified method named rrlPLS is developed to optimize the baseline fitting to our RRL spectra. To check their effectiveness, the four methods are tested by simulations and further verified using observed data sets. It turns out that the rrlPLS method, with optimized parameter $λ= 2 \times 10^8$ , reveals the most sensitive and reliable emission features in the RRL map. By injecting artificial line profiles into the real data cube, a further evaluation of profile distortion is conducted for rrlPLS. Comparing to simulated signals, the processed lines with low signal-to-noise ratio are less affected, of which the uncertainties are mainly caused by the rms noise. The rrlPLS method will be applied for baseline correction in future data processing pipeline of FAST RRL survey. Configured with proper parameters, the rrlPLS technique verified in this work may also be used for other spectroscopy projects.

astro-ph.GA↗

arXiv2022

Stochastic Weight Averaging Revisited

Averaging neural network weights sampled by a backbone stochastic gradient descent (SGD) is a simple yet effective approach to assist the backbone SGD in finding better optima, in terms of generalization. From a statistical perspective, weight averaging (WA) contributes to variance reduction. Recently, a well-established stochastic weight averaging (SWA) method is proposed, which is featured by the application of a cyclical or high constant (CHC) learning rate schedule (LRS) in generating weight samples for WA. Then a new insight on WA appears, which states that WA helps to discover wider optima and then leads to better generalization. We conduct extensive experimental studies for SWA, involving a dozen modern DNN model structures and a dozen benchmark open-source image, graph, and text datasets. We disentangle contributions of the WA operation and the CHC LRS for SWA, showing that the WA operation in SWA still contributes to variance reduction but does not always lead to wide optima. The experimental results indicate that there are global scale geometric structures in the DNN loss landscape. We then present an algorithm termed periodic SWA (PSWA) which makes use of a series of WA operations to discover the global geometric structures. PSWA outperforms its backbone SGD remarkably, providing experimental evidences for the existence of global geometric structures. Codes for reproducing the experimental results are available at https://github.com/ZJLAB-AMMI/PSWA.

cs.LG↗

arXiv2022

Learning by Consuming: Optimal Pricing with Endogenous Information Provision

We study the revenue-maximizing mechanism when a buyer's value evolves endogenously because of learning-by-consuming. A seller sells one unit of a divisible good, while the buyer relies on his private, rough valuation to choose his first-stage consumption level. Consuming more leads to a more precise valuation estimate, after which the buyer determines the second-stage consumption level. The optimum is a menu of try-and-decide contracts, consisting of a first-stage price-quantity pair and a second-stage per-unit price for the remaining quantity. In equilibrium, a higher first-stage valuation buyer pays more for higher first-stage consumption and enjoys a lower second-stage per-unit price. Methodologically, we deal with the difficulty that due to the failure of single-crossing condition, monotonicity in allocation plus the envelope condition is insufficient for incentive compatibility. Our results help to understand contracts about sequential consumption with the learning feature; e.g., leasing contracts for experience goods and trial sessions for certain courses.

econ.TH↗

arXiv2022

Highly active hydrogen evolution facilitated by topological surface states on a Pd/SnTe metal/topological crystalline insulator heterostructure

Recently, topological quantum materials have emerged as a promising electrocatalyst for hydrogen evolution reaction (HER). However, most of their performance largely lags behind noble metals such as benchmark platinum (Pt). In this work, a Pd(20nm)/SnTe(70nm) heterostructure, fabricated by molecular beam epitaxy and electron beam evaporation, is found to display much higher electrocatalytic activity than that of a pure Pd(20nm) thin film and even higher than that of a commercial Pt foil. This heterostructure adopts an extracted turnover frequency value more than two times higher than that of the Pd(20nm) thin film at a potential of 0.2 V, indicating a much higher intrinsic activity per Pd site. Density functional theory calculations show that the conventional d-band theory, which works well for many transition metal heterostructures, cannot explain the enhancement of electrocatalytic performance. Instead, we found that the topological surface states (TSSs) of the SnTe (001) underlayer play a key role; electrons transfer from both the Pd surface and the adsorbed H atoms to the TSSs of SnTe (001), resulting in weaker Pd-H binding strength and more favorable hydrogen adsorption free energies. Our work demonstrates for the first time that a metal/topological quantum material heterostructure could be a prominent catalyst to enjoy HER activity outperforming that of a commercial Pt foil and offers a promising direction to optimize the performance of electrocatalysts based on topological quantum materials.

cond-mat.mtrl-sci↗

arXiv2022

Multiple Similarity Drug-Target Interaction Prediction with Random Walks and Matrix Factorization

The discovery of drug-target interactions (DTIs) is a very promising area of research with great potential. The accurate identification of reliable interactions among drugs and proteins via computational methods, which typically leverage heterogeneous information retrieved from diverse data sources, can boost the development of effective pharmaceuticals. Although random walk and matrix factorization techniques are widely used in DTI prediction, they have several limitations. Random walk-based embedding generation is usually conducted in an unsupervised manner, while the linear similarity combination in matrix factorization distorts individual insights offered by different views. To tackle these issues, we take a multi-layered network approach to handle diverse drug and target similarities, and propose a novel optimization framework, called Multiple similarity DeepWalk-based Matrix Factorization (MDMF), for DTI prediction. The framework unifies embedding generation and interaction prediction, learning vector representations of drugs and targets that not only retain higher-order proximity across all hyper-layers and layer-specific local invariance, but also approximate the interactions with their inner product. Furthermore, we develop an ensemble method (MDMF2A) that integrates two instantiations of the MDMF model, optimizing the area under the precision-recall curve (AUPR) and the area under the receiver operating characteristic curve (AUC) respectively. The empirical study on real-world DTI datasets shows that our method achieves statistically significant improvement over current state-of-the-art approaches in four different settings. Moreover, the validation of highly ranked non-interacting pairs also demonstrates the potential of MDMF2A to discover novel DTIs.

q-bio.QM↗

arXiv2022

Intelligent decision-making method of TBM operating parameters based on multiple constraints and objective optimization

The decision-making of TBM operating parameters has an important guiding significance for TBM safe and efficient construction, and it has been one of the research hotpots in the field of TBM tunneling. For this purpose, this paper introduces rock-breaking rules into machine learning method, and a rock-machine mapping dual-driven by physical-rule and data-mining is established with high accuracy. This dual-driven mappings are subsequently used as objective function and constraints to build a decision-making method for TBM operating parameters. By searching the revolution per minute and penetration corresponding to the extremum of the objective function subject to the constraints, the optimal operating parameters can be obtained. This method is verified in the field of the Second Water Source Channel of Hangzhou, China, resulting in the average penetration rate increased by 11.3%, and the total cost decreased by 10.0%, which proves the practicability and effectiveness of the developed decision-making model.

cs.LG↗

arXiv2022

Two-Aspect Information Fusion Model For ABAW4 Multi-task Challenge

In this paper, we propose the solution to the Multi-Task Learning (MTL) Challenge of the 4th Affective Behavior Analysis in-the-wild (ABAW) competition. The task of ABAW is to predict frame-level emotion descriptors from videos: discrete emotional state; valence and arousal; and action units. Although researchers have proposed several approaches and achieved promising results in ABAW, current works in this task rarely consider interactions between different emotion descriptors. To this end, we propose a novel end to end architecture to achieve full integration of different types of information. Experimental results demonstrate the effectiveness of our proposed solution.

cs.CV↗

arXiv2022

Linear change and minutes variability of solar wind velocity revealed by FAST

Observation of Interplanetary Scintillation (IPS) provides an important and effective way to study the solar wind and the space weather. A series of IPS observations were conducted by the Five-hundred-meter Aperture Spherical radio Telescope (FAST). The extraordinary sensitivity and the wide frequency coverage make FAST an ideal platform for IPS studies. In this paper we present some first scientific results from FAST observations of IPS with the L-band receiver. Based on the solar wind velocity fitting values of FAST observations on September 26-28, 2020, we found that the velocity decreases with increasing frequency linearly, which has not yet been reported in literature. And we have also detected a variation of solar wind velocity on a timescale of 3-5 minutes, which imply the slow change of the background solar wind, a co-existence of high- and low-speed streams, or a reflect of the quasi-periodic electron-density fluctuations.

astro-ph.SR↗

arXiv2022

A DNS Tunnel Sliding Window Differential Detection Method Based on Normal Distribution Reasonable Range Filtering

A covert attack method often used by APT organizations is the DNS tunnel, which is used to pass information by constructing C2 networks. And they often use the method of frequently changing domain names and server IP addresses to evade monitoring, which makes it extremely difficult to detect them. However, they carry DNS tunnel information traffic in normal DNS communication, which inevitably brings anomalies in some statistical characteristics of DNS traffic, so that it would provide security personnel with the opportunity to find them. Based on the above considerations, this paper studies the statistical discovery methodology of typical DNS tunnel high-frequency query behavior. Firstly, we analyze the distribution of the DNS domain name length and times and finds that the DNS domain name length and times follow the normal distribution law. Secondly, based on this distribution law, we propose a method for detecting and discovering high-frequency DNS query behaviors of non-single domain names based on the statistical rules of domain name length and frequency and we also give three theorems as theoretical support. Thirdly, we design a sliding window difference scheme based on the above method. Experimental results show that our method has a higher detection rate. At the same time, since our method does not need to construct a data set, it has better practicability in detecting unknown DNS tunnels. This also shows that our detection method based on mathematical models can effectively avoid the dilemma for machine learning methods that must have useful training data sets, and has strong practical significance.

cs.NI↗

arXiv2022

Bayesian Negative Sampling for Recommendation

How to sample high quality negative instances from unlabeled data, i.e., negative sampling, is important for training implicit collaborative filtering and contrastive learning models. Although previous studies have proposed some approaches to sample informative instances, few has been done to discriminating false negative from true negative for unbiased negative sampling. On the basis of our order relation analysis of negatives' scores, we first derive the class conditional density of true negatives and that of false negatives. We next design a Bayesian classifier for negative classification, from which we define a model-agnostic posterior probability estimate of an instance being true negative as a quantitative negative signal measure. We also propose a Bayesian optimal sampling rule to sample high-quality negatives. The proposed Bayesian Negative Sampling (BNS) algorithm has a linear time complexity. Experimental studies validate the superiority of BNS over the peers in terms of better sampling quality and better recommendation performance.

cs.IR↗

arXiv2022

Towards Intrinsic Common Discriminative Features Learning for Face Forgery Detection using Adversarial Learning

Existing face forgery detection methods usually treat face forgery detection as a binary classification problem and adopt deep convolution neural networks to learn discriminative features. The ideal discriminative features should be only related to the real/fake labels of facial images. However, we observe that the features learned by vanilla classification networks are correlated to unnecessary properties, such as forgery methods and facial identities. Such phenomenon would limit forgery detection performance especially for the generalization ability. Motivated by this, we propose a novel method which utilizes adversarial learning to eliminate the negative effect of different forgery methods and facial identities, which helps classification network to learn intrinsic common discriminative features for face forgery detection. To leverage data lacking ground truth label of facial identities, we design a special identity discriminator based on similarity information derived from off-the-shelf face recognition model. With the help of adversarial learning, our face forgery detection model learns to extract common discriminative features through eliminating the effect of forgery methods and facial identities. Extensive experiments demonstrate the effectiveness of the proposed method under both intra-dataset and cross-dataset evaluation settings.

cs.CV↗

arXiv2022

Mergers prompted by dynamics in compact, multiple-star systems: a stellar-reduction case for the massive triple TIC 470710327

TIC 470710327, a massive compact hierarchical triple-star system, was recently identified by NASA's Transiting Exoplanet Survey Satellite (TESS). TIC 470710327 is comprised of a compact (1.10 d) circular eclipsing binary, with total mass $\approx 10.9-13.2\ \rm{M_{\odot}}$, and a more massive ($\approx 14-17\ \rm{M_{\odot}}$) eccentric non-eclipsing tertiary in a $52.04$ d orbit. Here we present a progenitor scenario for TIC 470710327 in which '2+2' quadruple dynamics result in Zeipel-Lidov-Kozai (ZLK) resonances that lead to a contact phase of the more massive binary. In this scenario, the two binary systems should form in a very similar manner, and dynamics trigger the merger of the more massive binary either during late phases of star formation or several Myr after the zero-age main sequence (ZAMS), when the stars begin to expand. Any evidence that the tertiary is a highly-magnetised ($\sim 1-10$ kG), slowly-rotating blue main-sequence star would hint towards a quadruple origin. Finally, our scenario suggests that the population of inclined, compact multiple-stellar systems is reduced into co-planar systems, via mergers, late during star formation or early in the main sequence. The elucidation of the origin of TIC 470710327 is crucial in our understanding of multiple massive-star formation and evolution.

astro-ph.SR↗

arXiv2022

Geometric effects induce anomalous size-dependent active transport in structured environments

Variations of transport efficiency in structured environments between distinct individuals in actively self-propelled systems is both hard to study and poorly understood. Here, we study the transport of a non-tumbling {\ecoli} strain, an active-matter archetype with intrinsic size variation but fairly uniform speed, through a periodic pillar array. We show that long-term transport switches from a trapping dominated state for shorter cells to a much more dispersive state for longer cells above a critical bacterial size set by the pillar array geometry. Using a combination of experiments and modeling, we show that this anomalous size-dependence arises from an enhancement of the escape rate from trapping for longer cells caused by nearby pillars. Our results show that geometric effects can lead to size being a sensitive tuning knob for transport in structured environments, with implications in general for active matter systems and, in particular, for the morphological adaptation of bacteria to structured habitats, spatial structuring of communities and for anti-biofouling materials design.

cond-mat.soft↗

arXiv2022

Consistent Relative Confidence and Label-Free Model Selection for Convolutional Neural Networks

In this paper, we are concerned with image classification with deep convolutional neural networks (CNNs). We focus on the following question: given a set of candidate CNN models, how to select the right one with the best generalization property for the current task? Current model selection methods all require access to a batch of labeled data for computing a pre-specified performance metric, such as the cross-entropy loss, the classification error rate and the negative log-likelihood. In many practical cases, labels are not available in time as labeling itself is a time-consuming and expensive task. To this end, we propose an approach to CNN model selection using only unlabeled data. We develop this method based on a principle termed consistent relative confidence. Experimental results on benchmark datasets demonstrate the effectiveness and efficiency of our method.

cs.CV↗