arXiv subjects

Bin Yang

Publications and source records attributed to Bin Yang.

At least 271 records · Page 15Linked to original sources

arXiv2023

Reinforcing User Retention in a Billion Scale Short Video Recommender System

Recently, short video platforms have achieved rapid user growth by recommending interesting content to users. The objective of the recommendation is to optimize user retention, thereby driving the growth of DAU (Daily Active Users). Retention is a long-term feedback after multiple interactions of users and the system, and it is hard to decompose retention reward to each item or a list of items. Thus traditional point-wise and list-wise models are not able to optimize retention. In this paper, we choose reinforcement learning methods to optimize the retention as they are designed to maximize the long-term performance. We formulate the problem as an infinite-horizon request-based Markov Decision Process, and our objective is to minimize the accumulated time interval of multiple sessions, which is equal to improving the app open frequency and user retention. However, current reinforcement learning algorithms can not be directly applied in this setting due to uncertainty, bias, and long delay time incurred by the properties of user retention. We propose a novel method, dubbed RLUR, to address the aforementioned challenges. Both offline and live experiments show that RLUR can significantly improve user retention. RLUR has been fully launched in Kuaishou app for a long time, and achieves consistent performance improvement on user retention and DAU.

cs.LG↗

arXiv2023

LSDM: Long-Short Diffeomorphic Motion for Weakly-Supervised Ultrasound Landmark Tracking

Accurate tracking of an anatomical landmark over time has been of high interests for disease assessment such as minimally invasive surgery and tumor radiation therapy. Ultrasound imaging is a promising modality benefiting from low-cost and real-time acquisition. However, generating a precise landmark tracklet is very challenging, as attempts can be easily distorted by different interference such as landmark deformation, visual ambiguity and partial observation. In this paper, we propose a long-short diffeomorphic motion network, which is a multi-task framework with a learnable deformation prior to search for the plausible deformation of landmark. Specifically, we design a novel diffeomorphism representation in both long and short temporal domains for delineating motion margins and reducing long-term cumulative tracking errors. To further mitigate local anatomical ambiguity, we propose an expectation maximisation motion alignment module to iteratively optimize both long and short deformation, aligning to the same directional and spatial representation. The proposed multi-task system can be trained in a weakly-supervised manner, which only requires few landmark annotations for tracking and zero annotation for long-short deformation learning. We conduct extensive experiments on two ultrasound landmark tracking datasets. Experimental results show that our proposed method can achieve better or competitive landmark tracking performance compared with other state-of-the-art tracking methods, with a strong generalization capability across different scanner types and different ultrasound modalities.

cs.CV↗

arXiv2022

Mid-IR Observations of IRAS, AKARI, WISE/NEOWISE and Subaru for Large Icy Asteroid (704) Interamnia: a New Perspective of Regolith Properties and Water Ice Fraction

(704) Interamnia is one of the largest asteroids that locates in the outer main-belt region, which may contain a large amount of water ice underneath its surface. We observe this asteroid using 8.2 m Subaru telescope at mid-infrared wavebands, and utilize thermophysical model for realistic surface layers (RSTPM) to analyze mid-infrared data from Subaru along with those of IRAS, AKARI and WISE/NEOWISE. We optimize the method to convert the WISE magnitude to thermal infrared flux with temperature dependent color corrections, which can provide significant references for main-belt asteroids at a large heliocentric distance with low surface temperature. We derive best-fitting thermal parameters of Interamnia - a mean regolith grain size of $190_{-180}^{+460}~\rm μm$, with a roughness of $0.30_{-0.17}^{+0.35}$ and RMS slope of $27_{-9}^{+13}$ degrees, thereby producing thermal inertia ranging from 9 to $92~\rm Jm^{-2}s^{-1/2}K^{-1}$ due to seasonal temperature variation. The geometric albedo and effective diameter are evaluated to be $0.0472_{-0.0031}^{+0.0033}$ and $339_{-11}^{+12}~\rm km$, respectively, being indicative of a bulk density of $1.86\pm0.63~\rm g/cm^3$. The low thermal inertia is consistent with typical B/C-type asteroids with $D\geq100$ km. The tiny regolith grain size suggests the presence of a fine regolith on the surface of Interamnia. Moreover, the seasonal and diurnal temperature distribution indicates that thermal features between southern and northern hemisphere appear to be very different. Finally, we present an estimation of volume fraction of water ice of $9\%\sim66\%$ from the published grain density and porosity of carbonaceous chondrites.

astro-ph.EP↗

arXiv2022

AutoPINN: When AutoML Meets Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINNs) have recently been proposed to solve scientific and engineering problems, where physical laws are introduced into neural networks as prior knowledge. With the embedded physical laws, PINNs enable the estimation of critical parameters, which are unobservable via physical tools, through observable variables. For example, Power Electronic Converters (PECs) are essential building blocks for the green energy transition. PINNs have been applied to estimate the capacitance, which is unobservable during PEC operations, using current and voltage, which can be observed easily during operations. The estimated capacitance facilitates self-diagnostics of PECs. Existing PINNs are often manually designed, which is time-consuming and may lead to suboptimal performance due to a large number of design choices for neural network architectures and hyperparameters. In addition, PINNs are often deployed on different physical devices, e.g., PECs, with limited and varying resources. Therefore, it requires designing different PINN models under different resource constraints, making it an even more challenging task for manual design. To contend with the challenges, we propose Automated Physics-Informed Neural Networks (AutoPINN), a framework that enables the automated design of PINNs by combining AutoML and PINNs. Specifically, we first tailor a search space that allows finding high-accuracy PINNs for PEC internal parameter estimation. We then propose a resource-aware search strategy to explore the search space to find the best PINN model under different resource constraints. We experimentally demonstrate that AutoPINN is able to find more accurate PINN models than human-designed, state-of-the-art PINN models using fewer resources.

cs.LG↗

arXiv2022

HALS: A Height-Aware Lidar Super-Resolution Framework for Autonomous Driving

Lidar sensors are costly yet critical for understanding the 3D environment in autonomous driving. High-resolution sensors provide more details about the surroundings because they contain more vertical beams, but they come at a much higher cost, limiting their inclusion in autonomous vehicles. Upsampling lidar pointclouds is a promising approach to gain the benefits of high resolution while maintaining an affordable cost. Although there exist many pointcloud upsampling frameworks, a consistent comparison of these works against each other on the same dataset using unified metrics is still missing. In the first part of this paper, we propose to benchmark existing methods on the Kitti dataset. In the second part, we introduce a novel lidar upsampling model, HALS: Height-Aware Lidar Super-resolution. HALS exploits the observation that lidar scans exhibit a height-aware range distribution and adopts a generator architecture with multiple upsampling branches of different receptive fields. HALS regresses polar coordinates instead of spherical coordinates and uses a surface-normal loss. Extensive experiments show that HALS achieves state-of-the-art performance on 3 real-world lidar datasets.

cs.CV↗

arXiv2022

Multi-Class Uncertainty Calibration via Mutual Information Maximization-based Binning

Post-hoc multi-class calibration is a common approach for providing high-quality confidence estimates of deep neural network predictions. Recent work has shown that widely used scaling methods underestimate their calibration error, while alternative Histogram Binning (HB) methods often fail to preserve classification accuracy. When classes have small prior probabilities, HB also faces the issue of severe sample-inefficiency after the conversion into K one-vs-rest class-wise calibration problems. The goal of this paper is to resolve the identified issues of HB in order to provide calibrated confidence estimates using only a small holdout calibration dataset for bin optimization while preserving multi-class ranking accuracy. From an information-theoretic perspective, we derive the I-Max concept for binning, which maximizes the mutual information between labels and quantized logits. This concept mitigates potential loss in ranking performance due to lossy quantization, and by disentangling the optimization of bin edges and representatives allows simultaneous improvement of ranking and calibration performance. To improve the sample efficiency and estimates from a small calibration set, we propose a shared class-wise (sCW) calibration strategy, sharing one calibrator among similar classes (e.g., with similar class priors) so that the training sets of their class-wise calibration problems can be merged to train the single calibrator. The combination of sCW and I-Max binning outperforms the state of the art calibration methods on various evaluation metrics across different benchmark datasets and models, using a small calibration set (e.g., 1k samples for ImageNet).

cs.LG↗

arXiv2022

Multivariate Time Series Forecasting with Dynamic Graph Neural ODEs

Multivariate time series forecasting has long received significant attention in real-world applications, such as energy consumption and traffic prediction. While recent methods demonstrate good forecasting abilities, they have three fundamental limitations. (i) Discrete neural architectures: Interlacing individually parameterized spatial and temporal blocks to encode rich underlying patterns leads to discontinuous latent state trajectories and higher forecasting numerical errors. (ii) High complexity: Discrete approaches complicate models with dedicated designs and redundant parameters, leading to higher computational and memory overheads. (iii) Reliance on graph priors: Relying on predefined static graph structures limits their effectiveness and practicability in real-world applications. In this paper, we address all the above limitations by proposing a continuous model to forecast $\textbf{M}$ultivariate $\textbf{T}$ime series with dynamic $\textbf{G}$raph neural $\textbf{O}$rdinary $\textbf{D}$ifferential $\textbf{E}$quations ($\texttt{MTGODE}$). Specifically, we first abstract multivariate time series into dynamic graphs with time-evolving node features and unknown graph structures. Then, we design and solve a neural ODE to complement missing graph topologies and unify both spatial and temporal message passing, allowing deeper graph propagation and fine-grained temporal information aggregation to characterize stable and precise latent spatial-temporal dynamics. Our experiments demonstrate the superiorities of $\texttt{MTGODE}$ from various perspectives on five time series benchmark datasets.

cs.LG↗

arXiv2022

Variation-based Cause Effect Identification

Mining genuine mechanisms underlying the complex data generation process in real-world systems is a fundamental step in promoting interpretability of, and thus trust in, data-driven models. Therefore, we propose a variation-based cause effect identification (VCEI) framework for causal discovery in bivariate systems from a single observational setting. Our framework relies on the principle of independence of cause and mechanism (ICM) under the assumption of an existing acyclic causal link, and offers a practical realization of this principle. Principally, we artificially construct two settings in which the marginal distributions of one covariate, claimed to be the cause, are guaranteed to have non-negligible variations. This is achieved by re-weighting samples of the marginal so that the resultant distribution is notably distinct from this marginal according to some discrepancy measure. In the causal direction, such variations are expected to have no impact on the effect generation mechanism. Therefore, quantifying the impact of these variations on the conditionals reveals the genuine causal direction. Moreover, we formulate our approach in the kernel-based maximum mean discrepancy, lifting all constraints on the data types of cause-and-effect covariates, and rendering such artificial interventions a convex optimization problem. We provide a series of experiments on real and synthetic data showing that VCEI is, in principle, competitive to other cause effect identification frameworks.

cs.AI↗

arXiv2022

Prompt Tuning for Parameter-efficient Medical Image Segmentation

Neural networks pre-trained on a self-supervision scheme have become the standard when operating in data rich environments with scarce annotations. As such, fine-tuning a model to a downstream task in a parameter-efficient but effective way, e.g. for a new set of classes in the case of semantic segmentation, is of increasing importance. In this work, we propose and investigate several contributions to achieve a parameter-efficient but effective adaptation for semantic segmentation on two medical imaging datasets. Relying on the recently popularized prompt tuning approach, we provide a prompt-able UNet (PUNet) architecture, that is frozen after pre-training, but adaptable throughout the network by class-dependent learnable prompt tokens. We pre-train this architecture with a dedicated dense self-supervision scheme based on assignments to online generated prototypes (contrastive prototype assignment, CPA) of a student teacher combination alongside a concurrent segmentation loss on a subset of classes. We demonstrate that the resulting neural network model is able to attenuate the gap between fully fine-tuned and parameter-efficiently adapted models on CT imaging datasets. As such, the difference between fully fine-tuned and prompt-tuned variants amounts to only 3.83 pp for the TCIA/BTCV dataset and 2.67 pp for the CT-ORG dataset in the mean Dice Similarity Coefficient (DSC, in %) while only prompt tokens, corresponding to 0.85% of the pre-trained backbone model with 6.8M frozen parameters, are adjusted. The code for this work is available on https://github.com/marcdcfischer/PUNet .

cs.CV↗

arXiv2022

Introducing Intermediate Domains for Effective Self-Training during Test-Time

Experiencing domain shifts during test-time is nearly inevitable in practice and likely results in a severe performance degradation. To overcome this issue, test-time adaptation continues to update the initial source model during deployment. A promising direction are methods based on self-training which have been shown to be well suited for gradual domain adaptation, since reliable pseudo-labels can be provided. In this work, we address two problems that exist when applying self-training in the setting of test-time adaptation. First, adapting a model to long test sequences that contain multiple domains can lead to error accumulation. Second, naturally, not all shifts are gradual in practice. To tackle these challenges, we introduce GTTA. By creating artificial intermediate domains that divide the current domain shift into a more gradual one, effective self-training through high quality pseudo-labels can be performed. To create the intermediate domains, we propose two independent variations: mixup and light-weight style transfer. We demonstrate the effectiveness of our approach on the continual and gradual corruption benchmarks, as well as ImageNet-R. To further investigate gradual shifts in the context of urban scene segmentation, we publish a new benchmark: CarlaTTA. It enables the exploration of several non-stationary domain shifts.

cs.CV↗

arXiv2022

Convergence and Complexity of an Adaptive Planewave Method for Eigenvalue Computations

In this paper, we study the adaptive planewave discretization for a cluster of eigenvalues of second-order elliptic partial differential equations. We first design an a posteriori error estimator and prove both the upper and lower bounds. Based on the a posteriori error estimator, we propose an adaptive planewave method. We then prove that the adaptive planewave approximations have the linear convergence rate and quasi-optimal complexity.

math.NA↗

arXiv2022

On-Board Pedestrian Trajectory Prediction Using Behavioral Features

This paper presents a novel approach to pedestrian trajectory prediction for on-board camera systems, which utilizes behavioral features of pedestrians that can be inferred from visual observations. Our proposed method, called Behavior-Aware Pedestrian Trajectory Prediction (BA-PTP), processes multiple input modalities, i.e. bounding boxes, body and head orientation of pedestrians as well as their pose, with independent encoding streams. The encodings of each stream are fused using a modality attention mechanism, resulting in a final embedding that is used to predict future bounding boxes in the image. In experiments on two datasets for pedestrian behavior prediction, we demonstrate the benefit of using behavioral features for pedestrian trajectory prediction and evaluate the effectiveness of the proposed encoding strategy. Additionally, we investigate the relevance of different behavioral features on the prediction performance based on an ablation study.

cs.CV↗

arXiv2022

Existence and regularity of pullback attractors for a non-autonomous diffusion equation with delay and nonlocal diffusion in time-dependent spaces

In this paper, we study the asymptotic behavior of solution to a non-autonomous diffusion equations with delay containing some hereditary characteristics and nonlocal diffusion in time-dependent space $C_{\mathcal{H}_{t}(Ω)}$. When the nonlinear function $f$ satisfies the polynomial growth of arbitrary order $p-1$ $(p \ge 2)$ and the external force $h \in L_{l o c}^{2}\left(\mathbb{R} ; H^{-1}(Ω)\right)$, we establish the existence and regularity of the time-dependent pullback attractors.

math.AP↗

arXiv2022

Existence and upper semicontinuity of time-dependent attractors for the non-autonomous nonlocal diffusion equations

In this paper, under some appropriate assumptions, we prove the existence of the minimal time-dependent pullback $\mathcal D_σ^{\mathcal{H}_{t}}$-attractors ${\mathcal{A}}_{\mathcal D_σ^{\mathcal{H}_{t}}}$ for the non-autonomous nonlocal diffusion equations in time-dependent space $\mathcal{H}_{t}(Ω)$. Next, in same phase space, using a priori estimate and energy methods we establish the existence of time-dependent pullback attractors $\left\{A_ξ(t)\right\}_{t \in \mathbb R}$ and the upper semicontinuity of $\left\{A_ξ(t)\right\}_{t \in \mathbb R}$ and the global attractor $A$ of the equation with $ξ= 0$, that is, $$ \lim_{ξ\rightarrow 0^{+}} \operatorname{dist}_{\mathcal H_{t}}\left(A_ξ(t), A \right)=0. $$

math.AP↗

arXiv2022

Towards Discriminative and Transferable One-Stage Few-Shot Object Detectors

Recent object detection models require large amounts of annotated data for training a new classes of objects. Few-shot object detection (FSOD) aims to address this problem by learning novel classes given only a few samples. While competitive results have been achieved using two-stage FSOD detectors, typically one-stage FSODs underperform compared to them. We make the observation that the large gap in performance between two-stage and one-stage FSODs are mainly due to their weak discriminability, which is explained by a small post-fusion receptive field and a small number of foreground samples in the loss function. To address these limitations, we propose the Few-shot RetinaNet (FSRN) that consists of: a multi-way support training strategy to augment the number of foreground samples for dense meta-detectors, an early multi-level feature fusion providing a wide receptive field that covers the whole anchor area and two augmentation techniques on query and source images to enhance transferability. Extensive experiments show that the proposed approach addresses the limitations and boosts both discriminability and transferability. FSRN is almost two times faster than two-stage FSODs while remaining competitive in accuracy, and it outperforms the state-of-the-art of one-stage meta-detectors and also some two-stage FSODs on the MS-COCO and PASCAL VOC benchmarks.

cs.CV↗

arXiv2022

Experts in the Loop: Conditional Variable Selection for Accelerating Post-Silicon Analysis Based on Deep Learning

Post-silicon validation is one of the most critical processes in modern semiconductor manufacturing. Specifically, correct and deep understanding in test cases of manufactured devices is key to enable post-silicon tuning and debugging. This analysis is typically performed by experienced human experts. However, with the fast development in semiconductor industry, test cases can contain hundreds of variables. The resulting high-dimensionality poses enormous challenges to experts. Thereby, some recent prior works have introduced data-driven variable selection algorithms to tackle these problems and achieved notable success. Nevertheless, for these methods, experts are not involved in training and inference phases, which may lead to bias and inaccuracy due to the lack of prior knowledge. Hence, this work for the first time aims to design a novel conditional variable selection approach while keeping experts in the loop. In this way, we expect that our algorithm can be more efficiently and effectively trained to identify the most critical variables under certain expert knowledge. Extensive experiments on both synthetic and real-world datasets from industry have been conducted and shown the effectiveness of our method.

cs.LG↗

arXiv2022

Deep Feature Selection Using a Novel Complementary Feature Mask

Feature selection has drawn much attention over the last decades in machine learning because it can reduce data dimensionality while maintaining the original physical meaning of features, which enables better interpretability than feature extraction. However, most existing feature selection approaches, especially deep-learning-based, often focus on the features with great importance scores only but neglect those with less importance scores during training as well as the order of important candidate features. This can be risky since some important and relevant features might be unfortunately ignored during training, leading to suboptimal solutions or misleading selections. In our work, we deal with feature selection by exploiting the features with less importance scores and propose a feature selection framework based on a novel complementary feature mask. Our method is generic and can be easily integrated into existing deep-learning-based feature selection approaches to improve their performance as well. Experiments have been conducted on benchmarking datasets and shown that the proposed method can select more representative and informative features than the state of the art.

cs.LG↗

arXiv2022

Spectral evolution of dark asteroid surfaces induced by space weathering over a decade

The surface of airless bodies like asteroids in the Solar System are known to be affected by space weathering. Experiments simulating space weathering are essential for studying the effects of this process on meteorite samples, but the problem is that the time spent to reproduce space weathering in these experiments is billions of times shorter than the actual phenomenon. In December 2010, the T-type asteroid 596 Scheila underwent a collision with a few-tens-of-meters impactor. A decade later, there is an opportunity to study how the surface layer of this asteroid is being altered by space weathering after the impact. To do so, we performed visible spectrophotometric and near-infrared spectroscopic observations of 596 Scheila. The acquired spectrum is consistent with those observed shortly after the 2010 impact event within the observational uncertainty range. This indicates that the surface color of dark asteroids is not noticeably changed by space weathering over a 10-year period. This study is the first to investigate color changes due to space weathering on an actual asteroid surface in the Solar System. Considering that fresh layers are regularly created on asteroid surfaces by collisions, we suggest a genetic link between D/T-type and dark (low albedo) X-complex asteroids and very red objects such as 269 Justitia, 732 Tjilaki (and 203 Pompeja). New observations show that 203 Pompeja has a X-type-like surface, with some local surface areas exhibiting a very red spectrum.

astro-ph.EP↗