arXiv subjects

Bin Li

Publications and source records attributed to Bin Li.

At least 469 records · Page 26Linked to original sources

arXiv2022

A Comprehensive Survey on Data-Efficient GANs in Image Generation

Generative Adversarial Networks (GANs) have achieved remarkable achievements in image synthesis. These successes of GANs rely on large scale datasets, requiring too much cost. With limited training data, how to stable the training process of GANs and generate realistic images have attracted more attention. The challenges of Data-Efficient GANs (DE-GANs) mainly arise from three aspects: (i) Mismatch Between Training and Target Distributions, (ii) Overfitting of the Discriminator, and (iii) Imbalance Between Latent and Data Spaces. Although many augmentation and pre-training strategies have been proposed to alleviate these issues, there lacks a systematic survey to summarize the properties, challenges, and solutions of DE-GANs. In this paper, we revisit and define DE-GANs from the perspective of distribution optimization. We conclude and analyze the challenges of DE-GANs. Meanwhile, we propose a taxonomy, which classifies the existing methods into three categories: Data Selection, GANs Optimization, and Knowledge Sharing. Last but not the least, we attempt to highlight the current problems and the future directions.

cs.CV↗

arXiv2022

Domain Discrepancy Aware Distillation for Model Aggregation in Federated Learning

Knowledge distillation has recently become popular as a method of model aggregation on the server for federated learning. It is generally assumed that there are abundant public unlabeled data on the server. However, in reality, there exists a domain discrepancy between the datasets of the server domain and a client domain, which limits the performance of knowledge distillation. How to improve the aggregation under such a domain discrepancy setting is still an open problem. In this paper, we first analyze the generalization bound of the aggregation model produced from knowledge distillation for the client domains, and then describe two challenges, server-to-client discrepancy and client-to-client discrepancy, brought to the aggregation model by the domain discrepancies. Following our analysis, we propose an adaptive knowledge aggregation algorithm FedD3A based on domain discrepancy aware distillation to lower the bound. FedD3A performs adaptive weighting at the sample level in each round of FL. For each sample in the server domain, only the client models of its similar domains will be selected for playing the teacher role. To achieve this, we show that the discrepancy between the server-side sample and the client domain can be approximately measured using a subspace projection matrix calculated on each client without accessing its raw data. The server can thus leverage the projection matrices from multiple clients to assign weights to the corresponding teacher models for each server-side sample. We validate FedD3A on two popular cross-domain datasets and show that it outperforms the compared competitors in both cross-silo and cross-device FL settings.

cs.LG↗

arXiv2022

Domain-Unified Prompt Representations for Source-Free Domain Generalization

Domain generalization (DG), aiming to make models work on unseen domains, is a surefire way toward general artificial intelligence. Limited by the scale and diversity of current DG datasets, it is difficult for existing methods to scale to diverse domains in open-world scenarios (e.g., science fiction and pixelate style). Therefore, the source-free domain generalization (SFDG) task is necessary and challenging. To address this issue, we propose an approach based on large-scale vision-language pretraining models (e.g., CLIP), which exploits the extensive domain information embedded in it. The proposed scheme generates diverse prompts from a domain bank that contains many more diverse domains than existing DG datasets. Furthermore, our method yields domain-unified representations from these prompts, thus being able to cope with samples from open-world domains. Extensive experiments on mainstream DG datasets, namely PACS, VLCS, OfficeHome, and DomainNet, show that the proposed method achieves competitive performance compared to state-of-the-art (SOTA) DG methods that require source domain data for training. Besides, we collect a small datasets consists of two domains to evaluate the open-world domain generalization ability of the proposed method. The source code and the dataset will be made publicly available at https://github.com/muse1998/Source-Free-Domain-Generalization

cs.CV↗

arXiv2022

Shape model and rotation acceleration of (1685) Toro and (85989) 1999 JD6 from optical observations

The Yarkovsky-O'Keefe-Radzievskii-Paddack (YORP) effect is a net torque caused by solar radiation directly reflected and thermally re-emitted from the surface of small asteroids and is considered to be crucial in their dynamical evolution. By long-term photometric observations of selected near-Earth asteroids, it's hoped to enlarge asteroid samples with a detected YORP effect to facilitate the development of a theoretical framework. Archived light-curve data are collected and photometric observations are made for (1685) Toro and (85989) 1999 JD6, which enables measurement of their YORP effect by inverting the light curve to fit observations from a convex shape model. For (1685) Toro, a YORP acceleration $\upsilon=(3.2\pm0.3)\times10^{-9}\ \rm{rad\cdot d^{-2}}(1σ error)$ is updated, which is consistent with previous YORP detection based on different light-curve data; for (85989) 1999 JD6, it is determined that the sidereal period is $7.667749\pm 0.000009$ h, the rotation pole direction locates is at $λ=232\pm 2^{\circ},\ β= -59\pm 1^{\circ}$, the acceleration is detected to be $\upsilon = (2.4\pm0.3)\times10^{-8}\ \rm{rad\cdot d^{-2}}(1σ error)$ and in addition to obtaining an excellent agreement between the observations and model. YORP should produce both spin-up and spin-down cases. However, including (85989) 1999 JD6, the $\rm{d}ω/\rm{d}t$ values of eleven near-Earth asteroids are positive totally, which suggests that there is either a bias in the sample of YORP detections or a real feature needs to be explained.

astro-ph.EP↗

arXiv2022

Clustering-Based Representation Learning through Output Translation and Its Application to Remote--Sensing Images

In supervised deep learning, learning good representations for remote--sensing images (RSI) relies on manual annotations. However, in the area of remote sensing, it is hard to obtain huge amounts of labeled data. Recently, self--supervised learning shows its outstanding capability to learn representations of images, especially the methods of instance discrimination. Comparing methods of instance discrimination, clustering--based methods not only view the transformations of the same image as ``positive" samples but also similar images. In this paper, we propose a new clustering-based method for representation learning. We first introduce a quantity to measure representations' discriminativeness and from which we show that even distribution requires the most discriminative representations. This provides a theoretical insight into why evenly distributing the images works well. We notice that only the even distributions that preserve representations' neighborhood relations are desirable. Therefore, we develop an algorithm that translates the outputs of a neural network to achieve the goal of evenly distributing the samples while preserving outputs' neighborhood relations. Extensive experiments have demonstrated that our method can learn representations that are as good as or better than the state of the art approaches, and that our method performs computationally efficiently and robustly on various RSI datasets.

cs.LG↗

arXiv2022

TODE-Trans: Transparent Object Depth Estimation with Transformer

Transparent objects are widely used in industrial automation and daily life. However, robust visual recognition and perception of transparent objects have always been a major challenge. Currently, most commercial-grade depth cameras are still not good at sensing the surfaces of transparent objects due to the refraction and reflection of light. In this work, we present a transformer-based transparent object depth estimation approach from a single RGB-D input. We observe that the global characteristics of the transformer make it easier to extract contextual information to perform depth estimation of transparent areas. In addition, to better enhance the fine-grained features, a feature fusion module (FFM) is designed to assist coherent prediction. Our empirical evidence demonstrates that our model delivers significant improvements in recent popular datasets, e.g., 25% gain on RMSE and 21% gain on REL compared to previous state-of-the-art convolutional-based counterparts in ClearGrasp dataset. Extensive results show that our transformer-based model enables better aggregation of the object's RGB and inaccurate depth information to obtain a better depth representation. Our code and the pre-trained model will be available at https://github.com/yuchendoudou/TODE.

cs.CV↗

arXiv2022

Phase Transitions and Superconductivity in Ternary Hydride Li$_2$SiH$_6$ at High Pressures

We predicted a new ternary hydride Li$_2$SiH$_6$ at high pressures. A systematic structure search in Li$_2$SiH$_6$ compound reveals novel stable phases with intriguing electronic and phonon properties. It is found that Li$_2$SiH$_6$ is dynamically stable from ambient pressure up to 400 GPa with three novel phases: P312, P$\bar{3}$, and P$\bar{6}$2m. The calculation of electron-phonon coupling combined with Bardeen-Cooper-Schrieffer's argument indicates that this compound may be a candidate for high $T_c$ superconductors under high pressures. In particular, the maximum $T_c$ of $P\bar{6}2m$-Li$_2$SiH$_6$ at 400 GPa reaches 56 K. These findings may pave the way for obtaining room temperature superconductors in dense hydrogen-rich compounds.

cond-mat.mtrl-sci↗

arXiv2022

A Q-band line survey towards Orion KL using the Tianma radio telescope

We have conducted a line survey towards Orion KL using the Q-band receiver of Tianma 65 m radio telescope (TMRT), covering 34.8--50 GHz with a velocity resolution between 0.79 km s$^{-1}$ and 0.55 km s$^{-1}$ respectively. The observations reach a sensitivity on the level of 1-8 mK, proving that the TMRT is sensitive for conducting deep line surveys. In total, 597 Gaussian features are extracted. Among them, 177 radio recombination lines (RRLs) are identified, including 126, 40 and 11 RRLs of hydrogen, helium and carbon, with a maximum $Δn$ of 16, 7, and 3, respectively. The carbon RRLs are confirmed to originate from photodissociation regions with a $V_{\rm LSR}\sim$9 km s$^{-1}$. In addition, 371 molecular transitions of 53 molecular species are identified. Twenty-one molecular species of this survey were not firmly detected in the Q band by Rizzo et al. (2017), including species such as H$_2$CS, HCOOH, C$_2$H$_5$OH, H$_2^{13}$CO, H$_2$CCO, CH$_3$CHO, CH$_2$OCH$_2$, HCN $v_2=1$, and CH$_3$OCHO $v_t=1$. In particular, the vibrationally excited states of ethyl cyanide (C$_2$H$_5$CN $v$13/$v$21) are for the first time firmly detected in the Q band. NH$_3$ (15,15) and (16,16) are identified, and they are so far the highest transitions of the NH$_3$ inversion lines detected towards Orion KL. All the identified lines can be reproduced by a radiative transfer model.

astro-ph.GA↗

arXiv2022

Combinatorial Procurement Auction in Social Networks

This paper studies one emerging procurement auction scenario where the market is constructed over the social networks. In a social network composed of many agents, smartphones or computers, one requester releases her requirement for goods or tasks to suppliers, then suppliers who have entered the market are also encouraged to invite some other suppliers to join and all the suppliers in the network could compete for the business. The key problem for this networked auction is about how to incentivize each node who have entered the sell not only to truthfully use her full ability, but also to forward the task to her neighbours. Auctions conducting over social networks have attracted considerable interests in recent years. However, most of the existing works focus on classic forward auctions. Moreover, there is no existing valid networked auction considering multiple goods/tasks. This work is the first to explore procurement auction for both homogeneous and heterogeneous goods or tasks in social networks. From both theoretical proof and experimental simulation, we proved that the proposed mechanisms are proved to be individual-rational and incentive-compatible, also both the cost of the system and the requester could get decreased.

cs.GT↗

arXiv2022

Charactering instrumental noises and stochastic gravitational wave signals from combined time-delay interferometry

LISA will detect gravitational waves (GWs) in the milli-Hz frequency band in space. Time-delay interferometry (TDI) is developed to suppress laser frequency noise beneath the acceleration noise and optical metrology noise. To identify stochastic GW signals, it would be required to characterize these noise components entangled in TDI data streams. In this work, we investigate noises characterization by combining the first-generation TDI channels from Michelson and Relay configurations. The Michelson channels are helpful to characterize acceleration noises in the lower frequency band, and the Relay configuration could effectively resolve optical path noises in the higher frequencies. Synergy could be achieved from their combination to determine these instrumental noises. Based on the characterized noises, we further reconstruct the power spectrum of noise in the selected TDI channel. Two cases are performed to characterize the spectrum shape of a stochastic GW signal. For a modeled signal, its parameter(s) could be directly estimated from the TDI data, and its spectrum could be recovered from the inferred values. And for an unexpected signal, its spectrum may be recognized and retrieved from noise-subtracted residual in which its power spectral density surpasses the noise level.

gr-qc↗

arXiv2022

Style Spectroscope: Improve Interpretability and Controllability through Fourier Analysis

Universal style transfer (UST) infuses styles from arbitrary reference images into content images. Existing methods, while enjoying many practical successes, are unable of explaining experimental observations, including different performances of UST algorithms in preserving the spatial structure of content images. In addition, methods are limited to cumbersome global controls on stylization, so that they require additional spatial masks for desired stylization. In this work, we provide a systematic Fourier analysis on a general framework for UST. We present an equivalent form of the framework in the frequency domain. The form implies that existing algorithms treat all frequency components and pixels of feature maps equally, except for the zero-frequency component. We connect Fourier amplitude and phase with Gram matrices and a content reconstruction loss in style transfer, respectively. Based on such equivalence and connections, we can thus interpret different structure preservation behaviors between algorithms with Fourier phase. Given the interpretations we have, we propose two manipulations in practice for structure preservation and desired stylization. Both qualitative and quantitative experiments demonstrate the competitive performance of our method against the state-of-the-art methods. We also conduct experiments to demonstrate (1) the abovementioned equivalence, (2) the interpretability based on Fourier amplitude and phase and (3) the controllability associated with frequency components.

cs.CV↗

arXiv2022

Overlooked Poses Actually Make Sense: Distilling Privileged Knowledge for Human Motion Prediction

Previous works on human motion prediction follow the pattern of building a mapping relation between the sequence observed and the one to be predicted. However, due to the inherent complexity of multivariate time series data, it still remains a challenge to find the extrapolation relation between motion sequences. In this paper, we present a new prediction pattern, which introduces previously overlooked human poses, to implement the prediction task from the view of interpolation. These poses exist after the predicted sequence, and form the privileged sequence. To be specific, we first propose an InTerPolation learning Network (ITP-Network) that encodes both the observed sequence and the privileged sequence to interpolate the in-between predicted sequence, wherein the embedded Privileged-sequence-Encoder (Priv-Encoder) learns the privileged knowledge (PK) simultaneously. Then, we propose a Final Prediction Network (FP-Network) for which the privileged sequence is not observable, but is equipped with a novel PK-Simulator that distills PK learned from the previous network. This simulator takes as input the observed sequence, but approximates the behavior of Priv-Encoder, enabling FP-Network to imitate the interpolation process. Extensive experimental results demonstrate that our prediction pattern achieves state-of-the-art performance on benchmarked H3.6M, CMU-Mocap and 3DPW datasets in both short-term and long-term predictions.

cs.CV↗

arXiv2022

A New Perspective on Stabilizing GANs training: Direct Adversarial Training

Generative Adversarial Networks (GANs) are the most popular image generation models that have achieved remarkable progress on various computer vision tasks. However, training instability is still one of the open problems for all GAN-based algorithms. Quite a number of methods have been proposed to stabilize the training of GANs, the focuses of which were respectively put on the loss functions, regularization and normalization technologies, training algorithms, and model architectures. Different from the above methods, in this paper, a new perspective on stabilizing GANs training is presented. It is found that sometimes the images produced by the generator act like adversarial examples of the discriminator during the training process, which may be part of the reason causing the unstable training of GANs. With this finding, we propose the Direct Adversarial Training (DAT) method to stabilize the training process of GANs. Furthermore, we prove that the DAT method is able to minimize the Lipschitz constant of the discriminator adaptively. The advanced performance of DAT is verified on multiple loss functions, network architectures, hyper-parameters, and datasets. Specifically, DAT achieves significant improvements of 11.5% FID on CIFAR-100 unconditional generation based on SSGAN, 10.5% FID on STL-10 unconditional generation based on SSGAN, and 13.2% FID on LSUN-Bedroom unconditional generation based on SSGAN. Code will be available at https://github.com/iceli1007/DAT-GAN

eess.IV↗

arXiv2022

FakeCLR: Exploring Contrastive Learning for Solving Latent Discontinuity in Data-Efficient GANs

Data-Efficient GANs (DE-GANs), which aim to learn generative models with a limited amount of training data, encounter several challenges for generating high-quality samples. Since data augmentation strategies have largely alleviated the training instability, how to further improve the generative performance of DE-GANs becomes a hotspot. Recently, contrastive learning has shown the great potential of increasing the synthesis quality of DE-GANs, yet related principles are not well explored. In this paper, we revisit and compare different contrastive learning strategies in DE-GANs, and identify (i) the current bottleneck of generative performance is the discontinuity of latent space; (ii) compared to other contrastive learning strategies, Instance-perturbation works towards latent space continuity, which brings the major improvement to DE-GANs. Based on these observations, we propose FakeCLR, which only applies contrastive learning on perturbed fake samples, and devises three related training techniques: Noise-related Latent Augmentation, Diversity-aware Queue, and Forgetting Factor of Queue. Our experimental results manifest the new state of the arts on both few-shot generation and limited-data generation. On multiple datasets, FakeCLR acquires more than 15% FID improvement compared to existing DE-GANs. Code is available at https://github.com/iceli1007/FakeCLR.

cs.CV↗

arXiv2022

Hybrid Spatial-Temporal Entropy Modelling for Neural Video Compression

For neural video codec, it is critical, yet challenging, to design an efficient entropy model which can accurately predict the probability distribution of the quantized latent representation. However, most existing video codecs directly use the ready-made entropy model from image codec to encode the residual or motion, and do not fully leverage the spatial-temporal characteristics in video. To this end, this paper proposes a powerful entropy model which efficiently captures both spatial and temporal dependencies. In particular, we introduce the latent prior which exploits the correlation among the latent representation to squeeze the temporal redundancy. Meanwhile, the dual spatial prior is proposed to reduce the spatial redundancy in a parallel-friendly manner. In addition, our entropy model is also versatile. Besides estimating the probability distribution, our entropy model also generates the quantization step at spatial-channel-wise. This content-adaptive quantization mechanism not only helps our codec achieve the smooth rate adjustment in single model but also improves the final rate-distortion performance by dynamic bit allocation. Experimental results show that, powered by the proposed entropy model, our neural codec can achieve 18.2% bitrate saving on UVG dataset when compared with H.266 (VTM) using the highest compression ratio configuration. It makes a new milestone in the development of neural video codec. The codes are at https://github.com/microsoft/DCVC.

eess.IV↗

arXiv2022

Secure UAV-to-Ground MIMO Communications: Joint Transceiver and Location Optimization

Unmanned aerial vehicles (UAVs) are foreseen to constitute promising airborne communication devices as a benefit of their superior channel quality. But UAV-to-ground (U2G) communications are vulnerable to eavesdropping. Hence, we conceive a sophisticated physical layer security solution for improving the secrecy rate of multi-antenna aided U2G systems. Explicitly, the secrecy rate of the U2G MIMO wiretap channels is derived by using random matrix theory. The resultant explicit expression is then applied in the joint optimization of the MIMO transceiver and the UAV location relying on an alternating optimization technique. Our numerical results show that the joint transceiver and location optimization conceived facilitates secure communications even in the challenging scenario, where the legitimate channel of confidential information is inferior to the eavesdropping channel.

cs.IT↗

arXiv2022

Digital Twin Assisted Task Offloading for Aerial Edge Computing and Networks

Considering the user mobility and unpredictable mobile edge computing (MEC) environments, this paper studies the intelligent task offloading problem in unmanned aerial vehicle (UAV)-enabled MEC with the assistance of digital twin (DT). We aim at minimizing the energy consumption of the entire MEC system by jointly optimizing mobile terminal users (MTUs) association, UAV trajectory, transmission power distribution and computation capacity allocation while respecting the constraints of mission maximum processing delays. Specifically, double deep Q-network (DDQN) algorithm stemming from deep reinforcement learning is first proposed to effectively solve the problem of MTUs association and UAV trajectory. Then, the closed-form expression is employed to handle the problem of transmission power distribution and the computation capacity allocation problem is further addressed via an iterative algorithm. Numerical results show that our proposed scheme is able to converge and significantly reduce the total energy consumption of the MEC system compared to the benchmark schemes.

cs.IT↗

arXiv2022

Scene-Aware Prompt for Multi-modal Dialogue Understanding and Generation

This paper introduces the schemes of Team LingJing's experiments in NLPCC-2022-Shared-Task-4 Multi-modal Dialogue Understanding and Generation (MDUG). The MDUG task can be divided into two phases: multi-modal context understanding and response generation. To fully leverage the visual information for both scene understanding and dialogue generation, we propose the scene-aware prompt for the MDUG task. Specifically, we utilize the multi-tasking strategy for jointly modelling the scene- and session- multi-modal understanding. The visual captions are adopted to aware the scene information, while the fixed-type templated prompt based on the scene- and session-aware labels are used to further improve the dialogue generation performance. Extensive experimental results show that the proposed method has achieved state-of-the-art (SOTA) performance compared with other competitive methods, where we rank the 1-st in all three subtasks in this MDUG competition.

cs.CL↗