arXiv subjects

Bin Ren

Publications and source records attributed to Bin Ren.

At least 109 records · Page 6Linked to original sources

arXiv2023

PI-Trans: Parallel-ConvMLP and Implicit-Transformation Based GAN for Cross-View Image Translation

For semantic-guided cross-view image translation, it is crucial to learn where to sample pixels from the source view image and where to reallocate them guided by the target view semantic map, especially when there is little overlap or drastic view difference between the source and target images. Hence, one not only needs to encode the long-range dependencies among pixels in both the source view image and target view semantic map but also needs to translate these learned dependencies. To this end, we propose a novel generative adversarial network, PI-Trans, which mainly consists of a novel Parallel-ConvMLP module and an Implicit Transformation module at multiple semantic levels. Extensive experimental results show that PI-Trans achieves the best qualitative and quantitative performance by a large margin compared to the state-of-the-art methods on two challenging datasets. The source code is available at https://github.com/Amazingren/PI-Trans.

cs.CV↗

arXiv2023

Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search

Though recent years have witnessed remarkable progress in single image super-resolution (SISR) tasks with the prosperous development of deep neural networks (DNNs), the deep learning methods are confronted with the computation and memory consumption issues in practice, especially for resource-limited platforms such as mobile devices. To overcome the challenge and facilitate the real-time deployment of SISR tasks on mobile, we combine neural architecture search with pruning search and propose an automatic search framework that derives sparse super-resolution (SR) models with high image quality while satisfying the real-time inference requirement. To decrease the search cost, we leverage the weight sharing strategy by introducing a supernet and decouple the search problem into three stages, including supernet construction, compiler-aware architecture and pruning search, and compiler-aware pruning ratio search. With the proposed framework, we are the first to achieve real-time SR inference (with only tens of milliseconds per frame) for implementing 720p resolution with competitive image quality (in terms of PSNR and SSIM) on mobile platforms (Samsung Galaxy S20).

eess.IV↗

arXiv2022

Deep Unsupervised Key Frame Extraction for Efficient Video Classification

Video processing and analysis have become an urgent task since a huge amount of videos (e.g., Youtube, Hulu) are uploaded online every day. The extraction of representative key frames from videos is very important in video processing and analysis since it greatly reduces computing resources and time. Although great progress has been made recently, large-scale video classification remains an open problem, as the existing methods have not well balanced the performance and efficiency simultaneously. To tackle this problem, this work presents an unsupervised method to retrieve the key frames, which combines Convolutional Neural Network (CNN) and Temporal Segment Density Peaks Clustering (TSDPC). The proposed TSDPC is a generic and powerful framework and it has two advantages compared with previous works, one is that it can calculate the number of key frames automatically. The other is that it can preserve the temporal information of the video. Thus it improves the efficiency of video classification. Furthermore, a Long Short-Term Memory network (LSTM) is added on the top of the CNN to further elevate the performance of classification. Moreover, a weight fusion strategy of different input networks is presented to boost the performance. By optimizing both video classification and key frame extraction simultaneously, we achieve better classification performance and higher efficiency. We evaluate our method on two popular datasets (i.e., HMDB51 and UCF101) and the experimental results consistently demonstrate that our strategy achieves competitive performance and efficiency compared with the state-of-the-art approaches.

cs.CV↗

arXiv2022

SPViT: Enabling Faster Vision Transformers via Soft Token Pruning

Recently, Vision Transformer (ViT) has continuously established new milestones in the computer vision field, while the high computation and memory cost makes its propagation in industrial production difficult. Pruning, a traditional model compression paradigm for hardware efficiency, has been widely applied in various DNN structures. Nevertheless, it stays ambiguous on how to perform exclusive pruning on the ViT structure. Considering three key points: the structural characteristics, the internal data pattern of ViTs, and the related edge device deployment, we leverage the input token sparsity and propose a computation-aware soft pruning framework, which can be set up on vanilla Transformers of both flatten and CNN-type structures, such as Pooling-based ViT (PiT). More concretely, we design a dynamic attention-based multi-head token selector, which is a lightweight module for adaptive instance-wise token selection. We further introduce a soft pruning technique, which integrates the less informative tokens generated by the selector module into a package token that will participate in subsequent calculations rather than being completely discarded. Our framework is bound to the trade-off between accuracy and computation constraints of specific edge devices through our proposed computation-aware training strategy. Experimental results show that our framework significantly reduces the computation cost of ViTs while maintaining comparable performance on image classification. Moreover, our framework can guarantee the identified model to meet resource specifications of mobile devices and FPGA, and even achieve the real-time execution of DeiT-T on mobile platforms. For example, our method reduces the latency of DeiT-T to 26 ms (26%$\sim $41% superior to existing works) on the mobile device with 0.25%$\sim $4% higher top-1 accuracy on ImageNet.

cs.CV↗

arXiv2022

SparCL: Sparse Continual Learning on the Edge

Existing work in continual learning (CL) focuses on mitigating catastrophic forgetting, i.e., model performance deterioration on past tasks when learning a new task. However, the training efficiency of a CL system is under-investigated, which limits the real-world application of CL systems under resource-limited scenarios. In this work, we propose a novel framework called Sparse Continual Learning(SparCL), which is the first study that leverages sparsity to enable cost-effective continual learning on edge devices. SparCL achieves both training acceleration and accuracy preservation through the synergy of three aspects: weight sparsity, data efficiency, and gradient sparsity. Specifically, we propose task-aware dynamic masking (TDM) to learn a sparse network throughout the entire CL process, dynamic data removal (DDR) to remove less informative training data, and dynamic gradient masking (DGM) to sparsify the gradient updates. Each of them not only improves efficiency, but also further mitigates catastrophic forgetting. SparCL consistently improves the training efficiency of existing state-of-the-art (SOTA) CL methods by at most 23X less training FLOPs, and, surprisingly, further improves the SOTA accuracy by at most 1.7%. SparCL also outperforms competitive baselines obtained from adapting SOTA sparse training methods to the CL setting in both efficiency and accuracy. We also evaluate the effectiveness of SparCL on a real mobile phone, further indicating the practical potential of our method.

cs.LG↗

arXiv2022

Survey: Exploiting Data Redundancy for Optimization of Deep Learning

Data redundancy is ubiquitous in the inputs and intermediate results of Deep Neural Networks (DNN). It offers many significant opportunities for improving DNN performance and efficiency and has been explored in a large body of work. These studies have scattered in many venues across several years. The targets they focus on range from images to videos and texts, and the techniques they use to detect and exploit data redundancy also vary in many aspects. There is not yet a systematic examination and summary of the many efforts, making it difficult for researchers to get a comprehensive view of the prior work, the state of the art, differences and shared principles, and the areas and directions yet to explore. This article tries to fill the void. It surveys hundreds of recent papers on the topic, introduces a novel taxonomy to put the various techniques into a single categorization framework, offers a comprehensive description of the main methods used for exploiting data redundancy in improving multiple kinds of DNNs on data, and points out a set of research opportunities for future to explore.

cs.LG↗

arXiv2022

A Clear View of a Cloudy Brown Dwarf Companion from High-Resolution Spectroscopy

Direct imaging studies have mainly used low-resolution spectroscopy ($R\sim20-100$) to study the atmospheres of giant exoplanets and brown dwarf companions, but the presence of clouds has often led to degeneracies in the retrieved atmospheric abundances (e.g. C/O, metallicity). This precludes clear insights into the formation mechanisms of these companions. The Keck Planet Imager and Characterizer (KPIC) uses adaptive optics and single-mode fibers to transport light into NIRSPEC ($R\sim35,000$ in $K$ band), and aims to address these challenges with high-resolution spectroscopy. Using an atmospheric retrieval framework based on petitRADTRANS, we analyze KPIC high-resolution spectrum ($2.29-2.49~μ$m) and archival low-resolution spectrum ($1-2.2~μ$m) of the benchmark brown dwarf HD 4747 B ($m=67.2\pm1.8~M_{\rm{Jup}}$, $a=10.0\pm0.2$ au, $T_{\rm eff}\approx1400$ K). We find that our measured C/O and metallicity for the companion from the KPIC high-resolution spectrum agree with that of its host star within $1-2σ$. The retrieved parameters from the $K$ band high-resolution spectrum are also independent of our choice of cloud model. In contrast, the retrieved parameters from the low-resolution spectrum are highly sensitive to our chosen cloud model. Finally, we detect CO, H$_2$O, and CH$_4$ (volume mixing ratio of log(CH$_4$)=$-4.82\pm0.23$) in this L/T transition companion with the KPIC data. The relative molecular abundances allow us to constrain the degree of chemical disequilibrium in the atmosphere of HD 4747 B, and infer a vertical diffusion coefficient that is at the upper limit predicted from mixing length theory.

astro-ph.EP↗

arXiv2022

Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution

Deep learning-based super-resolution (SR) has gained tremendous popularity in recent years because of its high image quality performance and wide application scenarios. However, prior methods typically suffer from large amounts of computations and huge power consumption, causing difficulties for real-time inference, especially on resource-limited platforms such as mobile devices. To mitigate this, we propose a compiler-aware SR neural architecture search (NAS) framework that conducts depth search and per-layer width search with adaptive SR blocks. The inference speed is directly taken into the optimization along with the SR loss to derive SR models with high image quality while satisfying the real-time inference requirement. Instead of measuring the speed on mobile devices at each iteration during the search process, a speed model incorporated with compiler optimizations is leveraged to predict the inference latency of the SR block with various width configurations for faster convergence. With the proposed framework, we achieve real-time SR inference for implementing 720p resolution with competitive SR performance (in terms of PSNR and SSIM) on GPU/DSP of mobile platforms (Samsung Galaxy S21).

cs.CV↗

arXiv2022

CoCoPIE XGen: A Full-Stack AI-Oriented Optimizing Framework

There is a growing demand for shifting the delivery of AI capability from data centers on the cloud to edge or end devices, exemplified by the fast emerging real-time AI-based apps running on smartphones, AR/VR devices, autonomous vehicles, and various IoT devices. The shift has however been seriously hampered by the large growing gap between DNN computing demands and the computing power on edge or end devices. This article presents the design of XGen, an optimizing framework for DNN designed to bridge the gap. XGen takes cross-cutting co-design as its first-order consideration. Its full-stack AI-oriented optimizations consist of a number of innovative optimizations at every layer of the DNN software stack, all designed in a cooperative manner. The unique technology makes XGen able to optimize various DNNs, including those with an extreme depth (e.g., BERT, GPT, other transformers), and generate code that runs several times faster than those from existing DNN frameworks, while delivering the same level of accuracy.

cs.LG↗

arXiv2022

ALMA Images the Eccentric HD 53143 Debris Disk

We present ALMA 1.3 mm observations of the HD~53143 debris disk - the first infrared or millimeter image produced of this ~1 Gyr-old solar-analogue. Previous HST STIS coronagraphic imaging did not detect flux along the minor axis of the disk which could suggest a face-on geometry with two 'clumps' of dust. These ALMA observations reveal a disk with a strikingly different structure. In order to fit models to the millimeter visibilities and constrain the uncertainties on the disk parameters, we adopt an MCMC approach. This is the most eccentric debris disk observed to date with a forced eccentricity of $0.21\pm0.02$, nearly twice that of the Fomalhaut debris disk, and also displays apocenter glow. Although this eccentric model fits the outer debris disk well, there are significant interior residuals remaining that may suggest a possible edge-on inner disk, which remains unresolved in these observations. Combined with the observed structure difference between HST and ALMA, these results suggest a potential previous scattering event or dynamical instability in this system. We also note that the stellar flux changes considerably over the course of our observations, suggesting flaring at millimeter wavelengths. Using simultaneous TESS observations, we determine the stellar rotation period to be $9.6\pm0.1$ days.

astro-ph.EP↗

arXiv2022

Real-Time Portrait Stylization on the Edge

In this work we demonstrate real-time portrait stylization, specifically, translating self-portrait into cartoon or anime style on mobile devices. We propose a latency-driven differentiable architecture search method, maintaining realistic generative quality. With our framework, we obtain $10\times$ computation reduction on the generative model and achieve real-time video stylization on off-the-shelf smartphone using mobile GPUs.

cs.CV↗

arXiv2022

Disk Evolution Study Through Imaging of Nearby Young Stars (DESTINYS): A Panchromatic View of DO Tau's Complex Kilo-au Environment

While protoplanetary disks are often treated as isolated systems in planet formation models, observations increasingly suggest that vigorous interactions between Class II disks and their environments are not rare. DO Tau is a T Tauri star that has previously been hypothesized to have undergone a close encounter with the HV Tau system. As part of the DESTINYS ESO Large Programme, we present new VLT/SPHERE polarimetric observations of DO Tau and combine them with archival HST scattered light images and ALMA observations of CO isotopologues and CS to map a network of complex structures. The SPHERE and ALMA observations show that the circumstellar disk is connected to arms extending out to several hundred au. HST and ALMA also reveal stream-like structures northeast of DO Tau, some of which are at least several thousand au long. These streams appear not to be gravitationally bound to DO Tau, and comparisons with previous Herschel far-IR observations suggest that the streams are part of a bridge-like structure connecting DO Tau and HV Tau. We also detect a fainter redshifted counterpart to a previously known blueshifted CO outflow. While some of DO Tau's complex structures could be attributed to a recent disk-disk encounter, they might be explained alternatively by interactions with remnant material from the star formation process. These panchromatic observations of DO Tau highlight the need to contextualize the evolution of Class II disks by examining processes occurring over a wide range of size scales.

astro-ph.SR↗

arXiv2022

Speed-ANN: Low-Latency and High-Accuracy Nearest Neighbor Search via Intra-Query Parallelism

Nearest Neighbor Search (NNS) has recently drawn a rapid increase of interest due to its core role in managing high-dimensional vector data in data science and AI applications. The interest is fueled by the success of neural embedding, where deep learning models transform unstructured data into semantically correlated feature vectors for data analysis, e.g., recommend popular items. Among several categories of methods for fast NNS, similarity graph is one of the most successful algorithmic trends. Several of the most popular and top-performing similarity graphs, such as NSG and HNSW, at their core employ best-first traversal along the underlying graph indices to search near neighbors. Maximizing the performance of the search is essential for many tasks, especially at the large-scale and high-recall regime. In this work, we provide an in-depth examination of the challenges of the state-of-the-art similarity search algorithms, revealing its challenges in leveraging multi-core processors to speed up the search efficiency. We also exploit whether similarity graph search is robust to deviation from maintaining strict order by allowing multiple walkers to simultaneously advance the search frontier. Based on our insights, we propose Speed-ANN, a parallel similarity search algorithm that exploits hidden intra-query parallelism and memory hierarchy that allows similarity search to take advantage of multiple CPU cores to significantly accelerate search speed while achieving high accuracy. We evaluate Speed-ANN on a wide range of datasets, ranging from million to billion data points, and show its shorter query latency than NSG and HNSW, respectively. Besides, with multicore support, we show that our approach offers faster search latency than highly-optimized GPU implementation and provides good scalability as the increase of the number of hardware resources (e.g., CPU cores) and graph sizes.

cs.DC↗

arXiv2021

Sensitivity of the Roman Coronagraph Instrument to Exozodiacal Dust

Exozodiacal dust, warm debris from comets and asteroids in and near the habitable zone of stellar systems, reveals the physical processes that shape planetary systems. Scattered light from this dust is also a source of background flux which must be overcome by future missions to image Earthlike planets. This study quantifies the sensitivity of the Nancy Grace Roman Space Telescope Coronagraph to light scattered by exozodi, the zodiacal dust around other stars. Using a sample of 149 nearby stars, previously selected for optimum detection of habitable exoplanets by space observatories, we find the maximum number of exozodiacal disks with observable \textit{inner} habitable zone boundaries is six and the number of observable outer habitable boundaries is 74. One zodi was defined as the visible-light surface brightness of 22 $m_{\rm V}\ $arcsec$^{-2}$ around a solar-mass star, approximating the scattered light brightness in visible light at the Earth-equivalent insolation. In the speckle limited case, where the signal-to-noise ratio is limited by speckle temporal stability rather than shot noise, the median $5σ$ sensitivity to habitable zone exozodi is 12 zodi per resolution element. This estimate is calculated at the inner-working angle of the coronagraph, for the current best estimate performance, neglecting margins on the uncertainty in instrument performance and including a post-processing speckle suppression factor. For an log-norm distribution of exozodi levels with a median exozodi of 3$\times$ the solar zodi, we find that the Roman Coronagraph would be able to make 5$σ$ detections of exozodiacal disks in scattered light from 13 systems with a 95\% confidence interval spanning 7-20 systems. This sensitivity allows Roman Coronagraph to complement ground-based measurements of exozodiacal thermal emission and constrain dust albedos.

astro-ph.EP↗

arXiv2021

DNNFusion: Accelerating Deep Neural Networks Execution with Advanced Operator Fusion

Deep Neural Networks (DNNs) have emerged as the core enabler of many major applications on mobile devices. To achieve high accuracy, DNN models have become increasingly deep with hundreds or even thousands of operator layers, leading to high memory and computational requirements for inference. Operator fusion (or kernel/layer fusion) is key optimization in many state-of-the-art DNN execution frameworks, such as TensorFlow, TVM, and MNN. However, these frameworks usually adopt fusion approaches based on certain patterns that are too restrictive to cover the diversity of operators and layer connections. Polyhedral-based loop fusion techniques, on the other hand, work on a low-level view of the computation without operator-level information, and can also miss potential fusion opportunities. To address this challenge, this paper proposes a novel and extensive loop fusion framework called DNNFusion. The basic idea of this work is to work at an operator view of DNNs, but expand fusion opportunities by developing a classification of both individual operators and their combinations. In addition, DNNFusion includes 1) a novel mathematical-property-based graph rewriting framework to reduce evaluation costs and facilitate subsequent operator fusion, 2) an integrated fusion plan generation that leverages the high-level analysis and accurate light-weight profiling, and 3) additional optimizations during fusion code generation. DNNFusion is extensively evaluated on 15 DNN models with varied types of tasks, model sizes, and layer counts. The evaluation results demonstrate that DNNFusion finds up to 8.8x higher fusion opportunities, outperforms four state-of-the-art DNN execution frameworks with 9.3x speedup. The memory requirement reduction and speedups can enable the execution of many of the target models on mobile devices and even make them part of a real-time application.

cs.LG↗

arXiv2021

Spiral Arm Pattern Motion in the SAO 206462 Protoplanetary Disk

Spiral arms have been observed in more than a dozen protoplanetary disks, yet the origin of nearly all systems is under debate. Multi-epoch monitoring of spiral arm morphology offers a dynamical way in distinguishing two leading arm formation mechanisms: companion-driven, and gravitational instability induction, since these mechanisms predict distinct motion patterns. By analyzing multi-epoch J-band observations of the SAO 206462 system using the SPHERE instrument on the Very Large Telescope (VLT) in 2015 and 2016, we measure the pattern motion for its two prominent spiral arms in polarized light. On one hand, if both arms are comoving, they can be driven by a planet at $86_{-13}^{+18}$ au on a circular orbit, with gravitational instability motion ruled out. On the other hand, they can be driven by two planets at $120_{-30}^{+30}$ au and $49_{-5}^{+6}$ au, offering a tentative evidence (3.0$σ$) that the two spirals are moving independently. The independent arm motion is possibly supported by our analysis of a re-reduction of archival observations using the NICMOS instrument onboard the Hubble Space Telescope (HST) in 1998 and 2005, yet artifacts including shadows can manifest spurious arm motion in HST observations. We expect future re-observations to better constrain the motion mechanism for the SAO 206462 spiral arms.

astro-ph.EP↗

arXiv2021

Improving Planet Detection with Disk Modeling: Keck/NIRC2 Imaging of the HD 34282 Single-armed Protoplanetary Disk

Formed in protoplanetary disks around young stars, giant planets can leave observational features such as spirals and gaps in their natal disks through planet-disk interactions. Although such features can indicate the existence of giant planets, protoplanetary disk signals can overwhelm the innate luminosity of planets. Therefore, in order to image planets that are embedded in disks, it is necessary to remove the contamination from the disks to reveal the planets possibly hiding within their natal environments. We observe and directly model the detected disk in the Keck/NIRC2 vortex coronagraph $L'$-band observations of the single-armed protoplanetary disk around HD 34282. Despite a non-detection of companions for HD 34282, this direct disk modeling improves planet detection sensitivity by up to a factor of 2 in flux ratio and ${\sim}10 M_{\rm Jupiter}$ in mass. This suggests that performing disk modeling can improve directly imaged planet detection limits in systems with visible scattered light disks, and can help to better constrain the occurrence rates of self-luminous planets in these systems.

astro-ph.EP↗

arXiv2021

Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration

Weight pruning is an effective model compression technique to tackle the challenges of achieving real-time deep neural network (DNN) inference on mobile devices. However, prior pruning schemes have limited application scenarios due to accuracy degradation, difficulty in leveraging hardware acceleration, and/or restriction on certain types of DNN layers. In this paper, we propose a general, fine-grained structured pruning scheme and corresponding compiler optimizations that are applicable to any type of DNN layer while achieving high accuracy and hardware inference performance. With the flexibility of applying different pruning schemes to different layers enabled by our compiler optimizations, we further probe into the new problem of determining the best-suited pruning scheme considering the different acceleration and accuracy performance of various pruning schemes. Two pruning scheme mapping methods, one is search-based and the other is rule-based, are proposed to automatically derive the best-suited pruning regularity and block size for each layer of any given DNN. Experimental results demonstrate that our pruning scheme mapping methods, together with the general fine-grained structured pruning scheme, outperform the state-of-the-art DNN optimization framework with up to 2.48$\times$ and 1.73$\times$ DNN inference acceleration on CIFAR-10 and ImageNet dataset without accuracy loss.

cs.LG↗