arXiv subjects

Kan Wu

Publications and source records attributed to Kan Wu.

At least 37 records · Page 2Linked to original sources

arXiv2023

TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance

In this paper, we propose a novel cross-modal distillation method, called TinyCLIP, for large-scale language-image pre-trained models. The method introduces two core techniques: affinity mimicking and weight inheritance. Affinity mimicking explores the interaction between modalities during distillation, enabling student models to mimic teachers' behavior of learning cross-modal feature alignment in a visual-linguistic affinity space. Weight inheritance transmits the pre-trained weights from the teacher models to their student counterparts to improve distillation efficiency. Moreover, we extend the method into a multi-stage progressive distillation to mitigate the loss of informative weights during extreme compression. Comprehensive experiments demonstrate the efficacy of TinyCLIP, showing that it can reduce the size of the pre-trained CLIP ViT-B/32 by 50%, while maintaining comparable zero-shot performance. While aiming for comparable performance, distillation with weight inheritance can speed up the training by 1.4 - 7.8 $\times$ compared to training from scratch. Moreover, our TinyCLIP ViT-8M/16, trained on YFCC-15M, achieves an impressive zero-shot top-1 accuracy of 41.1% on ImageNet, surpassing the original CLIP ViT-B/16 by 3.5% while utilizing only 8.9% parameters. Finally, we demonstrate the good transferability of TinyCLIP in various downstream tasks. Code and models will be open-sourced at https://aka.ms/tinyclip.

cs.CV↗

arXiv2023

WiscSort: External Sorting For Byte-Addressable Storage

We present WiscSort, a new approach to high-performance concurrent sorting for existing and future byte-addressable storage (BAS) devices. WiscSort carefully reduces writes, exploits random reads by splitting keys and values during sorting, and performs interference-aware scheduling with thread pool sizing to avoid I/O bandwidth degradation. We introduce the BRAID model which encompasses the unique characteristics of BAS devices. Many state-of-the-art sorting systems do not comply with the BRAID model and deliver sub-optimal performance, whereas WiscSort demonstrates the effectiveness of complying with BRAID. We show that WiscSort is 2-7x faster than competing approaches on a standard sort benchmark. We evaluate the effectiveness of key-value separation on different key-value sizes and compare our concurrency optimizations with various other concurrency models. Finally, we emulate generic BAS devices and show how our techniques perform well with various combinations of hardware properties.

cs.DB↗

arXiv2023

CBLab: Supporting the Training of Large-scale Traffic Control Policies with Scalable Traffic Simulation

Traffic simulation provides interactive data for the optimization of traffic control policies. However, existing traffic simulators are limited by their lack of scalability and shortage in input data, which prevents them from generating interactive data from traffic simulation in the scenarios of real large-scale city road networks. In this paper, we present \textbf{C}ity \textbf{B}rain \textbf{Lab}, a toolkit for scalable traffic simulation. CBLab consists of three components: CBEngine, CBData, and CBScenario. CBEngine is a highly efficient simulator supporting large-scale traffic simulation. CBData includes a traffic dataset with road network data of 100 cities all around the world. We also develop a pipeline to conduct a one-click transformation from raw road networks to input data of our traffic simulation. Combining CBEngine and CBData allows researchers to run scalable traffic simulations in the road network of real large-scale cities. Based on that, CBScenario implements an interactive environment and a benchmark for two scenarios of traffic control policies respectively, with which traffic control policies adaptable for large-scale urban traffic can be trained and tuned. To the best of our knowledge, CBLab is the first infrastructure supporting traffic control policy optimization in large-scale urban scenarios. CBLab has supported the City Brain Challenge @ KDD CUP 2021. The project is available on GitHub:~\url{https://github.com/CityBrainLab/CityBrainLab.git}.

physics.soc-ph↗

arXiv2023

Experimental observation of Kerr-Raman solitons in a normal-dispersion FP resonator

Different from the Kerr effect,stimulated Raman scattering (SRS) is a delayed response to molecular vibrations in materials. In microcavities, when driven in an anomalous group velocity dispersion (GVD) regime, SRS typically leads to self-frequency shift of solitons and generation of breather solitons which have been verified both theoretically and experimentally. However, when driven in a normal GVD regime, recent theoretical work predicts that SRS can cause the locking of switching waves (SWs) and thus support bright moving localized structures (LSs), which we term as Kerr-Raman solitons (KRSs). Limited by the design of suitable experimental parameters, experimental observation of the KRSs is not achieved yet. Here, we provide numerical investigation, and to our knowledge, the first experimental observation of these SRS enabled KRSs in a fiber Fabry-Perot (FP) resonator with ultra-low normal GVD. Such Kerr-Raman solitons exhibit localized temporal features with strong oscillations at ~13 THz local frequency on the top of a flat-top pulse. The corresponding spectrum is a low-noise and broadband Kerr comb with typical platicon-like spectrum in the center and two Raman Stokes and anti-Stokes peaks located near 13 THz away from the center. With such SRS enabled broadband Kerr comb, we have achieved a KRS spectrum with a repetition rate of ~3.68 GHz and a -40 dB spectral width of 260 nm. The corresponding comb tooth count is >9000, covering the S+C+L telecommunication bands. Moreover, the formation process of such KRSs is also revealed, and it is found that the GVD plays a key role in its generation. Our work will help to advance the study of the dynamics of optical frequency combs under the influence of SRS, as well as providing a broadband coherent mode-locked optical source for wide applications.

physics.optics↗

arXiv2022

Driving Style Recognition at First Impression for Online Trajectory Prediction

This paper proposes a new driving style recognition approach that allows autonomous vehicles (AVs) to perform trajectory predictions for surrounding vehicles with minimal data. Toward that end, we use a hybrid of offline and online methods in the proposed approach. We first learn typical driving styles with PCA and K-means algorithms in the offline part. After that, local Maximum-Likelihood techniques are used to perform online driving style recognition. We benchmarked our method on a real driving dataset against other methods in terms of the RMSE value of the predicted trajectory and the observed trajectory over a 5s duration. The proposed approach can reduce trajectory prediction error by up to 37.7\% compared to using the parameters from other literature and up to 24.4\% compared to not performing driving style recognition.

eess.SY↗

arXiv2022

Cornus: Atomic Commit for a Cloud DBMS with Storage Disaggregation (Extended Version)

Two-phase commit (2PC) is widely used in distributed databases to ensure the atomicity of distributed transactions. However, 2PC has two limitations. First, it requires two eager log writes on the critical path, which incurs significant latency. Second, when a coordinator fails, a participant may be blocked waiting for the coordinator's decision, leading to indefinitely long latency and low throughput. 2PC was originally designed for a shared-nothing architecture. We observe that the two problems above can be addressed in an emerging storage disaggregation architecture which provides compare-and-swap capability in the storage layer. We propose Cornus, an optimized 2PC protocol for Cloud DBMS with Storage Disaggregation. We present Cornus in detail with proofs and show how it addresses the two limitations in 2PC. We also deploy it on real storage services including Azure Blob Storage and Redis. Empirical evaluations show that Cornus can achieve up to 1.9x speedup in latency over 2PC.

cs.DB↗

arXiv2022

TinyViT: Fast Pretraining Distillation for Small Vision Transformers

Vision transformer (ViT) recently has drawn great attention in computer vision due to its remarkable model capability. However, most prevailing ViT models suffer from huge number of parameters, restricting their applicability on devices with limited resources. To alleviate this issue, we propose TinyViT, a new family of tiny and efficient small vision transformers pretrained on large-scale datasets with our proposed fast distillation framework. The central idea is to transfer knowledge from large pretrained models to small ones, while enabling small models to get the dividends of massive pretraining data. More specifically, we apply distillation during pretraining for knowledge transfer. The logits of large teacher models are sparsified and stored in disk in advance to save the memory cost and computation overheads. The tiny student transformers are automatically scaled down from a large pretrained model with computation and parameter constraints. Comprehensive experiments demonstrate the efficacy of TinyViT. It achieves a top-1 accuracy of 84.8% on ImageNet-1k with only 21M parameters, being comparable to Swin-B pretrained on ImageNet-21k while using 4.2 times fewer parameters. Moreover, increasing image resolutions, TinyViT can reach 86.5% accuracy, being slightly better than Swin-L while using only 11% parameters. Last but not the least, we demonstrate a good transfer ability of TinyViT on various downstream tasks. Code and models are available at https://github.com/microsoft/Cream/tree/main/TinyViT.

cs.CV↗

arXiv2022

Free-space point-to-multiplepoint optical frequency transfer with lens assisted integrated beam steering

We report on the realization of high-performance silica integrated two-dimensional lens assisted beam-steering (LABS) arrays along with the first-of-their-kind point-to-multiplepoint optical frequency transfer. {The LABS equips with $N$ antennas} and has the capability to produce arbitrary number of output beams with different output angles with the simple control complexity. We demonstrate that the LABS has 16 scanning angles, which can support {the access capability for the maximum of simultaneous 16 user nodes.} The coaxial configuration for transmitting and receiving the light as a monolithic transceiver allows us to reduce the out-of-loop phase noise significantly. Finally, the LABS-based non-blocking point-to-multiplepoint in-door free-space optical frequency transfer links with 24 m and 50 m free-space links are shown. After being compensated for the free-space link up to 50 m, the fractional frequency instability of $4.5\times10^{-17}$ and $7.7\times10^{-20}$ at the averaging time of 1 s and 20,000 s, respectively, can be achieved. The present work proves the potential application of the 2D LABS in free-space optical time-frequency transfer and provides a guidance for developing a chip-scale optical time-frequency transfer system.

physics.ins-det↗

arXiv2022

MiniViT: Compressing Vision Transformers with Weight Multiplexing

Vision Transformer (ViT) models have recently drawn much attention in computer vision due to their high model capability. However, ViT models suffer from huge number of parameters, restricting their applicability on devices with limited memory. To alleviate this problem, we propose MiniViT, a new compression framework, which achieves parameter reduction in vision transformers while retaining the same performance. The central idea of MiniViT is to multiplex the weights of consecutive transformer blocks. More specifically, we make the weights shared across layers, while imposing a transformation on the weights to increase diversity. Weight distillation over self-attention is also applied to transfer knowledge from large-scale ViT models to weight-multiplexed compact models. Comprehensive experiments demonstrate the efficacy of MiniViT, showing that it can reduce the size of the pre-trained Swin-B transformer by 48\%, while achieving an increase of 1.0\% in Top-1 accuracy on ImageNet. Moreover, using a single-layer of parameters, MiniViT is able to compress DeiT-B by 9.7 times from 86M to 9M parameters, without seriously compromising the performance. Finally, we verify the transferability of MiniViT by reporting its performance on downstream benchmarks. Code and models are available at here.

cs.CV↗

arXiv2021

Searching the Search Space of Vision Transformer

Vision Transformer has shown great visual representation power in substantial vision tasks such as recognition and detection, and thus been attracting fast-growing efforts on manually designing more effective architectures. In this paper, we propose to use neural architecture search to automate this process, by searching not only the architecture but also the search space. The central idea is to gradually evolve different search dimensions guided by their E-T Error computed using a weight-sharing supernet. Moreover, we provide design guidelines of general vision transformers with extensive analysis according to the space searching process, which could promote the understanding of vision transformer. Remarkably, the searched models, named S3 (short for Searching the Search Space), from the searched space achieve superior performance to recently proposed models, such as Swin, DeiT and ViT, when evaluated on ImageNet. The effectiveness of S3 is also illustrated on object detection, semantic segmentation and visual question answering, demonstrating its generality to downstream vision and vision-language tasks. Code and models will be available at https://github.com/microsoft/Cream.

cs.CV↗

arXiv2021

Erbium-doped lithium niobate thin film waveguide amplifier with 16 dB internal net gain

Erbium-doped lithium niobate on insulator (Er:LNOI) has attracted enormous interest as it provides gain and enables integrated amplifiers and lasers on the lithium niobate on insulator (LNOI) platform. We demonstrate a highly efficient waveguide amplifier on Er:LNOI. The 2.58-cm long amplifier can achieve 27.94 dB signal enhancement, 16.0 dB internal net gain (6.20 dB/cm), -8.84 dBm saturation power, 4.59 dB/mW power conversion efficiency, and 4.49 dB noise figure at 1531.6 nm. Besides, thorough investigation on the pumping wavelength, pumping scheme, output power and noise figure have been performed to provide a comprehensive understanding on this novel waveguide amplifier. This work will benefit the development of a powerful gain platform and can pave the way for a fully integrated photonic system on LNOI platform.

physics.optics↗

arXiv2021

Rethinking and Improving Relative Position Encoding for Vision Transformer

Relative position encoding (RPE) is important for transformer to capture sequence ordering of input tokens. General efficacy has been proven in natural language processing. However, in computer vision, its efficacy is not well studied and even remains controversial, e.g., whether relative position encoding can work equally well as absolute position? In order to clarify this, we first review existing relative position encoding methods and analyze their pros and cons when applied in vision transformers. We then propose new relative position encoding methods dedicated to 2D images, called image RPE (iRPE). Our methods consider directional relative distance modeling as well as the interactions between queries and relative position embeddings in self-attention mechanism. The proposed iRPE methods are simple and lightweight. They can be easily plugged into transformer blocks. Experiments demonstrate that solely due to the proposed encoding methods, DeiT and DETR obtain up to 1.5% (top-1 Acc) and 1.3% (mAP) stable improvements over their original versions on ImageNet and COCO respectively, without tuning any extra hyperparameters such as learning rate and weight decay. Our ablation and analysis also yield interesting findings, some of which run counter to previous understanding. Code and models are open-sourced at https://github.com/microsoft/Cream/tree/main/iRPE.

cs.CV↗

arXiv2021

A single-frequency single-resonator laser on erbium-doped lithium niobate on insulator

Erbium-doped lithium niobate on insulator (Er:LNOI) is a promising platform for photonic integrated circuits as it adds gain to the LNOI system and enables on-chip lasers and amplifiers. A challenge for Er:LNOI laser is to increase its output power while maintaining single-frequency and single (-transverse)-mode operation. In this work, we demonstrate that single-frequency and single-mode operation can be achieved even in a single multi-mode Er:LNOI microring by introducing mode-dependent loss and gain competition. In a single microring with a free spectral range of 192 GHz, we have achieved single-mode lasing with an output power of 2.1 microwatt, a side-mode suppression of 35.5 dB, and a linewidth of 1.27 MHz.

physics.optics↗

arXiv2021

A blind zone-suppressed hybrid beam steering for solid-state Lidar

We demonstrate a blind zone-suppressed and flash-emitting solid-state Lidar based on lens-assisted beam steering (LABS) technology. As a proof-of-concept demonstration, with a design of subwavelength-gap one-dimensional (1D) long-emitter array and multi-wavelength flash beam emitting, the device was measured to have 5%-blind zone suppression, 0.06°/point-deflection step and 4.2 microsecond-scanning speed. In time-of-flight (TOF) ranging experiments, Lidar systems have field of view of 11.3°* 8.1° (normal device) or 0.9°*8.1° (blind-zone suppressed device), far-field number of resolved points of 192 and a detection distance of 10 m. This work demonstrates the possibility that a new integrated beam-steering technology can be implemented in a Lidar without sacrificing other performance.

physics.app-ph↗

arXiv2021

Single-frequency integrated laser on erbium-doped lithium niobate on insulator

The erbium-doped Lithium niobate on insulator (Er:LNOI) platform has great promise in the application of telecommunication, microwave photonics, and quantum photonics due to its excellent electro-optic, piezo-electric, nonlinear nature as well as the gain characteristics in the telecommunication C-band. Here, we report a single-frequency Er:LNOI integrated laser based on dual-cavity structure. Facilitated by the Vernier effect and gain competition, the single-frequency laser can operate stably at 1531-nm wavelength with a 1484-nm pump laser. The output laser has a power of 0.31 uW, a linewidth of 1.2 MHz, and a side mode suppression ratio (SMSR) of 31 dB. Our work allows the direct integration of this laser source with existing LNOI components and paves the way for a fully integrated LNOI system.

physics.optics↗

arXiv2021

LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search

Object tracking has achieved significant progress over the past few years. However, state-of-the-art trackers become increasingly heavy and expensive, which limits their deployments in resource-constrained applications. In this work, we present LightTrack, which uses neural architecture search (NAS) to design more lightweight and efficient object trackers. Comprehensive experiments show that our LightTrack is effective. It can find trackers that achieve superior performance compared to handcrafted SOTA trackers, such as SiamRPN++ and Ocean, while using much fewer model Flops and parameters. Moreover, when deployed on resource-constrained mobile chipsets, the discovered trackers run much faster. For example, on Snapdragon 845 Adreno GPU, LightTrack runs $12\times$ faster than Ocean, while using $13\times$ fewer parameters and $38\times$ fewer Flops. Such improvements might narrow the gap between academic models and industrial deployments in object tracking task. LightTrack is released at https://github.com/researchmm/LightTrack.

cs.CV↗

arXiv2021

Releasing Locks As Early As You Can: Reducing Contention of Hotspots by Violating Two-Phase Locking (Extended Version)

Hotspots, a small set of tuples frequently read/written by a large number of transactions, cause contention in a concurrency control protocol. While a hotspot may comprise only a small fraction of a transaction's execution time, conventional strict two-phase locking allows a transaction to release lock only after the transaction completes, which leaves significant parallelism unexploited. Ideally, a concurrency control protocol serializes transactions only for the duration of the hotspots, rather than the duration of transactions. We observe that exploiting such parallelism requires violating two-phase locking. In this paper, we propose Bamboo, a new concurrency control protocol that can enable such parallelism by modifying the conventional two-phase locking, while maintaining the same guarantees in correctness. We thoroughly analyzed the effect of cascading aborts involved in reading uncommitted data and discussed optimizations that can be applied to further improve the performance. Our evaluation on TPC-C shows a performance improvement up to 4x compared to the best of pessimistic and optimistic baseline protocols. On synthetic workloads that contain a single hotspot, Bamboo achieves a speedup up to 19x over baselines.

cs.DB↗

arXiv2019

Harvesting Visual Objects from Internet Images via Deep Learning Based Objectness Assessment

The collection of internet images has been growing in an astonishing speed. It is undoubted that these images contain rich visual information that can be useful in many applications, such as visual media creation and data-driven image synthesis. In this paper, we focus on the methodologies for building a visual object database from a collection of internet images. Such database is built to contain a large number of high-quality visual objects that can help with various data-driven image applications. Our method is based on dense proposal generation and objectness-based re-ranking. A novel deep convolutional neural network is designed for the inference of proposal objectness, the probability of a proposal containing optimally-located foreground object. In our work, the objectness is quantitatively measured in regard of completeness and fullness, reflecting two complementary features of an optimal proposal: a complete foreground and relatively small background. Our experiments indicate that object proposals re-ranked according to the output of our network generally achieve higher performance than those produced by other state-of-the-art methods. As a concrete example, a database of over 1.2 million visual objects has been built using the proposed method, and has been successfully used in various data-driven image applications.

cs.CV↗