arXiv subjects

Ji Lin

Publications and source records attributed to Ji Lin.

At least 37 records · Page 2Linked to original sources

arXiv2023

Magnetic lump motion in saturated ferromagnetic films

In this paper, we study in detail the nonlinear propagation of magnetic soliton in a ferromagnetic film. The sample is magnetized to saturation by an external field perpendicular to film plane. A new generalized (2+1)-dimensional short-wave asymptotic model is derived. The bilinear-like forms of this equation are constructed, and exact magnetic line soliton solutions are exhibited. It is observed that a series of stable lumps can be generated by an unstable magnetic soliton under Gaussian disturbance. Such magnetic lumps are highly stable and can maintain their shapes and velocities during evolution or collision. The interaction between lump and magnetic soliton, as well as interaction between two lumps, are numerically investigated. We further discuss the nonlinear motion of lumps in ferrites with Gilbert-damping and inhomogeneous exchange effects. The results show that the Gilbert-damping effects make the amplitude and velocity of the magnetic lump decay exponentially during propagation. And the shock waves are generated from a lump when quenching the strength of inhomogeneous exchange.

nlin.PS↗

arXiv2023

Semi-derived Ringel-Hall algebras and Hall algebras of odd-periodic relative derived categories

Let $t$ be a positive integer and $\mathcal{A}$ a hereditary abelian category satisfying some finiteness conditions. We define the semi-derived Ringel-Hall algebra of $\mathcal{A}$ from the category $\mathcal{C}_{\mathbb{Z}/t}(\mathcal{A})$ of $\mathbb{Z}/t$-graded complexes and obtain a natural basis of the semi-derived Ringel-Hall algebra. Moreover, we describe the semi-derived Ringel-Hall algebra by the generators and defining relations. In particular, if $t$ is an odd integer, we show that there is an embedding of derived Hall algebra of the odd-periodic relative derived category in the extended semi-derived Ringel-Hall algebra.

math.RT↗

arXiv2023

A Multi-Arm Two-Stage (MATS) Design for Proof-of-Concept and Dose Optimization in Early-Phase Oncology Trials

The Project Optimus initiative by the FDA's Oncology Center of Excellence is widely viewed as a groundbreaking effort to change the $\textit{status quo}$ of conventional dose-finding strategies in oncology. Unlike in other therapeutic areas where multiple doses are evaluated thoroughly in dose ranging studies, early-phase oncology dose-finding studies are characterized by the practice of identifying a single dose, such as the maximum tolerated dose (MTD) or the recommended phase 2 dose (RP2D). Following the spirit of Project Optimus, we propose an Multi-Arm Two-Stage (MATS) design for proof-of-concept (PoC) and dose optimization that allows the evaluation of two selected doses from a dose-escalation trial. The design assess the higher dose first across multiple indications in the first stage, and adaptively enters the second stage for an indication if the higher dose exhibits promising anti-tumor activities. In the second stage, a randomized comparison between the higher and lower doses is conducted to achieve proof-of-concept (PoC) and dose optimization. A Bayesian hierarchical model governs the statistical inference and decision making by borrowing information across doses, indications, and stages. Our simulation studies show that the proposed MATS design yield desirable performance. An R Shiny application has been developed and made available at https://matsdesign.shinyapps.io/mats/.

stat.AP↗

arXiv2023

Quantum Borcherds-Bozec algebras via semi-derived Ringel-Hall algebras II: braid group actions

Based on the realization of quantum Borcherds-Bozec algebra $\widetilde{\mathbf{U}}$ and quantum generalized Kac-Moody algebra ${}^B\widetilde{\mathbf{U}}$ via semi-derived Ringel-Hall algebra of a quiver with loops, we deduce the braid group actions of $\widetilde{\mathbf{U}}$ introduced by Fan and Tong recently and establish braid group actions for ${}^B\widetilde{\mathbf{U}}$ by applying the BGP reflection functors to semi-derived Ringel-Hall algebras.

math.RT↗

arXiv2023

Integration of Efficacy Biomarkers Together with Toxicity Endpoints in Immune-Oncology Dose Finding Studies

The primary objective of phase I oncology studies is to establish the safety profile of a new treatment and determine the maximum tolerated dose (MTD). This is motivated by the development of cytotoxic agents based on the underlying assumption that the higher the dose, the greater the likelihood of efficacy and toxicity. However, evidence from the recent development of cancer immunotherapies that aim to stimulate patients' immune systems to fight cancer challenges this assumption, particularly further escalation after a certain dose level might not necessarily increase the efficacy. Dose escalation study of molecular targeted agents (MTA) often does not only rely on the safety profile. In this paper, we propose a simple and flexible model that uses multivariate Gaussian latent variables to integrate toxicity endpoint and efficacy biomarker. This model can be easily extended to incorporate additional immune biomarkers. By simultaneously considering multiple outcomes, the proposed method is better at identifying the biologically optimal dose, which results in better decision-making. Simulation studies showed that the proposed method has desirable operating characteristics by determining the target dose with an optimal risk-benefit trade-off. We have also implemented our proposed method in a user-friendly R Shiny tool.

stat.AP↗

arXiv2023

Offsite-Tuning: Transfer Learning without Full Model

Transfer learning is important for foundation models to adapt to downstream tasks. However, many foundation models are proprietary, so users must share their data with model owners to fine-tune the models, which is costly and raise privacy concerns. Moreover, fine-tuning large foundation models is computation-intensive and impractical for most downstream users. In this paper, we propose Offsite-Tuning, a privacy-preserving and efficient transfer learning framework that can adapt billion-parameter foundation models to downstream data without access to the full model. In offsite-tuning, the model owner sends a light-weight adapter and a lossy compressed emulator to the data owner, who then fine-tunes the adapter on the downstream data with the emulator's assistance. The fine-tuned adapter is then returned to the model owner, who plugs it into the full model to create an adapted foundation model. Offsite-tuning preserves both parties' privacy and is computationally more efficient than the existing fine-tuning methods that require access to the full model weights. We demonstrate the effectiveness of offsite-tuning on various large language and vision foundation models. Offsite-tuning can achieve comparable accuracy as full model fine-tuning while being privacy-preserving and efficient, achieving 6.5x speedup and 5.6x memory reduction. Code is available at https://github.com/mit-han-lab/offsite-tuning.

cs.CL↗

arXiv2022

Breakdown of the correspondence between the real-complex and delocalization-localization transitions in non-Hermitian quasicrystals

The correspondence between the real-complex transition in energy and delocalization-localization transition is well-established in a class of Aubry-Andr'e-Harper model with exponential non-Hermitian on-site potentials. In this paper, we study a generalized Aubry-Andr'e model with off-diagonal modulation and non-Hermitian on-site potential. We find that, when there exists an incommensurate off-diagonal modulation, the correspondence breaks down, although the extended phase is maintained in a wide parameter range of the strengths of the on-site potential and the off-diagonal hoppings. An additional intermediate phase with a non-Hermitian mobility edge emerges when the off-diagonal hoppings become commensurate. This phase is characterized by the real and complex sections of the energy spectrum corresponding to the extended and localized states. In this case, the aforementioned correspondence reappears due to the recovery of the PT-symmetry.

cond-mat.dis-nn↗

arXiv2022

Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications

Deep neural networks (DNNs) have achieved unprecedented success in the field of artificial intelligence (AI), including computer vision, natural language processing and speech recognition. However, their superior performance comes at the considerable cost of computational complexity, which greatly hinders their applications in many resource-constrained devices, such as mobile phones and Internet of Things (IoT) devices. Therefore, methods and techniques that are able to lift the efficiency bottleneck while preserving the high accuracy of DNNs are in great demand in order to enable numerous edge AI applications. This paper provides an overview of efficient deep learning methods, systems and applications. We start from introducing popular model compression methods, including pruning, factorization, quantization as well as compact model design. To reduce the large design cost of these manual solutions, we discuss the AutoML framework for each of them, such as neural architecture search (NAS) and automated pruning and quantization. We then cover efficient on-device training to enable user customization based on the local data on mobile devices. Apart from general acceleration techniques, we also showcase several task-specific accelerations for point cloud, video and natural language processing by exploiting their spatial sparsity and temporal/token redundancy. Finally, to support all these algorithmic advancements, we introduce the efficient deep learning system design from both software and hardware perspectives.

cs.LG↗

arXiv2022

Network Augmentation for Tiny Deep Learning

We introduce Network Augmentation (NetAug), a new training method for improving the performance of tiny neural networks. Existing regularization techniques (e.g., data augmentation, dropout) have shown much success on large neural networks by adding noise to overcome over-fitting. However, we found these techniques hurt the performance of tiny neural networks. We argue that training tiny models are different from large models: rather than augmenting the data, we should augment the model, since tiny models tend to suffer from under-fitting rather than over-fitting due to limited capacity. To alleviate this issue, NetAug augments the network (reverse dropout) instead of inserting noise into the dataset or the network. It puts the tiny model into larger models and encourages it to work as a sub-model of larger models to get extra supervision, in addition to functioning as an independent model. At test time, only the tiny model is used for inference, incurring zero inference overhead. We demonstrate the effectiveness of NetAug on image classification and object detection. NetAug consistently improves the performance of tiny models, achieving up to 2.2% accuracy improvement on ImageNet. On object detection, achieving the same level of performance, NetAug requires 41% fewer MACs on Pascal VOC and 38% fewer MACs on COCO than the baseline.

cs.CV↗

arXiv2021

GAN Compression: Efficient Architectures for Interactive Conditional GANs

Conditional Generative Adversarial Networks (cGANs) have enabled controllable image synthesis for many vision and graphics applications. However, recent cGANs are 1-2 orders of magnitude more compute-intensive than modern recognition CNNs. For example, GauGAN consumes 281G MACs per image, compared to 0.44G MACs for MobileNet-v3, making it difficult for interactive deployment. In this work, we propose a general-purpose compression framework for reducing the inference time and model size of the generator in cGANs. Directly applying existing compression methods yields poor performance due to the difficulty of GAN training and the differences in generator architectures. We address these challenges in two ways. First, to stabilize GAN training, we transfer knowledge of multiple intermediate representations of the original model to its compressed model and unify unpaired and paired learning. Second, instead of reusing existing CNN designs, our method finds efficient architectures via neural architecture search. To accelerate the search process, we decouple the model training and search via weight sharing. Experiments demonstrate the effectiveness of our method across different supervision settings, network architectures, and learning methods. Without losing image quality, we reduce the computation of CycleGAN by 21x, Pix2pix by 12x, MUNIT by 29x, and GauGAN by 9x, paving the way for interactive image synthesis.

cs.CV↗

arXiv2021

TSM: Temporal Shift Module for Efficient and Scalable Video Understanding on Edge Device

The explosive growth in video streaming requires video understanding at high accuracy and low computation cost. Conventional 2D CNNs are computationally cheap but cannot capture temporal relationships; 3D CNN-based methods can achieve good performance but are computationally intensive. In this paper, we propose a generic and effective Temporal Shift Module (TSM) that enjoys both high efficiency and high performance. The key idea of TSM is to shift part of the channels along the temporal dimension, thus facilitate information exchanged among neighboring frames. It can be inserted into 2D CNNs to achieve temporal modeling at zero computation and zero parameters. TSM offers several unique advantages. Firstly, TSM has high performance; it ranks the first on the Something-Something leaderboard upon submission. Secondly, TSM has high efficiency; it achieves a high frame rate of 74fps and 29fps for online video recognition on Jetson Nano and Galaxy Note8. Thirdly, TSM has higher scalability compared to 3D networks, enabling large-scale Kinetics training on 1,536 GPUs in 15 minutes. Lastly, TSM enables action concepts learning, which 2D networks cannot model; we visualize the category attention map and find that spatial-temporal action detector emerges during the training of classification tasks. The code is publicly available at https://github.com/mit-han-lab/temporal-shift-module.

cs.CV↗

arXiv2021

Cyclotron dynamics of a Bose-Einstein condensate in a quadruple-well potential with synthetic gauge fields

We investigate the cyclotron dynamics of Bose-Einstein condensate (BEC) in a quadruple-well potential with synthetic gauge fields. We use laser-assisted tunneling to generate large tunable effective magnetic fields for BEC. The mean position of BEC follows an orbit that simulated the cyclotron orbits of charged particles in a magnetic field. In the absence of atomic interaction, atom dynamics may exhibit periodic or quasi-periodic cyclotron orbits. In the presence of atomic interaction, the system may exhibit self-trapping, which depends on synthetic gauge fields and atomic interaction strength. In particular, the competition between synthetic gauge fields and atomic interaction leads to the generation of several discontinuous parameter windows for the transition to self-trapping, which is obviously different from that without synthetic gauge fields.

quant-ph↗

arXiv2021

PoD-BIN: A Probability of Decision Bayesian Interval Design for Time-to-Event Dose-Finding Trials with Multiple Toxicity Grades

We consider a Bayesian framework based on "probability of decision" for dose-finding trial designs. The proposed PoD-BIN design evaluates the posterior predictive probabilities of up-and-down decisions. In PoD-BIN, multiple grades of toxicity, categorized as the mild toxicity (MT) and dose-limiting toxicity (DLT), are modeled simultaneously, and the primary outcome of interests is time-to-toxicity for both MT and DLT. This allows the possibility of enrolling new patients when previously enrolled patients are still being followed for toxicity, thus potentially shortening trial length. The Bayesian decision rules in PoD-BIN utilize the probability of decisions to balance the need to speed up the trial and the risk of exposing patients to overly toxic doses. We demonstrate via numerical examples the resulting balance of speed and safety of PoD-BIN and compare to existing designs.

stat.ME↗

arXiv2021

Anycost GANs for Interactive Image Synthesis and Editing

Generative adversarial networks (GANs) have enabled photorealistic image synthesis and editing. However, due to the high computational cost of large-scale generators (e.g., StyleGAN2), it usually takes seconds to see the results of a single edit on edge devices, prohibiting interactive user experience. In this paper, we take inspirations from modern rendering software and propose Anycost GAN for interactive natural image editing. We train the Anycost GAN to support elastic resolutions and channels for faster image generation at versatile speeds. Running subsets of the full generator produce outputs that are perceptually similar to the full generator, making them a good proxy for preview. By using sampling-based multi-resolution training, adaptive-channel training, and a generator-conditioned discriminator, the anycost generator can be evaluated at various configurations while achieving better image quality compared to separately trained models. Furthermore, we develop new encoder training and latent code optimization techniques to encourage consistency between the different sub-generators during image projection. Anycost GAN can be executed at various cost budgets (up to 10x computation reduction) and adapt to a wide range of hardware and latency requirements. When deployed on desktop CPUs and edge devices, our model can provide perceptually similar previews at 6-12x speedup, enabling interactive image editing. The code and demo are publicly available: https://github.com/mit-han-lab/anycost-gan.

cs.CV↗

arXiv2020

Differentiable Augmentation for Data-Efficient GAN Training

The performance of generative adversarial networks (GANs) heavily deteriorates given a limited amount of training data. This is mainly because the discriminator is memorizing the exact training set. To combat it, we propose Differentiable Augmentation (DiffAugment), a simple method that improves the data efficiency of GANs by imposing various types of differentiable augmentations on both real and fake samples. Previous attempts to directly augment the training data manipulate the distribution of real images, yielding little benefit; DiffAugment enables us to adopt the differentiable augmentation for the generated samples, effectively stabilizes training, and leads to better convergence. Experiments demonstrate consistent gains of our method over a variety of GAN architectures and loss functions for both unconditional and class-conditional generation. With DiffAugment, we achieve a state-of-the-art FID of 6.80 with an IS of 100.8 on ImageNet 128x128 and 2-4x reductions of FID given 1,000 images on FFHQ and LSUN. Furthermore, with only 20% training data, we can match the top performance on CIFAR-10 and CIFAR-100. Finally, our method can generate high-fidelity images using only 100 images without pre-training, while being on par with existing transfer learning algorithms. Code is available at https://github.com/mit-han-lab/data-efficient-gans.

cs.CV↗

arXiv2020

MCUNet: Tiny Deep Learning on IoT Devices

Machine learning on tiny IoT devices based on microcontroller units (MCU) is appealing but challenging: the memory of microcontrollers is 2-3 orders of magnitude smaller even than mobile phones. We propose MCUNet, a framework that jointly designs the efficient neural architecture (TinyNAS) and the lightweight inference engine (TinyEngine), enabling ImageNet-scale inference on microcontrollers. TinyNAS adopts a two-stage neural architecture search approach that first optimizes the search space to fit the resource constraints, then specializes the network architecture in the optimized search space. TinyNAS can automatically handle diverse constraints (i.e.device, latency, energy, memory) under low search costs.TinyNAS is co-designed with TinyEngine, a memory-efficient inference library to expand the search space and fit a larger model. TinyEngine adapts the memory scheduling according to the overall network topology rather than layer-wise optimization, reducing the memory usage by 4.8x, and accelerating the inference by 1.7-3.3x compared to TF-Lite Micro and CMSIS-NN. MCUNet is the first to achieves >70% ImageNet top1 accuracy on an off-the-shelf commercial microcontroller, using 3.5x less SRAM and 5.7x less Flash compared to quantized MobileNetV2 and ResNet-18. On visual&audio wake words tasks, MCUNet achieves state-of-the-art accuracy and runs 2.4-3.4x faster than MobileNetV2 and ProxylessNAS-based solutions with 3.7-4.1x smaller peak SRAM. Our study suggests that the era of always-on tiny machine learning on IoT devices has arrived. Code and models can be found here: https://tinyml.mit.edu.

cs.CV↗

arXiv2020

Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution

Self-driving cars need to understand 3D scenes efficiently and accurately in order to drive safely. Given the limited hardware resources, existing 3D perception models are not able to recognize small instances (e.g., pedestrians, cyclists) very well due to the low-resolution voxelization and aggressive downsampling. To this end, we propose Sparse Point-Voxel Convolution (SPVConv), a lightweight 3D module that equips the vanilla Sparse Convolution with the high-resolution point-based branch. With negligible overhead, this point-based branch is able to preserve the fine details even from large outdoor scenes. To explore the spectrum of efficient 3D models, we first define a flexible architecture design space based on SPVConv, and we then present 3D Neural Architecture Search (3D-NAS) to search the optimal network architecture over this diverse design space efficiently and effectively. Experimental results validate that the resulting SPVNAS model is fast and accurate: it outperforms the state-of-the-art MinkowskiNet by 3.3%, ranking 1st on the competitive SemanticKITTI leaderboard. It also achieves 8x computation reduction and 3x measured speedup over MinkowskiNet with higher accuracy. Finally, we transfer our method to 3D object detection, and it achieves consistent improvements over the one-stage detection baseline on KITTI.

cs.CV↗

arXiv2020

Hardware-Centric AutoML for Mixed-Precision Quantization

Model quantization is a widely used technique to compress and accelerate deep neural network (DNN) inference. Emergent DNN hardware accelerators begin to support mixed precision (1-8 bits) to further improve the computation efficiency, which raises a great challenge to find the optimal bitwidth for each layer: it requires domain experts to explore the vast design space trading off among accuracy, latency, energy, and model size, which is both time-consuming and sub-optimal. Conventional quantization algorithm ignores the different hardware architectures and quantizes all the layers in a uniform way. In this paper, we introduce the Hardware-Aware Automated Quantization (HAQ) framework which leverages the reinforcement learning to automatically determine the quantization policy, and we take the hardware accelerator's feedback in the design loop. Rather than relying on proxy signals such as FLOPs and model size, we employ a hardware simulator to generate direct feedback signals (latency and energy) to the RL agent. Compared with conventional methods, our framework is fully automated and can specialize the quantization policy for different neural network architectures and hardware architectures. Our framework effectively reduced the latency by 1.4-1.95x and the energy consumption by 1.9x with negligible loss of accuracy compared with the fixed bitwidth (8 bits) quantization. Our framework reveals that the optimal policies on different hardware architectures (i.e., edge and cloud architectures) under different resource constraints (i.e., latency, energy, and model size) are drastically different. We interpreted the implication of different quantization policies, which offer insights for both neural network architecture design and hardware architecture design.

cs.CV↗