arXiv subjects

Bo Zhao

Publications and source records attributed to Bo Zhao.

At least 217 records · Page 12Linked to original sources

arXiv2023

Everyone's Preference Changes Differently: Weighted Multi-Interest Retrieval Model

User embeddings (vectorized representations of a user) are essential in recommendation systems. Numerous approaches have been proposed to construct a representation for the user in order to find similar items for retrieval tasks, and they have been proven effective in industrial recommendation systems as well. Recently people have discovered the power of using multiple embeddings to represent a user, with the hope that each embedding represents the user's interest in a certain topic. With multi-interest representation, it's important to model the user's preference over the different topics and how the preference change with time. However, existing approaches either fail to estimate the user's affinity to each interest or unreasonably assume every interest of every user fades with an equal rate with time, thus hurting the recall of candidate retrieval. In this paper, we propose the Multi-Interest Preference (MIP) model, an approach that not only produces multi-interest for users by using the user's sequential engagement more effectively but also automatically learns a set of weights to represent the preference over each embedding so that the candidates can be retrieved from each interest proportionally. Extensive experiments have been done on various industrial-scale datasets to demonstrate the effectiveness of our approach.

cs.IR↗

arXiv2023

Accelerated MR Fingerprinting with Low-Rank and Generative Subspace Modeling

Magnetic Resonance (MR) Fingerprinting is an emerging multi-parametric quantitative MR imaging technique, for which image reconstruction methods utilizing low-rank and subspace constraints have achieved state-of-the-art performance. However, this class of methods often suffers from an ill-conditioned model-fitting issue, which degrades the performance as the data acquisition lengths become short and/or the signal-to-noise ratio becomes low. To address this problem, we present a new image reconstruction method for MR Fingerprinting, integrating low-rank and subspace modeling with a deep generative prior. Specifically, the proposed method captures the strong spatiotemporal correlation of contrast-weighted time-series images in MR Fingerprinting via a low-rank factorization. Further, it utilizes an untrained convolutional generative neural network to represent the spatial subspace of the low-rank model, while estimating the temporal subspace of the model from simulated magnetization evolutions generated based on spin physics. Here the architecture of the generative neural network serves as an effective regularizer for the ill-conditioned inverse problem without additional spatial training data that are often expensive to acquire. The proposed formulation results in a non-convex optimization problem, for which we develop an algorithm based on variable splitting and alternating direction method of multipliers.We evaluate the performance of the proposed method with numerical simulations and in vivo experiments and demonstrate that the proposed method outperforms the state-of-the-art low-rank and subspace reconstruction.

eess.IV↗

arXiv2023

Toward Moiré-Free and Detail-Preserving Demosaicking

3D convolutions are commonly employed by demosaicking neural models, in the same way as solving other image restoration problems. Counter-intuitively, we show that 3D convolutions implicitly impede the RGB color spectra from exchanging complementary information, resulting in spectral-inconsistent inference of the local spatial high frequency components. As a consequence, shallow 3D convolution networks suffer the Moiré artifacts, but deep 3D convolutions cause over-smoothness. We analyze the fundamental difference between demosaicking and other problems that predict lost pixels between available ones (e.g., super-resolution reconstruction), and present the underlying reasons for the confliction between Moiré-free and detail-preserving. From the new perspective, our work decouples the common standard convolution procedure to spectral and spatial feature aggregations, which allow strengthening global communication in the spectral dimension while respecting local contrast in the spatial dimension. We apply our demosaicking model to two tasks: Joint Demosaicking-Denoising and Independently Demosaicking. In both applications, our model substantially alleviates artifacts such as Moiré and over-smoothness at similar or lower computational cost to currently top-performing models, as validated by diverse evaluations. Source code will be released along with paper publication.

cs.CV↗

arXiv2023

Accelerating Dataset Distillation via Model Augmentation

Dataset Distillation (DD), a newly emerging field, aims at generating much smaller but efficient synthetic training datasets from large ones. Existing DD methods based on gradient matching achieve leading performance; however, they are extremely computationally intensive as they require continuously optimizing a dataset among thousands of randomly initialized models. In this paper, we assume that training the synthetic data with diverse models leads to better generalization performance. Thus we propose two model augmentation techniques, i.e. using early-stage models and parameter perturbation to learn an informative synthetic set with significantly reduced training cost. Extensive experiments demonstrate that our method achieves up to 20x speedup and comparable performance on par with state-of-the-art methods.

cs.LG↗

arXiv2023

Symmetries, flat minima, and the conserved quantities of gradient flow

Empirical studies of the loss landscape of deep networks have revealed that many local minima are connected through low-loss valleys. Yet, little is known about the theoretical origin of such valleys. We present a general framework for finding continuous symmetries in the parameter space, which carve out low-loss valleys. Our framework uses equivariances of the activation functions and can be applied to different layer architectures. To generalize this framework to nonlinear neural networks, we introduce a novel set of nonlinear, data-dependent symmetries. These symmetries can transform a trained model such that it performs similarly on new samples, which allows ensemble building that improves robustness under certain adversarial attacks. We then show that conserved quantities associated with linear symmetries can be used to define coordinates along low-loss valleys. The conserved quantities help reveal that using common initialization methods, gradient flow only explores a small part of the global minimum. By relating conserved quantities to convergence rate and sharpness of the minimum, we provide insights on how initialization impacts convergence and generalizability.

cs.LG↗

arXiv2023

Controlled longitudinal spin-orbit separation of complex vector modes

Complex vector modes, entangled in spin and orbital angular momentum, are opening burgeoning opportunities for a wide variety of applications. Importantly, the flexible manipulation the various properties of such beams will pave the way to novel applications. As such, in this manuscript, we demonstrate a longitudinal spin-orbit separation of complex vector modes propagating in free space. To achieve this we employed the recently demonstrated circular Airy Gaussian vortex vector (CAGVV) modes, which feature a self-focusing property. More precisely, by properly manipulating the intrinsic parameters of CAGVV modes, the strong coupling between the two constituting orthogonal components of CAGVV mode undergo a spin-orbit separation along the propagation direction namely, while one polarisation component, focuses at a specific plane, the other focuses at a different plane. Such spin-orbit separation, which we demonstrated by numerical simulations and corroborated experimentally, can be adjusted on-demand by simply changing the initial parameters of CAGVV modes. Our findings will be of great relevance, for example in optical tweezers, to manipulate micro- or nano-particles at two different parallel planes.

physics.optics↗

arXiv2023

Tuning Optical Properties of Metamaterials by Mie Scattering for Efficient Sub-ambient Daytime Radiative Cooling

The management of the abundant eggshell biowaste produced worldwide has become a problematic issue due to the generated odor and microorganisms after directly disposing eggshell biowaste in landfills. Herein, we propose a novel method to convert the hazardous eggshell biowaste to valuable resources for energy management applications. Eggshell-based films are fabricated by embedding eggshell powders into polymer matrix to achieve highly efficient sub-ambient daytime radiative cooling. Benefiting from the Mie scattering of eggshell particles/air pores in the solar spectrum and strong emission of eggshell in the mid-infrared (mid-IR) range, the eggshell-based films present high reflection of 0.96 in the solar spectrum and high emission of 0.95 in the mid-IR range, resulting in significant average temperature drop of 5°C and 12°C below the ambient temperature during daytime and nighttime, respectively. Moreover, the eggshell-based films exhibit excellent flexibility and self-cleaning properties, which are beneficial for practical long-term outdoor applications. Our proposed design provides a novel means for an environmentally friendly and sustainable management of the eggshell biowaste.

physics.optics↗

arXiv2023

Symmetry Teleportation for Accelerated Optimization

Existing gradient-based optimization methods update parameters locally, in a direction that minimizes the loss function. We study a different approach, symmetry teleportation, that allows parameters to travel a large distance on the loss level set, in order to improve the convergence speed in subsequent steps. Teleportation exploits symmetries in the loss landscape of optimization problems. We derive loss-invariant group actions for test functions in optimization and multi-layer neural networks, and prove a necessary condition for teleportation to improve convergence rate. We also show that our algorithm is closely related to second order methods. Experimentally, we show that teleportation improves the convergence speed of gradient descent and AdaGrad for several optimization problems including test functions, multi-layer regressions, and MNIST classification.

cs.LG↗

arXiv2022

Dataset Condensation with Distribution Matching

Computational cost of training state-of-the-art deep models in many learning problems is rapidly increasing due to more sophisticated models and larger datasets. A recent promising direction for reducing training cost is dataset condensation that aims to replace the original large training set with a significantly smaller learned synthetic set while preserving the original information. While training deep models on the small set of condensed images can be extremely fast, their synthesis remains computationally expensive due to the complex bi-level optimization and second-order derivative computation. In this work, we propose a simple yet effective method that synthesizes condensed images by matching feature distributions of the synthetic and original training images in many sampled embedding spaces. Our method significantly reduces the synthesis cost while achieving comparable or better performance. Thanks to its efficiency, we apply our method to more realistic and larger datasets with sophisticated neural architectures and obtain a significant performance boost. We also show promising practical benefits of our method in continual learning and neural architecture search.

cs.LG↗

arXiv2022

Synthesizing Informative Training Samples with GAN

Remarkable progress has been achieved in synthesizing photo-realistic images with generative adversarial networks (GANs). Recently, GANs are utilized as the training sample generator when obtaining or storing real training data is expensive even infeasible. However, traditional GANs generated images are not as informative as the real training samples when being used to train deep neural networks. In this paper, we propose a novel method to synthesize Informative Training samples with GAN (IT-GAN). Specifically, we freeze a pre-trained GAN model and learn the informative latent vectors that correspond to informative training samples. The synthesized images are required to preserve information for training deep neural networks rather than visual reality or fidelity. Experiments verify that the deep neural networks can learn faster and achieve better performance when being trained with our IT-GAN generated images. We also show that our method is a promising solution to dataset condensation problem.

cs.LG↗

arXiv2022

Models of Rotating Infall for the B335 Protostar

Models of the protostellar source, B335, are developed using axisymmetric three-dimensional models to resolve conflicts found in one-dimensional models. The models are constrained by a large number of observations, including ALMA, Herschel, and Spitzer data. Observations of the protostellar source B335 with ALMA show red-shifted absorption against a central continuum source indicative of infall in the HCO$^+$ and HCN $J = 4\rightarrow 3$ transitions. The data are combined with a new estimate of the distance to provide strong constraints to three-dimensional radiative transfer models including a rotating, infalling envelope, outflow cavities, and a very small disk. The models favor ages since the initiation of collapse between $3 \times 10^4$ and $4 \times 10^4$ yr for both the continuum and the lines, resolving a conflict found in one-dimensional models. The models under-predict the continuum emission seen by ALMA, suggesting an additional component such as a pseudo-disk. The best-fitting model is used to convert variations in the 4.5 $μm$ flux in recent years into a model for a variation of a factor of 5-7 in luminosity over the last 8 years.

astro-ph.GA↗

arXiv2022

Increasing mass-to-flux ratio from the dense core to the protostellar envelope around the Class 0 protostar HH 211

To study transportation of magnetic flux from large to small scales in protostellar sources, we analyzed the Nobeyama 45-m N2H+ (1-0), JCMT 850 um polarization, and ALMA C18O (2-1) and 1.3 mm and 0.8 mm (polarized) continuum data of the Class 0 protostar HH 211. The magnetic field strength in the dense core on a 0.1 pc scale was estimated with the single-dish line and polarization data using the Davis-Chandrasekhar-Fermi method, and that in the protostellar envelope on a 600 au scale was estimated from the force balance between the gravity and magnetic field tension by analyzing the gas kinematics and magnetic field structures with the ALMA data. Our analysis suggests that from 0.1 pc to 600 au scales, the magnetic field strength increases from 40-107 uG to 0.3-1.2 mG with a scaling relation between the magnetic field strength and density of $B \propto ρ^{0.36\pm0.08}$, and the mass-to-flux ratio increases from 1.2-3.7 to 9.1-32.3. The increase in the mass-to-flux ratio could suggest that the magnetic field is partially decoupled from the neutral matter between 0.1 pc and 600 au scales, and hint at efficient ambipolar diffusion in the infalling protostellar envelope in HH 211, which is the dominant non-ideal magnetohydrodynamic effect considering the density on these scales. Thus, our results could support the scenario of efficient ambipolar diffusion enabling the formation of the 20 au Keplerian disk in HH 211.

astro-ph.SR↗

arXiv2022

MSRL: Distributed Reinforcement Learning with Dataflow Fragments

Reinforcement learning (RL) trains many agents, which is resource-intensive and must scale to large GPU clusters. Different RL training algorithms offer different opportunities for distributing and parallelising the computation. Yet, current distributed RL systems tie the definition of RL algorithms to their distributed execution: they hard-code particular distribution strategies and only accelerate specific parts of the computation (e.g. policy network updates) on GPU workers. Fundamentally, current systems lack abstractions that decouple RL algorithms from their execution. We describe MindSpore Reinforcement Learning (MSRL), a distributed RL training system that supports distribution policies that govern how RL training computation is parallelised and distributed on cluster resources, without requiring changes to the algorithm implementation. MSRL introduces the new abstraction of a fragmented dataflow graph, which maps Python functions from an RL algorithm's training loop to parallel computational fragments. Fragments are executed on different devices by translating them to low-level dataflow representations, e.g. computational graphs as supported by deep learning engines, CUDA implementations or multi-threaded CPU processes. We show that MSRL subsumes the distribution strategies of existing systems, while scaling RL training to 64 GPUs.

cs.LG↗

arXiv2022

Nowhere to Hide: A Lightweight Unsupervised Detector against Adversarial Examples

Although deep neural networks (DNNs) have shown impressive performance on many perceptual tasks, they are vulnerable to adversarial examples that are generated by adding slight but maliciously crafted perturbations to benign images. Adversarial detection is an important technique for identifying adversarial examples before they are entered into target DNNs. Previous studies to detect adversarial examples either targeted specific attacks or required expensive computation. How design a lightweight unsupervised detector is still a challenging problem. In this paper, we propose an AutoEncoder-based Adversarial Examples (AEAE) detector, that can guard DNN models by detecting adversarial examples with low computation in an unsupervised manner. The AEAE includes only a shallow autoencoder but plays two roles. First, a well-trained autoencoder has learned the manifold of benign examples. This autoencoder can produce a large reconstruction error for adversarial images with large perturbations, so we can detect significantly perturbed adversarial examples based on the reconstruction error. Second, the autoencoder can filter out the small noise and change the DNN's prediction on adversarial examples with small perturbations. It helps to detect slightly perturbed adversarial examples based on the prediction distance. To cover these two cases, we utilize the reconstruction error and prediction distance from benign images to construct a two-tuple feature set and train an adversarial detector using the isolation forest algorithm. We show empirically that the AEAE is unsupervised and inexpensive against the most state-of-the-art attacks. Through the detection in these two cases, there is nowhere to hide adversarial examples.

cs.LG↗

arXiv2022

The role of magnetic fields in the formation of protostars, disks, and outflows

We present our current understanding of the formation and early evolution of protostars, protoplanetary disks, and the driving of outflows as dictated by the interplay of magnetic fields and partially ionized gas in molecular cloud cores. In recent years, the field has witnessed enormous development through sub-millimeter observations which in turn have constrained models of protostar formation. As a result of these observations % that the observations provided, the state-of-the-art theoretical understanding of the formation and evolution of young stellar objects is described. In particular, we emphasize the importance of the coupling, decoupling, and re-coupling between weakly ionized gas and the magnetic field on appropriate scales. This highlights the complex and intimate relationship between gravitational collapse and magnetic fields in young protostars.

astro-ph.SR↗

arXiv2022

Light control with Weyl semimetals

Weyl semimetals are topological materials whose electron quasiparticles obey the Weyl equation. They possess many unusual properties that may lead to new applications. This is a tutorial review of the optical properties and applications of Weyl semimetals. We review the basic concepts and optical responses of Weyl semimetals, and survey their applications in optics and thermal photonics. We hope this pedagogical text will motivate further research on this emerging topic.

physics.optics↗

arXiv2022

Generation of arbitrary abruptly autofusing circular Airy Gaussian vortex vector modes

Complex vector modes represent a general state of light nonseparable in their spatial and polarization degrees of freedom, which have inspired a wide variety of novel applications and phenomena, such as their unexpected propagation behaviour. For example, they can propagate describing periodic polarization transitions, changing from one vector beam to another. Here, we put forward a novel class of vector modes with the capability to experience an abruptly autofocusing behaviour. To achieve such beams, we encode the spatial degree of freedom in the Circular Airy Gaussian vortex (CAGV) beams. We demonstrate the experimental generation of arbitrary CAGV vector beams and evince some of their properties, such as a rotation of intermodal phase. We anticipate that the fascinating properties of theses modes will prompt the development of novel applications associated to their autofocusing behaviour and polarization distribution.

physics.optics↗

arXiv2022

Preparation of a quantum degenerate mixture of $^{23}$Na$^{40}$K molecules and $^{40}$K atoms

We report on the preparation of a quantum degenerate mixture of $^{23}$Na$^{40}$K molecules and $^{40}$K atoms. A deeply degenerate atomic mixture of $^{23}$Na and $^{40}$K atoms with a large number ratio ($N_F/N_B\approx 6$) is prepared by the mode-matching loading of atoms from a cloverleaf-type magnetic trap into a large-volume horizontal optical dipole trap and evaporative cooling in a large-volume three-beam optical dipole trap. About $3.0\times10^4$ $^{23}$Na$^{40}$K ground-state molecules are created through magneto-association followed by stimulated adiabatic Raman passage. The 2D density distribution of the molecules is fit to the Fermi-Dirac distribution with $T/T_F\approx0.4-0.5$. In the atom-molecule mixture, the elastic collisions provide a thermalization mechanism for the molecules. In a few tens of milliseconds which are larger than the typical thermalization time, the degeneracy of the molecules is maintained, which may be due to the Pauli exclusion principle. The quantum degenerate mixture of $^{23}$Na$^{40}$K molecules and $^{40}$K atoms can be used to study strongly interacting atom-molecule mixtures and to prepare ultracold triatomic molecular gases.

physics.atom-ph↗