Searcharxiv⌕ Search

arXiv subjects

Yong Luo

Publications and source records attributed to Yong Luo.

At least 127 records · Page 7Linked to original sources
arXiv2022

Unpaired Image Captioning by Image-level Weakly-Supervised Visual Concept Recognition

The goal of unpaired image captioning (UIC) is to describe images without using image-caption pairs in the training phase. Although challenging, we except the task can be accomplished by leveraging a training set of images aligned with visual concepts. Most existing studies use off-the-shelf algorithms to obtain the visual concepts because the Bounding Box (BBox) labels or relationship-triplet labels used for the training are expensive to acquire. In order to resolve the problem in expensive annotations, we propose a novel approach to achieve cost-effective UIC. Specifically, we adopt image-level labels for the optimization of the UIC model in a weakly-supervised manner. For each image, we assume that only the image-level labels are available without specific locations and numbers. The image-level labels are utilized to train a weakly-supervised object recognition model to extract object information (e.g., instance) in an image, and the extracted instances are adopted to infer the relationships among different objects based on an enhanced graph neural network (GNN). The proposed approach achieves comparable or even better performance compared with previous methods without the expensive cost of annotations. Furthermore, we design an unrecognized object (UnO) loss combined with a visual concept reward to improve the alignment of the inferred object and relationship information with the images. It can effectively alleviate the issue encountered by existing UIC models about generating sentences with nonexistent objects. To the best of our knowledge, this is the first attempt to solve the problem of Weakly-Supervised visual concept recognition for UIC (WS-UIC) based only on image-level labels. Extensive experiments have been carried out to demonstrate that the proposed WS-UIC model achieves inspiring results on the COCO dataset while significantly reducing the cost of labeling.

cs.CV↗
arXiv2022

Hyper-relationship Learning Network for Scene Graph Generation

Generating informative scene graphs from images requires integrating and reasoning from various graph components, i.e., objects and relationships. However, current scene graph generation (SGG) methods, including the unbiased SGG methods, still struggle to predict informative relationships due to the lack of 1) high-level inference such as transitive inference between relationships and 2) efficient mechanisms that can incorporate all interactions of graph components. To address the issues mentioned above, we devise a hyper-relationship learning network, termed HLN, for SGG. Specifically, the proposed HLN stems from hypergraphs and two graph attention networks (GATs) are designed to infer relationships: 1) the object-relationship GAT or OR-GAT to explore interactions between objects and relationships, and 2) the hyper-relationship GAT or HR-GAT to integrate transitive inference of hyper-relationships, i.e., the sequential relationships between three objects for transitive reasoning. As a result, HLN significantly improves the performance of scene graph generation by integrating and reasoning from object interactions, relationship interactions, and transitive inference of hyper-relationships. We evaluate HLN on the most popular SGG dataset, i.e., the Visual Genome dataset, and the experimental results demonstrate its great superiority over recent state-of-the-art methods. For example, the proposed HLN improves the recall per relationship from 11.3\% to 13.1\%, and maintains the recall per image from 19.8\% to 34.9\%. We will release the source code and pretrained models on GitHub.

cs.CV↗
arXiv2022

Schema-Free Dependency Parsing via Sequence Generation

Dependency parsing aims to extract syntactic dependency structure or semantic dependency structure for sentences. Existing methods suffer the drawbacks of lacking universality or highly relying on the auxiliary decoder. To remedy these drawbacks, we propose to achieve universal and schema-free Dependency Parsing (DP) via Sequence Generation (SG) DPSG by utilizing only the pre-trained language model (PLM) without any auxiliary structures or parsing algorithms. We first explore different serialization designing strategies for converting parsing structures into sequences. Then we design dependency units and concatenate these units into the sequence for DPSG. Thanks to the high flexibility of the sequence generation, our DPSG can achieve both syntactic DP and semantic DP using a single model. By concatenating the prefix to indicate the specific schema with the sequence, our DPSG can even accomplish multi-schemata parsing. The effectiveness of our DPSG is demonstrated by the experiments on widely used DP benchmarks, i.e., PTB, CODT, SDP15, and SemEval16. DPSG achieves comparable results with the first-tier methods on all the benchmarks and even the state-of-the-art (SOTA) performance in CODT and SemEval16. This paper demonstrates our DPSG has the potential to be a new parsing paradigm. We will release our codes upon acceptance.

cs.CL↗
arXiv2021

Mass concentration and uniqueness of ground states for mass subcritical rotational nonlinear Schrödinger equations

This paper considers ground states of mass subcritical rotational nonlinear Schrödinger equation \begin{equation*} -Δu+V(x)u+iΩ(x^\perp\cdot\nabla u)=μu+ρ^{p-1}|u|^{p-1}u \,\ \text{in} \,\ \mathbb{R}^2, \end{equation*} where $V(x)$ is an external potential, $Ω>0$ characterizes the rotational velocity of the trap $V(x)$, $1 0$ describes the strength of the attractive interactions. It is shown that ground states of the above equation can be described equivalently by minimizers of the $L^2-$ constrained variational problem. We prove that minimizers exist for any $ρ\in(0,\infty)$ when $0<Ω<Ω^*$, where $0<Ω^*:=Ω^*(V)<\infty$ denotes the critical rotational velocity of $V(x)$. While $Ω>Ω^*$, there admits no minimizers for any $ρ\in(0,\infty)$. For fixed $0<Ω<Ω^*$, by using energy estimates and blow-up analysis, we also analyze the limit behavior of minimizers as $ρ\to\infty$. Finally, we prove that up to a constant phase, there exists a unique minimizer when $ρ>0$ is large enough and $Ω\in(0,Ω^*)$ is fixed.

math.AP↗
arXiv2021

Local Uniqueness of Ground States for Rotating Bose-Einstein Condensates with Attractive Interactions

We study ground states of two-dimensional Bose-Einstein condensates with attractive interactions in a trap $V(x)$ rotating at the velocity $Ω$. It is known that there exist a critical rotational velocity $0<Ω^*:=Ω^*(V)\leq \infty$ and a critical number $0<a^*<\infty$ such that for any rotational velocity $0\le Ω<Ω^*$, ground states exist if and only if the coupling constant $a$ satisfies $a<a^*$. For a general class of traps $V(x)$, which may not be symmetric, we prove in this paper that up to a constant phase, there exists a unique ground state as $a\nearrow a^*$, where $Ω\in(0,Ω^*)$ is fixed. This result extends essentially our recent uniqueness result, where only the radially symmetric traps $V(x)$ could be handled with.

math.AP↗
arXiv2021

Towards understanding the power of quantum kernels in the NISQ era

A key problem in the field of quantum computing is understanding whether quantum machine learning (QML) models implemented on noisy intermediate-scale quantum (NISQ) machines can achieve quantum advantages. Recently, Huang et al. [Nat Commun 12, 2631] partially answered this question by the lens of quantum kernel learning. Namely, they exhibited that quantum kernels can learn specific datasets with lower generalization error over the optimal classical kernel methods. However, most of their results are established on the ideal setting and ignore the caveats of near-term quantum machines. To this end, a crucial open question is: does the power of quantum kernels still hold under the NISQ setting? In this study, we fill this knowledge gap by exploiting the power of quantum kernels when the quantum system noise and sample error are considered. Concretely, we first prove that the advantage of quantum kernels is vanished for large size of datasets, few number of measurements, and large system noise. With the aim of preserving the superiority of quantum kernels in the NISQ era, we further devise an effective method via indefinite kernel learning. Numerical simulations accord with our theoretical results. Our work provides theoretical guidance of exploring advanced quantum kernels to attain quantum advantages on NISQ devices.

quant-ph↗
arXiv2021

On energy gap phenomena of the Whitney sphere and related problems

In this paper, we study Lagrangian submanifolds satisfying ${\rm \nabla^*} T=0$ introduced by Zhang \cite{Zh} in the complex space forms $N(4c)(c=0\ or \ 1)$, where $T ={\rm \nabla^*}\tilde{h}$ and $\tilde{h}$ is the Lagrangian trace-free second fundamental form. We obtain some Simons' type integral inequalities and rigidity theorems for such Lagrangian submanifolds. Moreover we study Lagrangian submanifolds in $\mathbb{C}^n$ satisfying $\nabla^*\nabla^*T=0$ and introduce a flow method related to them.

math.DG↗
arXiv2021

On energy gap phenomena of the Whitney spheres in $\mathbb{C}^n$ or $\mathbb{CP}^n$

In \cite{Zh} \cite{LY} Zhang, Luo and Yin initiated the study of Lagrangian submanifolds satisfying ${\rm \nabla^*} T=0$ or ${\rm \nabla^*\nabla^*}T=0$ in $\mathbb{C}^n$ or $\mathbb{CP}^n$, where $T ={\rm \nabla^*}\tilde{h}$ and $\tilde{h}$ is the Lagrangian trace-free second fundamental form. They proved several rigidity theorems for Lagrangian surfaces satisfying ${\rm \nabla^*} T=0$ or ${\rm \nabla^*\nabla^*}T=0$ in $\mathbb{C}^2$ under proper small energy assumption and gave new characterization of the Whitney spheres in $\mathbb{C}^2$. In this paper we extend these results to Lagrangian submanifolds in $\mathbb{C}^n$ of dimension $n\geq3$ and to Lagrangian submanifolds in $\mathbb{CP}^n$.

math.DG↗
arXiv2021

ModelPS: An Interactive and Collaborative Platform for Editing Pre-trained Models at Scale

AI engineering has emerged as a crucial discipline to democratize deep neural network (DNN) models among software developers with a diverse background. In particular, altering these DNN models in the deployment stage posits a tremendous challenge. In this research, we propose and develop a low-code solution, ModelPS (an acronym for "Model Photoshop"), to enable and empower collaborative DNN model editing and intelligent model serving. The ModelPS solution embodies two transformative features: 1) a user-friendly web interface for a developer team to share and edit DNN models pictorially, in a low-code fashion, and 2) a model genie engine in the backend to aid developers in customizing model editing configurations for given deployment requirements or constraints. Our case studies with a wide range of deep learning (DL) models show that the system can tremendously reduce both development and communication overheads with improved productivity.

cs.DC↗
arXiv2021

Existence and Asymptotic Behavior of Ground States for Rotating Bose-Einstein Condensates

We study ground states of two-dimensional Bose-Einstein condensates with repulsive ($a>0$) or attractive ($a<0$) interactions in a trap $V (x)$ rotating at the velocity $Ω$. It is known that there exist critical parameters $a^*>0$ and $Ω^*:=Ω^*(V(x))>0$ such that if $Ω>Ω^*$, then there is no ground state for any $a\in\R$; if $0\le Ω<Ω^*$, then ground states exist if and only if $a\in(-a^*,+\infty)$. As a completion of the existing results, in this paper, we focus on the critical case where $0<Ω=Ω^*<+\infty$ and classify the existence and nonexistence of ground states for $a\in\R$. Moreover, for a suitable class of radially symmetric traps $V(x)$, employing the inductive symmetry method, we prove that up to a constant phase, the ground states must be real-valued, unique and free of vortices as $Ω\searrow 0$, no matter whether the interactions of the condensates are repulsive or not.

math.AP↗
arXiv2021

Concentration Behavior of Ground States for $L^2$-Critical Schrödinger Equation with a Spatially Decaying Nonlinearity

We consider the following time-independent nonlinear $L^2$-critical Schrödinger equation \[ -Δu(x)+V(x)u(x)-a|x|^{-b}|u|^{1+\frac{4-2b}{N}}=μu(x)\,\ \hbox{in}\,\ \mathbb{R}^N, \] where $μ\in\mathbb{R}$, $a>0$, $N\geq 1$, $0 0$ such that minimizer exists for $0 a^*$. However if $a=a^*$, it is proved that whether minimizer exists depends sensitively on the value of $V(0)$. Moreover, when there is no minimizer at threshold $a^*$, we give a detailed concentration behavior of minimizers as $a\nearrow a^*$, based on which we finally prove that there is a unique minimizer as $a\nearrow a^*$.

math.AP↗
arXiv2021

On $n$-dimensional complete self-similar solutions to the mean curvature flow in $\mathbb{R}^{n+1}$ with nonnegative constant scalar curvature

As is well known, self-similar solutions to the mean curvature flow, including self-shrinkers, translating solitons and self-expanders, arise naturally in the singularity analysis of the mean curvature flow. Recently, Guo \cite{Guo} proved that $n$-dimensional compact self-shrinkers in $\mathbb{R}^{n+1}$ with scalar curvature bounded from above or below by some constant are isometric to the round sphere $\mathbb{S}^n(\sqrt{n})$, which implies that $n$-dimensional compact self-shrinkers in $\mathbb{R}^{n+1}$ with constant scalar curvature are isometric to the round sphere $\mathbb{S}^n(\sqrt{n})$(see also \cite{Hui1}). Complete classifications of $n$-dimensional translating solitons in $\mathbb{R}^{n+1}$ with nonnegative constant scalar curvature and of $n$-dimensional self-expanders in $\mathbb{R}^{n+1}$ with nonnegative constant scalar curvature were given by Martín, Savas-Halilaj and Smoczyk\cite{MSS} and Ancari and Cheng\cite{AC}, respectively. In this paper we give complete classifications of $n$-dimensional complete self-shrinkers in $\mathbb{R}^{n+1}$ with nonnegative constant scalar curvature. We will also give alternative proofs of the classification theorems due to Martín, Savas-Halilaj and Smoczyk \cite{MSS} and Ancari and Cheng\cite{AC}.

math.DG↗
arXiv2021

A Serverless Cloud-Fog Platform for DNN-Based Video Analytics with Incremental Learning

DNN-based video analytics have empowered many new applications (e.g., automated retail). Meanwhile, the proliferation of fog devices provides developers with more design options to improve performance and save cost. To the best of our knowledge, this paper presents the first serverless system that takes full advantage of the client-fog-cloud synergy to better serve the DNN-based video analytics. Specifically, the system aims to achieve two goals: 1) Provide the optimal analytics results under the constraints of lower bandwidth usage and shorter round-trip time (RTT) by judiciously managing the computational and bandwidth resources deployed in the client, fog, and cloud environment. 2) Free developers from tedious administration and operation tasks, including DNN deployment, cloud and fog's resource management. To this end, we implement a holistic cloud-fog system referred to as VPaaS (Video-Platform-as-a-Service). VPaaS adopts serverless computing to enable developers to build a video analytics pipeline by simply programming a set of functions (e.g., model inference), which are then orchestrated to process videos through carefully designed modules. To save bandwidth and reduce RTT, VPaaS provides a new video streaming protocol that only sends low-quality video to the cloud. The state-of-the-art (SOTA) DNNs deployed at the cloud can identify regions of video frames that need further processing at the fog ends. At the fog ends, misidentified labels in these regions can be corrected using a light-weight DNN model. To address the data drift issues, we incorporate limited human feedback into the system to verify the results and adopt incremental learning to improve our system continuously. The evaluation demonstrates that VPaaS is superior to several SOTA systems: it maintains high accuracy while reducing bandwidth usage by up to 21%, RTT by up to 62.5%, and cloud monetary cost by up to 50%.

cs.DC↗
arXiv2020

The Nonexistence of Vortices for Rotating Bose-Einstein Condensates with Attractive Interactions

This article is devoted to studying the model of two-dimensional attractive Bose-Einstein condensates in a trap $V(x)$ rotating at the velocity $Ω$. This model can be described by the complex-valued Gross-Pitaevskii energy functional. It is shown that there exists a critical rotational velocity $0<Ω^*:=Ω^*(V)\leq \infty$, depending on the general trap $V(x)$, such that for any rotational velocity $0\leq Ω<Ω^*$, minimizers (i.e., ground states) exist if and only if $a 0$ denotes the absolute product for the number of particles times the scattering length, and $w>0$ is the unique positive solution of $Δw-w+w^3=0$ in $\mathbb{R}^2$. If $V(x)=|x|^2$ and $ 0<Ω<Ω^*(=2)$ is fixed, we prove that, up to a constant phase, all minimizers must be real-valued, unique and free of vortices as $a \nearrow a^*$, by analyzing the refined limit behavior of minimizers and employing the non-degenerancy of $w$.

math.AP↗
arXiv2020

Rigidity theorems for minimal Lagrangian surfaces with Legendrian capillary boundary

In this note, we study minimal Lagrangian surfaces in $\mathbb{B}^4$ with Legendrian capillary boundary on $\mathbb{S}^3$. On the one hand, we prove that any minimal Lagrangian surface in $\mathbb{B}^4$ with Legendrian free boundary on $\mathbb{S}^3$ must be an equatorial plane disk. One the other hand, we show that any annulus type minimal Lagrangian surface in $\mathbb{B}^4$ with Legendrian capillary boundary on $\mathbb{S}^3$ must be congruent to one of the Lagrangian catenoids. These results confirm the conjecture proposed by Li, Wang and Weng (Sci. China Math., 2020).

math.DG↗
arXiv2020

Hysia: Serving DNN-Based Video-to-Retail Applications in Cloud

Combining \underline{v}ideo streaming and online \underline{r}etailing (V2R) has been a growing trend recently. In this paper, we provide practitioners and researchers in multimedia with a cloud-based platform named Hysia for easy development and deployment of V2R applications. The system consists of: 1) a back-end infrastructure providing optimized V2R related services including data engine, model repository, model serving and content matching; and 2) an application layer which enables rapid V2R application prototyping. Hysia addresses industry and academic needs in large-scale multimedia by: 1) seamlessly integrating state-of-the-art libraries including NVIDIA video SDK, Facebook faiss, and gRPC; 2) efficiently utilizing GPU computation; and 3) allowing developers to bind new models easily to meet the rapidly changing deep learning (DL) techniques. On top of that, we implement an orchestrator for further optimizing DL model serving performance. Hysia has been released as an open source project on GitHub, and attracted considerable attention. We have published Hysia to DockerHub as an official image for seamless integration and deployment in current cloud environments.

cs.MM↗
arXiv2020

Look, Read and Feel: Benchmarking Ads Understanding with Multimodal Multitask Learning

Given the massive market of advertising and the sharply increasing online multimedia content (such as videos), it is now fashionable to promote advertisements (ads) together with the multimedia content. It is exhausted to find relevant ads to match the provided content manually, and hence, some automatic advertising techniques are developed. Since ads are usually hard to understand only according to its visual appearance due to the contained visual metaphor, some other modalities, such as the contained texts, should be exploited for understanding. To further improve user experience, it is necessary to understand both the topic and sentiment of the ads. This motivates us to develop a novel deep multimodal multitask framework to integrate multiple modalities to achieve effective topic and sentiment prediction simultaneously for ads understanding. In particular, our model first extracts multimodal information from ads and learn high-level and comparable representations. The visual metaphor of the ad is decoded in an unsupervised manner. The obtained representations are then fed into the proposed hierarchical multimodal attention modules to learn task-specific representations for final prediction. A multitask loss function is also designed to train both the topic and sentiment prediction models jointly in an end-to-end manner. We conduct extensive experiments on the latest and large advertisement dataset and achieve state-of-the-art performance for both prediction tasks. The obtained results could be utilized as a benchmark for ads understanding.

cs.MM↗