arXiv subjects

Hao Zhang

Publications and source records attributed to Hao Zhang.

At least 1,027 records · Page 57Linked to original sources

arXiv2021

PhotoChat: A Human-Human Dialogue Dataset with Photo Sharing Behavior for Joint Image-Text Modeling

We present a new human-human dialogue dataset - PhotoChat, the first dataset that casts light on the photo sharing behavior in onlin emessaging. PhotoChat contains 12k dialogues, each of which is paired with a user photo that is shared during the conversation. Based on this dataset, we propose two tasks to facilitate research on image-text modeling: a photo-sharing intent prediction task that predicts whether one intends to share a photo in the next conversation turn, and a photo retrieval task that retrieves the most relevant photo according to the dialogue context. In addition, for both tasks, we provide baseline models using the state-of-the-art models and report their benchmark performances. The best image retrieval model achieves 10.4% recall@1 (out of 1000 candidates) and the best photo intent prediction model achieves 58.1% F1 score, indicating that the dataset presents interesting yet challenging real-world problems. We are releasing PhotoChat to facilitate future research work among the community.

cs.IR↗

arXiv2021

Learning Mesh Representations via Binary Space Partitioning Tree Networks

Polygonal meshes are ubiquitous, but have only played a relatively minor role in the deep learning revolution. State-of-the-art neural generative models for 3D shapes learn implicit functions and generate meshes via expensive iso-surfacing. We overcome these challenges by employing a classical spatial data structure from computer graphics, Binary Space Partitioning (BSP), to facilitate 3D learning. The core operation of BSP involves recursive subdivision of 3D space to obtain convex sets. By exploiting this property, we devise BSP-Net, a network that learns to represent a 3D shape via convex decomposition without supervision. The network is trained to reconstruct a shape using a set of convexes obtained from a BSP-tree built over a set of planes, where the planes and convexes are both defined by learned network weights. BSP-Net directly outputs polygonal meshes from the inferred convexes. The generated meshes are watertight, compact (i.e., low-poly), and well suited to represent sharp geometry. We show that the reconstruction quality by BSP-Net is competitive with those from state-of-the-art methods while using much fewer primitives. We also explore variations to BSP-Net including using a more generic decoder for reconstruction, more general primitives than planes, as well as training a generative model with variational auto-encoders. Code is available at https://github.com/czq142857/BSP-NET-original.

cs.CV↗

arXiv2021

Scalable higher-order exceptional surface with passive resonators

The sensitivity of perturbation sensing can be effectively enhanced with higher-order exceptional points due to the nonlinear response to frequency splitting. However, the experimental implementation is challenging since all the parameters need to be precisely prepared. The emergence of exceptional surface (ES) improves the robustness of the system to the external environment, while maintaining the same sensitivity. Here, we propose the first scalable protocol for realizing photonic high-order exceptional surface with passive resonators. By adding one or more additional passive resonators in the low-order ES photonic system, the 3- or arbitrary N-order ES is constructed and proved to be easily realized in experiment. We show that the sensitivity is enhanced and experimental demonstration is more resilent against the fabrication errors. The additional phase-modulation effect is also investigated.

physics.optics↗

arXiv2021

TM-NET: Deep Generative Networks for Textured Meshes

We introduce TM-NET, a novel deep generative model for synthesizing textured meshes in a part-aware manner. Once trained, the network can generate novel textured meshes from scratch or predict textures for a given 3D mesh, without image guidance. Plausible and diverse textures can be generated for the same mesh part, while texture compatibility between parts in the same shape is achieved via conditional generation. Specifically, our method produces texture maps for individual shape parts, each as a deformable box, leading to a natural UV map with minimal distortion. The network separately embeds part geometry (via a PartVAE) and part texture (via a TextureVAE) into their respective latent spaces, so as to facilitate learning texture probability distributions conditioned on geometry. We introduce a conditional autoregressive model for texture generation, which can be conditioned on both part geometry and textures already generated for other parts to achieve texture compatibility. To produce high-frequency texture details, our TextureVAE operates in a high-dimensional latent space via dictionary-based vector quantization. We also exploit transparencies in the texture as an effective means to model complex shape structures including topological details. Extensive experiments demonstrate the plausibility, quality, and diversity of the textures and geometries generated by our network, while avoiding inconsistency issues that are common to novel view synthesis methods.

cs.GR↗

arXiv2021

BalaGAN: Image Translation Between Imbalanced Domains via Cross-Modal Transfer

State-of-the-art image-to-image translation methods tend to struggle in an imbalanced domain setting, where one image domain lacks richness and diversity. We introduce a new unsupervised translation network, BalaGAN, specifically designed to tackle the domain imbalance problem. We leverage the latent modalities of the richer domain to turn the image-to-image translation problem, between two imbalanced domains, into a balanced, multi-class, and conditional translation problem, more resembling the style transfer setting. Specifically, we analyze the source domain and learn a decomposition of it into a set of latent modes or classes, without any supervision. This leaves us with a multitude of balanced cross-domain translation tasks, between all pairs of classes, including the target domain. During inference, the trained network takes as input a source image, as well as a reference or style image from one of the modes as a condition, and produces an image which resembles the source on the pixel-wise level, but shares the same mode as the reference. We show that employing modalities within the dataset improves the quality of the translated images, and that BalaGAN outperforms strong baselines of both unconditioned and style-transfer-based image-to-image translation methods, in terms of image quality and diversity.

cs.CV↗

arXiv2021

Dynamic radiomics: a new methodology to extract quantitative time-related features from tomographic images

The feature extraction methods of radiomics are mainly based on static tomographic images at a certain moment, while the occurrence and development of disease is a dynamic process that cannot be fully reflected by only static characteristics. This study proposes a new dynamic radiomics feature extraction workflow that uses time-dependent tomographic images of the same patient, focuses on the changes in image features over time, and then quantifies them as new dynamic features for diagnostic or prognostic evaluation. We first define the mathematical paradigm of dynamic radiomics and introduce three specific methods that can describe the transformation process of features over time. Three different clinical problems are used to validate the performance of the proposed dynamic feature with conventional 2D and 3D static features.

eess.IV↗

arXiv2021

A Novel Automatic Modulation Classification Scheme Based on Multi-Scale Networks

Automatic modulation classification enables intelligent communications and it is of crucial importance in today's and future wireless communication networks. Although many automatic modulation classification schemes have been proposed, they cannot tackle the intra-class diversity problem caused by the dynamic changes of the wireless communication environment. In order to overcome this problem, inspired by face recognition, a novel automatic modulation classification scheme is proposed by using the multi-scale network in this paper. Moreover, a novel loss function that combines the center loss and the cross entropy loss is exploited to learn both discriminative and separable features in order to further improve the classification performance. Extensive simulation results demonstrate that our proposed automatic modulation classification scheme can achieve better performance than the benchmark schemes in terms of the classification accuracy. The influence of the network parameters and the loss function with the two-stage training strategy on the classification accuracy of our proposed scheme are investigated.

eess.SP↗

arXiv2021

Team Assignment for Heterogeneous Multi-Robot Sensor Coverage through Graph Representation Learning

Sensor coverage is the critical multi-robot problem of maximizing the detection of events in an environment through the deployment of multiple robots. Large multi-robot systems are often composed of simple robots that are typically not equipped with a complete set of sensors, so teams with comprehensive sensing abilities are required to properly cover an area. Robots also exhibit multiple forms of relationships (e.g., communication connections or spatial distribution) that need to be considered when assigning robot teams for sensor coverage. To address this problem, in this paper we introduce a novel formulation of sensor coverage by multi-robot systems with heterogeneous relationships as a graph representation learning problem. We propose a principled approach based on the mathematical framework of regularized optimization to learn a unified representation of the multi-robot system from the graphs describing the heterogeneous relationships and to identify the learned representation's underlying structure in order to assign the robots to teams. To evaluate the proposed approach, we conduct extensive experiments on simulated multi-robot systems and a physical multi-robot system as a case study, demonstrating that our approach is able to effectively assign teams for heterogeneous multi-robot sensor coverage.

cs.RO↗

arXiv2021

Adaptation to Team Composition Changes for Heterogeneous Multi-Robot Sensor Coverage

We consider the problem of multi-robot sensor coverage, which deals with deploying a multi-robot team in an environment and optimizing the sensing quality of the overall environment. As real-world environments involve a variety of sensory information, and individual robots are limited in their available number of sensors, successful multi-robot sensor coverage requires the deployment of robots in such a way that each individual team member's sensing quality is maximized. Additionally, because individual robots have varying complements of sensors and both robots and sensors can fail, robots must be able to adapt and adjust how they value each sensing capability in order to obtain the most complete view of the environment, even through changes in team composition. We introduce a novel formulation for sensor coverage by multi-robot teams with heterogeneous sensing capabilities that maximizes each robot's sensing quality, balancing the varying sensing capabilities of individual robots based on the overall team composition. We propose a solution based on regularized optimization that uses sparsity-inducing terms to ensure a robot team focuses on all possible event types, and which we show is proven to converge to the optimal solution. Through extensive simulation, we show that our approach is able to effectively deploy a multi-robot team to maximize the sensing quality of an environment, responding to failures in the multi-robot team more robustly than non-adaptive approaches.

cs.RO↗

arXiv2021

Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learning

Pollux improves scheduling performance in deep learning (DL) clusters by adaptively co-optimizing inter-dependent factors both at the per-job level and at the cluster-wide level. Most existing schedulers expect users to specify the number of resources for each job, often leading to inefficient resource use. Some recent schedulers choose job resources for users, but do so without awareness of how DL training can be re-optimized to better utilize the provided resources. Pollux simultaneously considers both aspects. By monitoring the status of each job during training, Pollux models how their goodput (a novel metric we introduce that combines system throughput with statistical efficiency) would change by adding or removing resources. Leveraging these information, Pollux dynamically (re-)assigns resources to improve cluster-wide goodput, while respecting fairness and continually optimizing each DL job to better utilize those resources. In experiments with real DL jobs and with trace-driven simulations, Pollux reduces average job completion times by 37-50% relative to state-of-the-art DL schedulers, even when they are provided with ideal resource and training configurations for every job. Pollux promotes fairness among DL jobs competing for resources based on a more meaningful measure of useful job progress, and reveals a new opportunity for reducing DL cost in cloud environments. Pollux is implemented and publicly available as part of an open-source project at https://github.com/petuum/adaptdl.

cs.DC↗

arXiv2021

Experimental realization of sensitivity enhancement and suppression with exceptional surfaces

By preparing a sensor system around isolated exceptional points, one can obtain a great enhancement of the sensitivity benefiting from the non-Hermiticity. However, this comes at the cost of reduction of the flexibility of the system, which is critical for practical applications. By generalizing the exceptional points to exceptional surfaces, it has been theoretically proposed recently that enhanced sensitivity and flexibility can be combined. Here, we experimentally demonstrate an exceptional surface in a non-Hermitian photonic sensing system, which is composed of a whispering-gallery-mode microresonator and two nanofiber waveguides, resulting in a unidirectional coupling between two degenerate counter-propagating modes with an external optical isolator. The system is simple, robust, and can be easily operated around an exceptional surface. On the one hand, we observe sensitivity enhancement by monitoring the resonant frequency splitting caused by small perturbations. This demonstration of exceptional-surface-enhanced sensitivity paves the way for practical non-Hermitian sensing applications. On the other hand, we also show the suppression of frequency splitting around the exceptional surface for the first time.

physics.optics↗

arXiv2021

Parallel Attention Network with Sequence Matching for Video Grounding

Given a video, video grounding aims to retrieve a temporal moment that semantically corresponds to a language query. In this work, we propose a Parallel Attention Network with Sequence matching (SeqPAN) to address the challenges in this task: multi-modal representation learning, and target moment boundary prediction. We design a self-guided parallel attention module to effectively capture self-modal contexts and cross-modal attentive information between video and text. Inspired by sequence labeling tasks in natural language processing, we split the ground truth moment into begin, inside, and end regions. We then propose a sequence matching strategy to guide start/end boundary predictions using region labels. Experimental results on three datasets show that SeqPAN is superior to state-of-the-art methods. Furthermore, the effectiveness of the self-guided parallel attention module and the sequence matching module is verified.

cs.CL↗

arXiv2021

3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics

We introduce 3D-FRONT (3D Furnished Rooms with layOuts and semaNTics), a new, large-scale, and comprehensive repository of synthetic indoor scenes highlighted by professionally designed layouts and a large number of rooms populated by high-quality textured 3D models with style compatibility. From layout semantics down to texture details of individual objects, our dataset is freely available to the academic community and beyond. Currently, 3D-FRONT contains 18,968 rooms diversely furnished by 3D objects, far surpassing all publicly available scene datasets. In addition, the 13,151 furniture objects all come with high-quality textures. While the floorplans and layout designs are directly sourced from professional creations, the interior designs in terms of furniture styles, color, and textures have been carefully curated based on a recommender system we develop to attain consistent styles as expert designs. Furthermore, we release Trescope, a light-weight rendering tool, to support benchmark rendering of 2D images and annotations from 3D-FRONT. We demonstrate two applications, interior scene synthesis and texture synthesis, that are especially tailored to the strengths of our new dataset. The project page is at: https://tianchi.aliyun.com/specials/promotion/alibaba-3d-scene-dataset.

cs.CV↗

arXiv2021

Video Corpus Moment Retrieval with Contrastive Learning

Given a collection of untrimmed and unsegmented videos, video corpus moment retrieval (VCMR) is to retrieve a temporal moment (i.e., a fraction of a video) that semantically corresponds to a given text query. As video and text are from two distinct feature spaces, there are two general approaches to address VCMR: (i) to separately encode each modality representations, then align the two modality representations for query processing, and (ii) to adopt fine-grained cross-modal interaction to learn multi-modal representations for query processing. While the second approach often leads to better retrieval accuracy, the first approach is far more efficient. In this paper, we propose a Retrieval and Localization Network with Contrastive Learning (ReLoCLNet) for VCMR. We adopt the first approach and introduce two contrastive learning objectives to refine video encoder and text encoder to learn video and text representations separately but with better alignment for VCMR. The video contrastive learning (VideoCL) is to maximize mutual information between query and candidate video at video-level. The frame contrastive learning (FrameCL) aims to highlight the moment region corresponds to the query at frame-level, within a video. Experimental results show that, although ReLoCLNet encodes text and video separately for efficiency, its retrieval accuracy is comparable with baselines adopting cross-modal interaction learning.

cs.CL↗

arXiv2021

Understanding Deep MIMO Detection

Incorporating deep learning (DL) into multiple-input multiple-output (MIMO) detection has been deemed as a promising technique for future wireless communications. However, most DL-based detection algorithms are lack of theoretical interpretation on internal mechanisms and could not provide general guidance on network design. In this paper, we analyze the performance of DL-based MIMO detection to better understand its strengths and weaknesses. We investigate two different architectures: a data-driven DL detector with a neural network activated by rectifier linear unit (ReLU) function and a model-driven DL detector from unfolding a traditional iterative detection algorithm. We demonstrate that data-driven DL detector asymptotically approaches to the maximum a posterior (MAP) detector in various scenarios but requires enough training samples to converge in time-varying channels. On the other hand, the model-driven DL detector utilizes model expert knowledge to alleviate the impact of channels and establish a relatively reliable detection method with a small set of training data. Due to its model specific property, the performance of model-driven DL detector is largely determined by the underlying iterative detection algorithm, which is usually suboptimal compared to the MAP detector. Simulation results confirm our analytical results and demonstrate the effectiveness of DL-based MIMO detection for both linear and nonlinear signal systems.

eess.SP↗

arXiv2021

Roof-GAN: Learning to Generate Roof Geometry and Relations for Residential Houses

This paper presents Roof-GAN, a novel generative adversarial network that generates structured geometry of residential roof structures as a set of roof primitives and their relationships. Given the number of primitives, the generator produces a structured roof model as a graph, which consists of 1) primitive geometry as raster images at each node, encoding facet segmentation and angles; 2) inter-primitive colinear/coplanar relationships at each edge; and 3) primitive geometry in a vector format at each node, generated by a novel differentiable vectorizer while enforcing the relationships. The discriminator is trained to assess the primitive raster geometry, the primitive relationships, and the primitive vector geometry in a fully end-to-end architecture. Qualitative and quantitative evaluations demonstrate the effectiveness of our approach in generating diverse and realistic roof models over the competing methods with a novel metric proposed in this paper for the task of structured geometry generation. Code and data are available at https://github.com/yi-ming-qian/roofgan .

cs.CV↗

arXiv2021

Prompt acceleration of the $μ^+$ beam in a donut wakefield driven by a shaped Laguerre-Gaussian laser pulse

The recent experimental data of anomalous magnetic moments strongly indicate the existence of new physics beyond the standard model. An energetic $μ^+$ beam is a potential option to the expected neutrino factories, the future muon colliders and the $μ$SR(the spin rotation, resonance and relaxation) technology. It is proposed a prompt acceleration scheme of the $μ^+$ beam in a donut wakefield driven by a shaped Laguerre-Gaussian (LG) laser pulse. The forward part of the donut wakefield can accelerate and also focus positive particle beams effectively. The LG laser is shaped by a near-critical-density plasma. The shaped LG laser has the shorter rise time and can enlarge the acceleration field. The acceleration field driven by a shaped LG laser pulse is six times higher than that driven by a normal LG laser pulse. The simulation results show that the $ μ^+$ bunch can be accelerated from $200\mathrm{MeV}$ to 2GeV and the transversal size of the $μ^+$ bunch is also focused from initial $ω_0=5μm$ to $ω=1μm$ within several picoseconds.

physics.plasm-ph↗

arXiv2021

Model Averaging Estimation for Partially Linear Functional Score Models

This paper is concerned with model averaging estimation for partially linear functional score models. These models predict a scalar response using both parametric effect of scalar predictors and non-parametric effect of a functional predictor. Within this context, we develop a Mallows-type criterion for choosing weights. The resulting model averaging estimator is proved to be asymptotically optimal under certain regularity conditions in terms of achieving the smallest possible squared error loss. Simulation studies demonstrate its superiority or comparability to information criterion score-based model selection and averaging estimators. The proposed procedure is also applied to two real data sets for illustration. That the components of nonparametric part are unobservable leads to a more complicated situation than ordinary partially linear models (PLM) and a different theoretical derivation from those of PLM.

stat.ME↗