arXiv subjects

Hao Wang

Publications and source records attributed to Hao Wang.

At least 1,495 records · Page 83Linked to original sources

arXiv2020

Neural Network Design: Learning from Neural Architecture Search

Neural Architecture Search (NAS) aims to optimize deep neural networks' architecture for better accuracy or smaller computational cost and has recently gained more research interests. Despite various successful approaches proposed to solve the NAS task, the landscape of it, along with its properties, are rarely investigated. In this paper, we argue for the necessity of studying the landscape property thereof and propose to use the so-called Exploratory Landscape Analysis (ELA) techniques for this goal. Taking a broad set of designs of the deep convolutional network, we conduct extensive experimentation to obtain their performance. Based on our analysis of the experimental results, we observed high similarities between well-performing architecture designs, which is then used to significantly narrow the search space to improve the efficiency of any NAS algorithm. Moreover, we extract the ELA features over the NAS landscapes on three common image classification data sets, MNIST, Fashion, and CIFAR-10, which shows that the NAS landscape can be distinguished for those three data sets. Also, when comparing to the ELA features of the well-known Black-Box Optimization Benchmarking (BBOB) problem set, we found out that the NAS landscapes surprisingly form a new problem class on its own, which can be separated from all $24$ BBOB problems. Given this interesting observation, we, therefore, state the importance of further investigation on selecting an efficient optimizer for the NAS landscape as well as the necessity of augmenting the current benchmark problem set.

cs.LG↗

arXiv2020

Painting with Hue, Saturation, and Brightness Control by Nanoscale 3D Printing

Varying only the in-plane or out-of-plane dimensions of nanostructures produces a wide range of colourful elements in metasurfaces and thin films. However, achieving shades of grey and control of colour saturation remains challenging. Here, we introduce a hybrid approach to colour generation based on the tuning of nanostructure geometry in all three dimensions. Through two-photon polymerization lithography, we systematically investigated colour generation from the simple single nanopillar geometry made of low-refractive-index material; realizing grayscale and full colour palettes with control of hue, saturation, brightness through tuning of height, diameter, and periodicity of nanopillars. Arbitrary colourful and grayscale images were painted by mapping desired prints to precisely controllable parameters during 3D printing. We extend our understanding of the scattering properties of the low-refractive-index nanopillar to demonstrate grayscale inversion and colour desaturation, with steganography at the level of single nanopillars.

physics.optics↗

arXiv2020

Energy Storage Arbitrage in Real-Time Markets via Reinforcement Learning

In this paper, we derive a temporal arbitrage policy for storage via reinforcement learning. Real-time price arbitrage is an important source of revenue for storage units, but designing good strategies have proven to be difficult because of the highly uncertain nature of the prices. Instead of current model predictive or dynamic programming approaches, we use reinforcement learning to design an optimal arbitrage policy. This policy is learned through repeated charge and discharge actions performed by the storage unit through updating a value matrix. We design a reward function that does not only reflect the instant profit of charge/discharge decisions but also incorporate the history information. Simulation results demonstrate that our designed reward function leads to significant performance improvement compared with existing algorithms.

eess.SY↗

arXiv2020

Blockchain-Empowered Socially Optimal Transactive Energy System: Framework and Implementation

Transactive energy plays a key role in the operation and energy management of future power systems. However, the conventional operational mechanism, which follows a centralized design, is often less secure, vulnerable to malicious behaviors, and suffers from privacy leakage. In this work, we introduce blockchain technology in transactive energy to address these challenges. Specifically, we develop a novel blockchain-based transactive energy framework for prosumers and design a decentralized energy trading algorithm that matches the operation of the underlying blockchain system. We prove that the trading algorithm improves the individual benefit and guarantees the socially optimal performance, and thus incentivizes prosumers to join the transactive energy platform. Moreover, we evaluate the feasibility of the transactive energy platform throughout the implementation of a small-scale network of Internet of Things (IoT) devices and extensive simulations using real-world data. Our results show that this blockchain-based transactive energy platform is feasible in practice, and the decentralized trading algorithm reduces the user's individual cost by up to 77% and lowers the overall cost by 24%.

eess.SY↗

arXiv2020

A High Accuracy Electrical Stopping Power Prediction Model based on Deep Learning Algorithm and its Applications

Energy loss of energetic ions in solid is crucial in many field, and accurate prediction of the ion stopping power is a long-time goal. Though great efforts have been made, it is still very difficult to find a universal prediction model to accurately calculate the ion stopping power in distinct target materials. Deep learning algorithm is a newly emerged method to solve multi-factors physical problems and can mine the deeply implicit relations among parameters, which make it a powerful tool in energy loss prediction. In this work, we developed an energy loss prediction model based on deep learning. When experimental data are available, our model can give predictions with an average absolute difference close to 5.7%, which is in the same level compared with other widely used programs e.g. SRIM. In the regime without experimental data, our model still can maintain a high performance, and has higher reliability compared with the existing models. The ion range of Au ions in SiC can be calculated with a relative error of 0.6~25% for ions in the energy range of 700~10'000 keV, which is much better than the results calculated by SRIM. Moreover, our model support the reciprocity conjecture of ion stopping power in solid proposed by P. Sigmund, which has been known for a long time but can hardly been proved by any of the existing stopping power models. This high-accuracy energy loss prediction model is very important for the research of ion-solid interaction mechanism and enormous relevant applications of energetic ions, such as in semiconductor fabrications, nuclear energy systems and the space facilities.

physics.app-ph↗

arXiv2020

COSEA: Convolutional Code Search with Layer-wise Attention

Semantic code search, which aims to retrieve code snippets relevant to a given natural language query, has attracted many research efforts with the purpose of accelerating software development. The huge amount of online publicly available code repositories has prompted the employment of deep learning techniques to build state-of-the-art code search models. Particularly, they leverage deep neural networks to embed codes and queries into a unified semantic vector space and then use the similarity between code's and query's vectors to approximate the semantic correlation between code and the query. However, most existing studies overlook the code's intrinsic structural logic, which indeed contains a wealth of semantic information, and fails to capture intrinsic features of codes. In this paper, we propose a new deep learning architecture, COSEA, which leverages convolutional neural networks with layer-wise attention to capture the valuable code's intrinsic structural logic. To further increase the learning efficiency of COSEA, we propose a variant of contrastive loss for training the code search model, where the ground-truth code should be distinguished from the most similar negative sample. We have implemented a prototype of COSEA. Extensive experiments over existing public datasets of Python and SQL have demonstrated that COSEA can achieve significant improvements over state-of-the-art methods on code search tasks.

cs.SE↗

arXiv2020

Deep-learning interatomic potential for irradiation damage simulations in MoS2 with ab initial accuracy

Potentials that could accurately describe the irradiation damage processes are highly desired to figure out the atomic-level response of various newly-discovered materials under irradiation environments. In this work, we introduce a deep-learning interatomic potential for monolayer MoS2 by combining all-electron calculations, an active-learning sampling method and a hybrid deep-learning model. This potential could not only give an overall good performance on the predictions of near-equilibrium material properties including lattice constants, elastic coefficients, energy stress curves, phonon spectra, defect formation energy and displacement threshold, but also reproduce the ab initial irradiation damage processes with high quality. Further irradiation simulations indicate that one single highenergy ion could generate a large nanopore with a diameter of more than 2 nm, or a series of multiple nanopores, which is qualitatively verified by the subsequent 500 keV Au+ ion irradiation experiments. This work provides a promising and feasible approach to simulate irradiation effects in enormous newly-discovered materials with unprecedented accuracy.

cond-mat.mtrl-sci↗

arXiv2020

Towards Natural Bilingual and Code-Switched Speech Synthesis Based on Mix of Monolingual Recordings and Cross-Lingual Voice Conversion

Recent state-of-the-art neural text-to-speech (TTS) synthesis models have dramatically improved intelligibility and naturalness of generated speech from text. However, building a good bilingual or code-switched TTS for a particular voice is still a challenge. The main reason is that it is not easy to obtain a bilingual corpus from a speaker who achieves native-level fluency in both languages. In this paper, we explore the use of Mandarin speech recordings from a Mandarin speaker, and English speech recordings from another English speaker to build high-quality bilingual and code-switched TTS for both speakers. A Tacotron2-based cross-lingual voice conversion system is employed to generate the Mandarin speaker's English speech and the English speaker's Mandarin speech, which show good naturalness and speaker similarity. The obtained bilingual data are then augmented with code-switched utterances synthesized using a Transformer model. With these data, three neural TTS models -- Tacotron2, Transformer and FastSpeech are applied for building bilingual and code-switched TTS. Subjective evaluation results show that all the three systems can produce (near-)native-level speech in both languages for each of the speaker.

cs.SD↗

arXiv2020

Remote Sensing Image Scene Classification with Self-Supervised Paradigm under Limited Labeled Samples

With the development of deep learning, supervised learning methods perform well in remote sensing images (RSIs) scene classification. However, supervised learning requires a huge number of annotated data for training. When labeled samples are not sufficient, the most common solution is to fine-tune the pre-training models using a large natural image dataset (e.g. ImageNet). However, this learning paradigm is not a panacea, especially when the target remote sensing images (e.g. multispectral and hyperspectral data) have different imaging mechanisms from RGB natural images. To solve this problem, we introduce new self-supervised learning (SSL) mechanism to obtain the high-performance pre-training model for RSIs scene classification from large unlabeled data. Experiments on three commonly used RSIs scene classification datasets demonstrated that this new learning paradigm outperforms the traditional dominant ImageNet pre-trained model. Moreover, we analyze the impacts of several factors in SSL on RSIs scene classification tasks, including the choice of self-supervised signals, the domain difference between the source and target dataset, and the amount of pre-training data. The insights distilled from our studies can help to foster the development of SSL in the remote sensing community. Since SSL could learn from unlabeled massive RSIs which are extremely easy to obtain, it will be a potentially promising way to alleviate dependence on labeled samples and thus efficiently solve many problems, such as global mapping.

cs.CV↗

arXiv2020

Contract-based Time-of-use Pricing for Energy Storage Investment

Time-of-use (ToU) pricing is widely used by the electricity utility. A carefully designed ToU pricing can incentivize end-users' energy storage deployment, which helps shave the system peak load and reduce the system social cost. However, the optimization of ToU pricing is highly non-trivial, and an improperly designed ToU pricing may lead to storage investments that are far from the social optimum. In this paper, we aim at designing the optimal ToU pricing, jointly considering the social cost of the utility and the storage investment decisions of users. Since the storage investment costs are users' private information, we design low-complexity contracts to elicit the necessary information and induce the proper behavior of users' storage investment. The proposed contracts only specify three contract items, which guides users of arbitrarily many different storage-cost types to invest in full, partial, or no storage capacity with respect to their peak demands. Our contracts can achieve the social optimum when the utility knows the aggregate demand of each storage-cost type (but not the individual user's type). When the utility only knows the distribution of each storage-cost type's demand, our contracts can lead to a near-optimal solution. The gap with the social optimum is as small as 1.5% based on the simulations using realistic data. We also show that the proposed contracts can reduce the system social cost by over 30%, compared with no storage investment benchmark.

eess.SY↗

arXiv2020

Classical nucleation theory of ice nucleation: second-order correction of thermodynamic parameters

Accurate estimate of nucleation rate is crucial for the study of ice nucleation and ice-promoting/anti-freeze strategies. Within the framework of Classical Nucleation Theory (CNT), the estimate of ice nucleation rate is very sensitive to thermodynamic parameters, such as chemical potential difference between water and ice $Δμ$ and ice-water interfacial free energy $γ$. However, even today, there are still many contradictions and approximations in the estimating of these thermodynamic parameters, introducing large uncertainty to the estimate of the ice nucleation rate. Herein, starting from the basic concepts, for a general solid-liquid crystallization system, we expand the Gibbs-Thomson (GT) equation to second order, and derive the second-order analytical formulas of $Δμ$, $γ$ and nucleation barrier $ΔG$ with combining molecular dynamics (MD) simulations. These formulas describe well the temperature dependence of these thermodynamic parameters. Our results can provide a method of estimating $Δμ$, $γ$ and $ΔG$.

cond-mat.soft↗

arXiv2020

Does "Fans Economy" Work for Chinese Pop Music Industry?

China has become one of the largest entertainment markets in the world in recent years. Due to the success of Xiaomi, many Chinese pop music industry entrepreneurs believe "Fans Economy" works in the pop music industry. "Fans Economy" is based on the assumption that pop music consumer market could be segmented based on artists. Each music artist has its own exclusive loyal fans. In this paper, we provide an insightful study of the pop music artists and fans social network. Particularly, we segment the pop music consumer market and pop music artists respectively. Our results show that due to the Matthew Effect and limited diversity of consumer market, "Fans Economy" does not work for the Chinese pop music industry.

cs.SI↗

arXiv2020

Blockchain-based Privacy Preservation for 5G-enabled Drone Communications

5G-enabled drones have potential applications in a variety of both military and civilian settings (e.g., monitoring and tracking of individuals in demonstrations and/or enforcing of social / physical distancing during pandemics such as COVID-19). Such applications generally involve the collection and dissemination of (massive) data from the drones to remote data centres for storage and analysis, for example via 5G networks. Consequently, there are security and privacy considerations underpinning 5G-enabled drone communications. We posit the potential of leveraging blockchain to facilitate privacy preservation, and therefore in this article we will review existing blockchain-based solutions after introducing the architecture for 5G-enabled drone communications and blockchain. We will also review existing legislation and data privacy regulations that need to be considered in the design of blockchain-based solutions, as well as identifying potential challenges and open issues which will hopefully inform future research agenda.

cs.NI↗

arXiv2020

Detection Defense Against Adversarial Attacks with Saliency Map

It is well established that neural networks are vulnerable to adversarial examples, which are almost imperceptible on human vision and can cause the deep models misbehave. Such phenomenon may lead to severely inestimable consequences in the safety and security critical applications. Existing defenses are trend to harden the robustness of models against adversarial attacks, e.g., adversarial training technology. However, these are usually intractable to implement due to the high cost of re-training and the cumbersome operations of altering the model architecture or parameters. In this paper, we discuss the saliency map method from the view of enhancing model interpretability, it is similar to introducing the mechanism of the attention to the model, so as to comprehend the progress of object identification by the deep networks. We then propose a novel method combined with additional noises and utilize the inconsistency strategy to detect adversarial examples. Our experimental results of some representative adversarial attacks on common datasets including ImageNet and popular models show that our method can detect all the attacks with high detection success rate effectively. We compare it with the existing state-of-the-art technique, and the experiments indicate that our method is more general.

cs.LG↗

arXiv2020

Structure-Aware Generation Network for Recipe Generation from Images

Sharing food has become very popular with the development of social media. For many real-world applications, people are keen to know the underlying recipes of a food item. In this paper, we are interested in automatically generating cooking instructions for food. We investigate an open research task of generating cooking instructions based on only food images and ingredients, which is similar to the image captioning task. However, compared with image captioning datasets, the target recipes are long-length paragraphs and do not have annotations on structure information. To address the above limitations, we propose a novel framework of Structure-aware Generation Network (SGN) to tackle the food recipe generation task. Our approach brings together several novel ideas in a systematic framework: (1) exploiting an unsupervised learning approach to obtain the sentence-level tree structure labels before training; (2) generating trees of target recipes from images with the supervision of tree structure labels learned from (1); and (3) integrating the inferred tree structures with the recipe generation procedure. Our proposed model can produce high-quality and coherent recipes, and achieve the state-of-the-art performance on the benchmark Recipe1M dataset.

cs.CV↗

arXiv2020

Continuously Indexed Domain Adaptation

Existing domain adaptation focuses on transferring knowledge between domains with categorical indices (e.g., between datasets A and B). However, many tasks involve continuously indexed domains. For example, in medical applications, one often needs to transfer disease analysis and prediction across patients of different ages, where age acts as a continuous domain index. Such tasks are challenging for prior domain adaptation methods since they ignore the underlying relation among domains. In this paper, we propose the first method for continuously indexed domain adaptation. Our approach combines traditional adversarial adaptation with a novel discriminator that models the encoding-conditioned domain index distribution. Our theoretical analysis demonstrates the value of leveraging the domain index to generate invariant features across a continuous range of domains. Our empirical results show that our approach outperforms the state-of-the-art domain adaption methods on both synthetic and real-world medical datasets.

cs.LG↗

arXiv2020

Liquid crystal phases with unusual structures and physical properties formed by acute-angle bent-core molecules

Liquid crystals formed by acute-angle bent-core (ABC) molecules with a 1,7 naphthalene central core show an intriguing phase behavior with the nematic phase accompanied by poorly understood additional phases. In this work, we characterize the physical properties of an ABC material, such as birefringence, dielectric permittivities, elastic constants, and surface alignment, and present X-ray diffraction and transmission electron microscopy studies of their ordering. The ABC molecular shape resembling the letter $λ$ yields a very small splay elastic constant in the uniaxial nematic phase and results in the formation of a tetragonal positionally ordered columnar phase consisting of molecular columns with a uniform uniaxial director that can be bent but not splayed.

cond-mat.soft↗

arXiv2020

Domain-specific Communication Optimization for Distributed DNN Training

Communication overhead poses an important obstacle to distributed DNN training and draws increasing attention in recent years. Despite continuous efforts, prior solutions such as gradient compression/reduction, compute/communication overlapping and layer-wise flow scheduling, etc., are still coarse-grained and insufficient for an efficient distributed training especially when the network is under pressure. We present DLCP, a novel solution exploiting the domain-specific properties of deep learning to optimize communication overhead of DNN training in a fine-grained manner. At its heart, DLCP comprises of several key innovations beyond prior work: e.g., it exploits {\em bounded loss tolerance} of SGD-based training to improve tail communication latency which cannot be avoided purely through gradient compression. It then performs fine-grained packet-level prioritization and dropping, as opposed to flow-level scheduling, based on layers and magnitudes of gradients to further speedup model convergence without affecting accuracy. In addition, it leverages inter-packet order-independency to perform per-packet load balancing without causing classical re-ordering issues. DLCP works with both Parameter Server and collective communication routines. We have implemented DLCP with commodity switches, integrated it with various training frameworks including TensorFlow, MXNet and PyTorch, and deployed it in our small-scale testbed with 10 Nvidia V100 GPUs. Our testbed experiments and large-scale simulations show that DLCP delivers up to $84.3\%$ additional training acceleration over the best existing solutions.

cs.DC↗