arXiv subjects

Hao Zhu

Publications and source records attributed to Hao Zhu.

At least 235 records · Page 13Linked to original sources

arXiv2021

Dependency Induction Through the Lens of Visual Perception

Most previous work on grammar induction focuses on learning phrasal or dependency structure purely from text. However, because the signal provided by text alone is limited, recently introduced visually grounded syntax models make use of multimodal information leading to improved performance in constituency grammar induction. However, as compared to dependency grammars, constituency grammars do not provide a straightforward way to incorporate visual information without enforcing language-specific heuristics. In this paper, we propose an unsupervised grammar induction model that leverages word concreteness and a structural vision-based heuristic to jointly learn constituency-structure and dependency-structure grammars. Our experiments find that concreteness is a strong indicator for learning dependency grammars, improving the direct attachment score (DAS) by over 50\% as compared to state-of-the-art models trained on pure text. Next, we propose an extension of our model that leverages both word concreteness and visual semantic role labels in constituency and dependency parsing. Our experiments show that the proposed extension outperforms the current state-of-the-art visually grounded models in constituency parsing even with a smaller grammar size.

cs.CL↗

arXiv2021

Why Existing Machine Learning Methods Fails At Extracting the Information of Future Returns Out of Historical Sctock Prices : the Curve-Shape-Feature and Non-Curve-Shape-Feature Modes

The financial time series analysis is important access to touch the complex laws of financial markets. Among many goals of the financial time series analysis, one is to construct a model that can extract the information of the future return out of the known historical stock data, such as stock price, financial news, and e.t.c. To design such a model, prior knowledge on how the future return is correlated with the historical stock prices is needed. In this work, we focus on the issue: in what mode the future return is correlated with the historical stock prices. We manually design several financial time series where the future return is correlated with the historical stock prices in pre-designed modes, namely the curve-shape-feature (CSF) and the non-curve-shape-feature (NCSF) modes. In the CSF mode, the future return can be extracted from the curve shapes of the historical stock prices. By applying various kinds of existing algorithms on those pre-designed time series and real financial time series, we show that: (1) the major information of the future return is not contained in the curve-shape features of historical stock prices. That is, the future return is not mainly correlated with the historical stock prices in the CSF mode. (2) Various kinds of existing machine learning algorithms are good at extracting the curveshape features in the historical stock prices and thus are inappropriate for financial time series analysis although they are successful in the image recognition and natural language processing. That is, new models handling the NCSF series are needed in the financial time series analysis.

cs.CE↗

arXiv2021

REFINE: Random RangE FInder for Network Embedding

Network embedding approaches have recently attracted considerable interest as they learn low-dimensional vector representations of nodes. Embeddings based on the matrix factorization are effective but they are usually computationally expensive due to the eigen-decomposition step. In this paper, we propose a Random RangE FInder based Network Embedding (REFINE) algorithm, which can perform embedding on one million of nodes (YouTube) within 30 seconds in a single thread. REFINE is 10x faster than ProNE, which is 10-400x faster than other methods such as LINE, DeepWalk, Node2Vec, GraRep, and Hope. Firstly, we formulate our network embedding approach as a skip-gram model, but with an orthogonal constraint, and we reformulate it into the matrix factorization problem. Instead of using randomized tSVD (truncated SVD) as other methods, we employ the Randomized Blocked QR decomposition to obtain the node representation fast. Moreover, we design a simple but efficient spectral filter for network enhancement to obtain higher-order information for node representation. Experimental results prove that REFINE is very efficient on datasets of different sizes (from thousand to million of nodes/edges) for node classification, while enjoying a good performance.

cs.LG↗

arXiv2021

Enhancing the Spatio-temporal Observability of Grid-Edge Resources in Distribution Grids

Enhancing the spatio-temporal observability of distributed energy resources (DERs) is crucial for achieving secure and efficient operations in distribution grids. This paper puts forth a joint recovery framework for residential loads by leveraging the complimentary strengths of heterogeneous measurements in real time. The proposed framework integrates low-resolution smart meter data collected at every load node with fast-sampled feeder-level measurements from limited number of distribution phasor measurement units. To address the lack of data, we exploit two key characteristics for the loads and DERs, namely the sparse changes due to infrequent activities of appliances and electric vehicles (EVs) and the locational dependence of solar photovoltaic (PV) generation. Accordingly, meaningful regularization terms are introduced to cast a convex load recovery problem, which will be further simplified to reduce the computational complexity. The load recovery solutions can be utilized to identify the EV charging events at each load node and to infer the total behind-the-meter PV output. Numerical tests using real-world data have demonstrated the effectiveness of the proposed approaches in enhancing the visibility of grid-edge DERs.

eess.SP↗

arXiv2021

CIGLI: Conditional Image Generation from Language & Image

Multi-modal generation has been widely explored in recent years. Current research directions involve generating text based on an image or vice versa. In this paper, we propose a new task called CIGLI: Conditional Image Generation from Language and Image. Instead of generating an image based on text as in text-image generation, this task requires the generation of an image from a textual description and an image prompt. We designed a new dataset to ensure that the text description describes information from both images, and that solely analyzing the description is insufficient to generate an image. We then propose a novel language-image fusion model which improves the performance over two established baseline methods, as evaluated by quantitative (automatic) and qualitative (human) evaluations. The code and dataset is available at https://github.com/vincentlux/CIGLI.

cs.CV↗

arXiv2021

Detailed Avatar Recovery from Single Image

This paper presents a novel framework to recover \emph{detailed} avatar from a single image. It is a challenging task due to factors such as variations in human shapes, body poses, texture, and viewpoints. Prior methods typically attempt to recover the human body shape using a parametric-based template that lacks the surface details. As such resulting body shape appears to be without clothing. In this paper, we propose a novel learning-based framework that combines the robustness of the parametric model with the flexibility of free-form 3D deformation. We use the deep neural networks to refine the 3D shape in a Hierarchical Mesh Deformation (HMD) framework, utilizing the constraints from body joints, silhouettes, and per-pixel shading information. Our method can restore detailed human body shapes with complete textures beyond skinned models. Experiments demonstrate that our method has outperformed previous state-of-the-art approaches, achieving better accuracy in terms of both 2D IoU number and 3D metric distance.

cs.CV↗

arXiv2021

Graph Convolutional Network with Generalized Factorized Bilinear Aggregation

Although Graph Convolutional Networks (GCNs) have demonstrated their power in various applications, the graph convolutional layers, as the most important component of GCN, are still using linear transformations and a simple pooling step. In this paper, we propose a novel generalization of Factorized Bilinear (FB) layer to model the feature interactions in GCNs. FB performs two matrix-vector multiplications, that is, the weight matrix is multiplied with the outer product of the vector of hidden features from both sides. However, the FB layer suffers from the quadratic number of coefficients, overfitting and the spurious correlations due to correlations between channels of hidden representations that violate the i.i.d. assumption. Thus, we propose a compact FB layer by defining a family of summarizing operators applied over the quadratic term. We analyze proposed pooling operators and motivate their use. Our experimental results on multiple datasets demonstrate that the GFB-GCN is competitive with other methods for text classification.

cs.LG↗

arXiv2021

Few-shot Language Coordination by Modeling Theory of Mind

$\textit{No man is an island.}$ Humans communicate with a large community by coordinating with different interlocutors within short conversations. This ability has been understudied by the research on building neural communicative agents. We study the task of few-shot $\textit{language coordination}$: agents quickly adapting to their conversational partners' language abilities. Different from current communicative agents trained with self-play, we require the lead agent to coordinate with a $\textit{population}$ of agents with different linguistic abilities, quickly adapting to communicate with unseen agents in the population. This requires the ability to model the partner's beliefs, a vital component of human communication. Drawing inspiration from theory-of-mind (ToM; Premack& Woodruff (1978)), we study the effect of the speaker explicitly modeling the listeners' mental states. The speakers, as shown in our experiments, acquire the ability to predict the reactions of their partner, which helps it generate instructions that concisely express its communicative goal. We examine our hypothesis that the instructions generated with ToM modeling yield better communication performance in both a referential game and a language navigation task. Positive results from our experiments hint at the importance of explicitly modeling communication as a socio-pragmatic progress.

cs.CL↗

arXiv2021

Graph Neural Networks for Learning Real-Time Prices in Electricity Market

Solving the optimal power flow (OPF) problem in real-time electricity market improves the efficiency and reliability in the integration of low-carbon energy resources into the power grids. To address the scalability and adaptivity issues of existing end-to-end OPF learning solutions, we propose a new graph neural network (GNN) framework for predicting the electricity market prices from solving OPFs. The proposed GNN-for-OPF framework innovatively exploits the locality property of prices and introduces physics-aware regularization, while attaining reduced model complexity and fast adaptivity to varying grid topology. Numerical tests have validated the learning efficiency and adaptivity improvements of our proposed method over existing approaches.

cs.LG↗

arXiv2021

Substation-Level Grid Topology Optimization Using Bus Splitting

Operations of substation circuit breakers are important for maintenance needs and topology reconfiguration in power systems. Bus splitting is one type of topology change where the two bus bars at a substation can become electrically disconnected under certain actions of circuit breakers. Because these events involve detailed substation modeling, they are typically not considered in routine power system operation and control. In this paper, an improved substation-level topology optimization framework is developed by expanding traditional line switching decisions by breaker-level bus splitting, which can further reduce grid congestion and generation costs. A tight McCormick relaxation is proposed to reformulate the bilinear terms in the resultant optimization problem to linear inequality constraints. Thus, a tractable mixed-integer linear program reformulation is attained that allows for efficient solutions in real-time operations. Numerical studies on the IEEE 14-bus and 118-bus systems demonstrate the computational performance and economic benefits of the proposed topology optimization approach.

math.OC↗

arXiv2021

Efficient Identification of Bus Split Events Using Synchrophasor Data

Accurate grid topology information is of paramount importance for routine power system operations, while the growing availability of synchrophasor data offers the opportunity to identify topology changes in real time. Identification of bus split events, where the substation becomes electrically disconnected, is becoming increasingly important for maintaining the security of power systems. This paper aims to provide an efficient modeling and monitoring framework for bus split events by using a concise bus-branch representation. The linear sensitivity analysis is first performed to quickly evaluate the grid-wide impact of such events. Furthermore, the synchrophasor data enabled identification problem is formulated by matching the changes in bus phase angles (and possibly line flows). To address the resultant bilinear multiplication involving the binary connectivity variables, the McCormick relaxation technique is leveraged to attain an equivalent mixed-integer linear program reformulation that is efficiently solvable. Numerical studies on the IEEE 14-bus and 300-bus systems demonstrate the validity and efficiency of the proposed identification algorithm towards real-time implementation.

math.OC↗

arXiv2020

Achieving Optimal Output Consensus for Discrete-time Linear Multi-agent Systems with Disturbance Rejection

In this paper, an optimal output consensus problem is studied for discrete-time linear multiagent systems subject to external disturbances. Each agent is assigned with a local cost function which is known only to itself. Distributed protocols are to be designed to guarantee an output consensus for these high-order agents and meanwhile minimize the aggregate cost as the sum of these local costs. To overcome the difficulties brought by high-order dynamics and external disturbances, we develop an embedded design and constructively present a distributed rule to solve this problem. The proposed control includes three terms: an optimal signal generator under a directed information graph, an observer-based compensator to reject these disturbances, and a reference tracking controller for these linear agents. It is shown to solve the formulated problem with some mild assumptions. A numerical example is also provided to illustrate the effectiveness of our proposed distributed control laws.

eess.SY↗

arXiv2020

Imbalance Robust Softmax for Deep Embeeding Learning

Deep embedding learning is expected to learn a metric space in which features have smaller maximal intra-class distance than minimal inter-class distance. In recent years, one research focus is to solve the open-set problem by discriminative deep embedding learning in the field of face recognition (FR) and person re-identification (re-ID). Apart from open-set problem, we find that imbalanced training data is another main factor causing the performance degradation of FR and re-ID, and data imbalance widely exists in the real applications. However, very little research explores why and how data imbalance influences the performance of FR and re-ID with softmax or its variants. In this work, we deeply investigate data imbalance in the perspective of neural network optimisation and feature distribution about softmax. We find one main reason of performance degradation caused by data imbalance is that the weights (from the penultimate fully-connected layer) are far from their class centers in feature space. Based on this investigation, we propose a unified framework, Imbalance-Robust Softmax (IR-Softmax), which can simultaneously solve the open-set problem and reduce the influence of data imbalance. IR-Softmax can generalise to any softmax and its variants (which are discriminative for open-set problem) by directly setting the weights as their class centers, naturally solving the data imbalance problem. In this work, we explicitly re-formulate two discriminative softmax (A-Softmax and AM-Softmax) under the framework of IR-Softmax. We conduct extensive experiments on FR databases (LFW, MegaFace) and re-ID database (Market-1501, Duke), and IR-Softmax outperforms many state-of-the-art methods.

cs.CV↗

arXiv2020

EnsemFDet: An Ensemble Approach to Fraud Detection based on Bipartite Graph

Fraud detection is extremely critical for e-commerce business. It is the intent of the companies to detect and prevent fraud as early as possible. Existing fraud detection methods try to identify unexpected dense subgraphs and treat related nodes as suspicious. Spectral relaxation-based methods solve the problem efficiently but hurt the performance due to the relaxed constraints. Besides, many methods cannot be accelerated with parallel computation or control the number of returned suspicious nodes because they provide a set of subgraphs with diverse node sizes. These drawbacks affect the real-world applications of existing methods. In this paper, we propose an Ensemble-based Fraud Detection (EnsemFDet) method to scale up fraud detection in bipartite graphs by decomposing the original problem into subproblems on small-sized subgraphs. By oversampling the graph and solving the subproblems, the ensemble approach further votes suspicious nodes without sacrificing the prediction accuracy. Extensive experiments have been done on real transaction data from JD.com, which is one of the world's largest e-commerce platforms. Experimental results demonstrate the effectiveness, practicability, and scalability of EnsemFDet. More specifically, EnsemFDet is up to 100x faster than the state-of-the-art methods due to its parallelism with all aspects of data.

cs.LG↗

arXiv2020

Lets Play Music: Audio-driven Performance Video Generation

We propose a new task named Audio-driven Per-formance Video Generation (APVG), which aims to synthesizethe video of a person playing a certain instrument guided bya given music audio clip. It is a challenging task to gener-ate the high-dimensional temporal consistent videos from low-dimensional audio modality. In this paper, we propose a multi-staged framework to achieve this new task to generate realisticand synchronized performance video from given music. Firstly,we provide both global appearance and local spatial informationby generating the coarse videos and keypoints of body and handsfrom a given music respectively. Then, we propose to transformthe generated keypoints to heatmap via a differentiable spacetransformer, since the heatmap offers more spatial informationbut is harder to generate directly from audio. Finally, wepropose a Structured Temporal UNet (STU) to extract bothintra-frame structured information and inter-frame temporalconsistency. They are obtained via graph-based structure module,and CNN-GRU based high-level temporal module respectively forfinal video generation. Comprehensive experiments validate theeffectiveness of our proposed framework.

cs.CV↗

arXiv2020

AOT: Appearance Optimal Transport Based Identity Swapping for Forgery Detection

Recent studies have shown that the performance of forgery detection can be improved with diverse and challenging Deepfakes datasets. However, due to the lack of Deepfakes datasets with large variance in appearance, which can be hardly produced by recent identity swapping methods, the detection algorithm may fail in this situation. In this work, we provide a new identity swapping algorithm with large differences in appearance for face forgery detection. The appearance gaps mainly arise from the large discrepancies in illuminations and skin colors that widely exist in real-world scenarios. However, due to the difficulties of modeling the complex appearance mapping, it is challenging to transfer fine-grained appearances adaptively while preserving identity traits. This paper formulates appearance mapping as an optimal transport problem and proposes an Appearance Optimal Transport model (AOT) to formulate it in both latent and pixel space. Specifically, a relighting generator is designed to simulate the optimal transport plan. It is solved via minimizing the Wasserstein distance of the learned features in the latent space, enabling better performance and less computation than conventional optimization. To further refine the solution of the optimal transport plan, we develop a segmentation game to minimize the Wasserstein distance in the pixel space. A discriminator is introduced to distinguish the fake parts from a mix of real and fake image patches. Extensive experiments reveal that the superiority of our method when compared with state-of-the-art methods and the ability of our generated data to improve the performance of face forgery detection.

cs.CV↗

arXiv2020

Multi-agent Trajectory Prediction with Fuzzy Query Attention

Trajectory prediction for scenes with multiple agents and entities is a challenging problem in numerous domains such as traffic prediction, pedestrian tracking and path planning. We present a general architecture to address this challenge which models the crucial inductive biases of motion, namely, inertia, relative motion, intents and interactions. Specifically, we propose a relational model to flexibly model interactions between agents in diverse environments. Since it is well-known that human decision making is fuzzy by nature, at the core of our model lies a novel attention mechanism which models interactions by making continuous-valued (fuzzy) decisions and learning the corresponding responses. Our architecture demonstrates significant performance gains over existing state-of-the-art predictive models in diverse domains such as human crowd trajectories, US freeway traffic, NBA sports data and physics datasets. We also present ablations and augmentations to understand the decision-making process and the source of gains in our model.

cs.LG↗

arXiv2020

Using Graph Neural Networks for Mass Spectrometry Prediction

Detecting and quantifying products of cellular metabolism using Mass Spectrometry (MS) has already shown great promise in many biological and biomedical applications. The biggest challenge in metabolomics is annotation, where measured spectra are assigned chemical identities. Despite advances, current methods provide limited annotation for measured spectra. Here, we explore using graph neural networks (GNNs) to predict the spectra. The input to our model is a molecular graph. The model is trained and tested on the NIST 17 LC-MS dataset. We compare our results to NEIMS, a neural network model that utilizes molecular fingerprints as inputs. Our results show that GNN-based models offer higher performance than NEIMS. Importantly, we show that ranking results heavily depend on the candidate set size and on the similarity of the candidates to the target molecule, thus highlighting the need for consistent, well-characterized evaluation protocols for this domain.

cs.LG↗