arXiv subjects

Ting Chen

Publications and source records attributed to Ting Chen.

At least 163 records · Page 9Linked to original sources

arXiv2019

Unsupervised Inductive Graph-Level Representation Learning via Graph-Graph Proximity

We introduce a novel approach to graph-level representation learning, which is to embed an entire graph into a vector space where the embeddings of two graphs preserve their graph-graph proximity. Our approach, UGRAPHEMB, is a general framework that provides a novel means to performing graph-level embedding in a completely unsupervised and inductive manner. The learned neural network can be considered as a function that receives any graph as input, either seen or unseen in the training set, and transforms it into an embedding. A novel graph-level embedding generation mechanism called Multi-Scale Node Attention (MSNA), is proposed. Experiments on five real graph datasets show that UGRAPHEMB achieves competitive accuracy in the tasks of graph classification, similarity ranking, and graph visualization.

cs.LG↗

arXiv2019

Pre-Training Graph Neural Networks for Generic Structural Feature Extraction

Graph neural networks (GNNs) are shown to be successful in modeling applications with graph structures. However, training an accurate GNN model requires a large collection of labeled data and expressive features, which might be inaccessible for some applications. To tackle this problem, we propose a pre-training framework that captures generic graph structural information that is transferable across tasks. Our framework can leverage the following three tasks: 1) denoising link reconstruction, 2) centrality score ranking, and 3) cluster preserving. The pre-training procedure can be conducted purely on the synthetic graphs, and the pre-trained GNN is then adapted for downstream applications. With the proposed pre-training procedure, the generic structural information is learned and preserved, thus the pre-trained GNN requires less amount of labeled data and fewer domain-specific features to achieve high performance on different downstream tasks. Comprehensive experiments demonstrate that our proposed framework can significantly enhance the performance of various tasks at the level of node, link, and graph.

cs.LG↗

arXiv2019

An Optimal Game Approach for Heterogeneous Vehicular Network Selection with Varying Network Performance

Most conventional heterogeneous network selection strategies applied in heterogeneous vehicular network regard the performance of each network constant in various traffic scenarios. This assumption leads such strategies to be ineffective in the real-world performance-changing scenarios. To solve this problem, we propose an optimal game approach for heterogeneous vehicular network selection under conditions in which the performance parameters of some networks are changing. Terminals attempting to switch to the network with higher evaluation is formulated as a multi-play non-cooperative game. Heterogeneous vehicular network characteristics are thoroughly accounted for to adjust the game strategy and adapt to the vehicular environment for stability and rapid convergence. A multi-play non-cooperative game model is built to formulate network selection. A probabilistic strategy is used to gradually drive players toward convergence to prevent instability. Furthermore, a system prototype was built at the Connected and Automated Vehicle Test bed of Chang'an University (CAVTest). Its corresponding test results indicate that the proposed approach can effectively suppress the ping-pong effect caused by massive handoffs due to varying network performance and thus well outperforms the single-play strategy.

cs.NI↗

arXiv2019

On Self Modulation for Generative Adversarial Networks

Training Generative Adversarial Networks (GANs) is notoriously challenging. We propose and study an architectural modification, self-modulation, which improves GAN performance across different data sets, architectures, losses, regularizers, and hyperparameter settings. Intuitively, self-modulation allows the intermediate feature maps of a generator to change as a function of the input noise vector. While reminiscent of other conditioning techniques, it requires no labeled data. In a large-scale empirical study we observe a relative decrease of $5\%-35\%$ in FID. Furthermore, all else being equal, adding this modification to the generator leads to improved performance in $124/144$ ($86\%$) of the studied settings. Self-modulation is a simple architectural change that requires no additional parameter tuning, which suggests that it can be applied readily to any GAN.

cs.LG↗

arXiv2019

Self-Supervised GANs via Auxiliary Rotation Loss

Conditional GANs are at the forefront of natural image synthesis. The main drawback of such models is the necessity for labeled data. In this work we exploit two popular unsupervised learning techniques, adversarial training and self-supervision, and take a step towards bridging the gap between conditional and unconditional GANs. In particular, we allow the networks to collaborate on the task of representation learning, while being adversarial with respect to the classic GAN game. The role of self-supervision is to encourage the discriminator to learn meaningful feature representations which are not forgotten during training. We test empirically both the quality of the learned image representations, and the quality of the synthesized images. Under the same conditions, the self-supervised GAN attains a similar performance to state-of-the-art conditional counterparts. Finally, we show that this approach to fully unsupervised learning can be scaled to attain an FID of 23.4 on unconditional ImageNet generation.

cs.LG↗

arXiv2018

Self-Supervised GAN to Counter Forgetting

GANs involve training two networks in an adversarial game, where each network's task depends on its adversary. Recently, several works have framed GAN training as an online or continual learning problem. We focus on the discriminator, which must perform classification under an (adversarially) shifting data distribution. When trained on sequential tasks, neural networks exhibit \emph{forgetting}. For GANs, discriminator forgetting leads to training instability. To counter forgetting, we encourage the discriminator to maintain useful representations by adding a self-supervision. Conditional GANs have a similar effect using labels. However, our self-supervised GAN does not require labels, and closes the performance gap between conditional and unconditional models. We show that, in doing so, the self-supervised discriminator learns better representations than regular GANs.

cs.LG↗

arXiv2018

Learning K-way D-dimensional Discrete Codes for Compact Embedding Representations

Conventional embedding methods directly associate each symbol with a continuous embedding vector, which is equivalent to applying a linear transformation based on a "one-hot" encoding of the discrete symbols. Despite its simplicity, such approach yields the number of parameters that grows linearly with the vocabulary size and can lead to overfitting. In this work, we propose a much more compact K-way D-dimensional discrete encoding scheme to replace the "one-hot" encoding. In the proposed "KD encoding", each symbol is represented by a $D$-dimensional code with a cardinality of $K$, and the final symbol embedding vector is generated by composing the code embedding vectors. To end-to-end learn semantically meaningful codes, we derive a relaxed discrete optimization approach based on stochastic gradient descent, which can be generally applied to any differentiable computational graph with an embedding layer. In our experiments with various applications from natural language processing to graph convolutional networks, the total size of the embedding layer can be reduced up to 98\% while achieving similar or better performance.

cs.LG↗

arXiv2018

Topological Band Engineering of Graphene Nanoribbons

Topological insulators (TIs) are an emerging class of materials that host highly robust in-gap surface/interface states while maintaining an insulating bulk. While most notable scientific advancements in this field have been focused on TIs and related topological crystalline insulators in 2D and 3D, more recent theoretical work has predicted the existence of 1D symmetry-protected topological phases in graphene nanoribbons (GNRs). The topological phase of these laterally-confined, semiconducting strips of graphene is determined by their width, edge shape, and the terminating unit cell, and is characterized by a Z2 invariant (similar to 1D solitonic systems). Interfaces between topologically distinct GNRs characterized by different Z2 are predicted to support half-filled in-gap localized electronic states which can, in principle, be utilized as a tool for material engineering. Here we present the rational design and experimental realization of a topologically-engineered GNR superlattice that hosts a 1D array of such states, thus generating otherwise inaccessible electronic structure. This strategy also enables new end states to be engineered directly into the termini of the 1D GNR superlattice. Atomically-precise topological GNR superlattices were synthesized from molecular precursors on a Au(111) surface under ultra-high vacuum (UHV) conditions and characterized by low temperature scanning tunneling microscopy (STM) and spectroscopy (STS). Our experimental results and first-principles calculations reveal that the frontier band structure of these GNR superlattices is defined purely by the coupling between adjacent topological interface states. This novel manifestation of 1D topological phases presents an entirely new route to band engineering in 1D materials based on precise control of their electronic topology, and is a promising platform for future studies of 1D quantum spin physics.

cond-mat.mes-hall↗

arXiv2018

HeteroMed: Heterogeneous Information Network for Medical Diagnosis

With the recent availability of Electronic Health Records (EHR) and great opportunities they offer for advancing medical informatics, there has been growing interest in mining EHR for improving quality of care. Disease diagnosis due to its sensitive nature, huge costs of error, and complexity has become an increasingly important focus of research in past years. Existing studies model EHR by capturing co-occurrence of clinical events to learn their latent embeddings. However, relations among clinical events carry various semantics and contribute differently to disease diagnosis which gives precedence to a more advanced modeling of heterogeneous data types and relations in EHR data than existing solutions. To address these issues, we represent how high-dimensional EHR data and its rich relationships can be suitably translated into HeteroMed, a heterogeneous information network for robust medical diagnosis. Our modeling approach allows for straightforward handling of missing values and heterogeneity of data. HeteroMed exploits metapaths to capture higher level and semantically important relations contributing to disease diagnosis. Furthermore, it employs a joint embedding framework to tailor clinical event representations to the disease diagnosis goal. To the best of our knowledge, this is the first study to use Heterogeneous Information Network for modeling clinical data and disease diagnosis. Experimental results of our study show superior performance of HeteroMed compared to prior methods in prediction of exact diagnosis codes and general disease cohorts. Moreover, HeteroMed outperforms baseline models in capturing similarities of clinical events which are examined qualitatively through case studies.

cs.AI↗

arXiv2018

Iterated and Mixed Weak Norms with Applications to Geometric Inequalities

In this paper, we consider a new weak norm, iterated weak norm in Lebesgue spaces with mixed norms. We study properties of the mixed weak norm and the iterated weak norm and present the relationship between the two weak norms. Even for the ordinary Lebesgue spaces, the two weak norms are not equivalent and any one of them can not control the other one. We give some convergence and completeness results for both weak norms. We study the convergence in truncated norm, which is a substitution of the convergence in measure for mixed Lebesgue spaces. And we give a characterization of the convergence in truncated norm. We show that Hölder's inequality is not always true on mixed weak spaces and we give a complete characterization of indices which admit Hölder's inequality. As applications, we establish some geometric inequalities related to fractional integration in mixed weak spaces and in iterated weak spaces which essentially generalize the Hardy-Littlewood-Sobolev inequality.

math.FA↗

arXiv2017

Learning K-way D-dimensional Discrete Code For Compact Embedding Representations

Embedding methods such as word embedding have become pillars for many applications containing discrete structures. Conventional embedding methods directly associate each symbol with a continuous embedding vector, which is equivalent to applying linear transformation based on "one-hot" encoding of the discrete symbols. Despite its simplicity, such approach yields number of parameters that grows linearly with the vocabulary size and can lead to overfitting. In this work we propose a much more compact K-way D-dimensional discrete encoding scheme to replace the "one-hot" encoding. In "KD encoding", each symbol is represented by a $D$-dimensional code, and each of its dimension has a cardinality of $K$. The final symbol embedding vector can be generated by composing the code embedding vectors. To learn the semantically meaningful code, we derive a relaxed discrete optimization technique based on stochastic gradient descent. By adopting the new coding system, the efficiency of parameterization can be significantly improved (from linear to logarithmic), and this can also mitigate the over-fitting problem. In our experiments with language modeling, the number of embedding parameters can be reduced by 97\% while achieving similar or better performance.

cs.LG↗

arXiv2017

Joint Text Embedding for Personalized Content-based Recommendation

Learning a good representation of text is key to many recommendation applications. Examples include news recommendation where texts to be recommended are constantly published everyday. However, most existing recommendation techniques, such as matrix factorization based methods, mainly rely on interaction histories to learn representations of items. While latent factors of items can be learned effectively from user interaction data, in many cases, such data is not available, especially for newly emerged items. In this work, we aim to address the problem of personalized recommendation for completely new items with text information available. We cast the problem as a personalized text ranking problem and propose a general framework that combines text embedding with personalized recommendation. Users and textual content are embedded into latent feature space. The text embedding function can be learned end-to-end by predicting user interactions with items. To alleviate sparsity in interaction data, and leverage large amount of text data with little or no user interactions, we further propose a joint text embedding model that incorporates unsupervised text embedding with a combination module. Experimental results show that our model can significantly improve the effectiveness of recommendation systems on real-world datasets.

cs.IR↗

arXiv2017

On Sampling Strategies for Neural Network-based Collaborative Filtering

Recent advances in neural networks have inspired people to design hybrid recommendation algorithms that can incorporate both (1) user-item interaction information and (2) content information including image, audio, and text. Despite their promising results, neural network-based recommendation algorithms pose extensive computational costs, making it challenging to scale and improve upon. In this paper, we propose a general neural network-based recommendation framework, which subsumes several existing state-of-the-art recommendation algorithms, and address the efficiency issue by investigating sampling strategies in the stochastic gradient descent training for the framework. We tackle this issue by first establishing a connection between the loss functions and the user-item interaction bipartite graph, where the loss function terms are defined on links while major computation burdens are located at nodes. We call this type of loss functions "graph-based" loss functions, for which varied mini-batch sampling strategies can have different computational costs. Based on the insight, three novel sampling strategies are proposed, which can significantly improve the training efficiency of the proposed framework (up to $\times 30$ times speedup in our experiments), as well as improving the recommendation performance. Theoretical analysis is also provided for both the computational cost and the convergence. We believe the study of sampling strategies have further implications on general graph-based loss functions, and would also enable more research under the neural network-based recommendation framework.

cs.LG↗

arXiv2017

HSEARCH: fast and accurate protein sequence motif search and clustering

Protein motifs are conserved fragments occurred frequently in protein sequences. They have significant functions, such as active site of an enzyme. Search and clustering protein sequence motifs are computational intensive. Most existing methods are not fast enough to analyze large data sets for motif finding or achieve low accuracy for motif clustering. We present a new protein sequence motif finding and clustering algorithm, called HSEARCH. It converts fixed length protein sequences to data points in high dimensional space, and applies locality-sensitive hashing to fast search homologous protein sequences for a motif. HSEARCH is significantly faster than the brute force algorithm for protein motif finding and achieves high accuracy for protein motif clustering.

q-bio.GN↗

arXiv2016

Ideology Detection for Twitter Users with Heterogeneous Types of Links

The problem of ideology detection is to study the latent (political) placement for people, which is traditionally studied on politicians according to their voting behaviors. Recently, more and more studies begin to address the ideology detection problem for ordinary users based on their online behaviors that can be captured by social media, e.g., Twitter. As far as we are concerned, however, the vast majority of the existing methods on ideology detection on social media have oversimplified the problem as a binary classification problem (i.e., liberal vs. conservative). Moreover, though social links can play a critical role in deciding one's ideology, most of the existing work ignores the heterogeneous types of links in social media. In this paper we propose to detect \emph{numerical} ideology positions for Twitter users, according to their \emph{follow}, \emph{mention}, and \emph{retweet} links to a selected set of politicians. A unified probabilistic model is proposed that can (1) explain the reasons why links are built among people in terms of their ideology, (2) integrate heterogeneous types of links together in determining people's ideology, and (3) automatically learn the quality of each type of links in deciding one's ideology. Experiments have demonstrated the advantages of our model in terms of both ranking and political leaning classification accuracy. It is shown that (1) using multiple types of links is better than using any single type of links alone to determine one's ideology, and (2) our model is even more superior than baselines when dealing with people that are sparsely linked in one type of links. We also show that the detected ideology for Twitter users aligns with our intuition quite well.

cs.SI↗

arXiv2016

Task-Guided and Path-Augmented Heterogeneous Network Embedding for Author Identification

In this paper, we study the problem of author identification under double-blind review setting, which is to identify potential authors given information of an anonymized paper. Different from existing approaches that rely heavily on feature engineering, we propose to use network embedding approach to address the problem, which can automatically represent nodes into lower dimensional feature vectors. However, there are two major limitations in recent studies on network embedding: (1) they are usually general-purpose embedding methods, which are independent of the specific tasks; and (2) most of these approaches can only deal with homogeneous networks, where the heterogeneity of the network is ignored. Hence, challenges faced here are two folds: (1) how to embed the network under the guidance of the author identification task, and (2) how to select the best type of information due to the heterogeneity of the network. To address the challenges, we propose a task-guided and path-augmented heterogeneous network embedding model. In our model, nodes are first embedded as vectors in latent feature space. Embeddings are then shared and jointly trained according to task-specific and network-general objectives. We extend the existing unsupervised network embedding to incorporate meta paths in heterogeneous networks, and select paths according to the specific task. The guidance from author identification task for network embedding is provided both explicitly in joint training and implicitly during meta path selection. Our experiments demonstrate that by using path-augmented network embedding with task guidance, our model can obtain significantly better accuracy at identifying the true authors comparing to existing methods.

cs.LG↗

arXiv2016

Entity Embedding-based Anomaly Detection for Heterogeneous Categorical Events

Anomaly detection plays an important role in modern data-driven security applications, such as detecting suspicious access to a socket from a process. In many cases, such events can be described as a collection of categorical values that are considered as entities of different types, which we call heterogeneous categorical events. Due to the lack of intrinsic distance measures among entities, and the exponentially large event space, most existing work relies heavily on heuristics to calculate abnormal scores for events. Different from previous work, we propose a principled and unified probabilistic model APE (Anomaly detection via Probabilistic pairwise interaction and Entity embedding) that directly models the likelihood of events. In this model, we embed entities into a common latent space using their observed co-occurrence in different events. More specifically, we first model the compatibility of each pair of entities according to their embeddings. Then we utilize the weighted pairwise interactions of different entity types to define the event probability. Using Noise-Contrastive Estimation with "context-dependent" noise distribution, our model can be learned efficiently regardless of the large event space. Experimental results on real enterprise surveillance data show that our methods can accurately detect abnormal events compared to other state-of-the-art abnormal detection techniques.

cs.LG↗

arXiv2016

A Lattice Study of $(\bar{D}_1 D^{*})^\pm$ Near-threshold Scattering

In this exploratory lattice study, low-energy near threshold scattering of the $(\bar{D}_1 D^{*})^\pm$ meson system is analyzed using lattice QCD with $N_f=2$ twisted mass fermion configurations. Both s-wave ($J^P=0^-$) and p-wave ($J^P=1^+$) channels are investigated. It is found that the interaction between the two charmed mesons is attractive near the threshold in both channels. This calculation provides some hints in the searching of resonances or bound states around the threshold of $(\bar{D}_1 D^{*})^\pm$ system.

hep-lat↗