arXiv subjects

Yang Shen

Publications and source records attributed to Yang Shen.

At least 127 records · Page 7Linked to original sources

arXiv2022

Spectral Thermal Spreading Resistance of Wide Bandgap Semiconductors in Ballistic-Diffusive Regime

To develop efficient thermal management strategies for wide bandgap (WBG) semiconductor devices, it is essential to have a clear understanding of the heat transport process within the device and accurately predict the junction temperature. In this paper, we used the phonon Monte Carlo (MC) method with the phonon dispersion of various typical WBG semiconductors, including GaN, SiC, AlN, and \ce{β-Ga_2O_3}, to investigate the thermal spreading resistance in a ballistic-diffusive regime. It was found that when compared with Fourier's law-based predictions, the increase in the thermal resistance caused by ballistic effects was strongly related to different phonon dispersions. Based on the model deduced under the gray-medium approximation and the results of dispersion MC, we obtained a thermal resistance model that can well address the issues of thermal spreading and ballistic effects, and the influences of phonon dispersion. The model can be easily coupled with FEM based thermal analysis and applied to different materials. This paper can provide a clearer understanding of the influences of phonon dispersion on the thermal transport process, and it can be useful for the prediction of junction temperatures and the development of thermal management strategies for WBG semiconductor devices.

physics.app-ph↗

arXiv2022

Bringing Your Own View: Graph Contrastive Learning without Prefabricated Data Augmentations

Self-supervision is recently surging at its new frontier of graph learning. It facilitates graph representations beneficial to downstream tasks; but its success could hinge on domain knowledge for handcraft or the often expensive trials and errors. Even its state-of-the-art representative, graph contrastive learning (GraphCL), is not completely free of those needs as GraphCL uses a prefabricated prior reflected by the ad-hoc manual selection of graph data augmentations. Our work aims at advancing GraphCL by answering the following questions: How to represent the space of graph augmented views? What principle can be relied upon to learn a prior in that space? And what framework can be constructed to learn the prior in tandem with contrastive learning? Accordingly, we have extended the prefabricated discrete prior in the augmentation set, to a learnable continuous prior in the parameter space of graph generators, assuming that graph priors per se, similar to the concept of image manifolds, can be learned by data generation. Furthermore, to form contrastive views without collapsing to trivial solutions due to the prior learnability, we have leveraged both principles of information minimization (InfoMin) and information bottleneck (InfoBN) to regularize the learned priors. Eventually, contrastive learning, InfoMin, and InfoBN are incorporated organically into one framework of bi-level optimization. Our principled and automated approach has proven to be competitive against the state-of-the-art graph self-supervision methods, including GraphCL, on benchmarks of small graphs; and shown even better generalizability on large-scale graphs, without resorting to human expertise or downstream validation. Our code is publicly released at https://github.com/Shen-Lab/GraphCL_Automated.

cs.LG↗

arXiv2021

Algorithmic insights on continual learning from fruit flies

Continual learning in computational systems is challenging due to catastrophic forgetting. We discovered a two layer neural circuit in the fruit fly olfactory system that addresses this challenge by uniquely combining sparse coding and associative learning. In the first layer, odors are encoded using sparse, high dimensional representations, which reduces memory interference by activating non overlapping populations of neurons for different odors. In the second layer, only the synapses between odor activated neurons and the output neuron associated with the odor are modified during learning; the rest of the weights are frozen to prevent unrelated memories from being overwritten. We show empirically and analytically that this simple and lightweight algorithm significantly boosts continual learning performance. The fly associative learning algorithm is strikingly similar to the classic perceptron learning algorithm, albeit two modifications, which we show are critical for reducing catastrophic forgetting. Overall, fruit flies evolved an efficient lifelong learning algorithm, and circuit mechanisms from neuroscience can be translated to improve machine computation.

cs.LG↗

arXiv2021

Mean-Variance Portfolio Selection in Contagious Markets

We consider a mean-variance portfolio selection problem in a financial market with contagion risk. The risky assets follow a jump-diffusion model, in which jumps are driven by a multivariate Hawkes process with mutual-excitation effect. The mutual-excitation feature of the Hawkes process captures the contagion risk in the sense that each price jump of an asset increases the likelihood of future jumps not only in the same asset but also in other assets. We apply the stochastic maximum principle, backward stochastic differential equation theory, and linear-quadratic control technique to solve the problem and obtain the efficient strategy and efficient frontier in semi-closed form, subject to a non-local partial differential equation. Numerical examples are provided to illustrate our results.

q-fin.MF↗

arXiv2021

Highly sensitive strain sensor from topological-structure modulated dielectric elastic nanocomposites

Flexible strain sensors are critical to several potential intelligent applications, such as human-machine interfaces, soft robotics, human motion detection, and safety monitoring of components. Stretchable functional materials are important components of strain sensors, and they are still major challenges for high performance strain sensors. Herein, we demonstrate a novel strategy of designing and optimizing flexible strain sensor by developing topological structure modulated high permittivity elastic nanocomposite. The topological structure with three-phase percolative nano-nanonetworks produces synergistic effects of space charge enhancement and local electric field modulation, and it gives rise to an ultrahigh dielectric permittivity (113.4, at 1 kHz, over 1500% enhancement than that of commercial elastic polyurethane matrix) and excellent comprehensive electromechanical performance, and the optimal comprehensive electromechanical performance reaches to 542.91 MPa-1, which is over 9-fold than that of commercial polyurethane elastic film. An interdigital capacitive strain sensor is designed using the topological structured elastic dielectric nanocomposite. It possesses high initial capacitance density and positive capacitance response with stain, achieving high signal-to-noise ratio, high capacitance response sensitivity, and wide linear range, and it breaks through disadvantages of negative sensitivity and narrow linear range for conventional interdigital strain sensors. The prepared integrated strain sensor arrays are able to measure local strain of convoluted surfaces and monitor the motion of soft actuators in real time, and they would make conditions for intelligent control systems and the study of morphological intelligence.

physics.app-ph↗

arXiv2021

Graph Contrastive Learning Automated

Self-supervised learning on graph-structured data has drawn recent interest for learning generalizable, transferable and robust representations from unlabeled graphs. Among many, graph contrastive learning (GraphCL) has emerged with promising representation learning performance. Unfortunately, unlike its counterpart on image data, the effectiveness of GraphCL hinges on ad-hoc data augmentations, which have to be manually picked per dataset, by either rules of thumb or trial-and-errors, owing to the diverse nature of graph data. That significantly limits the more general applicability of GraphCL. Aiming to fill in this crucial gap, this paper proposes a unified bi-level optimization framework to automatically, adaptively and dynamically select data augmentations when performing GraphCL on specific graph data. The general framework, dubbed JOint Augmentation Optimization (JOAO), is instantiated as min-max optimization. The selections of augmentations made by JOAO are shown to be in general aligned with previous "best practices" observed from handcrafted tuning: yet now being automated, more flexible and versatile. Moreover, we propose a new augmentation-aware projection head mechanism, which will route output features through different projection heads corresponding to different augmentations chosen at each training step. Extensive experiments demonstrate that JOAO performs on par with or sometimes better than the state-of-the-art competitors including GraphCL, on multiple graph datasets of various scales and types, yet without resorting to any laborious dataset-specific tuning on augmentation selection. We release the code at https://github.com/Shen-Lab/GraphCL_Automated.

cs.LG↗

arXiv2021

Fold2Seq: A Joint Sequence(1D)-Fold(3D) Embedding-based Generative Model for Protein Design

Designing novel protein sequences for a desired 3D topological fold is a fundamental yet non-trivial task in protein engineering. Challenges exist due to the complex sequence--fold relationship, as well as the difficulties to capture the diversity of the sequences (therefore structures and functions) within a fold. To overcome these challenges, we propose Fold2Seq, a novel transformer-based generative framework for designing protein sequences conditioned on a specific target fold. To model the complex sequence--structure relationship, Fold2Seq jointly learns a sequence embedding using a transformer and a fold embedding from the density of secondary structural elements in 3D voxels. On test sets with single, high-resolution and complete structure inputs for individual folds, our experiments demonstrate improved or comparable performance of Fold2Seq in terms of speed, coverage, and reliability for sequence design, when compared to existing state-of-the-art methods that include data-driven deep generative models and physics-based RosettaDesign. The unique advantages of fold-based Fold2Seq, in comparison to a structure-based deep model and RosettaDesign, become more evident on three additional real-world challenges originating from low-quality, incomplete, or ambiguous input structures. Source code and data are available at https://github.com/IBM/fold2seq.

cs.LG↗

arXiv2021

Graph Contrastive Learning with Augmentations

Generalizable, transferrable, and robust representation learning on graph-structured data remains a challenge for current graph neural networks (GNNs). Unlike what has been developed for convolutional neural networks (CNNs) for image data, self-supervised learning and pre-training are less explored for GNNs. In this paper, we propose a graph contrastive learning (GraphCL) framework for learning unsupervised representations of graph data. We first design four types of graph augmentations to incorporate various priors. We then systematically study the impact of various combinations of graph augmentations on multiple datasets, in four different settings: semi-supervised, unsupervised, and transfer learning as well as adversarial attacks. The results show that, even without tuning augmentation extents nor using sophisticated GNN architectures, our GraphCL framework can produce graph representations of similar or better generalizability, transferrability, and robustness compared to state-of-the-art methods. We also investigate the impact of parameterized graph augmentation extents and patterns, and observe further performance gains in preliminary experiments. Our codes are available at https://github.com/Shen-Lab/GraphCL.

cs.LG↗

arXiv2021

Mean-Variance Investment and Risk Control Strategies -- A Time-Consistent Approach via A Forward Auxiliary Process

We consider an optimal investment and risk control problem for an insurer under the mean-variance (MV) criterion. By introducing a deterministic auxiliary process defined forward in time, we formulate an alternative time-consistent problem related to the original MV problem, and obtain the optimal strategy and the value function to the new problem in closed-form. We compare our formulation and optimal strategy to those under the precommitment and game-theoretic framework. Numerical studies show that, when the financial market is negatively correlated with the risk process, optimal investment may involve short selling the risky asset and, if that happens, a less risk averse insurer short sells more risky asset.

q-fin.PM↗

arXiv2020

Synergistical Electroluminescent Manipulation for Efficient Blue Perovskite Light-Emitting Diodes Through Interfacial Nucleation Seeding

The low efficiency of metal halide perovskite light-emitting diodes (PeLEDs) with blue emission block their potential applications in large-area full-color displays and solid-state lighting. A delicate control over the entire electroluminescence process is indispensable to overcome the efficiency limitations of blue PeLEDs. Here, we demonstrate an efficient device architecture to synergistically reduce the energetic losses during electron-photon conversion and boost the extraction of trapped light in the device. An interfacial nucleation seeding scheme is proposed to control the crystallization process of highly emissive perovskite nanocrystals and suppress the trap-mediated non-radiative recombination losses due to interfacial hydrogen bonding interactions. This manipulation results in a record external quantum efficiency (EQE) of 12.8% for blue PeLEDs emitting at 486 nm, along with the improved spectral stability and operation lifetime. Additionally, the maximum EQE reaches 16.8% after combining an internal outcoupling structure without spectral distortion, which can be further raised to 27.5% when using a lens-based structure on top of the device. We anticipate that our work provides an effective method for its application in high-performance PeLEDs.

physics.app-ph↗

arXiv2020

Cross-Modality Protein Embedding for Compound-Protein Affinity and Contact Prediction

Compound-protein pairs dominate FDA-approved drug-target pairs and the prediction of compound-protein affinity and contact (CPAC) could help accelerate drug discovery. In this study we consider proteins as multi-modal data including 1D amino-acid sequences and (sequence-predicted) 2D residue-pair contact maps. We empirically evaluate the embeddings of the two single modalities in their accuracy and generalizability of CPAC prediction (i.e. structure-free interpretable compound-protein affinity prediction). And we rationalize their performances in both challenges of embedding individual modalities and learning generalizable embedding-label relationship. We further propose two models involving cross-modality protein embedding and establish that the one with cross interaction (thus capturing correlations among modalities) outperforms SOTAs and our single modality models in affinity, contact, and binding-site predictions for proteins never seen in the training set.

q-bio.BM↗

arXiv2020

Optimal controls of stochastic differential equations with jumps and random coefficients: Stochastic Hamilton-Jacobi-Bellman equations with jumps

In this paper, we study the following nonlinear backward stochastic integral partial differential equation with jumps \begin{equation*} \left\{ \begin{split} -d V(t,x) =&\displaystyle\inf_{u\in U}\bigg\{H(t,x,u, DV(t,x),D Φ(t,x), D^2 V(t,x),\int_E \left(\mathcal I V(t,e,x,u)+Ψ(t,x+g(t,e,x,u))\right)l(t,e)ν(de)) \\ &+\displaystyle\int_{E}\big[\mathcal I V(t,e,x,u)-\displaystyle (g(t, e,x,u), D V(t,x))\big]ν(d e)+\int_{E}\big[\mathcal I Ψ(t,e,x,u)\big]ν(d e)\bigg\}dt\\ &-Φ(t,x)dW(t)-\displaystyle\int_{E} Ψ(t, e,x)\tildeμ(d e,dt),\\ V(T,x)=& \ h(x), \end{split} \right. \end{equation*} where $\tilde μ$ is a Poisson random martingale measure, $W$ is a Brownian motion, and $\mathcal I$ is a non-local operator to be specified later. The function $H$ is a given random mapping, which arises from a corresponding non-Markovian optimal control problem. This equation appears as the stochastic Hamilton-Jacobi-Bellman equation, which characterizes the value function of the optimal control problem with a recursive utility cost functional. The solution to the equation is a predictable triplet of random fields $(V,Φ,Ψ)$. We show that the value function, under some regularity assumptions, is the solution to the stochastic HJB equation; and a classical solution to this equation is the value function and gives the optimal control. With some additional assumptions on the coefficients, an existence and uniqueness result in the sense of Sobolev space is shown by recasting the backward stochastic partial integral differential equation with jumps as a backward stochastic evolution equation in Hilbert spaces with Poisson jumps.

math.OC↗

arXiv2020

A universal method for depositing patterned materials in-situ

Current techniques of patterned material deposition require separate steps for patterning and material deposition. The complexity and harsh working conditions post serious limitations for fabrication. Here, we introduce a novel single-step and easy-to-adapt method that can deposit materials in-situ. Its unique methodology is based on the semiconductor nanoparticle assisted photon-induced chemical reduction and optical trapping. This universal mechanism can be used for depositing a large selection of materials including metals, insulators and magnets, with quality on par with current technologies. Patterning with several materials together with optical-diffraction-limited resolution accuracy can be achieved from macroscopic to microscopic scale. Furthermore, the setup is naturally compatible with optical microscopy based measurements, thus sample characterisation and material deposition can be realised in-situ. Various devices fabricated with this method in 2D or 3D show it is ready for deployment in practical applications. This revolutionary method will provide a distinct tool in material technology.

cond-mat.mes-hall↗

arXiv2020

L$^2$-GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks

Graph convolution networks (GCN) are increasingly popular in many applications, yet remain notoriously hard to train over large graph datasets. They need to compute node representations recursively from their neighbors. Current GCN training algorithms suffer from either high computational costs that grow exponentially with the number of layers, or high memory usage for loading the entire graph and node embeddings. In this paper, we propose a novel efficient layer-wise training framework for GCN (L-GCN), that disentangles feature aggregation and feature transformation during training, hence greatly reducing time and memory complexities. We present theoretical analysis for L-GCN under the graph isomorphism framework, that L-GCN leads to as powerful GCNs as the more costly conventional training algorithm does, under mild conditions. We further propose L$^2$-GCN, which learns a controller for each layer that can automatically adjust the training epochs per layer in L-GCN. Experiments show that L-GCN is faster than state-of-the-arts by at least an order of magnitude, with a consistent of memory usage not dependent on dataset size, while maintaining comparable prediction performance. With the learned controller, L$^2$-GCN can further cut the training time in half. Our codes are available at https://github.com/Shen-Lab/L2-GCN.

cs.LG↗

arXiv2020

When Does Self-Supervision Help Graph Convolutional Networks?

Self-supervision as an emerging technique has been employed to train convolutional neural networks (CNNs) for more transferrable, generalizable, and robust representation learning of images. Its introduction to graph convolutional networks (GCNs) operating on graph data is however rarely explored. In this study, we report the first systematic exploration and assessment of incorporating self-supervision into GCNs. We first elaborate three mechanisms to incorporate self-supervision into GCNs, analyze the limitations of pretraining & finetuning and self-training, and proceed to focus on multi-task learning. Moreover, we propose to investigate three novel self-supervised learning tasks for GCNs with theoretical rationales and numerical comparisons. Lastly, we further integrate multi-task self-supervision into graph adversarial training. Our results show that, with properly designed task forms and incorporation mechanisms, self-supervision benefits GCNs in gaining more generalizability and robustness. Our codes are available at https://github.com/Shen-Lab/SS-GCNs.

cs.LG↗

arXiv2019

Explainable Deep Relational Networks for Predicting Compound-Protein Affinities and Contacts

Predicting compound-protein affinity is critical for accelerating drug discovery. Recent progress made by machine learning focuses on accuracy but leaves much to be desired for interpretability. Through molecular contacts underlying affinities, our large-scale interpretability assessment finds commonly-used attention mechanisms inadequate. We thus formulate a hierarchical multi-objective learning problem whose predicted contacts form the basis for predicted affinities. We further design a physics-inspired deep relational network, DeepRelations, with intrinsically explainable architecture. Specifically, various atomic-level contacts or "relations" lead to molecular-level affinity prediction. And the embedded attentions are regularized with predicted structural contexts and supervised with partially available training contacts. DeepRelations shows superior interpretability to the state-of-the-art: without compromising affinity prediction, it boosts the AUPRC of contact prediction 9.5, 16.9, 19.3 and 5.7-fold for the test, compound-unique, protein-unique, and both-unique sets, respectively. Our study represents the first dedicated model development and systematic model assessment for interpretable machine learning of compound-protein affinity.

q-bio.BM↗

arXiv2019

Energy-based Graph Convolutional Networks for Scoring Protein Docking Models

Structural information about protein-protein interactions, often missing at the interactome scale, is important for mechanistic understanding of cells and rational discovery of therapeutics. Protein docking provides a computational alternative to predict such information. However, ranking near-native docked models high among a large number of candidates, often known as the scoring problem, remains a critical challenge. Moreover, estimating model quality, also known as the quality assessment problem, is rarely addressed in protein docking. In this study the two challenging problems in protein docking are regarded as relative and absolute scoring, respectively, and addressed in one physics-inspired deep learning framework. We represent proteins' and encounter complexes' 3D structures as intra- and inter-molecular residue contact graphs with atom-resolution node and edge features. And we propose a novel graph convolutional kernel that pool interacting nodes' features through edge features so that generalized interaction energies can be learned directly from graph data. The resulting energy-based graph convolutional networks (EGCN) with multi-head attention are trained to predict intra- and inter-molecular energies, binding affinities, and quality measures (interface RMSD) for encounter complexes. Compared to a state-of-the-art scoring function for model ranking, EGCN has significantly improved ranking for a CAPRI test set involving homology docking; and is comparable for Score_set, a CAPRI benchmark set generated by diverse community-wide docking protocols not known to training data. For Score_set quality assessment, EGCN shows about 27% improvement to our previous efforts. Directly learning from 3D structure data in graph representation, EGCN represents the first successful development of graph convolutional networks for protein docking.

q-bio.BM↗

arXiv2019

Learning to Optimize in Swarms

Learning to optimize has emerged as a powerful framework for various optimization and machine learning tasks. Current such "meta-optimizers" often learn in the space of continuous optimization algorithms that are point-based and uncertainty-unaware. To overcome the limitations, we propose a meta-optimizer that learns in the algorithmic space of both point-based and population-based optimization algorithms. The meta-optimizer targets at a meta-loss function consisting of both cumulative regret and entropy. Specifically, we learn and interpret the update formula through a population of LSTMs embedded with sample- and feature-level attentions. Meanwhile, we estimate the posterior directly over the global optimum and use an uncertainty measure to help guide the learning process. Empirical results over non-convex test functions and the protein-docking application demonstrate that this new meta-optimizer outperforms existing competitors.

cs.LG↗