arXiv subjects

Zhiyuan Zhang

Publications and source records attributed to Zhiyuan Zhang.

At least 163 records · Page 9Linked to original sources

arXiv2022

Recent Advances in Design of Electrocatalysts for High-Current-Density Water Splitting

Electrochemical water splitting technology for producing "green hydrogen" is important for the global mission of carbon neutrality. Electrocatalysts with decent performance at high current densities play a central role in the industrial implementation of this technology. The field has advanced immensely in recent years, as witnessed by many types of catalysts have been designed and synthesized which work at industrially-relevant current densities (> 200 mA cm-2). Note that the activity and stability of catalysts can be influenced by their local reaction environment, which are closely related to the current density. By discussing recent advances in this field, we summarize several key aspects that affect the catalytic performance for high-current-density electrocatalysis, including dimensionality of catalysts, surface chemistry, electron transport path, morphology, and catalyst-electrolyte interplay. We highlight the multiscale design strategy that considers these aspects comprehensively for developing high-current-density catalysts. We also put forward out perspectives on the future directions in this emerging field.

physics.chem-ph↗

arXiv2022

A Ta-TaS2 monolithic catalyst with robust and metallic interface for superior hydrogen evolution

The use of highly active and robust catalysts is crucial for producing green hydrogen by water electrolysis as we strive to achieve global carbon neutrality. Noble metals like platinum are currently used in industry for the hydrogen evolution reaction (HER), but suffer from scarcity, high price and unsatisfied performance and stability at large current density, restricting their large scale implementations. Here we report the synthesis of a new type of monolithic catalyst (MC) consisting of a metal disulfide (e.g., TaS2) catalyst vertically bonded to a conductive substrate of the same metal by strong covalent bonds. These features give the MC a mechanically robust and electrically near zero resistance interface, leading to an outstanding HER performance including rapid charge transfer and excellent durability, together with a low overpotential of 398 mV to achieve a current density of 2,000 mA cm-2 as required by industry. The Ta TaS2 MC has a negligible performance decay after 200 h operation at large current densities. In light of its unique interface and the various choice of metal elements giving the same structure, such monolithic materials may have broad uses besides catalysis.

cond-mat.mtrl-sci↗

arXiv2021

Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning

Recent pretrained language models extend from millions to billions of parameters. Thus the need to fine-tune an extremely large pretrained model with a limited training corpus arises in various downstream tasks. In this paper, we propose a straightforward yet effective fine-tuning technique, Child-Tuning, which updates a subset of parameters (called child network) of large pretrained models via strategically masking out the gradients of the non-child network during the backward process. Experiments on various downstream tasks in GLUE benchmark show that Child-Tuning consistently outperforms the vanilla fine-tuning by 1.5~8.6 average score among four different pretrained models, and surpasses the prior fine-tuning techniques by 0.6~1.3 points. Furthermore, empirical results on domain transfer and task transfer show that Child-Tuning can obtain better generalization performance by large margins.

cs.CL↗

arXiv2021

A durable and efficient electrocatalyst for saline water splitting with current density exceeding 2000 mA cm -2

Water electrolysis is promising for industrial hydrogen production to achieve a sustainable and green hydrogen economy, but the high cost of the technology limits its market share. Developing efficient yet economic electrocatalysts is crucial to decrease the cost of electricity and electrolytic cell. Meanwhile, electrolysis in seawater electrolyte can further reduce feedstock cost. Here we synthesize a type of electrocatalyst where trace precious metals are strongly anchored on corrosion-resistive matrix. As an example, the produced Pt/Ni-Mo electrocatalyst only needs an overpotential of 113 mV to reach an ultrahigh current density of 2000 mA cm-2 in saline-alkaline electrolyte, standing as the best performance so far. It shows high activity and long durability in various electrolytes and under harsh conditions, including strong alkaline and simulated seawater electrolytes, and under elevated temperatures up to 80 degree Celsius). This electrocatalyst is produced on a large scale at low cost and shows good performance in a commercial membrane electrode assembly stack, demonstrating its feasibility for practical water electrolysis

physics.chem-ph↗

arXiv2021

Stabilized Hydroxide Mediated Nickel-Based Electrocatalysts for High Current Density Hydrogen Evolution in Alkaline Media

Large scale production of hydrogen by electrochemical water splitting is considered as a promising technology to address critical energy challenges caused by the extensive use of fossil fuels. Although nonprecious nickel-based catalysts work well at low current densities, they need large overpotentials at high current densities that hinders their potential applications in practical industry. Here we report a hydroxide-mediated nickel based electrocatalyst for high current density hydrogen evolution, which delivers a current density of 1000 mA cm-2 at an overpotential of 98 mV. Combined X-ray absorption spectroscopy and high resolution X-ray photoelectron spectroscopy results show charge redistribution of nickel sites caused by Mo and surface FeOx clusters, which can stabilize the surface nickel hydroxide at high current densities for promoting water dissociation step. Such catalyst is synthesized at the metre scale and shows a current density of 500 mA cm-2 at 1.56 V in the overall water splitting, which demonstrate its potential for practical use. This work highlights a charge-engineering strategy for rational design of catalysts that work well at high current densities.

physics.chem-ph↗

arXiv2021

Quantum resonant tunnelling enhances hydrogen bond rotation of confined water

Many studies have revealed that confined water chain flipping is closely related to the spatial size and even quantum effects of the confinement environment. Here, we show that these are not the only factors that affect the flipping process of a confined water chain. First-principles calculations and analyses confirm that quantum tunnelling effects from the water chain itself, especially resonant tunnelling, enhance the hydrogen bond rotation process. Importantly, resonant tunnelling can result in tunnelling rotation of hydrogen bonds with a probability close to 1 with only 0.597 eV provided energy. Compared to sequential tunnelling, resonant tunnelling dominants water chain flipping at temperatures up to 20 K higher. Additionally, the ratio of the resonant tunnelling probability to the thermal disturbance probability at 200 K is at least ten times larger than that of sequential tunnelling, which further illustrates the enhancement of hydrogen bond rotation brought about by resonant tunnelling.

physics.bio-ph↗

arXiv2021

Uniform Embeddings for Robinson Similarity Matrices

A Robinson similarity matrix is a symmetric matrix where the entry values on all rows and columns increase toward the diagonal. Decompose the Robinson matrix into the sum of k {0, 1}-matrices, then these k {0, 1}-matrices are the adjacency matrices of a set of nested unit interval graphs. Previous studies show that unit interval graphs coincide with indifference graphs. An indifference graph has an embedding that maps each vertex to a real number, where two vertices are adjacent if their embedding is within a fixed threshold distance. In this thesis, consider k different threshold distances, we study the problem of finding an embedding that, simultaneously and with respect to each threshold distance, embeds the k indifference graphs corresponding to the k adjacency matrices. This is called a uniform embedding of a Robinson matrix with respect to the k threshold distances. We give a sufficient and necessary condition on Robinson matrices that have a uniform embedding, which is derived from paths in an associated graph. We also give an efficient combinatorial algorithm to find a uniform embedding or give proof that it does not exist, for the case where k = 2.

math.CO↗

arXiv2021

Rethinking Skip Connection with Layer Normalization in Transformers and ResNets

Skip connection, is a widely-used technique to improve the performance and the convergence of deep neural networks, which is believed to relieve the difficulty in optimization due to non-linearity by propagating a linear component through the neural network layers. However, from another point of view, it can also be seen as a modulating mechanism between the input and the output, with the input scaled by a pre-defined value one. In this work, we investigate how the scale factors in the effectiveness of the skip connection and reveal that a trivial adjustment of the scale will lead to spurious gradient exploding or vanishing in line with the deepness of the models, which could be addressed by normalization, in particular, layer normalization, which induces consistent improvements over the plain skip connection. Inspired by the findings, we further propose to adaptively adjust the scale of the input by recursively applying skip connection with layer normalization, which promotes the performance substantially and generalizes well across diverse tasks including both machine translation and image classification datasets.

cs.LG↗

arXiv2021

Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models

Recent studies have revealed a security threat to natural language processing (NLP) models, called the Backdoor Attack. Victim models can maintain competitive performance on clean samples while behaving abnormally on samples with a specific trigger word inserted. Previous backdoor attacking methods usually assume that attackers have a certain degree of data knowledge, either the dataset which users would use or proxy datasets for a similar task, for implementing the data poisoning procedure. However, in this paper, we find that it is possible to hack the model in a data-free way by modifying one single word embedding vector, with almost no accuracy sacrificed on clean samples. Experimental results on sentiment analysis and sentence-pair classification tasks show that our method is more efficient and stealthier. We hope this work can raise the awareness of such a critical security risk hidden in the embedding layers of NLP models. Our code is available at https://github.com/lancopku/Embedding-Poisoning.

cs.CL↗

arXiv2021

Learning Robust Representation for Clustering through Locality Preserving Variational Discriminative Network

Clustering is one of the fundamental problems in unsupervised learning. Recent deep learning based methods focus on learning clustering oriented representations. Among those methods, Variational Deep Embedding achieves great success in various clustering tasks by specifying a Gaussian Mixture prior to the latent space. However, VaDE suffers from two problems: 1) it is fragile to the input noise; 2) it ignores the locality information between the neighboring data points. In this paper, we propose a joint learning framework that improves VaDE with a robust embedding discriminator and a local structure constraint, which are both helpful to improve the robustness of our model. Experiment results on various vision and textual datasets demonstrate that our method outperforms the state-of-the-art baseline models in all metrics. Further detailed analysis shows that our proposed model is very robust to the adversarial inputs, which is a desirable property for practical applications.

cs.LG↗

arXiv2021

Spatial Structure Engineering in Enhancing Performance of Mosaic Electrocatalysts

Understanding the mechanism and developing strategies toward efficient electrocatalysis at gas-liquidsolid interfaces are important yet challenging. In the past decades, researchers have devoted many efforts to improving catalyst activity by modulating electronic properties of catalysts in terms of chemical components and physical features. Here we develop a mosaic catalyst strategy to improve activity of electrocatalysts by engineering their spatial structures. Taking Pt catalyst as an example, the mosaic Pt leads to high catalytic performance, showing a specific activity 11 times higher than uniform Pt films for hydrogen evolution reaction (HER), as well as higher current densities than commercial Pt/C and uniform Pt films. Such a strategy is found to be general to other catalysts (e.g., twodimensional PtS) and other reactions (e.g., oxygen evolution reaction). The improved catalytic performance of the mosaic catalysts is attributed to enhanced mass transferability and local electric field, both are determined by the occupation ratio of catalysts. Our work shines new light on manipulating electrocatalysis from the perspective of the spatial structure of catalyst, which would guide the design of efficient catalysts for heterogeneous reactions.

physics.app-ph↗

arXiv2020

Exploring the Vulnerability of Deep Neural Networks: A Study of Parameter Corruption

We argue that the vulnerability of model parameters is of crucial value to the study of model robustness and generalization but little research has been devoted to understanding this matter. In this work, we propose an indicator to measure the robustness of neural network parameters by exploiting their vulnerability via parameter corruption. The proposed indicator describes the maximum loss variation in the non-trivial worst-case scenario under parameter corruption. For practical purposes, we give a gradient-based estimation, which is far more effective than random corruption trials that can hardly induce the worst accuracy degradation. Equipped with theoretical support and empirical validation, we are able to systematically investigate the robustness of different model parameters and reveal vulnerability of deep neural networks that has been rarely paid attention to before. Moreover, we can enhance the models accordingly with the proposed adversarial corruption-resistant training, which not only improves the parameter robustness but also translates into accuracy elevation.

cs.LG↗

arXiv2020

RRCN: A Reinforced Random Convolutional Network based Reciprocal Recommendation Approach for Online Dating

Recently, the reciprocal recommendation, especially for online dating applications, has attracted more and more research attention. Different from conventional recommendation problems, the reciprocal recommendation aims to simultaneously best match users' mutual preferences. Intuitively, the mutual preferences might be affected by a few key attributes that users like or dislike. Meanwhile, the interactions between users' attributes and their key attributes are also important for key attributes selection. Motivated by these observations, in this paper we propose a novel reinforced random convolutional network (RRCN) approach for the reciprocal recommendation task. In particular, we technically propose a novel random CNN component that can randomly convolute non-adjacent features to capture their interaction information and learn feature embeddings of key attributes to make the final recommendation. Moreover, we design a reinforcement learning based strategy to integrate with the random CNN component to select salient attributes to form the candidate set of key attributes. We evaluate the proposed RRCN against a number of both baselines and the state-of-the-art approaches on two real-world datasets, and the promising results have demonstrated the superiority of RRCN against the compared approaches in terms of a number of evaluation criteria.

cs.IR↗

arXiv2020

Memorized Sparse Backpropagation

Neural network learning is usually time-consuming since backpropagation needs to compute full gradients and backpropagate them across multiple layers. Despite its success of existing works in accelerating propagation through sparseness, the relevant theoretical characteristics remain under-researched and empirical studies found that they suffer from the loss of information contained in unpropagated gradients. To tackle these problems, this paper presents a unified sparse backpropagation framework and provides a detailed analysis of its theoretical characteristics. Analysis reveals that when applied to a multilayer perceptron, our framework essentially performs gradient descent using an estimated gradient similar enough to the true gradient, resulting in convergence in probability under certain conditions. Furthermore, a simple yet effective algorithm named memorized sparse backpropagation (MSBP) is proposed to remedy the problem of information loss by storing unpropagated gradients in memory for learning in the next steps. Experimental results demonstrate that the proposed MSBP is effective to alleviate the information loss in traditional sparse backpropagation while achieving comparable acceleration.

cs.LG↗

arXiv2020

A short-range metastable defect in the double layer ice

Although the phase of water has extensively investigated whether there exists a defect distorting only locally the structure still under debate. Here we report a localized 5775 defect phase presented in the double layer ice on the Au (111) surface, which is a metastable structure with 5- and 7-membered rings compared with a perfect hexagonal one. Without altering the total number of the hydrogen bonds of the ice, the defect only introduces 0.08 Å molecular displacement and 3.27% interaction energy change outside the defected area. Such defect also exists without Au support but causes a larger lattice relaxation or smaller interaction energy change. The excessively high barrier as well as the low quantum tunneling and thermodynamic probabilities hinder the formation of the defect by post-grown isomerization from the perfect to the defected structure. This finding indicates that the defected ice is stable, and the defect can be formed during the ice growth stage.

cond-mat.mtrl-sci↗

arXiv2020

Global Context Aware Convolutions for 3D Point Cloud Understanding

Recent advances in deep learning for 3D point clouds have shown great promises in scene understanding tasks thanks to the introduction of convolution operators to consume 3D point clouds directly in a neural network. Point cloud data, however, could have arbitrary rotations, especially those acquired from 3D scanning. Recent works show that it is possible to design point cloud convolutions with rotation invariance property, but such methods generally do not perform as well as translation-invariant only convolution. We found that a key reason is that compared to point coordinates, rotation-invariant features consumed by point cloud convolution are not as distinctive. To address this problem, we propose a novel convolution operator that enhances feature distinction by integrating global context information from the input point cloud to the convolution. To this end, a globally weighted local reference frame is constructed in each point neighborhood in which the local point set is decomposed into bins. Anchor points are generated in each bin to represent global shape features. A convolution can then be performed to transform the points and anchor features into final rotation-invariant features. We conduct several experiments on point cloud classification, part segmentation, shape retrieval, and normals estimation to evaluate our convolution, which achieves state-of-the-art accuracy under challenging rotations.

cs.CV↗

arXiv2020

High-Throughput Production of Cheap Mineral-Based 2D Electrocatalysts for High-Current-Density Hydrogen Evolution

The high-throughput scalable production of cheap, efficient and durable electrocatalysts that work well at high current densities demanded by industry is a great challenge for the large-scale implementation of electrochemical technologies. Here we report the production of a 2D MoS2-based ink-type electrocatalyst by a scalable top-down exfoliation technique followed by a simple heat treatment. The catalyst shows a high current density of 1000 mA cm^-2 at an overpotential of 454 mV for the hydrogen evolution reaction (HER) without the need of iR correction, as well as good stability over 24 hours. Using the same method, we have, for the first time, produced a cheap MoS2 mineral-based catalyst and found that it had an excellent performance for high-current-density HER. Noteworthy, production rate of this MoS2-based catalyst is one to two orders of magnitude higher than those previously reported. In addition, the price of the MoS2 mineral is five orders of magnitude lower than commercial Pt catalysts, making the MoS2 mineral-based catalyst cheap, and the ink-type catalyst dispersions can be easily integrated with other technologies for large-scale catalyst electrode preparation. These advantages indicate the huge potentials of this method and mineral-based cheap and abundant natural resources as catalysts in the electrochemical technologies.

physics.app-ph↗

arXiv2020

Shop The Look: Building a Large Scale Visual Shopping System at Pinterest

As online content becomes ever more visual, the demand for searching by visual queries grows correspondingly stronger. Shop The Look is an online shopping discovery service at Pinterest, leveraging visual search to enable users to find and buy products within an image. In this work, we provide a holistic view of how we built Shop The Look, a shopping oriented visual search system, along with lessons learned from addressing shopping needs. We discuss topics including core technology across object detection and visual embeddings, serving infrastructure for realtime inference, and data labeling methodology for training/evaluation data collection and human evaluation. The user-facing impacts of our system design choices are measured through offline evaluations, human relevance judgements, and online A/B experiments. The collective improvements amount to cumulative relative gains of over 160% in end-to-end human relevance judgements and over 80% in engagement. Shop The Look is deployed in production at Pinterest.

cs.CV↗