arXiv subjects

Zilong Wang

Publications and source records attributed to Zilong Wang.

At least 109 records · Page 6Linked to original sources

arXiv2023

Towards Zero-shot Relation Extraction in Web Mining: A Multimodal Approach with Relative XML Path

The rapid growth of web pages and the increasing complexity of their structure poses a challenge for web mining models. Web mining models are required to understand the semi-structured web pages, particularly when little is known about the subject or template of a new page. Current methods migrate language models to the web mining by embedding the XML source code into the transformer or encoding the rendered layout with graph neural networks. However, these approaches do not take into account the relationships between text nodes within and across pages. In this paper, we propose a new approach, ReXMiner, for zero-shot relation extraction in web mining. ReXMiner encodes the shortest relative paths in the Document Object Model (DOM) tree which is a more accurate and efficient signal for key-value pair extraction within a web page. It also incorporates the popularity of each text node by counting the occurrence of the same text node across different web pages. We use the contrastive learning to address the issue of sparsity in relation extraction. Extensive experiments on public benchmarks show that our method, ReXMiner, outperforms the state-of-the-art baselines in the task of zero-shot relation extraction in web mining.

cs.CL↗

arXiv2023

A Computational Model of Children's Learning and Use of Probabilities Across Different Ages

Recent empirical work has shown that human children are adept at learning and reasoning with probabilities. Here, we model a recent experiment investigating the development of school-age children's non-symbolic probability reasoning ability using the Neural Probability Learner and Sampler (NPLS) system. We demonstrate that NPLS can accurately simulate children's probability judgments at different ages, tasks and difficulty levels to discriminate two probabilistic choices through accurate probability learning and sampling. An extension of NPLS using a skewed heuristic distribution can also model children's tendency to wrongly select the outcome with more favorable items but less likely to draw the favorable ones when the probabilistic choices are similar. We discuss the roles of two model parameters that can be adjusted to simulate the probability matching versus probability maximization phenomena in children, and why frequency biases children's probabilistic judgments.

q-bio.NC↗

arXiv2023

Delving into E-Commerce Product Retrieval with Vision-Language Pre-training

E-commerce search engines comprise a retrieval phase and a ranking phase, where the first one returns a candidate product set given user queries. Recently, vision-language pre-training, combining textual information with visual clues, has been popular in the application of retrieval tasks. In this paper, we propose a novel V+L pre-training method to solve the retrieval problem in Taobao Search. We design a visual pre-training task based on contrastive learning, outperforming common regression-based visual pre-training tasks. In addition, we adopt two negative sampling schemes, tailored for the large-scale retrieval task. Besides, we introduce the details of the online deployment of our proposed method in real-world situations. Extensive offline/online experiments demonstrate the superior performance of our method on the retrieval task. Our proposed method is employed as one retrieval channel of Taobao Search and serves hundreds of millions of users in real time.

cs.IR↗

arXiv2023

FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement

With the increasing data volume, there is a trend of using large-scale pre-trained models to store the knowledge into an enormous number of model parameters. The training of these models is composed of lots of dense algebras, requiring a huge amount of hardware resources. Recently, sparsely-gated Mixture-of-Experts (MoEs) are becoming more popular and have demonstrated impressive pretraining scalability in various downstream tasks. However, such a sparse conditional computation may not be effective as expected in practical systems due to the routing imbalance and fluctuation problems. Generally, MoEs are becoming a new data analytics paradigm in the data life cycle and suffering from unique challenges at scales, complexities, and granularities never before possible. In this paper, we propose a novel DNN training framework, FlexMoE, which systematically and transparently address the inefficiency caused by dynamic dataflow. We first present an empirical analysis on the problems and opportunities of training MoE models, which motivates us to overcome the routing imbalance and fluctuation problems by a dynamic expert management and device placement mechanism. Then we introduce a novel scheduling module over the existing DNN runtime to monitor the data flow, make the scheduling plans, and dynamically adjust the model-to-hardware mapping guided by the real-time data traffic. A simple but efficient heuristic algorithm is exploited to dynamically optimize the device placement during training. We have conducted experiments on both NLP models (e.g., BERT and GPT) and vision models (e.g., Swin). And results show FlexMoE can achieve superior performance compared with existing systems on real-world workloads -- FlexMoE outperforms DeepSpeed by 1.70x on average and up to 2.10x, and outperforms FasterMoE by 1.30x on average and up to 1.45x.

cs.DC↗

arXiv2023

MAKE: Vision-Language Pre-training based Product Retrieval in Taobao Search

Taobao Search consists of two phases: the retrieval phase and the ranking phase. Given a user query, the retrieval phase returns a subset of candidate products for the following ranking phase. Recently, the paradigm of pre-training and fine-tuning has shown its potential in incorporating visual clues into retrieval tasks. In this paper, we focus on solving the problem of text-to-multimodal retrieval in Taobao Search. We consider that users' attention on titles or images varies on products. Hence, we propose a novel Modal Adaptation module for cross-modal fusion, which helps assigns appropriate weights on texts and images across products. Furthermore, in e-commerce search, user queries tend to be brief and thus lead to significant semantic imbalance between user queries and product titles. Therefore, we design a separate text encoder and a Keyword Enhancement mechanism to enrich the query representations and improve text-to-multimodal matching. To this end, we present a novel vision-language (V+L) pre-training methods to exploit the multimodal information of (user query, product title, product image). Extensive experiments demonstrate that our retrieval-specific pre-training model (referred to as MAKE) outperforms existing V+L pre-training methods on the text-to-multimodal retrieval task. MAKE has been deployed online and brings major improvements on the retrieval system of Taobao Search.

cs.IR↗

arXiv2023

Improved Differential-neural Cryptanalysis for Round-reduced Simeck32/64

In CRYPTO 2019, Gohr presented differential-neural cryptanalysis by building the differential distinguisher with a neural network, achieving practical 11-, and 12-round key recovery attack for Speck32/64. Inspired by this framework, we develop the Inception neural network that is compatible with the round function of Simeck to improve the accuracy of the neural distinguishers, thus improving the accuracy of (9-12)-round neural distinguishers for Simeck32/64. To provide solid baselines for neural distinguishers, we compute the full distribution of differences induced by one specific input difference up to 13-round Simeck32/64. Moreover, the performance of the DDT-based distinguishers in multiple ciphertext pairs is evaluated. Compared with the DDT-based distinguishers, the 9-, and 10-round neural distinguishers achieve better accuracy. Also, an in-depth analysis of the wrong key response profile revealed that the 12-th and 13-th bits of the subkey have little effect on the score of the neural distinguisher, thereby accelerating key recovery attacks. Finally, an enhanced 15-round and the first practical 16-, and 17-round attacks are implemented for Simeck32/64, and the success rate of both the 15-, and 16-round attacks is almost 100%.

cs.CR↗

arXiv2022

2D Human Pose Estimation with Explicit Anatomical Keypoints Structure Constraints

Recently, human pose estimation mainly focuses on how to design a more effective and better deep network structure as human features extractor, and most designed feature extraction networks only introduce the position of each anatomical keypoint to guide their training process. However, we found that some human anatomical keypoints kept their topology invariance, which can help to localize them more accurately when detecting the keypoints on the feature map. But to the best of our knowledge, there is no literature that has specifically studied it. Thus, in this paper, we present a novel 2D human pose estimation method with explicit anatomical keypoints structure constraints, which introduces the topology constraint term that consisting of the differences between the distance and direction of the keypoint-to-keypoint and their groundtruth in the loss object. More importantly, our proposed model can be plugged in the most existing bottom-up or top-down human pose estimation methods and improve their performance. The extensive experiments on the benchmark dataset: COCO keypoint dataset, show that our methods perform favorably against the most existing bottom-up and top-down human pose estimation methods, especially for Lite-HRNet, when our model is plugged into it, its AP scores separately raise by 2.9\% and 3.3\% on COCO val2017 and test-dev2017 datasets.

cs.CV↗

arXiv2022

MGDoc: Pre-training with Multi-granular Hierarchy for Document Image Understanding

Document images are a ubiquitous source of data where the text is organized in a complex hierarchical structure ranging from fine granularity (e.g., words), medium granularity (e.g., regions such as paragraphs or figures), to coarse granularity (e.g., the whole page). The spatial hierarchical relationships between content at different levels of granularity are crucial for document image understanding tasks. Existing methods learn features from either word-level or region-level but fail to consider both simultaneously. Word-level models are restricted by the fact that they originate from pure-text language models, which only encode the word-level context. In contrast, region-level models attempt to encode regions corresponding to paragraphs or text blocks into a single embedding, but they perform worse with additional word-level features. To deal with these issues, we propose MGDoc, a new multi-modal multi-granular pre-training framework that encodes page-level, region-level, and word-level information at the same time. MGDoc uses a unified text-visual encoder to obtain multi-modal features across different granularities, which makes it possible to project the multi-granular features into the same hyperspace. To model the region-word correlation, we design a cross-granular attention mechanism and specific pre-training tasks for our model to reinforce the model of learning the hierarchy between regions and words. Experiments demonstrate that our proposed model can learn better features that perform well across granularities and lead to improvements in downstream tasks.

cs.CV↗

arXiv2022

Resonance effects in Brunel harmonic generation in thin film organic semiconductors

Organic semiconductors have attracted extensive attention due to their excellent optical and electronic properties. Here, we present an experimental and theoretical study of Brunel harmonic generation in two types of porphyrin thin films: tetraphenylporphyrin (TPP) and its organometallic complex derivative Zinc tetraphenylporphyrin (ZnTPP). Our results show that the $π$-$π^\ast$ excitation of the porphyrin ringsystem plays a major role in the harmonic generation process. We uncovered the contribution of an interband process to Brunel harmonic generation. In particular, the resonant ($S_0 \rightarrow S_2$ transition) enhanced multiphoton excitation is found to lead to an early onset of non-perturbative behavior for the 5th harmonic. Similar resonance effects are expected in Brunel harmonic generation with other organic materials.

physics.optics↗

arXiv2022

Formulating Few-shot Fine-tuning Towards Language Model Pre-training: A Pilot Study on Named Entity Recognition

Fine-tuning pre-trained language models has recently become a common practice in building NLP models for various tasks, especially few-shot tasks. We argue that under the few-shot setting, formulating fine-tuning closer to the pre-training objectives shall be able to unleash more benefits from the pre-trained language models. In this work, we take few-shot named entity recognition (NER) for a pilot study, where existing fine-tuning strategies are much different from pre-training. We propose a novel few-shot fine-tuning framework for NER, FFF-NER. Specifically, we introduce three new types of tokens, "is-entity", "which-type" and bracket, so we can formulate the NER fine-tuning as (masked) token prediction or generation, depending on the choice of pre-trained language models. In our experiments, we apply FFF-NER to fine-tune both BERT and BART for few-shot NER on several benchmark datasets and observe significant improvements over existing fine-tuning strategies, including sequence labeling, prototype meta-learning, and prompt-based approaches. We further perform a series of ablation studies, showing few-shot NER performance is strongly correlated with the similarity between fine-tuning and pre-training.

cs.CL↗

arXiv2022

A Neural Model of Number Comparison with Surprisingly Robust Generalization

We propose a relatively simple computational neural-network model of number comparison. Training on comparisons of the integers 1-9 enable the model to efficiently and accurately simulate a wide range of phenomena, including distance and ratio effects and robust generalization to multidigit integers, negative numbers, and decimal numbers. An accompanying logical model of number comparison provides further insights into the workings of number comparison and its relation to the Arabic number system. These models provide a rational basis for the psychology of number comparison and the ability of neural networks to efficiently learn a powerful system with robust generalization.

q-bio.NC↗

arXiv2022

Linear and Nonlinear Optical Properties of Iridium Nanoparticles by Atomic Layer deposition

Nonlinear optical phenomena enable novel photonic and optoelectronic applications. Especially metallic nanoparticles and thin films with nonlinear optical properties offer the potential for micro-optical system integration. For this purpose, new nonlinear materials need to be continuously identified, investigated, and utilized for nonlinear optical applications. While noble metal nanoparticles, nanostructures, and thin films of Ag and Au were widely studied, iridium (Ir) nanoparticles and ultra-thin films have not been investigated yet. Here, we present a combined theoretical and experimental study on the linear and nonlinear optical properties of Ir nanoparticles deposited by atomic layer deposition (ALD). Linear optical constants, i.e., the effective refractive index n and extinction coefficient k, were evaluated at different growth stages of nanoparticle formation. Both linear and nonlinear optical properties of these Ir ALD coatings were calculated theoretically using Bruggeman and Maxwell-Garnett theories. The third-order susceptibility of Ir nanoparticle samples was experimentally investigated using the Z-scan technique. Overall, our studies demonstrate the potential of ultrathin Ir NPs as an alternative nonlinear optical material at an atomic scale.

physics.optics↗

arXiv2022

Systematic Constructions of Bent-Negabent Functions, 2-Rotation Symmetric Bent-Negabent Functions and Their Duals

Bent-negabent functions have many important properties for their application in cryptography since they have the flat absolute spectrum under the both Walsh-Hadamard transform and nega-Hadamard transform. In this paper, we present four new systematic constructions of bent-negabent functions on $4k, 8k, 4k+2$ and $8k+2$ variables, respectively, by modifying the truth tables of two classes of quadratic bent-negabent functions with simple form. The algebraic normal forms and duals of these constructed functions are also determined. We further identify necessary and sufficient conditions for those bent-negabent functions which have the maximum algebraic degree. At last, by modifying the truth tables of a class of quadratic 2-rotation symmetric bent-negabent functions, we present a construction of 2-rotation symmetric bent-negabent functions with any possible algebraic degrees. Considering that there are probably no bent-negabent functions in the rotation symmetric class, it is the first significant attempt to construct bent-negabent functions in the generalized rotation symmetric class.

cs.IT↗

arXiv2022

Non-standard Golay Complementary Sequence Pair over QAM

We generalize the three-stage process for constructing and enumerating Golay array and sequence pairs given in 2008 by Frank Fiedler et al. [A multi-dimensional approach to the construction and enumeration of Golay complementary sequences, Journal of Combinatorial Theory, Series A 115 (2008) 753-776] to $4^{q}$-QAM constellation based on para-unitary matrix method, which partly solves their open questions. Our work not only includes the main part of known results of Golay complementary sequences over $4^{q}$-QAM based on Boolean functions and standard Golay sequence pairs over QPSK, but also generates new Golay complementary arrays (sequences) over $4^{q}$-QAM based on non-standard Golay array pairs over QPSK.

cs.IT↗

arXiv2022

Boolean Functions of Binary Type-II and Type-II/III Complementary Array Pair

The sequence pairs of length $2^{m}$ projected from complementary array pairs of Type-II of size $\mathbf{2}^{(m)}$ and mixed Type-II/III and of size $\mathbf{2}^{(m-1)}\times2$ are complementary sequence pairs Type-II and Type-III respectively. An exhaustive search for binary Type-II and Type-III complementary sequence pairs of small lengths $2^{m}$ ($m=1,2,3,4$) shows that they are all projected from the aforementioned complementary array pairs, whose algebraic normal forms satisfy specified expressions. It's natural to ask whether the conclusion holds for all $m$. In this paper, we proved that these expressions of algebraic normal forms determine all the binary complementary array pairs of Type-II of size $\mathbf{2}^{(m)}$ and mixed Type-II/III of size $\mathbf{2}^{(m-1)}\times2$ respectively.

cs.IT↗

arXiv2022

The $q$-ary Golay complementary arrays of size $\mathbf{2}^{(m)}$ are standard

To find the non-standard binary Golay complementary sequences (GCSs) of length $2^{m}$ or theoretically prove the nonexistence of them are still open. Since it has been shown that all the standard $q$-ary (where $q$ is even) GCSs of length $2^m$ can be obtained by standard $q$-ary Golay complementary array pair (GAP) of dimension $m$ and size $2\times 2 \times \cdots \times 2$ (abbreviated to size $\mathbf{2}^{(m)}$), it's natural to ask whether all the $q$-ary GAP of size $\mathbf{2}^{(m)}$ are standard. We give a positive answer to this question.

cs.IT↗

arXiv2022

CFL: Cluster Federated Learning in Large-scale Peer-to-Peer Networks

Federated learning (FL) has sparked extensive interest in exploiting the private data on clients' local devices. However, the parameter server setting of FL not only has high bandwidth requirements, but also poses data privacy issues and a single point of failure. In this paper, we propose an efficient and privacy-preserving protocol, dubbed CFL, which is the first fine-grained global model training for FL in large-scale peer-to-peer (P2P) networks. Unlike previous FL in P2P networks, CFL aggregates local model update parameters hierarchically, which improves the communication efficiency facing large amounts of clients. Also, the aggregation in CFL is performed in a secure manner by introducing the authenticated encryption scheme, whose key is established through a random pairwise key scheme enhanced by a proposed voting-based key revocation mechanism. Rigorous analyses show that CFL guarantees the privacy and data integrity and authenticity of local model update parameters under two widespread threat models. More importantly, the proposed key revocation mechanism can effectively resist hijack attacks, thereby ensuring the confidentiality of the communication keys. Ingenious experiments on the Trec06p and Trec07 datasets show that the global model trained by CFL has good classification accuracy, model generalization, and rapid convergence rate, and the dropout-robustness of the system is achieved. Compared to the first global model training protocol for FL in P2P networks, PPT, CFL improves communication efficiency by 43.25%. Also, CFL outperforms PPT in terms of computational efficiency.

cs.CR↗

arXiv2022

Dap-FL: Federated Learning flourishes by adaptive tuning and secure aggregation

Federated learning (FL), an attractive and promising distributed machine learning paradigm, has sparked extensive interest in exploiting tremendous data stored on ubiquitous mobile devices. However, conventional FL suffers severely from resource heterogeneity, as clients with weak computational and communication capability may be unable to complete local training using the same local training hyper-parameters. In this paper, we propose Dap-FL, a deep deterministic policy gradient (DDPG)-assisted adaptive FL system, in which local learning rates and local training epochs are adaptively adjusted by all resource-heterogeneous clients through locally deployed DDPG-assisted adaptive hyper-parameter selection schemes. Particularly, the rationality of the proposed hyper-parameter selection scheme is confirmed through rigorous mathematical proof. Besides, due to the thoughtlessness of security consideration of adaptive FL systems in previous studies, we introduce the Paillier cryptosystem to aggregate local models in a secure and privacy-preserving manner. Rigorous analyses show that the proposed Dap-FL system could guarantee the security of clients' private local models against chosen-plaintext attacks and chosen-message attacks in a widely used honest-but-curious participants and active adversaries security model. In addition, through ingenious and extensive experiments, the proposed Dap-FL achieves higher global model prediction accuracy and faster convergence rates than conventional FL, and the comprehensiveness of the adjusted local training hyper-parameters is validated. More importantly, experimental results also show that the proposed Dap-FL achieves higher model prediction accuracy than two state-of-the-art RL-assisted FL methods, i.e., 6.03% higher than DDPG-based FL and 7.85% higher than DQN-based FL.

cs.CR↗