arXiv subjects

Zhi Chen

Publications and source records attributed to Zhi Chen.

At least 217 records · Page 12Linked to original sources

arXiv2022

DFM: Dialogue Foundation Model for Universal Large-Scale Dialogue-Oriented Task Learning

Building a universal conversational agent has been a long-standing goal of the dialogue research community. Most previous works only focus on a small set of dialogue tasks. In this work, we aim to build a unified dialogue foundation model (DFM) which can be used to solve massive diverse dialogue tasks. To achieve this goal, a large-scale well-annotated dialogue dataset with rich task diversity (DialogZoo) is collected. We introduce a framework to unify all dialogue tasks and propose novel auxiliary self-supervised tasks to achieve stable training of DFM on the highly diverse large scale DialogZoo corpus. Experiments show that, compared with models of the same size, DFM can achieve state-of-the-art or competitive performance on very rich cross-domain downstream dialogue tasks. This demonstrates that DFM largely extends the ability of unified dialogue pre-trained model.

cs.CL↗

arXiv2022

How to See Hidden Patterns in Metamaterials with Interpretable Machine Learning

Machine learning models can assist with metamaterials design by approximating computationally expensive simulators or solving inverse design problems. However, past work has usually relied on black box deep neural networks, whose reasoning processes are opaque and require enormous datasets that are expensive to obtain. In this work, we develop two novel machine learning approaches to metamaterials discovery that have neither of these disadvantages. These approaches, called shape-frequency features and unit-cell templates, can discover 2D metamaterials with user-specified frequency band gaps. Our approaches provide logical rule-based conditions on metamaterial unit-cells that allow for interpretable reasoning processes, and generalize well across design spaces of different resolutions. The templates also provide design flexibility where users can almost freely design the fine resolution features of a unit-cell without affecting the user's desired band gap.

cs.LG↗

arXiv2022

TimberTrek: Exploring and Curating Sparse Decision Trees with Interactive Visualization

Given thousands of equally accurate machine learning (ML) models, how can users choose among them? A recent ML technique enables domain experts and data scientists to generate a complete Rashomon set for sparse decision trees--a huge set of almost-optimal interpretable ML models. To help ML practitioners identify models with desirable properties from this Rashomon set, we develop TimberTrek, the first interactive visualization system that summarizes thousands of sparse decision trees at scale. Two usage scenarios highlight how TimberTrek can empower users to easily explore, compare, and curate models that align with their domain knowledge and values. Our open-source tool runs directly in users' computational notebooks and web browsers, lowering the barrier to creating more responsible ML models. TimberTrek is available at the following public demo link: https://poloclub.github.io/timbertrek.

cs.HC↗

arXiv2022

Phase Transitions and Superconductivity in Ternary Hydride Li$_2$SiH$_6$ at High Pressures

We predicted a new ternary hydride Li$_2$SiH$_6$ at high pressures. A systematic structure search in Li$_2$SiH$_6$ compound reveals novel stable phases with intriguing electronic and phonon properties. It is found that Li$_2$SiH$_6$ is dynamically stable from ambient pressure up to 400 GPa with three novel phases: P312, P$\bar{3}$, and P$\bar{6}$2m. The calculation of electron-phonon coupling combined with Bardeen-Cooper-Schrieffer's argument indicates that this compound may be a candidate for high $T_c$ superconductors under high pressures. In particular, the maximum $T_c$ of $P\bar{6}2m$-Li$_2$SiH$_6$ at 400 GPa reaches 56 K. These findings may pave the way for obtaining room temperature superconductors in dense hydrogen-rich compounds.

cond-mat.mtrl-sci↗

arXiv2022

OPAL: Ontology-Aware Pretrained Language Model for End-to-End Task-Oriented Dialogue

This paper presents an ontology-aware pretrained language model (OPAL) for end-to-end task-oriented dialogue (TOD). Unlike chit-chat dialogue models, task-oriented dialogue models fulfill at least two task-specific modules: dialogue state tracker (DST) and response generator (RG). The dialogue state consists of the domain-slot-value triples, which are regarded as the user's constraints to search the domain-related databases. The large-scale task-oriented dialogue data with the annotated structured dialogue state usually are inaccessible. It prevents the development of the pretrained language model for the task-oriented dialogue. We propose a simple yet effective pretraining method to alleviate this problem, which consists of two pretraining phases. The first phase is to pretrain on large-scale contextual text data, where the structured information of the text is extracted by the information extracting tool. To bridge the gap between the pretraining method and downstream tasks, we design two pretraining tasks: ontology-like triple recovery and next-text generation, which simulates the DST and RG, respectively. The second phase is to fine-tune the pretrained model on the TOD data. The experimental results show that our proposed method achieves an exciting boost and get competitive performance even without any TOD data on CamRest676 and MultiWOZ benchmarks.

cs.CL↗

arXiv2022

Profiling Television Watching Behaviour Using Bayesian Hierarchical Joint Models for Time-to-Event and Count Data

Customer churn prediction is a valuable task in many industries. In telecommunications it presents great challenges, given the high dimensionality of the data, and how difficult it is to identify underlying frustration signatures, which may represent an important driver regarding future churn behaviour. Here, we propose a novel Bayesian hierarchical joint model that is able to characterise customer profiles based on how many events take place within different television watching journeys, and how long it takes between events. The model drastically reduces the dimensionality of the data from thousands of observations per customer to 11 customer-level parameter estimates and random effects. We test our methodology using data from 40 BT customers (20 active and 20 who eventually cancelled their subscription) whose TV watching behaviours were recorded from October to December 2019, totalling approximately half a million observations. Employing different machine learning techniques using the parameter estimates and random effects from the Bayesian hierarchical model as features yielded up to 92\% accuracy predicting churn, associated with 100\% true positive rates and false positive rates as low as 14\% on a validation set. Our proposed methodology represents an efficient way of reducing the dimensionality of the data, while at the same time maintaining high descriptive and predictive capabilities. We provide code to implement the Bayesian model at https://github.com/rafamoral/profiling_tv_watching_behaviour.

stat.ME↗

arXiv2022

Federated Zero-Shot Learning for Visual Recognition

Zero-shot learning is a learning regime that recognizes unseen classes by generalizing the visual-semantic relationship learned from the seen classes. To obtain an effective ZSL model, one may resort to curating training samples from multiple sources, which may inevitably raise the privacy concerns about data sharing across different organizations. In this paper, we propose a novel Federated Zero-Shot Learning FedZSL framework, which learns a central model from the decentralized data residing on edge devices. To better generalize to previously unseen classes, FedZSL allows the training data on each device sampled from the non-overlapping classes, which are far from the i.i.d. that traditional federated learning commonly assumes. We identify two key challenges in our FedZSL protocol: 1) the trained models are prone to be biased to the locally observed classes, thus failing to generalize to the unseen classes and/or seen classes appeared on other devices; 2) as each category in the training data comes from a single source, the central model is highly vulnerable to model replacement (backdoor) attacks. To address these issues, we propose three local objectives for visual-semantic alignment and cross-device alignment through relation distillation, which leverages the normalized class-wise covariance to regularize the consistency of the prediction logits across devices. To defend against the backdoor attacks, a feature magnitude defending technique is proposed. As malicious samples are less correlated to the given semantic attributes, the visual features of low magnitude will be discarded to stabilize model updates. The effectiveness and robustness of FedZSL are demonstrated by extensive experiments conducted on three zero-shot benchmark datasets.

cs.CV↗

arXiv2022

LPFS: Learnable Polarizing Feature Selection for Click-Through Rate Prediction

In industry, feature selection is a standard but necessary step to search for an optimal set of informative feature fields for efficient and effective training of deep Click-Through Rate (CTR) models. Most previous works measure the importance of feature fields by using their corresponding continuous weights from the model, then remove the feature fields with small weight values. However, removing many features that correspond to small but not exact zero weights will inevitably hurt model performance and not be friendly to hot-start model training. There is also no theoretical guarantee that the magnitude of weights can represent the importance, thus possibly leading to sub-optimal results if using these methods. To tackle this problem, we propose a novel Learnable Polarizing Feature Selection (LPFS) method using a smoothed-$\ell^0$ function in literature. Furthermore, we extend LPFS to LPFS++ by our newly designed smoothed-$\ell^0$-liked function to select a more informative subset of features. LPFS and LPFS++ can be used as gates inserted at the input of the deep network to control the active and inactive state of each feature. When training is finished, some gates are exact zero, while others are around one, which is particularly favored by the practical hot-start training in the industry, due to no damage to the model performance before and after removing the features corresponding to exact-zero gates. Experiments show that our methods outperform others by a clear margin, and have achieved great A/B test results in KuaiShou Technology.

cs.IR↗

arXiv2022

On Multiple-Antenna Techniques for Physical-Layer Range Security in the Terahertz Band

Terahertz (THz) communications have naturally promising physical layer security (PLS) performance in the angular domain due to the high directivity feature brought by the ultra-massive multiple-antenna techniques. However, traditional multiple-antenna techniques fail to combat eavesdroppers residing in the THz beam sector, even when the communication distances of legitimate users and eavesdroppers are different. This THz range security challenge motivates us to study new multiple-antenna techniques to provide THz range and angular security. In this paper, we first conduct a theoretical analysis of the secrecy capacity of the multiple antenna channel under the range security scenario. Based on this, the frequency diverse array, as a candidate multiple-antenna technique, is proven ineffective in addressing the range security problem. Then, motivated by the theoretical analysis, a novel widely-spaced array and beamforming design for THz range security are proposed, which realize communications in the near-field regions. A non-constrained optimum approaching (NCOA) algorithm is developed to achieve the optimal secrecy rate. Simulation results illustrate that under the range security scenario where the eavesdropper is inside the beam sector, our proposed widely-spaced antenna communication scheme can ensure a 6 bps/Hz secrecy rate when the transmit power is 10 dBm and the propagation distance is 10 m.

cs.IT↗

arXiv2022

UniDU: Towards A Unified Generative Dialogue Understanding Framework

With the development of pre-trained language models, remarkable success has been witnessed in dialogue understanding (DU). However, current DU approaches usually employ independent models for each distinct DU task without considering shared knowledge across different DU tasks. In this paper, we propose a unified generative dialogue understanding framework, named {\em UniDU}, to achieve effective information exchange across diverse DU tasks. Here, we reformulate all DU tasks into a unified prompt-based generative model paradigm. More importantly, a novel model-agnostic multi-task training strategy (MATS) is introduced to dynamically adapt the weights of diverse tasks for best knowledge sharing during training, based on the nature and available data of each task. Experiments on ten DU datasets covering five fundamental DU tasks show that the proposed UniDU framework largely outperforms task-specific well-designed methods on all tasks. MATS also reveals the knowledge-sharing structure of these tasks. Finally, UniDU obtains promising performance in the unseen dialogue domain, showing the great potential for generalization.

cs.CL↗

arXiv2022

Improving Multi-Interest Network with Stable Learning

Modeling users' dynamic preferences from historical behaviors lies at the core of modern recommender systems. Due to the diverse nature of user interests, recent advances propose the multi-interest networks to encode historical behaviors into multiple interest vectors. In real scenarios, the corresponding items of captured interests are usually retrieved together to get exposure and collected into training data, which produces dependencies among interests. Unfortunately, multi-interest networks may incorrectly concentrate on subtle dependencies among captured interests. Misled by these dependencies, the spurious correlations between irrelevant interests and targets are captured, resulting in the instability of prediction results when training and test distributions do not match. In this paper, we introduce the widely used Hilbert-Schmidt Independence Criterion (HSIC) to measure the degree of independence among captured interests and empirically show that the continuous increase of HSIC may harm model performance. Based on this, we propose a novel multi-interest network, named DEep Stable Multi-Interest Learning (DESMIL), which tries to eliminate the influence of subtle dependencies among captured interests via learning weights for training samples and make model concentrate more on underlying true causation. We conduct extensive experiments on public recommendation datasets, a large-scale industrial dataset and the synthetic datasets which simulate the out-of-distribution data. Experimental results demonstrate that our proposed DESMIL outperforms state-of-the-art models by a significant margin. Besides, we also conduct comprehensive model analysis to reveal the reason why DESMIL works to a certain extent.

cs.IR↗

arXiv2022

GSMFlow: Generation Shifts Mitigating Flow for Generalized Zero-Shot Learning

Generalized Zero-Shot Learning (GZSL) aims to recognize images from both the seen and unseen classes by transferring semantic knowledge from seen to unseen classes. It is a promising solution to take the advantage of generative models to hallucinate realistic unseen samples based on the knowledge learned from the seen classes. However, due to the generation shifts, the synthesized samples by most existing methods may drift from the real distribution of the unseen data. To address this issue, we propose a novel flow-based generative framework that consists of multiple conditional affine coupling layers for learning unseen data generation. Specifically, we discover and address three potential problems that trigger the generation shifts, i.e., semantic inconsistency, variance collapse, and structure disorder. First, to enhance the reflection of the semantic information in the generated samples, we explicitly embed the semantic information into the transformation in each conditional affine coupling layer. Second, to recover the intrinsic variance of the real unseen features, we introduce a boundary sample mining strategy with entropy maximization to discover more difficult visual variants of semantic prototypes and hereby adjust the decision boundary of the classifiers. Third, a relative positioning strategy is proposed to revise the attribute embeddings, guiding them to fully preserve the inter-class geometric structure and further avoid structure disorder in the semantic space. Extensive experimental results on four GZSL benchmark datasets demonstrate that GSMFlow achieves the state-of-the-art performance on GZSL.

cs.CV↗

arXiv2022

Cooperative Beamforming for RIS-Aided Cell-Free Massive MIMO Networks

The combination of cell-free massive multiple-input multiple-output (CF-mMIMO) and reconfigurable intelligent surface (RIS) is envisioned as a promising paradigm to improve network capacity and enhance coverage capability. However, to reap full benefits of RIS-aided CF-mMIMO, the main challenge is to efficiently design cooperative beamforming (CBF) at base stations (BSs), RISs, and users. Firstly, we investigate the fractional programing to convert the weighted sum-rate (WSR) maximization problem into a tractable optimization problem. Then, the alternating optimization framework is employed to decompose the transformed problem into a sequence of subproblems, i.e., hybrid BF (HBF) at BSs, passive BF at RISs, and combining at users. In particular, the alternating direction method of multipliers algorithm is utilized to solve the HBF subproblem at BSs. Concretely, the analog BF design with unit-modulus constraints is solved by the manifold optimization (MO) while we obtain a closed-form solution to the digital BF design that is essentially a convex least-square problem. Additionally, the passive BF at RISs and the analog combining at users are designed by primal-dual subgradient and MO methods. Moreover, considering heavy communication costs in conventional CF-mMIMO systems, we propose a partially-connected CF-mMIMO (P-CF-mMIMO) framework to decrease the number of connections among BSs and users. To better compromise WSR performance and network costs, we formulate the BS selection problem in the P-CF-mMIMO system as a binary integer quadratic programming (BIQP) problem, and develop a relaxed linear approximation algorithm to handle this BIQP problem. Finally, numerical results demonstrate superiorities of our proposed algorithms over baseline counterparts.

cs.IT↗

arXiv2022

Nanoscale Reconfigurable Intelligent Surface Design and Performance Analysis for Terahertz Communications

Terahertz (THz) communications have been envisioned as a promising enabler to provide ultra-high data transmission for sixth generation (6G) wireless networks. To tackle the blockage vulnerability brought by severe attenuation and poor diffraction of THz waves, a nanoscale reconfigurable intelligent surface (NRIS) is developed to smartly manipulate the propagation directions of incident THz waves. In this paper, the electric properties of the graphene are investigated by revealing the relationship between conductivity and applied voltages, and then an efficient hardware structure of electrically-controlled NRIS is designed based on Fabry-Perot resonance model. Particularly, the phase response of NRIS can be programmed up to 306.82 degrees. To analyze the hardware performance, we jointly design the passive and active beamforming for NRIS aided THz communication system. Particularly, an adaptive gradient descent (A-GD) algorithm is developed to optimize the phase shift matrix of NRIS by dynamically updating the step size during the iterative process. Finally, numerical results demonstrate the effectiveness of our designed hardware architecture as well as the developed algorithm.

cs.IT↗

arXiv2022

Data-Driven Chance Constrained Programs over Wasserstein Balls

We provide an exact deterministic reformulation for data-driven chance constrained programs over Wasserstein balls. For individual chance constraints as well as joint chance constraints with right-hand side uncertainty, our reformulation amounts to a mixed-integer conic program. In the special case of a Wasserstein ball with the $1$-norm or the $\infty$-norm, the cone is the nonnegative orthant, and the chance constrained program can be reformulated as a mixed-integer linear program. Our reformulation compares favourably to several state-of-the-art data-driven optimization schemes in our numerical experiments.

math.OC↗

arXiv2022

An Analytical Range-Angle Dependent Beam Focusing Model for Terahertz Linear Antenna Array

This paper considers a scenario in which the Terahertz (THz) transmitter equipped with a linear antenna array wishes to focus its beam to a desired spatial region in the array near-field. The goal is to compute the achievable spatial region and determine how the system parameters such as the carrier frequency, the array dimension and the user's location affect its beam focusing performance. First, based on a theorem from analytic geometry, we show that the achievable focusing spatial region constitutes a rotated ellipse, with the x and y coordinates denoting the range and angle, respectively. In this way, the determination of the spatial region is reduced to a problem of deriving the coverage of an ellipse. The achievable coverage is then obtained in closed form, and the construction of carrier frequency offsets that can analytically control the beam focusing performance is provided. Numerical results validate the theoretical findings and demonstrate the performance of the proposed method.

cs.IT↗

arXiv2022

SegDiscover: Visual Concept Discovery via Unsupervised Semantic Segmentation

Visual concept discovery has long been deemed important to improve interpretability of neural networks, because a bank of semantically meaningful concepts would provide us with a starting point for building machine learning models that exhibit intelligible reasoning process. Previous methods have disadvantages: either they rely on labelled support sets that incorporate human biases for objects that are "useful," or they fail to identify multiple concepts that occur within a single image. We reframe the concept discovery task as an unsupervised semantic segmentation problem, and present SegDiscover, a novel framework that discovers semantically meaningful visual concepts from imagery datasets with complex scenes without supervision. Our method contains three important pieces: generating concept primitives from raw images, discovering concepts by clustering in the latent space of a self-supervised pretrained encoder, and concept refinement via neural network smoothing. Experimental results provide evidence that our method can discover multiple concepts within a single image and outperforms state-of-the-art unsupervised methods on complex datasets such as Cityscapes and COCO-Stuff. Our method can be further used as a neural network explanation tool by comparing results obtained by different encoders.

cs.CV↗

arXiv2022

SC^2-PCR: A Second Order Spatial Compatibility for Efficient and Robust Point Cloud Registration

In this paper, we present a second order spatial compatibility (SC^2) measure based method for efficient and robust point cloud registration (PCR), called SC^2-PCR. Firstly, we propose a second order spatial compatibility (SC^2) measure to compute the similarity between correspondences. It considers the global compatibility instead of local consistency, allowing for more distinctive clustering between inliers and outliers at early stage. Based on this measure, our registration pipeline employs a global spectral technique to find some reliable seeds from the initial correspondences. Then we design a two-stage strategy to expand each seed to a consensus set based on the SC^2 measure matrix. Finally, we feed each consensus set to a weighted SVD algorithm to generate a candidate rigid transformation and select the best model as the final result. Our method can guarantee to find a certain number of outlier-free consensus sets using fewer samplings, making the model estimation more efficient and robust. In addition, the proposed SC^2 measure is general and can be easily plugged into deep learning based frameworks. Extensive experiments are carried out to investigate the performance of our method. Code will be available at \url{https://github.com/ZhiChen902/SC2-PCR}.

cs.CV↗