arXiv subjects

Zihao Chen

Publications and source records attributed to Zihao Chen.

89 records · Page 5Linked to original sources

arXiv2023

Break The Spell Of Total Correlation In betaTCVAE

In the absence of artificial labels, the independent and dependent features in the data are cluttered. How to construct the inductive biases of the model to flexibly divide and effectively contain features with different complexity is the main focal point of unsupervised disentangled representation learning. This paper proposes a new iterative decomposition path of total correlation and explains the disentangled representation ability of VAE from the perspective of model capacity allocation. The newly developed objective function combines latent variable dimensions into joint distribution while relieving the independence constraints of marginal distributions in combination, leading to latent variables with a more manipulable prior distribution. The novel model enables VAE to adjust the parameter capacity to divide dependent and independent data features flexibly. Experimental results on various datasets show an interesting relevance between model capacity and the latent variable grouping size, called the "V"-shaped best ELBO trajectory. Additionally, we empirically demonstrate that the proposed method obtains better disentangling performance with reasonable parameter capacity allocation.

cs.LG↗

arXiv2023

Understanding the limitation of Total Correlation Estimation Based on Mutual Information Bounds

The total correlation(TC) is a crucial index to measure the correlation between marginal distribution in multidimensional random variables, and it is frequently applied as an inductive bias in representation learning. Previous research has shown that the TC value can be estimated using mutual information boundaries through decomposition. However, we found through theoretical derivation and qualitative experiments that due to the use of importance sampling in the decomposition process, the bias of TC value estimated based on MI bounds will be amplified when the proposal distribution in the sampling differs significantly from the target distribution. To reduce estimation bias issues, we propose a TC estimation correction model based on supervised learning, which uses the training iteration loss sequence of the TC estimator based on MI bounds as input features to output the true TC value. Experiments show that our proposed method can improve the accuracy of TC estimation and eliminate the variance generated by the TC estimation process.

stat.ME↗

arXiv2023

An Interpretable Loan Credit Evaluation Method Based on Rule Representation Learner

The interpretability of model has become one of the obstacles to its wide application in the high-stake fields. The usual way to obtain interpretability is to build a black-box first and then explain it using the post-hoc methods. However, the explanations provided by the post-hoc method are not always reliable. Instead, we design an intrinsically interpretable model based on RRL(Rule Representation Learner) for the Lending Club dataset. Specifically, features can be divided into three categories according to their characteristics of themselves and build three sub-networks respectively, each of which is similar to a neural network with a single hidden layer but can be equivalently converted into a set of rules. During the training, we learned tricks from previous research to effectively train binary weights. Finally, our model is compared with the tree-based model. The results show that our model is much better than the interpretable decision tree in performance and close to other black-box, which is of practical significance to both financial institutions and borrowers. More importantly, our model is used to test the correctness of the explanations generated by the post-hoc method, the results show that the post-hoc method is not always reliable.

cs.LG↗

arXiv2023

JCSE: Contrastive Learning of Japanese Sentence Embeddings and Its Applications

Contrastive learning is widely used for sentence representation learning. Despite this prevalence, most studies have focused exclusively on English and few concern domain adaptation for domain-specific downstream tasks, especially for low-resource languages like Japanese, which are characterized by insufficient target domain data and the lack of a proper training strategy. To overcome this, we propose a novel Japanese sentence representation framework, JCSE (derived from ``Contrastive learning of Sentence Embeddings for Japanese''), that creates training data by generating sentences and synthesizing them with sentences available in a target domain. Specifically, a pre-trained data generator is finetuned to a target domain using our collected corpus. It is then used to generate contradictory sentence pairs that are used in contrastive learning for adapting a Japanese language model to a specific task in the target domain. Another problem of Japanese sentence representation learning is the difficulty of evaluating existing embedding methods due to the lack of benchmark datasets. Thus, we establish a comprehensive Japanese Semantic Textual Similarity (STS) benchmark on which various embedding models are evaluated. Based on this benchmark result, multiple embedding methods are chosen and compared with JCSE on two domain-specific tasks, STS in a clinical domain and information retrieval in an educational domain. The results show that JCSE achieves significant performance improvement surpassing direct transfer and other training strategies. This empirically demonstrates JCSE's effectiveness and practicability for downstream tasks of a low-resource language.

cs.CL↗

arXiv2023

Simultaneously realizing thermal and electromagnetic cloaking by multi-physical null medium

Simultaneously manipulating multiple physical fields plays an important role in the increasingly complex integrated systems, aerospace equipment, biochemical productions, etc. For on-chip systems with high integration level (e.g., electronic/photonic chips and radio-frequency/microwave circuits), where both electromagnetic information/energy transporting and heat dissipation/recovery need to be considered, the precise and efficient control of the propagation of electromagnetic waves and heat fluxes simultaneously is particularly important. In this study, we propose a graphical designing method based on thermal-electromagnetic null medium to simultaneously control the propagation of electromagnetic waves and thermal fields according to the pre-designed paths. A thermal-electromagnetic cloak, which can create a cloaking effect on both electromagnetic waves and thermal fields simultaneously, is designed by thermal-electromagnetic surface transformation and verified by both numerical simulations and experimental measurements. The thermal-electromagnetic surface transformation proposed in this study provides a new methodology for simultaneous controlling on electromagnetic and temperature fields, which can be used to realize a series of novel thermal-electromagnetic devices such as thermal-electromagnetic shifter, splitter, bender, multiplexer and mode converter. The designed thermal-electromagnetic cloak opens up new ways to create a concealed region where any object within it does not create any disturbance to the external electromagnetic waves and temperature fields simultaneously, which may have significant applications in improving thermal-electromagnetic compatibility problem, protecting of thermal-electromagnetic sensitive components, and improving efficiency of energy usage for complex on-chip systems.

physics.app-ph↗

arXiv2022

Active beam steering enabled by photonic crystal surface emitting laser

Emitting light towards on-demand directions is important for various optoelectronic applications, such as optical communication, displaying, and ranging. However, almost all existing directional emitters are assemblies of passive optical antennae and external light sources, which are usually bulky, fragile, and with unendurable loss of light power. Here we theoretically propose and experimentally demonstrate a new conceptual design of directional emitter, by using a single surface-emitting laser source itself to achieve dynamically controlled beam steering. The laser is built on photonic crystals that operates near the band edges in the continuum. By shrinking laser sizes into tens-of-wavelength, the optical modes quantize in three-dimensional momentum space, and each of them directionally radiates towards the far-field. Further utilizing the luminescence spectrum shifting effect under current injection, we consecutively select a sequence of modes into lasing action and show the laser maintaining in single mode operation with linewidths at a minimum of $1.8$ MHz and emitting power of $\sim$ ten milliwatts, and we demonstrate fast beam steering across a range of $3.2^\circ \times 4^\circ$ in a time scale of $500$ nanoseconds. Our work proposes a novel method for on-chip active beam steering, which could pave the way for the development of automotive, industrial, and robotic applications.

physics.optics↗

arXiv2022

Magnitude-image based data-consistent deep learning method for MRI super resolution

Magnetic Resonance Imaging (MRI) is important in clinic to produce high resolution images for diagnosis, but its acquisition time is long for high resolution images. Deep learning based MRI super resolution methods can reduce scan time without complicated sequence programming, but may create additional artifacts due to the discrepancy between training data and testing data. Data consistency layer can improve the deep learning results but needs raw k-space data. In this work, we propose a magnitude-image based data consistency deep learning MRI super resolution method to improve super resolution images' quality without raw k-space data. Our experiments show that the proposed method can improve NRMSE and SSIM of super resolution images compared to the same Convolutional Neural Network (CNN) block without data consistency module.

eess.IV↗

arXiv2022

Low-threshold nanolasers based on miniaturized bound states in the continuum

The pursuit of compact lasers with low-thresholds has imposed strict requirements on tight light confinements with minimized radiation losses. Bound states in the continuum (BICs) have been recently demonstrated as an effective mechanism to trap light along the out-of-plane direction, paving the way to low-threshold lasers. To date, most reported BIC lasers are still bulky due to the absence of in-plane light confinement. In this work, we combine BICs and photonic band gaps to realize three-dimensional (3D) light confinements, as referred to miniaturized (mini-) BICs. Together with 3D carrier confinements provided by quantum dots (QDs) as optical gain materials, we have realized highly-compact active BIC resonators with a record-high quality ($Q$) factor up to 32500, which enables single-mode continuous wave (CW) lasing with the lowest threshold of 80 W/cm$^{2}$ among the reported BIC lasers. In addidtion, our photon statistics measurements under both CW and pulsed excitations confirm the occurence of the phase transition from spontaneous emission to stimulated emission, further suggesting that conventional criteria of input-output and linewidth are not sufficient for claiming nanoscale lasing. Our work reveal a via path towards compact BIC lasers with ultra-low power consumption and potentially boost the applications in cavity quantum electrodynamics (QEDs), nonlinear optics and integrated photonics.

physics.optics↗

arXiv2022

Where is VALDO? VAscular Lesions Detection and segmentatiOn challenge at MICCAI 2021

Imaging markers of cerebral small vessel disease provide valuable information on brain health, but their manual assessment is time-consuming and hampered by substantial intra- and interrater variability. Automated rating may benefit biomedical research, as well as clinical assessment, but diagnostic reliability of existing algorithms is unknown. Here, we present the results of the \textit{VAscular Lesions DetectiOn and Segmentation} (\textit{Where is VALDO?}) challenge that was run as a satellite event at the international conference on Medical Image Computing and Computer Aided Intervention (MICCAI) 2021. This challenge aimed to promote the development of methods for automated detection and segmentation of small and sparse imaging markers of cerebral small vessel disease, namely enlarged perivascular spaces (EPVS) (Task 1), cerebral microbleeds (Task 2) and lacunes of presumed vascular origin (Task 3) while leveraging weak and noisy labels. Overall, 12 teams participated in the challenge proposing solutions for one or more tasks (4 for Task 1 - EPVS, 9 for Task 2 - Microbleeds and 6 for Task 3 - Lacunes). Multi-cohort data was used in both training and evaluation. Results showed a large variability in performance both across teams and across tasks, with promising results notably for Task 1 - EPVS and Task 2 - Microbleeds and not practically useful results yet for Task 3 - Lacunes. It also highlighted the performance inconsistency across cases that may deter use at an individual level, while still proving useful at a population level.

cs.CV↗

arXiv2022

Experimental study of secure quantum key distribution with source and detection imperfections

The quantum key distribution (QKD), guaranteed by the principle of quantum physics, is a promising solution for future secure information and communication technology. However, device imperfections compromise the security of real-life QKD systems, restricting the wide deployment of QKD. This study reports a decoy-state BB84 QKD experiment that considers both source and detection imperfections. In particular, we achieved a rigorous finite-key security bound over fiber links of up to 75 km by applying a systematic performance analysis. Furthermore, our study considers more device imperfections than most previous experiments, and the proposed theory can be extended to other discrete-variable QKD systems. These features constitute a crucial step toward securing QKD with imperfect practical devices.

quant-ph↗

arXiv2022

Data-Consistent Non-Cartesian Deep Subspace Learning for Efficient Dynamic MR Image Reconstruction

Non-Cartesian sampling with subspace-constrained image reconstruction is a popular approach to dynamic MRI, but slow iterative reconstruction limits its clinical application. Data-consistent (DC) deep learning can accelerate reconstruction with good image quality, but has not been formulated for non-Cartesian subspace imaging. In this study, we propose a DC non-Cartesian deep subspace learning framework for fast, accurate dynamic MR image reconstruction. Four novel DC formulations are developed and evaluated: two gradient decent approaches, a directly solved approach, and a conjugate gradient approach. We applied a U-Net model with and without DC layers to reconstruct T1-weighted images for cardiac MR Multitasking (an advanced multidimensional imaging method), comparing our results to the iteratively reconstructed reference. Experimental results show that the proposed framework significantly improves reconstruction accuracy over the U-Net model without DC, while significantly accelerating the reconstruction over conventional iterative reconstruction.

eess.IV↗

arXiv2021

DeepShift: Towards Multiplication-Less Neural Networks

The high computation, memory, and power budgets of inferring convolutional neural networks (CNNs) are major bottlenecks of model deployment to edge computing platforms, e.g., mobile devices and IoT. Moreover, training CNNs is time and energy-intensive even on high-grade servers. Convolution layers and fully connected layers, because of their intense use of multiplications, are the dominant contributor to this computation budget. We propose to alleviate this problem by introducing two new operations: convolutional shifts and fully-connected shifts which replace multiplications with bitwise shift and sign flipping during both training and inference. During inference, both approaches require only 5 bits (or less) to represent the weights. This family of neural network architectures (that use convolutional shifts and fully connected shifts) is referred to as DeepShift models. We propose two methods to train DeepShift models: DeepShift-Q which trains regular weights constrained to powers of 2, and DeepShift-PS that trains the values of the shifts and sign flips directly. Very close accuracy, and in some cases higher accuracy, to baselines are achieved. Converting pre-trained 32-bit floating-point baseline models of ResNet18, ResNet50, VGG16, and GoogleNet to DeepShift and training them for 15 to 30 epochs, resulted in Top-1/Top-5 accuracies higher than that of the original model. Last but not least, we implemented the convolutional shifts and fully connected shift GPU kernels and showed a reduction in latency time of 25% when inferring ResNet18 compared to unoptimized multiplication-based GPU kernels. The code can be found at https://github.com/mostafaelhoushi/DeepShift.

cs.LG↗

arXiv2021

Observation of miniaturized bound states in the continuum with ultra-high quality factors

Light trapping is a constant pursuit in photonics because of its importance in science and technology. Many mechanisms have been explored, including the use of mirrors made of materials or structures that forbid outgoing waves, and bound states in the continuum that are mirror-less but based on topology. Here we report a compound method, combing mirrors and bound states in the continuum in an optimized way, to achieve a class of on-chip optical cavities that have high quality factors and small modal volumes. Specifically, light is trapped in the transverse direction by the photonic band gap of the lateral hetero-structure and confined in the vertical direction by the constellation of multiple bound states in the continuum. As a result, unlike most bound states in the continuum found in photonic crystal slabs that are de-localized Bloch modes, we achieve light-trapping in all three dimensions and experimentally demonstrate quality factors as high as $Q = 1.09 \times 10^6$ and modal volumes as low as $V = 3.56~ μm^3$ in the telecommunication regime. We further prove the robustness of our method through the statistical study of multiple fabricated devices. Our work provides a new method of light trapping, which can find potential applications in photonic integration, nonlinear optics and quantum computing.

physics.optics↗

arXiv2020

CATCH: Context-based Meta Reinforcement Learning for Transferrable Architecture Search

Neural Architecture Search (NAS) achieved many breakthroughs in recent years. In spite of its remarkable progress, many algorithms are restricted to particular search spaces. They also lack efficient mechanisms to reuse knowledge when confronting multiple tasks. These challenges preclude their applicability, and motivate our proposal of CATCH, a novel Context-bAsed meTa reinforcement learning (RL) algorithm for transferrable arChitecture searcH. The combination of meta-learning and RL allows CATCH to efficiently adapt to new tasks while being agnostic to search spaces. CATCH utilizes a probabilistic encoder to encode task properties into latent context variables, which then guide CATCH's controller to quickly "catch" top-performing networks. The contexts also assist a network evaluator in filtering inferior candidates and speed up learning. Extensive experiments demonstrate CATCH's universality and search efficiency over many other widely-recognized algorithms. It is also capable of handling cross-domain architecture search as competitive networks on ImageNet, COCO, and Cityscapes are identified. This is the first work to our knowledge that proposes an efficient transferrable NAS solution while maintaining robustness across various settings.

cs.LG↗

arXiv2019

Accelerating Training using Tensor Decomposition

Tensor decomposition is one of the well-known approaches to reduce the latency time and number of parameters of a pre-trained model. However, in this paper, we propose an approach to use tensor decomposition to reduce training time of training a model from scratch. In our approach, we train the model from scratch (i.e., randomly initialized weights) with its original architecture for a small number of epochs, then the model is decomposed, and then continue training the decomposed model till the end. There is an optional step in our approach to convert the decomposed architecture back to the original architecture. We present results of using this approach on both CIFAR10 and Imagenet datasets, and show that there can be upto 2x speed up in training time with accuracy drop of upto 1.5% only, and in other cases no accuracy drop. This training acceleration approach is independent of hardware and is expected to have similar speed ups on both CPU and GPU platforms.

cs.CV↗

arXiv2016

Communication Lower Bounds for Distributed Convex Optimization: Partition Data on Features

Recently, there has been an increasing interest in designing distributed convex optimization algorithms under the setting where the data matrix is partitioned on features. Algorithms under this setting sometimes have many advantages over those under the setting where data is partitioned on samples, especially when the number of features is huge. Therefore, it is important to understand the inherent limitations of these optimization problems. In this paper, with certain restrictions on the communication allowed in the procedures, we develop tight lower bounds on communication rounds for a broad class of non-incremental algorithms under this setting. We also provide a lower bound on communication rounds for a class of (randomized) incremental algorithms.

cs.LG↗

arXiv2016

A Proximal Stochastic Quasi-Newton Algorithm

In this paper, we discuss the problem of minimizing the sum of two convex functions: a smooth function plus a non-smooth function. Further, the smooth part can be expressed by the average of a large number of smooth component functions, and the non-smooth part is equipped with a simple proximal mapping. We propose a proximal stochastic second-order method, which is efficient and scalable. It incorporates the Hessian in the smooth part of the function and exploits multistage scheme to reduce the variance of the stochastic gradient. We prove that our method can achieve linear rate of convergence.

cs.LG↗