arXiv subjects

Ziyu Liu

Publications and source records attributed to Ziyu Liu.

At least 91 records · Page 5Linked to original sources

arXiv2023

A multifractal analysis for uniform level sets with constraints on word appearance

Given a topologically transitive subshift of finite type, the $u$-dimension of the $α$-level set of the quotient of Birkhoff sums is a well-studied topic. In this paper, we consider the uniform $α$-level set, which is a subset of the $α$-level set. We shall show that the $α$-level set and the uniform $α$-level set have the same $u$-dimension. Moreover, we also consider two types of subsets of the $α$-level set, whose elements are sequences satisfying some constraints on their subwords. We shall show that for $α$ not on the boundary of the dimension spectrum, these subsets also have the same $u$-dimension as the $α$-level set. As an application, we shall perform a multifractal analysis of the Hoelder regularity of a Gibbs measure on a one-dimensional manifold.

math.DS↗

arXiv2023

Introducing Shape Prior Module in Diffusion Model for Medical Image Segmentation

Medical image segmentation is critical for diagnosing and treating spinal disorders. However, the presence of high noise, ambiguity, and uncertainty makes this task highly challenging. Factors such as unclear anatomical boundaries, inter-class similarities, and irrational annotations contribute to this challenge. Achieving both accurate and diverse segmentation templates is essential to support radiologists in clinical practice. In recent years, denoising diffusion probabilistic modeling (DDPM) has emerged as a prominent research topic in computer vision. It has demonstrated effectiveness in various vision tasks, including image deblurring, super-resolution, anomaly detection, and even semantic representation generation at the pixel level. Despite the robustness of existing diffusion models in visual generation tasks, they still struggle with discrete masks and their various effects. To address the need for accurate and diverse spine medical image segmentation templates, we propose an end-to-end framework called VerseDiff-UNet, which leverages the denoising diffusion probabilistic model (DDPM). Our approach integrates the diffusion model into a standard U-shaped architecture. At each step, we combine the noise-added image with the labeled mask to guide the diffusion direction accurately towards the target region. Furthermore, to capture specific anatomical a priori information in medical images, we incorporate a shape a priori module. This module efficiently extracts structural semantic information from the input spine images. We evaluate our method on a single dataset of spine images acquired through X-ray imaging. Our results demonstrate that VerseDiff-UNet significantly outperforms other state-of-the-art methods in terms of accuracy while preserving the natural features and variations of anatomy.

eess.IV↗

arXiv2023

Relation between the eventual continuity and the e-property for Markov-Feller semigroups

We investigate the relation between the e-property and the eventual continuity, or called the asymptotic equicontinuity, which is a generalization of the e-property. We prove that, for any discrete-time or strongly continuous continuous-time eventually continuous Markov-Feller semigroup with an ergodic measure, if the interior of the support of the ergodic measure is nonempty, then the e-property is satisfied on the interior of the support. In particular, it implies that, restricted on the support of each ergodic measure, the e-property and the eventual continuity are equivalent for the discrete-time and the strongly continuous continuous-time Markov-Feller semigroups.

math.PR↗

arXiv2022

On Privacy and Personalization in Cross-Silo Federated Learning

While the application of differential privacy (DP) has been well-studied in cross-device federated learning (FL), there is a lack of work considering DP and its implications for cross-silo FL, a setting characterized by a limited number of clients each containing many data subjects. In cross-silo FL, usual notions of client-level DP are less suitable as real-world privacy regulations typically concern the in-silo data subjects rather than the silos themselves. In this work, we instead consider an alternative notion of silo-specific sample-level DP, where silos set their own privacy targets for their local examples. Under this setting, we reconsider the roles of personalization in federated learning. In particular, we show that mean-regularized multi-task learning (MR-MTL), a simple personalization framework, is a strong baseline for cross-silo FL: under stronger privacy requirements, silos are incentivized to federate more with each other to mitigate DP noise, resulting in consistent improvements relative to standard baseline methods. We provide an empirical study of competing methods as well as a theoretical characterization of MR-MTL for mean estimation, highlighting the interplay between privacy and cross-silo data heterogeneity. Our work serves to establish baselines for private cross-silo FL as well as identify key directions of future work in this area.

cs.LG↗

arXiv2022

Motley: Benchmarking Heterogeneity and Personalization in Federated Learning

Personalized federated learning considers learning models unique to each client in a heterogeneous network. The resulting client-specific models have been purported to improve metrics such as accuracy, fairness, and robustness in federated networks. However, despite a plethora of work in this area, it remains unclear: (1) which personalization techniques are most effective in various settings, and (2) how important personalization truly is for realistic federated applications. To better answer these questions, we propose Motley, a benchmark for personalized federated learning. Motley consists of a suite of cross-device and cross-silo federated datasets from varied problem domains, as well as thorough evaluation metrics for better understanding the possible impacts of personalization. We establish baselines on the benchmark by comparing a number of representative personalized federated learning methods. These initial results highlight strengths and weaknesses of existing approaches, and raise several open questions for the community. Motley aims to provide a reproducible means with which to advance developments in personalized and heterogeneity-aware federated learning, as well as the related areas of transfer learning, meta-learning, and multi-task learning.

cs.LG↗

arXiv2022

The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation

We consider training models on private data that are distributed across user devices. To ensure privacy, we add on-device noise and use secure aggregation so that only the noisy sum is revealed to the server. We present a comprehensive end-to-end system, which appropriately discretizes the data and adds discrete Gaussian noise before performing secure aggregation. We provide a novel privacy analysis for sums of discrete Gaussians and carefully analyze the effects of data quantization and modular summation arithmetic. Our theoretical guarantees highlight the complex tension between communication, privacy, and accuracy. Our extensive experimental results demonstrate that our solution is essentially able to match the accuracy to central differential privacy with less than 16 bits of precision per value.

cs.LG↗

arXiv2022

Domain Textures in the Fractional Quantum Hall Effect

Impacts of domain textures on low-lying neutral excitations in the bulk of fractional quantum Hall effect (FQHE) systems are probed by resonant inelastic light scattering. We demonstrate that large domains of quantum fluids support long-wavelength neutral collective excitations with well-defined wave vector (momentum) dispersion that could be interpreted by theories for uniform phases. Access to dispersive low-lying neutral collective modes in large domains of FQHE fluids such as long wavelength magnetorotons at filling factor v=1/3 offer significant experimental access to strong electron correlation physics in the FQHE.

cond-mat.mes-hall↗

arXiv2021

The Skellam Mechanism for Differentially Private Federated Learning

We introduce the multi-dimensional Skellam mechanism, a discrete differential privacy mechanism based on the difference of two independent Poisson random variables. To quantify its privacy guarantees, we analyze the privacy loss distribution via a numerical evaluation and provide a sharp bound on the Rényi divergence between two shifted Skellam distributions. While useful in both centralized and distributed privacy applications, we investigate how it can be applied in the context of federated learning with secure aggregation under communication constraints. Our theoretical findings and extensive experimental evaluations demonstrate that the Skellam mechanism provides the same privacy-accuracy trade-offs as the continuous Gaussian mechanism, even when the precision is low. More importantly, Skellam is closed under summation and sampling from it only requires sampling from a Poisson distribution -- an efficient routine that ships with all machine learning and data analysis software packages. These features, along with its discrete nature and competitive privacy-accuracy trade-offs, make it an attractive practical alternative to the newly introduced discrete Gaussian mechanism.

cs.LG↗

arXiv2021

ECG-Based Heart Arrhythmia Diagnosis Through Attentional Convolutional Neural Networks

Electrocardiography (ECG) signal is a highly applied measurement for individual heart condition, and much effort have been endeavored towards automatic heart arrhythmia diagnosis based on machine learning. However, traditional machine learning models require large investment of time and effort for raw data preprocessing and feature extraction, as well as challenged by poor classification performance. Here, we propose a novel deep learning model, named Attention-Based Convolutional Neural Networks (ABCNN) that taking advantage of CNN and multi-head attention, to directly work on the raw ECG signals and automatically extract the informative dependencies for accurate arrhythmia detection. To evaluate the proposed approach, we conduct extensive experiments over a benchmark ECG dataset. Our main task is to find the arrhythmia from normal heartbeats and, at the meantime, accurately recognize the heart diseases from five arrhythmia types. We also provide convergence analysis of ABCNN and intuitively show the meaningfulness of extracted representation through visualization. The experimental results show that the proposed ABCNN outperforms the widely used baselines, which puts one step closer to intelligent heart disease diagnosis system.

eess.SP↗

arXiv2021

A Photonics-based superheterodyne RF reception approach

A novel photonics-based RF reception approach is proposed as a competitive solution to meet the current challenges of photonic-based approaches and to realize high performances at the same time. The proposed approach adopts the superheterodyne configuration by a combination manner of electronic techniques and photonic techniques, including the ultrawideband generation of optical LO, the two-stage photonic superheterodyne frequency conversion and the real-time IF compensation. An engineering prototype has been developed and its performance has been evaluated in the laboratory environment. The experiment results preliminarily verify the feasibility of the proposed approach and its engineering potential. The typical performances are as follows: 0.1 GHz~ 45GHz operation spectrum range (>40 GHz), 900 MHz instantaneous bandwidth, 101 dBHz2/3 SFDR and 130 dBHz LDR, image rejections of ~80 dB for 1st frequency conversion and >90 dB for 2nd frequency conversion.

physics.app-ph↗

arXiv2021

Provenance-based Data Skipping (TechReport)

Database systems analyze queries to determine upfront which data is needed for answering them and use indexes and other physical design techniques to speed-up access to that data. However, for important classes of queries, e.g., HAVING and top-k queries, it is impossible to determine up-front what data is relevant. To overcome this limitation, we develop provenance-based data skipping (PBDS), a novel approach that generates provenance sketches to concisely encode what data is relevant for a query. Once a provenance sketch has been captured it is used to speed up subsequent queries. PBDS can exploit physical design artifacts such as indexes and zone maps. Our approach significantly improves performance for both disk-based and main-memory database systems.

cs.DB↗

arXiv2021

A topological photonic ring-resonator for on-chip channel filters

A topologically protected ring-resonator formed in valley photonic crystals is proposed and fabricated on a silicon slab. The unidirectional transmission and robustness against structure defects of its resonant modes are illustrated. Coupled with topological waveguides, the topological ring is functioned as notch and channel-drop filters. The work opens up a new avenue for developing advanced chip-integrated photonic circuits with attributes of topological photonics.

physics.optics↗

arXiv2021

Observation of flat bands in gated semiconductor artificial graphene

Flat bands near M points in the Brillouin zone are key features of honeycomb symmetry in artificial graphene (AG) where electrons may condense into novel correlated phases. Here we report the observation of van Hove singularity doublet of AG in GaAs quantum well transistors, which presents the evidence of flat bands in semiconductor AG. Two emerging peaks in photoluminescence spectra tuned by backgate voltages probe the singularity doublet of AG flat bands, and demonstrate their accessibility to the Fermi level. As the Fermi level crosses the doublet, the spectra display dramatic stability against electron density, indicating interplays between electron-electron interactions and honeycomb symmetry. Our results provide a new flexible platform to explore intriguing flat band physics.

cond-mat.mes-hall↗

arXiv2020

Learning Implicit Credit Assignment for Cooperative Multi-Agent Reinforcement Learning

We present a multi-agent actor-critic method that aims to implicitly address the credit assignment problem under fully cooperative settings. Our key motivation is that credit assignment among agents may not require an explicit formulation as long as (1) the policy gradients derived from a centralized critic carry sufficient information for the decentralized agents to maximize their joint action value through optimal cooperation and (2) a sustained level of exploration is enforced throughout training. Under the centralized training with decentralized execution (CTDE) paradigm, we achieve the former by formulating the centralized critic as a hypernetwork such that a latent state representation is integrated into the policy gradients through its multiplicative association with the stochastic policies; to achieve the latter, we derive a simple technique called adaptive entropy regularization where magnitudes of the entropy gradients are dynamically rescaled based on the current policy stochasticity to encourage consistent levels of exploration. Our algorithm, referred to as LICA, is evaluated on several benchmarks including the multi-agent particle environments and a set of challenging StarCraft II micromanagement tasks, and we show that LICA significantly outperforms previous methods.

cs.LG↗

arXiv2020

Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition

Spatial-temporal graphs have been widely used by skeleton-based action recognition algorithms to model human action dynamics. To capture robust movement patterns from these graphs, long-range and multi-scale context aggregation and spatial-temporal dependency modeling are critical aspects of a powerful feature extractor. However, existing methods have limitations in achieving (1) unbiased long-range joint relationship modeling under multi-scale operators and (2) unobstructed cross-spacetime information flow for capturing complex spatial-temporal dependencies. In this work, we present (1) a simple method to disentangle multi-scale graph convolutions and (2) a unified spatial-temporal graph convolutional operator named G3D. The proposed multi-scale aggregation scheme disentangles the importance of nodes in different neighborhoods for effective long-range modeling. The proposed G3D module leverages dense cross-spacetime edges as skip connections for direct information propagation across the spatial-temporal graph. By coupling these proposals, we develop a powerful feature extractor named MS-G3D based on which our model outperforms previous state-of-the-art methods on three large-scale datasets: NTU RGB+D 60, NTU RGB+D 120, and Kinetics Skeleton 400.

cs.CV↗

arXiv2019

LightMC: A Dynamic and Efficient Multiclass Decomposition Algorithm

Multiclass decomposition splits a multiclass classification problem into a series of independent binary learners and recomposes them by combining their outputs to reconstruct the multiclass classification results. Three widely-used realizations of such decomposition methods are One-Versus-All (OVA), One-Versus-One (OVO), and Error-Correcting-Output-Code (ECOC). While OVA and OVO are quite simple, both of them assume all classes are orthogonal which neglect the latent correlation between classes in real-world. Error-Correcting-Output-Code (ECOC) based decomposition methods, on the other hand, are more preferable due to its integration of the correlation among classes. However, the performance of existing ECOC-based methods highly depends on the design of coding matrix and decoding strategy. Unfortunately, it is quite uncertain and time-consuming to discover an effective coding matrix with appropriate decoding strategy. To address this problem, we propose LightMC, an efficient dynamic multiclass decomposition algorithm. Instead of using fixed coding matrix and decoding strategy, LightMC uses a differentiable decoding strategy, which enables it to dynamically optimize the coding matrix and decoding strategy, toward increasing the overall accuracy of multiclass classification, via back propagation jointly with the training of base learners in an iterative way. Empirical experimental results on several public large-scale multiclass classification datasets have demonstrated the effectiveness of LightMC in terms of both good accuracy and high efficiency.

cs.LG↗

arXiv2019

Error Analysis for the Particle Filter: Methods and Theoretical Support

The particle filter is a popular Bayesian filtering algorithm for use in cases where the state-space model is nonlinear and/or the random terms (initial state or noises) are non-Gaussian distributed. We study the behavior of the error in the particle filter algorithm as the number of particles gets large. After a decomposition of the error into two terms, we show that the difference between the estimator and the conditional mean is asymptotically normal when the resampling is done at every step in the filtering process. Two nonlinear/non-Gaussian examples are tested to verify this conclusion.

stat.CO↗

arXiv2019

Towards Understanding Chinese Checkers with Heuristics, Monte Carlo Tree Search, and Deep Reinforcement Learning

The game of Chinese Checkers is a challenging traditional board game of perfect information that differs from other traditional games in two main aspects: first, unlike Chess, all checkers remain indefinitely in the game and hence the branching factor of the search tree does not decrease as the game progresses; second, unlike Go, there are also no upper bounds on the depth of the search tree since repetitions and backward movements are allowed. Therefore, even in a restricted game instance, the state-space of the game can still be unbounded, making it challenging for a computer program to excel. In this work, we present an approach that effectively combines the use of heuristics, Monte Carlo tree search, and deep reinforcement learning for building a Chinese Checkers agent without the use of any human game-play data. Experiment results show that our agent is competent under different scenarios and reaches the level of experienced human players.

cs.LG↗