arXiv subjects

Jia Liu

Publications and source records attributed to Jia Liu.

At least 307 records · Page 17Linked to original sources

arXiv2021

Generative Steganography with Kerckhoffs' Principle

The distortion in steganography that usually comes from the modification or recoding on the cover image during the embedding process leaves the steganalyzer with possibility of discriminating. Faced with such a risk, we propose generative steganography with Kerckhoffs' principle (GSK) in this letter. In GSK, the secret messages are generated by a cover image using a generator rather than embedded into the cover, thus resulting in no modifications in the cover. To ensure the security, the generators are trained to meet Kerckhoffs' principle based on generative adversarial networks (GAN). Everything about the GSK system, except the extraction key, is public knowledge for the receivers. The secret messages can be outputted by the generator if and only if the extraction key and the cover image are both inputted. In the generator training procedures, there are two GANs, Message- GAN and Cover-GAN, designed to work jointly making the generated results under the control of the extraction key and the cover image. We provide experimental results on the training process and give an example of the working process by adopting a generator trained on MNIST, which demonstrate that GSK can use a cover image without any modification to generate messages, and without the extraction key or the cover image, only meaningless results would be obtained.

cs.MM↗

arXiv2021

STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learning

Federated Learning (FL) refers to the paradigm where multiple worker nodes (WNs) build a joint model by using local data. Despite extensive research, for a generic non-convex FL problem, it is not clear, how to choose the WNs' and the server's update directions, the minibatch sizes, and the local update frequency, so that the WNs use the minimum number of samples and communication rounds to achieve the desired solution. This work addresses the above question and considers a class of stochastic algorithms where the WNs perform a few local updates before communication. We show that when both the WN's and the server's directions are chosen based on a stochastic momentum estimator, the algorithm requires $\tilde{\mathcal{O}}(ε^{-3/2})$ samples and $\tilde{\mathcal{O}}(ε^{-1})$ communication rounds to compute an $ε$-stationary solution. To the best of our knowledge, this is the first FL algorithm that achieves such {\it near-optimal} sample and communication complexities simultaneously. Further, we show that there is a trade-off curve between local update frequencies and local minibatch sizes, on which the above sample and communication complexities can be maintained. Finally, we show that for the classical FedAvg (a.k.a. Local SGD, which is a momentum-less special case of the STEM), a similar trade-off curve exists, albeit with worse sample and communication complexities. Our insights on this trade-off provides guidelines for choosing the four important design elements for FL algorithms, the update frequency, directions, and minibatch sizes to achieve the best performance.

cs.LG↗

arXiv2021

Transformation Importance with Applications to Cosmology

Machine learning lies at the heart of new possibilities for scientific discovery, knowledge generation, and artificial intelligence. Its potential benefits to these fields requires going beyond predictive accuracy and focusing on interpretability. In particular, many scientific problems require interpretations in a domain-specific interpretable feature space (e.g. the frequency domain) whereas attributions to the raw features (e.g. the pixel space) may be unintelligible or even misleading. To address this challenge, we propose TRIM (TRansformation IMportance), a novel approach which attributes importances to features in a transformed space and can be applied post-hoc to a fully trained model. TRIM is motivated by a cosmological parameter estimation problem using deep neural networks (DNNs) on simulated data, but it is generally applicable across domains/models and can be combined with any local interpretation method. In our cosmology example, combining TRIM with contextual decomposition shows promising results for identifying which frequencies a DNN uses, helping cosmologists to understand and validate that the model learns appropriate physical features rather than simulation artifacts.

stat.ML↗

arXiv2021

CFedAvg: Achieving Efficient Communication and Fast Convergence in Non-IID Federated Learning

Federated learning (FL) is a prevailing distributed learning paradigm, where a large number of workers jointly learn a model without sharing their training data. However, high communication costs could arise in FL due to large-scale (deep) learning models and bandwidth-constrained connections. In this paper, we introduce a communication-efficient algorithmic framework called CFedAvg for FL with non-i.i.d. datasets, which works with general (biased or unbiased) SNR-constrained compressors. We analyze the convergence rate of CFedAvg for non-convex functions with constant and decaying learning rates. The CFedAvg algorithm can achieve an $\mathcal{O}(1 / \sqrt{mKT} + 1 / T)$ convergence rate with a constant learning rate, implying a linear speedup for convergence as the number of workers increases, where $K$ is the number of local steps, $T$ is the number of total communication rounds, and $m$ is the total worker number. This matches the convergence rate of distributed/federated learning without compression, thus achieving high communication efficiency while not sacrificing learning accuracy in FL. Furthermore, we extend CFedAvg to cases with heterogeneous local steps, which allows different workers to perform a different number of local steps to better adapt to their own circumstances. The interesting observation in general is that the noise/variance introduced by compressors does not affect the overall convergence rate order for non-i.i.d. FL. We verify the effectiveness of our CFedAvg algorithm on three datasets with two gradient compression schemes of different compression ratios.

cs.LG↗

arXiv2021

Searching for lepton portal dark matter with colliders and gravitational waves

We study the lepton portal dark matter (DM) model in which the relic abundance is determined by the portal coupling among the Majorana fermion DM candidate $χ$, the singlet charged scalar mediator $S^\pm$ and the Standard Model (SM) right-handed lepton. The direct and indirect searches are not sensitive to this model. This article studies the lepton portal coupling as well as the scalar portal coupling (between $S^\pm$ and SM Higgs boson), as the latter is generally allowed in the Lagrangian. The inclusion of scalar portal coupling not only significantly enhances the LHC reach via the $gg\to h^*\to S^+S^-$ process, but also provides a few novel signal channels, such as the exotic decays and coupling deviations of the Higgs boson, offering new opportunities to probe the model. In addition, we also study the Drell-Yan production of $S^+S^-$ at future lepton colliders, and find out that the scenario where one $S^\pm$ is off-shell can be used to measure the lepton portal coupling directly. In particular, we are interested in the possibility that the scalar potential triggers a first-order phase transition and hence provides the stochastic gravitational wave (GW) signals. In this case, the terrestrial collider experiments and space-based GW detectors serve as complementary approaches to probe the model.

hep-ph↗

arXiv2021

Incentivized Bandit Learning with Self-Reinforcing User Preferences

In this paper, we investigate a new multi-armed bandit (MAB) online learning model that considers real-world phenomena in many recommender systems: (i) the learning agent cannot pull the arms by itself and thus has to offer rewards to users to incentivize arm-pulling indirectly; and (ii) if users with specific arm preferences are well rewarded, they induce a "self-reinforcing" effect in the sense that they will attract more users of similar arm preferences. Besides addressing the tradeoff of exploration and exploitation, another key feature of this new MAB model is to balance reward and incentivizing payment. The goal of the agent is to maximize the total reward over a fixed time horizon $T$ with a low total payment. Our contributions in this paper are two-fold: (i) We propose a new MAB model with random arm selection that considers the relationship of users' self-reinforcing preferences and incentives; and (ii) We leverage the properties of a multi-color Polya urn with nonlinear feedback model to propose two MAB policies termed "At-Least-$n$ Explore-Then-Commit" and "UCB-List". We prove that both policies achieve $O(log T)$ expected regret with $O(log T)$ expected payment over a time horizon $T$. We conduct numerical simulations to demonstrate and verify the performances of these two policies and study their robustness under various settings.

cs.LG↗

arXiv2021

A Sum-of-Ratios Multi-Dimensional-Knapsack Decomposition for DNN Resource Scheduling

In recent years, to sustain the resource-intensive computational needs for training deep neural networks (DNNs), it is widely accepted that exploiting the parallelism in large-scale computing clusters is critical for the efficient deployments of DNN training jobs. However, existing resource schedulers for traditional computing clusters are not well suited for DNN training, which results in unsatisfactory job completion time performance. The limitations of these resource scheduling schemes motivate us to propose a new computing cluster resource scheduling framework that is able to leverage the special layered structure of DNN jobs and significantly improve their job completion times. Our contributions in this paper are three-fold: i) We develop a new resource scheduling analytical model by considering DNN's layered structure, which enables us to analytically formulate the resource scheduling optimization problem for DNN training in computing clusters; ii) Based on the proposed performance analytical model, we then develop an efficient resource scheduling algorithm based on the widely adopted parameter-server architecture using a sum-of-ratios multi-dimensional-knapsack decomposition (SMD) method to offer strong performance guarantee; iii) We conduct extensive numerical experiments to demonstrate the effectiveness of the proposed schedule algorithm and its superior performance over the state of the art.

cs.DC↗

arXiv2021

GT-STORM: Taming Sample, Communication, and Memory Complexities in Decentralized Non-Convex Learning

Decentralized nonconvex optimization has received increasing attention in recent years in machine learning due to its advantages in system robustness, data privacy, and implementation simplicity. However, three fundamental challenges in designing decentralized optimization algorithms are how to reduce their sample, communication, and memory complexities. In this paper, we propose a \underline{g}radient-\underline{t}racking-based \underline{sto}chastic \underline{r}ecursive \underline{m}omentum (GT-STORM) algorithm for efficiently solving nonconvex optimization problems. We show that to reach an $ε^2$-stationary solution, the total number of sample evaluations of our algorithm is $\tilde{O}(m^{1/2}ε^{-3})$ and the number of communication rounds is $\tilde{O}(m^{-1/2}ε^{-3})$, which improve the $O(ε^{-4})$ costs of sample evaluations and communications for the existing decentralized stochastic gradient algorithms. We conduct extensive experiments with a variety of learning models, including non-convex logistical regression and convolutional neural networks, to verify our theoretical findings. Collectively, our results contribute to the state of the art of theories and algorithms for decentralized network optimization.

cs.LG↗

arXiv2021

Challenges for a QCD Axion at the 10 MeV Scale

We report on an interesting realization of the QCD axion, with mass in the range $\mathcal{O}(10)$ MeV. It has previously been shown that although this scenario is stringently constrained from multiple sources, the model remains viable for a range of parameters that leads to an explanation of the Atomki experiment anomaly. In this article we study in more detail the additional constraints proceeding from recent low energy experiments and study the compatibility of the allowed parameter space with the one leading to consistency of the most recent measurements of the electron anomalous magnetic moment and the fine structure constant. We further provide an ultraviolet completion of this axion variant and show the conditions under which it may lead to the observed quark masses and CKM mixing angles, and remain consistent with experimental constraints on the extended scalar sector appearing in this Standard Model extension. In particular, the decay of the Standard Model-like Higgs boson into two light axions may be relevant and leads to a novel Higgs boson signature that may be searched for at the LHC in the near future.

hep-ph↗

arXiv2021

High speed and reconfigurable optronic neural network with digital nonlinear activation

With its unique parallel processing capability, optical neural network has shown low-power consumption in image recognition and speech processing. At present, the manufacturing technology of programmable photonic chip is not mature, and the realization of optical neural network in free-space is still a hot spot of intelligent optical computing. In this article, based on MNIST datasets and 4f system, three-layer optical neural networks are constructed, whose recognition accuracy can reach 93.66%. Our network is programmable, high speed, reconfigurable and is better than the existing free-space optical neural network in terms of spatial complexity.

cs.ET↗

arXiv2021

Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning

Federated learning (FL) is a distributed machine learning architecture that leverages a large number of workers to jointly learn a model with decentralized data. FL has received increasing attention in recent years thanks to its data privacy protection, communication efficiency and a linear speedup for convergence in training (i.e., convergence performance increases linearly with respect to the number of workers). However, existing studies on linear speedup for convergence are only limited to the assumptions of i.i.d. datasets across workers and/or full worker participation, both of which rarely hold in practice. So far, it remains an open question whether or not the linear speedup for convergence is achievable under non-i.i.d. datasets with partial worker participation in FL. In this paper, we show that the answer is affirmative. Specifically, we show that the federated averaging (FedAvg) algorithm (with two-sided learning rates) on non-i.i.d. datasets in non-convex settings achieves a convergence rate $\mathcal{O}(\frac{1}{\sqrt{mKT}} + \frac{1}{T})$ for full worker participation and a convergence rate $\mathcal{O}(\frac{\sqrt{K}}{\sqrt{nT}} + \frac{1}{T})$ for partial worker participation, where $K$ is the number of local steps, $T$ is the number of total communication rounds, $m$ is the total worker number and $n$ is the worker number in one communication round if for partial worker participation. Our results also reveal that the local steps in FL could help the convergence and show that the maximum number of local steps can be improved to $T/m$ in full worker participation. We conduct extensive experiments on MNIST and CIFAR-10 to verify our theoretical results.

cs.LG↗

arXiv2021

Radio-frequency Dark Photon Dark Matter across the Sun

Dark photon as an ultralight dark matter candidate can interact with the Standard Model particles via kinetic mixing. We propose to search for the ultralight dark photon dark matter using radio telescopes with solar observations. The dark photon dark matter can efficiently convert into photons in the outermost region of the solar atmosphere, the solar corona, where the plasma mass of photons is close to the dark photon rest mass. Due to the strong resonant conversion and benefiting from the short distance between the Sun and the Earth, the radio telescopes can lead the dark photon search sensitivity in the mass range of $4 \times 10^{-8} - 4\times 10^{-6} \, \rm{eV}$, corresponding to the frequency $10 - 1000 \, {\rm MHz}$. As a promising example, the operating radio telescope LOFAR can reach the kinetic mixing $ε\sim 10^{-13}$ ($10^{-14}$) within 1 (100) hour solar observations. The future experiment SKA phase 1 can reach $ε\sim 10^{-16} - 10^{-14}$ with $1$ hour solar observations.

hep-ph↗

arXiv2021

Interaction induced non-reciprocal three-level quantum transport

Besides its fundamental importance, non-reciprocity has also found many potential applications in quantum technology. Recently, many quantum systems have been proposed to realize non-reciprocity, but stable non-reciprocal process is still experimentally difficult in general, due to the needed cyclical interactions in artificial systems or operational difficulties in solid state materials. Here, we propose a new kind of interaction induced non-reciprocal operation, based on the conventional STIRAP setup, which removes the experimental difficulty of requiring cyclical interaction, and thus it is directly implementable in various quantum systems. Furthermore, we also illustrate our proposal on a chain of three coupled superconducting transmons, which can lead to a non-reciprocal circulator with high fidelity without a ring coupling configuration as in the previous schemes or implementations. Therefore, our protocol provides a promising way to explore fundamental non-reciprocal quantum physics as well as realize non-reciprocal quantum device.

quant-ph↗

arXiv2021

Data Poisoning Attacks and Defenses to Crowdsourcing Systems

A key challenge of big data analytics is how to collect a large volume of (labeled) data. Crowdsourcing aims to address this challenge via aggregating and estimating high-quality data (e.g., sentiment label for text) from pervasive clients/users. Existing studies on crowdsourcing focus on designing new methods to improve the aggregated data quality from unreliable/noisy clients. However, the security aspects of such crowdsourcing systems remain under-explored to date. We aim to bridge this gap in this work. Specifically, we show that crowdsourcing is vulnerable to data poisoning attacks, in which malicious clients provide carefully crafted data to corrupt the aggregated data. We formulate our proposed data poisoning attacks as an optimization problem that maximizes the error of the aggregated data. Our evaluation results on one synthetic and two real-world benchmark datasets demonstrate that the proposed attacks can substantially increase the estimation errors of the aggregated data. We also propose two defenses to reduce the impact of malicious clients. Our empirical results show that the proposed defenses can substantially reduce the estimation errors of the data poisoning attacks.

cs.CR↗

arXiv2021

Near octave-spanning perfect soliton crystals in AlN microresonators

The perfect soliton crystal (PSC) was recently discovered as an extraordinary Kerr soliton state with regularly distributed soliton pulses and enhanced comb line power spaced by multiples of the cavity free spectral ranges (FSRs). The modulation of continuous-wave excitation in optical microresonators and the tunable repetition rate characteristic will significantly enhance and extend the application potential of soliton microcombs for self-referencing comb source, terahertz wave generation, and arbitrary waveform generation. However, the reported PSC spectrum is generally narrow. Here, we demonstrate the deterministic accessing of versatile perfect soliton crystals in the AlN microresonators (FSR ~374 GHz), featuring a broad spectral range up to 0.96 of an octave-span (1170-2300 nm) and terahertz repetition rates (up to ~1.87 THz). The measured 60-fs short pulses and low-noise characteristics confirms the high coherence of the PSCs

physics.optics↗

arXiv2021

$κ$TNG: Effect of Baryonic Processes on Weak Lensing with IllustrisTNG Simulations

We study the effect of baryonic processes on weak lensing (WL) observables with a suite of mock WL maps, the $κ$TNG, based on the cosmological hydrodynamic simulations IllustrisTNG. We quantify the baryonic effects on the WL angular power spectrum, one-point probability distribution function (PDF), and number counts of peaks and minima. We also show the redshift evolution of the effects, which is a key to distinguish the effect of baryons from fundamental physics such as dark energy, dark matter, and massive neutrinos. We find that baryonic processes reduce the small-scale power, suppress the tails of the PDF, peak and minimum counts, and change the total number of peaks and minima. We compare our results to existing semi-analytic models and hydrodynamic simulations, and discuss the source of discrepancies. The $κ$TNG suite includes 10,000 realisations of $5 \times 5 \, \mathrm{deg}^2$ maps for 40 source redshifts up to $z_s = 2.6$, well covering the range of interest for existing and upcoming weak lensing surveys. We also produce the $κ$TNG-Dark suite of maps, generated based on the corresponding dark matter only IllustrisTNG simulations. Our mock maps are suitable for developing analytic models that incorporate the effect of baryons, but also particularly useful for studies that rely on mass maps, such as non-Gaussian statistics and machine learning with convolutional neural networks. The suite of mock maps is publicly available at Columbia Lensing (http://columbialensing.org).

astro-ph.CO↗

arXiv2021

Faithful Simulation and Detection of Quantum Spin Hall Effect on Superconducting Circuits

Topological states of quantum matter have inspired both fascinating physics findings and exciting opportunities for applications. Due to the over-complicated structure of, as well as interactions between, real materials, a faithful quantum simulation of topological matter is very important in deepening our understanding of these states. This requirement puts the quantum superconducting circuits system as a good option for mimicking topological materials, owing to their flexible tunability and fine controllability. As a typical example herein, we realize a $\mathbb{Z}_2$-type topological insulator featuring the quantum spin Hall effect state, using a coupled system of transmission-line resonators and transmons. The single-excitation eigenstates of each unit cell are used as a pseudo-spin 1/2 system. The boundary of the topological phase transition is fixed in the phase diagram. Topological edge states are shown, which can be experimentally verified by detecting the population at the boundary of the plane. Compared to the previous simulations, this compositional system is fairly controllable, stable and less limited. Therefore, our scheme provides a reliable platform for faithful quantum simulations of topological matter.

quant-ph↗

arXiv2021

Multi-objective Search of Robust Neural Architectures against Multiple Types of Adversarial Attacks

Many existing deep learning models are vulnerable to adversarial examples that are imperceptible to humans. To address this issue, various methods have been proposed to design network architectures that are robust to one particular type of adversarial attacks. It is practically impossible, however, to predict beforehand which type of attacks a machine learn model may suffer from. To address this challenge, we propose to search for deep neural architectures that are robust to five types of well-known adversarial attacks using a multi-objective evolutionary algorithm. To reduce the computational cost, a normalized error rate of a randomly chosen attack is calculated as the robustness for each newly generated neural architecture at each generation. All non-dominated network architectures obtained by the proposed method are then fully trained against randomly chosen adversarial attacks and tested on two widely used datasets. Our experimental results demonstrate the superiority of optimized neural architectures found by the proposed approach over state-of-the-art networks that are widely used in the literature in terms of the classification accuracy under different adversarial attacks.

cs.LG↗