arXiv subjects

Jie Ren

Publications and source records attributed to Jie Ren.

At least 199 records · Page 11Linked to original sources

arXiv2022

Geometric thermodynamic uncertainty relation in periodically driven thermoelectric heat engine

Thermodynamic uncertainty relation, quantifying a trade-off among average current, the associated fluctuation (precision), and entropy production (cost), has been formulated in nonequilibrium steady state and various stochastic systems. Herein, we study the thermodynamic uncertainty relation in generic thermoelectric heat engines under a periodic control protocol, by uncovering the underlying Berry-phase-like contribution. We show that our thermodynamic uncertainty relation breaks the seminal steady-state results, originating from the non-vanishing geometric effect. Furthermore, by deriving the consequent trade-off relation binding efficiency, power, and constancy, we prove that the periodically driven thermoelectric heat engines can generally outperform the steady-state analogies. The general bounds are illustrated by an analytically solvable two-terminal single quantum dot heat engine under the periodic modulation. Our work provides a geometric framework in bounding and optimizing a wide range of periodically driven thermoelectric thermal machines.

cond-mat.mes-hall↗

arXiv2022

Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning

High-quality estimates of uncertainty and robustness are crucial for numerous real-world applications, especially for deep learning which underlies many deployed ML systems. The ability to compare techniques for improving these estimates is therefore very important for research and practice alike. Yet, competitive comparisons of methods are often lacking due to a range of reasons, including: compute availability for extensive tuning, incorporation of sufficiently many baselines, and concrete documentation for reproducibility. In this paper we introduce Uncertainty Baselines: high-quality implementations of standard and state-of-the-art deep learning methods on a variety of tasks. As of this writing, the collection spans 19 methods across 9 tasks, each with at least 5 metrics. Each baseline is a self-contained experiment pipeline with easily reusable and extendable components. Our goal is to provide immediate starting points for experimentation with new methods or applications. Additionally we provide model checkpoints, experiment outputs as Python notebooks, and leaderboards for comparing results. Code available at https://github.com/google/uncertainty-baselines.

cs.LG↗

arXiv2021

Deformed Symmetry Structures and Quantum Many-body Scar Subspaces

A quantum many-body scar system usually contains a special non-thermal subspace (approximately) decoupled from the rest of the Hilbert space. In this work, we propose a general structure called deformed symmetric spaces for the decoupled subspaces hosting quantum many-body scars, which are irreducible sectors of simple Lie groups transformed by matrix-product operators (or projected entangled pair operators), of which the entanglement entropies are proved to obey sub-volume-law scaling and thus violate the eigenstate thermalization hypothesis. A deformed symmetric space, in general, is required to have at least a U(1) sub-Lie-group symmetry to allow coherent periodic dynamics from certain low-entangled initial states. We enumerate several possible deforming transformations based on the sub-group symmetry requirement and recover many existing models whose scar states are not connected by symmetry. In particular, a two-dimensional scar model is proposed, which hosts a periodic dynamical trajectory on which all states are topologically ordered.

cond-mat.str-el↗

arXiv2021

A Unified Game-Theoretic Interpretation of Adversarial Robustness

This paper provides a unified view to explain different adversarial attacks and defense methods, i.e. the view of multi-order interactions between input variables of DNNs. Based on the multi-order interaction, we discover that adversarial attacks mainly affect high-order interactions to fool the DNN. Furthermore, we find that the robustness of adversarially trained DNNs comes from category-specific low-order interactions. Our findings provide a potential method to unify adversarial perturbations and robustness, which can explain the existing defense methods in a principle way. Besides, our findings also make a revision of previous inaccurate understanding of the shape bias of adversarially learned features.

cs.LG↗

arXiv2021

A Unified Game-Theoretic Interpretation of Adversarial Robustness

This paper provides a unified view to explain different adversarial attacks and defense methods, \emph{i.e.} the view of multi-order interactions between input variables of DNNs. Based on the multi-order interaction, we discover that adversarial attacks mainly affect high-order interactions to fool the DNN. Furthermore, we find that the robustness of adversarially trained DNNs comes from category-specific low-order interactions. Our findings provide a potential method to unify adversarial perturbations and robustness, which can explain the existing defense methods in a principle way. Besides, our findings also make a revision of previous inaccurate understanding of the shape bias of adversarially learned features.

cs.LG↗

arXiv2021

Detecting an axion-like particle with machine learning at the LHC

Axion-like particles (ALPs) appear in various new physics models with spontaneous global symmetry breaking. When the ALP mass is in the range of MeV to GeV, the cosmology and astrophysics bounds are so far quite weak. In this work, we investigate such light ALPs through the ALP-strahlung production processes $pp \to W^\pm a, Z a$ with the sequential decay $a \to γγ$ at the 14 TeV LHC with an integrated luminosity of 3000 fb$^{-1}$ (HL-LHC). Building on the concept of jet image which uses calorimeter towers as the pixels of the image and measures a jet as an image, we investigate the potential of machine learning techniques based on convolutional neural network (CNN) to identify the highly boosted ALPs which decay to a pair of highly collimated photons. With the CNN tagging algorithm, we demonstrate that our approach can extend current LHC sensitivity and probe the ALP mass range from 0.3~GeV to 5~GeV. The obtained bounds are stronger than the existing limits on the ALP-photon coupling.

hep-ph↗

arXiv2021

ESOD:Edge-based Task Scheduling for Object Detection

Object Detection on the mobile system is a challenge in terms of everything. Nowadays, many object detection models have been designed, and most of them concentrate on precision. However, the computation burden of those models on mobile systems is unacceptable. Researchers have designed some lightweight networks for mobiles by sacrificing precision. We present a novel edge-based task scheduling framework for object detection (termed as ESOD). In detail, we train a DNN model (termed as pre-model) to predict which object detection model to use for the coming task and offloads to which edge servers by physical characteristics of the image task (e.g., brightness, saturation). The results show that ESOD can reduce latency and energy consumption by an average of 22.13% and 29.60% and improve the mAP to 45.8(with 0.9 mAP better), respectively, compared with the SOTA DETR model.

cs.CV↗

arXiv2021

Observation of Geometric Heat Pump Effect in Periodic Driven Thermal Diffusion

The concept of geometry works as an overarching framework underlying a wide range of transport phenomena. Particularly, the geometric phase effect in classical and quantum heat pump has been attracting much attention in microscopic systems. Here, we formulate theoretically the geometric heat pump effect in macroscopic driven diffusive systems. Upon modulation protocols, the nontrivial geometric curvature in the parameter space universally induces an additional pumped heat, beyond the constraint of hot-to-cold flowing. Furthermore, we set up a minimum experiment and indeed observe a non-vanishing directional heat flow across the driven system, despite keeping zero thermal bias between two time-dependent thermal reservoirs at every instant. We verify that in analogy to the geometric phase effect, the geometric pumped heat during each driving cycle is independent of driving periods in the adiabatic limit and coincides with theoretical predictions, thus validating its geometric origin. These results about geometric heat pump effect could have potential implications for designing and implementing nonreciprocal and topological thermal meta-devices under spatiotemporal modulations.

cond-mat.stat-mech↗

arXiv2021

Cellular Network Radio Propagation Modeling with Deep Convolutional Neural Networks

Radio propagation modeling and prediction is fundamental for modern cellular network planning and optimization. Conventional radio propagation models fall into two categories. Empirical models, based on coarse statistics, are simple and computationally efficient, but are inaccurate due to oversimplification. Deterministic models, such as ray tracing based on physical laws of wave propagation, are more accurate and site specific. But they have higher computational complexity and are inflexible to utilize site information other than traditional global information system (GIS) maps. In this article we present a novel method to model radio propagation using deep convolutional neural networks and report significantly improved performance compared to conventional models. We also lay down the framework for data-driven modeling of radio propagation and enable future research to utilize rich and unconventional information of the site, e.g. satellite photos, to provide more accurate and flexible models.

cs.IT↗

arXiv2021

Tuning nonequilibrium heat current and two-photon statistics via composite qubit-resonator interaction

Quantum thermal transport and two-photon statistics serve as two representative nonequilibrium features in circuit quantum electrodynamics systems. Here, we investigate quantum heat flow and two-photon correlation function at steady-state in a composite qubit-resonator model, where one qubit shows both transverse and longitudinal couplings to a single-mode optical resonator. With weak qubit-resonator interaction, we unravel two microscopic transport pictures, i.e., cotunneling and cyclic heat exchange processes, corresponding to transverse and longitudinal couplings respectively. At strong qubit-resonator coupling, the heat current exhibits nonmonotonic behavior by increasing qubit-resonator coupling strength, which tightly relies on the scattering processes between the qubit and corresponding thermal bath. Furthermore, the longitudinal coupling is preferred to enhance heat current in strong qubit-resonator coupling regime. For two-photon correlation function, it exhibits an antibunching-to-bunching transition, which is mainly dominated by the modulation of energy gap between the first and second excited eigenstates. Our results are expected to deepen the understanding of nonequilibrium thermal transport and nonclassical photon radiation based on the circuit quantum electrodynamics platform.

quant-ph↗

arXiv2021

Learning to Remove: Towards Isotropic Pre-trained BERT Embedding

Pre-trained language models such as BERT have become a more common choice of natural language processing (NLP) tasks. Research in word representation shows that isotropic embeddings can significantly improve performance on downstream tasks. However, we measure and analyze the geometry of pre-trained BERT embedding and find that it is far from isotropic. We find that the word vectors are not centered around the origin, and the average cosine similarity between two random words is much higher than zero, which indicates that the word vectors are distributed in a narrow cone and deteriorate the representation capacity of word embedding. We propose a simple, and yet effective method to fix this problem: remove several dominant directions of BERT embedding with a set of learnable weights. We train the weights on word similarity tasks and show that processed embedding is more isotropic. Our method is evaluated on three standardized tasks: word similarity, word analogy, and semantic textual similarity. In all tasks, the word embedding processed by our method consistently outperforms the original embedding (with average improvement of 13% on word analogy and 16% on semantic textual similarity) and two baseline methods. Our method is also proven to be more robust to changes of hyperparameter.

cs.CL↗

arXiv2021

Exploring the Limits of Out-of-Distribution Detection

Near out-of-distribution detection (OOD) is a major challenge for deep neural networks. We demonstrate that large-scale pre-trained transformers can significantly improve the state-of-the-art (SOTA) on a range of near OOD tasks across different data modalities. For instance, on CIFAR-100 vs CIFAR-10 OOD detection, we improve the AUROC from 85% (current SOTA) to more than 96% using Vision Transformers pre-trained on ImageNet-21k. On a challenging genomics OOD detection benchmark, we improve the AUROC from 66% to 77% using transformers and unsupervised pre-training. To further improve performance, we explore the few-shot outlier exposure setting where a few examples from outlier classes may be available; we show that pre-trained transformers are particularly well-suited for outlier exposure, and that the AUROC of OOD detection on CIFAR-100 vs CIFAR-10 can be improved to 98.7% with just 1 image per OOD class, and 99.46% with 10 images per OOD class. For multi-modal image-text pre-trained transformers such as CLIP, we explore a new way of using just the names of outlier classes as a sole source of information without any accompanying images, and show that this outperforms previous SOTA on standard vision OOD benchmark tasks.

cs.LG↗

arXiv2021

Proceedings of ICML 2021 Workshop on Theoretic Foundation, Criticism, and Application Trend of Explainable AI

This is the Proceedings of ICML 2021 Workshop on Theoretic Foundation, Criticism, and Application Trend of Explainable AI. Deep neural networks (DNNs) have undoubtedly brought great success to a wide range of applications in computer vision, computational linguistics, and AI. However, foundational principles underlying the DNNs' success and their resilience to adversarial attacks are still largely missing. Interpreting and theorizing the internal mechanisms of DNNs becomes a compelling yet controversial topic. This workshop pays a special interest in theoretic foundations, limitations, and new application trends in the scope of XAI. These issues reflect new bottlenecks in the future development of XAI.

cs.LG↗

arXiv2021

Sparse group variable selection for gene-environment interactions in the longitudinal study

Penalized variable selection for high dimensional longitudinal data has received much attention as accounting for the correlation among repeated measurements and providing additional and essential information for improved identification and prediction performance. Despite the success, in longitudinal studies the potential of penalization methods is far from fully understood for accommodating structured sparsity. In this article, we develop a sparse group penalization method to conduct the bi-level gene-environment (G$\times$E) interaction study under the repeatedly measured phenotype. Within the quadratic inference function (QIF) framework, the proposed method can achieve simultaneous identification of main and interaction effects on both the group and individual level. Simulation studies have shown that the proposed method outperforms major competitors. In the case study of asthma data from the Childhood Asthma Management Program (CAMP), we conduct G$\times$E study by using high dimensional SNP data as the Genetic factor and the longitudinal trait, forced expiratory volume in one second (FEV1), as phenotype. Our method leads to improved prediction and identification of main and interaction effects with important implications.

stat.ME↗

arXiv2021

Cycle Flux Ranking of Network Analysis in Quantum Thermal Device

Manipulating quantum thermal transport relies on uncovering the principle working cycles of quantum devices. Here, we apply the cycle flux ranking of network analysis to nonequilibrium thermal devices described by graphs of quantum state transitions. To excavate the principal mechanism out of complex transport behaviors, we decompose the quantum-transition network into cycles, calculate the cycle flux by algebraic graph theory, and pick out the dominant cycles with top-ranked fluxes, i.e., the cycle trajectories with highest probabilities. We demonstrate the cycle flux ranking in typical quantum device models, such as a thermal-drag spin-Seebeck pump, and a quantum thermal transistor as thermal switch or heat amplifier. The dominant cycle trajectories indeed elucidate the principal working mechanisms of those quantum devices. The cycle flux analysis provides an alternative perspective that naturally describes the working cycle corresponding to the main functionality of quantum thermal devices, which would further guide the device optimization with desired performance

quant-ph↗

arXiv2021

Sawtooth Factorial Topic Embeddings Guided Gamma Belief Network

Hierarchical topic models such as the gamma belief network (GBN) have delivered promising results in mining multi-layer document representations and discovering interpretable topic taxonomies. However, they often assume in the prior that the topics at each layer are independently drawn from the Dirichlet distribution, ignoring the dependencies between the topics both at the same layer and across different layers. To relax this assumption, we propose sawtooth factorial topic embedding guided GBN, a deep generative model of documents that captures the dependencies and semantic similarities between the topics in the embedding space. Specifically, both the words and topics are represented as embedding vectors of the same dimension. The topic matrix at a layer is factorized into the product of a factor loading matrix and a topic embedding matrix, the transpose of which is set as the factor loading matrix of the layer above. Repeating this particular type of factorization, which shares components between adjacent layers, leads to a structure referred to as sawtooth factorization. An auto-encoding variational inference network is constructed to optimize the model parameter via stochastic gradient descent. Experiments on big corpora show that our models outperform other neural topic models on extracting deeper interpretable topics and deriving better document representations.

cs.IR↗

arXiv2021

Geometric Heat Pump: Controlling Thermal Transport with Time-dependent Modulations

The second law of thermodynamics dictates that heat simultaneously flows from the hot to cold bath on average. To go beyond this picture, a range of works in the past decade show that, other than the average dynamical heat flux determined by instantaneous thermal bias, a non-trivial flux contribution of intrinsic geometric origin is generally present in temporally driven systems. This additional heat flux provides a free lunch for the pumped heat and could even drive heat against the bias. We review here the emergence and development of this so called ``geometric heat pump'', originating from the topological geometric phase effect, and cover various quantum and classical transport systems with different internal dynamics. The generalization from the adiabatic to the non-adiabatic regime and the application of control theory are also discussed. Then, we briefly discuss the symmetry restriction on the heat pump effect, such as duality, supersymmetry and time-reversal symmetry. Finally, we examine open problems concerning the geometric heat pump process and elucidate their prospective significance in devising thermal machines with high performance.

cond-mat.stat-mech↗

arXiv2021

A Game-Theoretic Taxonomy of Visual Concepts in DNNs

In this paper, we rethink how a DNN encodes visual concepts of different complexities from a new perspective, i.e. the game-theoretic multi-order interactions between pixels in an image. Beyond the categorical taxonomy of objects and the cognitive taxonomy of textures and shapes, we provide a new taxonomy of visual concepts, which helps us interpret the encoding of shapes and textures, in terms of concept complexities. In this way, based on multi-order interactions, we find three distinctive signal-processing behaviors of DNNs encoding textures. Besides, we also discover the flexibility for a DNN to encode shapes is lower than the flexibility of encoding textures. Furthermore, we analyze how DNNs encode outlier samples, and explore the impacts of network architectures on interactions. Additionally, we clarify the crucial role of the multi-order interactions in real-world applications. The code will be released when the paper is accepted.

cs.LG↗