arXiv subjects

Jun Luo

Publications and source records attributed to Jun Luo.

At least 163 records · Page 9Linked to original sources

arXiv2022

A Simple Decentralized Cross-Entropy Method

Cross-Entropy Method (CEM) is commonly used for planning in model-based reinforcement learning (MBRL) where a centralized approach is typically utilized to update the sampling distribution based on only the top-$k$ operation's results on samples. In this paper, we show that such a centralized approach makes CEM vulnerable to local optima, thus impairing its sample efficiency. To tackle this issue, we propose Decentralized CEM (DecentCEM), a simple but effective improvement over classical CEM, by using an ensemble of CEM instances running independently from one another, and each performing a local improvement of its own sampling distribution. We provide both theoretical and empirical analysis to demonstrate the effectiveness of this simple decentralized approach. We empirically show that, compared to the classical centralized approach using either a single or even a mixture of Gaussian distributions, our DecentCEM finds the global optimum much more consistently thus improves the sample efficiency. Furthermore, we plug in our DecentCEM in the planning problem of MBRL, and evaluate our approach in several continuous control environments, with comparison to the state-of-art CEM based MBRL approaches (PETS and POPLIN). Results show sample efficiency improvement by simply replacing the classical CEM module with our DecentCEM module, while only sacrificing a reasonable amount of computational cost. Lastly, we conduct ablation studies for more in-depth analysis. Code is available at https://github.com/vincentzhang/decentCEM

cs.LG↗

arXiv2022

Colossal Nernst power factor in topological semimetal NbSb$_2$

Today solid-state cooling technologies below liquid nitrogen boiling temperature (77 K), crucial to quantum information technology and probing quantum state of matter, are greatly limited due to the lack of good thermoelectric and/or thermomagnetic materials. Here, we report the discovery of colossal Nernst power factor of 3800$\times$10$^{-4}$ W m$^{-1}$ K$^{-2}$ under 5 T at 25 K and high Nernst figure-of-merit of 71$\times$10$^{-4}$ K$^{-1}$ under 5 T at 20 K in topological semimetal NbSb$_2$ single crystals. The observed high thermomagnetic performance is attributed to large Nernst thermopower and longitudinal electrical conductivity, and relatively low transverse thermal conductivity. The large and unsaturated Nernst thermopower is the result of the combination of highly desirable electronic structures of NbSb$_2$ having compensated high mobility electrons and holes near Fermi level and strong phonon-drag effect. This discovery opens an avenue for exploring material option for the solid-state heat pumping below liquid nitrogen temperature.

cond-mat.mtrl-sci↗

arXiv2022

NeurIPS 2022 Competition: Driving SMARTS

Driving SMARTS is a regular competition designed to tackle problems caused by the distribution shift in dynamic interaction contexts that are prevalent in real-world autonomous driving (AD). The proposed competition supports methodologically diverse solutions, such as reinforcement learning (RL) and offline learning methods, trained on a combination of naturalistic AD data and open-source simulation platform SMARTS. The two-track structure allows focusing on different aspects of the distribution shift. Track 1 is open to any method and will give ML researchers with different backgrounds an opportunity to solve a real-world autonomous driving challenge. Track 2 is designed for strictly offline learning methods. Therefore, direct comparisons can be made between different methods with the aim to identify new promising research directions. The proposed setup consists of 1) realistic traffic generated using real-world data and micro simulators to ensure fidelity of the scenarios, 2) framework accommodating diverse methods for solving the problem, and 3) baseline method. As such it provides a unique opportunity for the principled investigation into various aspects of autonomous vehicle deployment.

cs.RO↗

arXiv2022

Build generally reusable agent-environment interaction models

This paper tackles the problem of how to pre-train a model and make it generally reusable backbones for downstream task learning. In pre-training, we propose a method that builds an agent-environment interaction model by learning domain invariant successor features from the agent's vast experiences covering various tasks, then discretize them into behavior prototypes which result in an embodied set structure. To make the model generally reusable for downstream task learning, we propose (1) embodied feature projection that retains previous knowledge by projecting the new task's observation-action pair to the embodied set structure and (2) projected Bellman updates which add learning plasticity for the new task setting. We provide preliminary results that show downstream task learning based on a pre-trained embodied set structure can handle unseen changes in task objectives, environmental dynamics and sensor modalities.

cs.LG↗

arXiv2022

High-performance non-Fermi-liquid metallic thermoelectric materials

Searching for high-performance thermoelectric (TE) materials in the paradigm of narrow-bandgap semiconductors has lasted for nearly 70 years and is obviously hampered by a bottleneck of research now. Here we report on the discovery of a few metallic compounds, TiFexCu2x-1Sb and TiFe1.33Sb, showing the thermopower exceeding many TE semiconductors and the dimensionless figure of merits comparable with the state-of-the-art TE materials. A quasi-linear temperature (T) dependence of electrical resistivity in 2 K - 700 K and the logarithmic T-dependent electronic specific heat at low temperature are also observed to coexist with the high thermopower, highlighting the strong intercoupling of the non-Fermi-liquid (NFL) quantum critical behavior of electrons with TE transports. Electronic structure analysis reveals the existence of fluctuating Fe-eg-related local magnetic moments, Fe-Fe antiferromagnetic (AFM) interaction at the nearest 4c-4d sites, and two-fold degenerate eg orbitals antiferromagnetically coupled with the dual-type itinerant electrons close to the Fermi level, all of which infer to a competition between the AFM ordering and Kondo-like spin compensation as well as a parallel two-channel Kondo effect. These effects are both strongly meditated by the structural disorder due to the random filling of Fe/Cu at the equivalent 4c/4d sites of the Heusler crystal lattice. The magnetic susceptibility deviates from ideal antiferromagnetism but can be fitted well by x(T) = 1/(θ + BTα), seemingly being consistent with the quantum critical scenario of strong local correlation as discussed before. Our work not only breaks the dilemma that the promising TE materials should be heavily-doped semiconductors, but also demonstrates the correlation among high TE performance, NFL quantum criticality, and magnetic fluctuation, which opens up new directions for future research.

cond-mat.str-el↗

arXiv2022

Connectedness and local cut points of generalized Sierpinski carpets

We investigate a homeomorphism problem on a class of self-similar sets called generalized Sierpinski carpets (or shortly GSCs). It follows from two well-known results by Hata and Whyburn that a connected GSC is homeomorphic to the standard Sierpinski carpet if and only if it has no local cut points. On the one hand, we show that to determine whether a given GSC is connected, it suffices to iterate the initial pattern twice. On the other hand, we obtain two criteria: (1) for a connected GSC to have cut points, (2) for a connected GSC with no cut points to have local cut points. With these two criteria, we characterize all GSCs that are homeomorphic to the standard Sierpinski carpet. Our results on cut points and local cut points hold for Baranski carpets, too. Moreover, we extend the connectedness result to Baranski sponges. Thus, we also characterize when a Baranski carpet is homeomorphic to the standard GSC.

math.GN↗

arXiv2022

How is Vaping Framed on Online Knowledge Dissemination Platforms?

We analyze 1,888 articles and 1,119,453 vaping posts to study how vaping is framed across multiple knowledge dissemination platforms (Wikipedia, Quora, Medium, Reddit, Stack Exchange, wikiHow). We use various NLP techniques to understand these differences. For example, n-grams, emotion recognition, and question answering results indicate that Medium, Quora, and Stack Exchange are appropriate venues for those looking to transition from smoking to vaping. Other platforms (Reddit, wikiHow) are more for vaping hobbyists and may not sufficiently dissuade youth vaping. Conversely, Wikipedia may exaggerate vaping harms, dissuading smokers from transitioning. A strength of our work is how the different techniques we have applied validate each other. Based on our results, we provide several recommendations. Stakeholders may utilize our findings to design informational tools to reinforce or mitigate vaping (mis)perceptions online.

cs.SI↗

arXiv2022

First-principles study of oxygen vacancy defects in orthorhombic Hf$_{0.5}$Zr$_{0.5}$O$_2$/SiO$_2$/Si gate stack

The gate defect of the ferroelectric HfO$_2$-based Si field-effect transistor (Si FeFET) plays a dominant role in its reliability issue. The first-principles calculations are an effective method for the atomic-scale understanding of gate defects. However, the first-principles study on the defects of FeFET gate stacks, i.e., metal/orthorhombic-Hf$_{0.5}$Zr$_{0.5}$O$_2$/SiO$_2$/Si structure, has not been reported so far. The key challenge is the construction of metal/orthorhombic-Hf$_{0.5}$Zr$_{0.5}$O$_2$/SiO$_2$/Si gate stack models. Here, we use the Hf$_{0.5}$Zr$_{0.5}$O$_2$(130) high-index crystal face as the orthorhombic ferroelectric layer and construct a robust atomic structure of the orthorhombic-Hf$_{0.5}$Zr$_{0.5}$O$_2$/SiO$_2$/Si gate stack without any gap states. Its high structural stability is ascribed to the insulated interface. The calculated band offsets show that this gate structure is of the type-I band alignment. Furthermore, the formation energies and charge transition levels (CTLs) of defects reveal that the oxygen vacancy defects are more favorable to form compared with other defects such as oxygen interstitial and Hf/Zr vacancy, and their CTLs are mainly localized near the Si conduction band minimum and valence band maximum, in agreement with the reported experimental results. The oxygen vacancy defects are responsible for charge trapping/de-trapping behavior in Si FeFET. This work provides an insight into gate defects and paves the way to carry out the first-principles study of ferroelectric HfO$_2$-based Si FeFET.

cond-mat.mtrl-sci↗

arXiv2022

Understanding and Mitigating the Limitations of Prioritized Experience Replay

Prioritized Experience Replay (ER) has been empirically shown to improve sample efficiency across many domains and attracted great attention; however, there is little theoretical understanding of why such prioritized sampling helps and its limitations. In this work, we take a deep look at the prioritized ER. In a supervised learning setting, we show the equivalence between the error-based prioritized sampling method for mean squared error and uniform sampling for cubic power loss. We then provide theoretical insight into why it improves convergence rate upon uniform sampling during early learning. Based on the insight, we further point out two limitations of the prioritized ER method: 1) outdated priorities and 2) insufficient coverage of the sample space. To mitigate the limitations, we propose our model-based stochastic gradient Langevin dynamics sampling method. We show that our method does provide states distributed close to an ideal prioritized sampling distribution estimated by the brute-force method, which does not suffer from the two limitations. We conduct experiments on both discrete and continuous control problems to show our approach's efficacy and examine the practical implication of our method in an autonomous driving application.

cs.AI↗

arXiv2022

Adapt to Adaptation: Learning Personalization for Cross-Silo Federated Learning

Conventional federated learning (FL) trains one global model for a federation of clients with decentralized data, reducing the privacy risk of centralized training. However, the distribution shift across non-IID datasets, often poses a challenge to this one-model-fits-all solution. Personalized FL aims to mitigate this issue systematically. In this work, we propose APPLE, a personalized cross-silo FL framework that adaptively learns how much each client can benefit from other clients' models. We also introduce a method to flexibly control the focus of training APPLE between global and local objectives. We empirically evaluate our method's convergence and generalization behaviors, and perform extensive experiments on two benchmark datasets and two medical imaging datasets under two non-IID settings. The results show that the proposed personalized FL framework, APPLE, achieves state-of-the-art performance compared to several other personalized FL approaches in the literature. The code is publicly available at https://github.com/ljaiverson/pFL-APPLE.

cs.LG↗

arXiv2022

Design and testing of LGAD sensor with shallow carbon implantation

The low gain avalanche detectors (LGADs) are thin sensors with fast charge collection which in combination with internal gain deliver an outstanding time resolution of about 30 ps. High collision rates and consequent large particle rates crossing the detectors at the upgraded Large Hadron Collider (LHC) in 2028 will lead to radiation damage and deteriorated performance of the LGADs. The main consequence of radiation damage is loss of gain layer doping (acceptor removal) which requires an increase of bias voltage to compensate for the loss of charge collection efficiency and consequently time resolution. The Institute of High Energy Physics (IHEP), Chinese Academy of Sciences (CAS) has developed a process based on the Institute of Microelectronics (IME), CAS capability to enrich the gain layer with carbon to reduce the acceptor removal effect by radiation. After 1 MeV neutron equivalent fluence of 2.5$\times$10$^{15}$ n$_{eq}$/cm$^{2}$, which is the maximum fluence to which sensors will be exposed at ATLAS High Granularity Timing Detector (HGTD), the IHEP-IME second version (IHEP-IMEv2) 50 $μ$m LGAD sensors already deliver adequate charge collection > 4 fC and time resolution < 50 ps at voltages < 400 V. The operation voltages of these 50 $μ$m devices are well below those at which single event burnout may occur.

physics.ins-det↗

arXiv2022

Probing the Effect of Selection Bias on Generalization: A Thought Experiment

Learned systems in the domain of visual recognition and cognition impress in part because even though they are trained with datasets many orders of magnitude smaller than the full population of possible images, they exhibit sufficient generalization to be applicable to new and previously unseen data. Since training data sets typically represent small sampling of a domain, the possibility of bias in their composition is very real. But what are the limits of generalization given such bias, and up to what point might it be sufficient for a real problem task? Although many have examined issues regarding generalization, this question may require examining the data itself. Here, we focus on the characteristics of the training data that may play a role. Other disciplines have grappled with these problems, most interestingly epidemiology, where experimental bias is a critical concern. The range and nature of data biases seen clinically are really quite relatable to learned vision systems. One obvious way to deal with bias is to ensure a large enough training set, but this might be infeasible for many domains. Another approach might be to perform a statistical analysis of the actual training set, to determine if all aspects of the domain are fairly captured. This too is difficult, in part because the full set of variables might not be known, or perhaps not even knowable. Here, we try a different approach in the tradition of the Thought Experiment, whose most famous instance may be Schrödinger's Cat. There are many types of bias as will be seen, but we focus only on one, selection bias. The point of the thought experiment is not to demonstrate problems with all learned systems. Rather, this might be a simple theoretical tool to probe into bias during data collection to highlight deficiencies that might then deserve extra attention either in data collection or system development.

cs.CV↗

arXiv2022

What makes useful auxiliary tasks in reinforcement learning: investigating the effect of the target policy

Auxiliary tasks have been argued to be useful for representation learning in reinforcement learning. Although many auxiliary tasks have been empirically shown to be effective for accelerating learning on the main task, it is not yet clear what makes useful auxiliary tasks. Some of the most promising results are on the pixel control, reward prediction, and the next state prediction auxiliary tasks; however, the empirical results are mixed, showing substantial improvements in some cases and marginal improvements in others. Careful investigations of how auxiliary tasks help the learning of the main task is necessary. In this paper, we take a step studying the effect of the target policies on the usefulness of the auxiliary tasks formulated as general value functions. General value functions consist of three core elements: 1) policy 2) cumulant 3) continuation function. Our focus on the role of the target policy of the auxiliary tasks is motivated by the fact that the target policy determines the behavior about which the agent wants to make a prediction and the state-action distribution that the agent is trained on, which further affects the main task learning. Our study provides insights about questions such as: Does a greedy policy result in bigger improvement gains compared to other policies? Is it best to set the auxiliary task policy to be the same as the main task policy? Does the choice of the target policy have a substantial effect on the achieved performance gain or simple strategies for setting the policy, such as using a uniformly random policy, work as well? Our empirical results suggest that: 1) Auxiliary tasks with the greedy policy tend to be useful. 2) Most policies, including a uniformly random policy, tend to improve over the baseline. 3) Surprisingly, the main task policy tends to be less useful compared to other policies.

cs.AI↗

arXiv2022

On continuous extension of conformal homeomorphisms of infinitely connected planar domains

We consider conformal homeomorphisms $φ$ of generalized Jordan domains $U$ onto planar domains $Ω$ %, possibly {\bf infinitely connected}, that satisfy both of the next two conditions: (1) at most countably many boundary components of $Ω$ are non-degenerate and their diameters have a finite sum; (2) either the degenerate boundary components of $Ω$ or those of $U$ form a set of sigma-finite linear measure. We prove that $φ$ continuously extends to the closure of $U$ if and only if every boundary component of $Ω$ is locally connected. This generalizes the Carathéodory's Continuity Theorem and leads us to a new generalization of the well known Osgood-Taylor-Carathéodory Theorem. There are three issues that are noteworthy. Firstly, none of the above conditions (1) and (2) can be removed. Secondly, %no further requirements concerning $U$ or $Ω$ are needed. So our results remain valid for non-cofat domains and do not follow from the extension results, of a similar nature, that are obtained in very recent studies on the conformal rigidity of circle domains. Finally, when $φ$ does extend continuously to the closure of $U$, the boundary of $Ω$ is a Peano compactum. Therefore, we also show that the following properties are equivalent for any planar domain $Ω$: (1) The boundary of $Ω$ is a Peano compactum. (2) $Ω$ has Property S. (3) Every point on the boundary of $Ω$ is locally accessible. (4) Every point on the boundary of $Ω$ is locally sequentially accessible. (5) $Ω$ is finitely connected at the boundary. (6) The completion of $Ω$ under the Mazurkiewicz distance is compact. \noindent This provides new generalizations of earlier partial results that are restricted to special cases, when additional assumptions on the topology of $U$ or its boundary are required.

math.CV↗

arXiv2022

LatentFormer: Multi-Agent Transformer-Based Interaction Modeling and Trajectory Prediction

Multi-agent trajectory prediction is a fundamental problem in autonomous driving. The key challenges in prediction are accurately anticipating the behavior of surrounding agents and understanding the scene context. To address these problems, we propose LatentFormer, a transformer-based model for predicting future vehicle trajectories. The proposed method leverages a novel technique for modeling interactions among dynamic objects in the scene. Contrary to many existing approaches which model cross-agent interactions during the observation time, our method additionally exploits the future states of the agents. This is accomplished using a hierarchical attention mechanism where the evolving states of the agents autoregressively control the contributions of past trajectories and scene encodings in the final prediction. Furthermore, we propose a multi-resolution map encoding scheme that relies on a vision transformer module to effectively capture both local and global scene context to guide the generation of more admissible future trajectories. We evaluate the proposed method on the nuScenes benchmark dataset and show that our approach achieves state-of-the-art performance and improves upon trajectory metrics by up to 40%. We further investigate the contributions of various components of the proposed technique via extensive ablation studies.

cs.CV↗

arXiv2022

Offline Learning of Counterfactual Predictions for Real-World Robotic Reinforcement Learning

We consider real-world reinforcement learning (RL) of robotic manipulation tasks that involve both visuomotor skills and contact-rich skills. We aim to train a policy that maps multimodal sensory observations (vision and force) to a manipulator's joint velocities under practical considerations. We propose to use offline samples to learn a set of general value functions (GVFs) that make counterfactual predictions from the visual inputs. We show that combining the offline learned counterfactual predictions with force feedbacks in online policy learning allows efficient reinforcement learning given only a terminal (success/failure) reward. We argue that the learned counterfactual predictions form a compact and informative representation that enables sample efficiency and provides auxiliary reward signals that guide online explorations towards contact-rich states. Various experiments in simulation and real-world settings were performed for evaluation. Recordings of the real-world robot training can be found via https://sites.google.com/view/realrl.

cs.RO↗

arXiv2022

Drawing Inductor Layout with a Reinforcement Learning Agent: Method and Application for VCO Inductors

Design of Voltage-Controlled Oscillator (VCO) inductors is a laborious and time-consuming task that is conventionally done manually by human experts. In this paper, we propose a framework for automating the design of VCO inductors, using Reinforcement Learning (RL). We formulate the problem as a sequential procedure, where wire segments are drawn one after another, until a complete inductor is created. We then employ an RL agent to learn to draw inductors that meet certain target specifications. In light of the need to tweak the target specifications throughout the circuit design cycle, we also develop a variant in which the agent can learn to quickly adapt to draw new inductors for moderately different target specifications. Our empirical results show that the proposed framework is successful at automatically generating VCO inductors that meet or exceed the target specification.

cs.AI↗

arXiv2022

Low Gain Avalanche Detectors with Good Time Resolution Developed by IHEP and IME for ATLAS HGTD project

This paper shows the simulation and test results of 50um thick Low Gain Avalanche Detectors (LGAD) sensors designed by the Institute of High Energy Physics (IHEP) and fabricated by the Institute of Microelectronics of the Chinese Academy of Sciences (IME). Three wafers have been produced with four different gain layer implant doses each. Different production processes, including variation in the n++ layer implant energy and carbon co-implantation were used. Test results show that the IHEP-IME sensors with the higher dose of gain layer have lower breakdown voltages and higher gain layer voltages from capacitance-voltage properties, which are consistent with the TCAD simulation. Beta test results show that the time resolution of IHEP-IME sensors is better than 35ps when operated at high voltage and the collected charges of IHEP-IME sensors are larger than 15fC before irradiation, which fulfill the required specifications of sensors before irradiations for the ATLAS HGTD project.

physics.ins-det↗