arXiv subjects

Yuhao Zhou

Publications and source records attributed to Yuhao Zhou.

At least 127 records · Page 7Linked to original sources

arXiv2022

Fast Instrument Learning with Faster Rates

We investigate nonlinear instrumental variable (IV) regression given high-dimensional instruments. We propose a simple algorithm which combines kernelized IV methods and an arbitrary, adaptive regression algorithm, accessed as a black box. Our algorithm enjoys faster-rate convergence and adapts to the dimensionality of informative latent features, while avoiding an expensive minimax optimization procedure, which has been necessary to establish similar guarantees. It further brings the benefit of flexible machine learning models to quasi-Bayesian uncertainty quantification, likelihood-based model selection, and model averaging. Simulation studies demonstrate the competitive performance of our method.

stat.ML↗

arXiv2022

DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps

Diffusion probabilistic models (DPMs) are emerging powerful generative models. Despite their high-quality generation performance, DPMs still suffer from their slow sampling as they generally need hundreds or thousands of sequential function evaluations (steps) of large neural networks to draw a sample. Sampling from DPMs can be viewed alternatively as solving the corresponding diffusion ordinary differential equations (ODEs). In this work, we propose an exact formulation of the solution of diffusion ODEs. The formulation analytically computes the linear part of the solution, rather than leaving all terms to black-box ODE solvers as adopted in previous works. By applying change-of-variable, the solution can be equivalently simplified to an exponentially weighted integral of the neural network. Based on our formulation, we propose DPM-Solver, a fast dedicated high-order solver for diffusion ODEs with the convergence order guarantee. DPM-Solver is suitable for both discrete-time and continuous-time DPMs without any further training. Experimental results show that DPM-Solver can generate high-quality samples in only 10 to 20 function evaluations on various datasets. We achieve 4.70 FID in 10 function evaluations and 2.87 FID in 20 function evaluations on the CIFAR10 dataset, and a $4\sim 16\times$ speedup compared with previous state-of-the-art training-free samplers on various datasets.

cs.LG↗

arXiv2022

A Semismooth Newton based Augmented Lagrangian Method for Nonsmooth Optimization on Matrix Manifolds

This paper is devoted to studying an augmented Lagrangian method for solving a class of manifold optimization problems, which have nonsmooth objective functions and nonlinear constraints. Under the constant positive linear dependence condition on manifolds, we show that the proposed method converges to a stationary point of the nonsmooth manifold optimization problem. Moreover, we propose a globalized semismooth Newton method to solve the augmented Lagrangian subproblem on manifolds efficiently. The local superlinear convergence of the manifold semismooth Newton method is also established under some suitable conditions. We also prove that the semismoothness on submanifolds can be inherited from that in the ambient manifold. Finally, numerical experiments on compressed modes and (constrained) sparse principal component analysis illustrate the advantages of the proposed method.

math.OC↗

arXiv2022

DeFTA: A Plug-and-Play Decentralized Replacement for FedAvg

Federated learning (FL) is identified as a crucial enabler for large-scale distributed machine learning (ML) without the need for local raw dataset sharing, substantially reducing privacy concerns and alleviating the isolated data problem. In reality, the prosperity of FL is largely due to a centralized framework called FedAvg, in which workers are in charge of model training and servers are in control of model aggregation. However, FedAvg's centralized worker-server architecture has raised new concerns, be it the low scalability of the cluster, the risk of data leakage, and the failure or even defection of the central server. To overcome these problems, we propose Decentralized Federated Trusted Averaging (DeFTA), a decentralized FL framework that serves as a plug-and-play replacement for FedAvg, instantly bringing better security, scalability, and fault-tolerance to the federated learning process after installation. In principle, it fundamentally resolves the above-mentioned issues from an architectural perspective without compromises or tradeoffs, primarily consisting of a new model aggregating formula with theoretical performance analysis, and a decentralized trust system (DTS) to greatly improve system robustness. Note that since DeFTA is an alternative to FedAvg at the framework level, \textit{prevalent algorithms published for FedAvg can be also utilized in DeFTA with ease}. Extensive experiments on six datasets and six basic models suggest that DeFTA not only has comparable performance with FedAvg in a more realistic setting, but also achieves great resilience even when 66% of workers are malicious. Furthermore, we also present an asynchronous variant of DeFTA to endow it with more powerful usability.

cs.DC↗

arXiv2022

Fuse Local and Global Semantics in Representation Learning

We propose Fuse Local and Global Semantics in Representation Learning (FLAGS) to generate richer representations. FLAGS aims at extract both global and local semantics from images to benefit various downstream tasks. It shows promising results under common linear evaluation protocol. We also conduct detection and segmentation on PASCAL VOC and COCO to show the representations extracted by FLAGS are transferable.

cs.CV↗

arXiv2021

Robust Online Detection in Serially Correlated Directed Network

As the complexity of production processes increases, the diversity of data types drives the development of network monitoring technology. This paper mainly focuses on an online algorithm to detect serially correlated directed networks robustly and sensitively. First, we consider a transition probability matrix to resolve the double correlation of primary data. Further, since the sum of each row of the transition probability matrix is one, it standardizes the data, facilitating subsequent modeling. Then we extend the spring length based method to the multivariate case and propose an adaptive cumulative sum (CUSUM) control chart on the strength of a weighted statistic to monitor directed networks. This novel approach assumes only that the process observation is associated with nearby points without any parametric time series model, which is in line with reality. Simulation results and a real example from metro transportation demonstrate the superiority of our design.

stat.AP↗

arXiv2021

Coronal rain in randomly heated arcades

Adopting the MPI-AMRVAC code, we present a 2.5-dimensional magnetohydrodynamic (MHD) simulation, which includes thermal conduction and radiative cooling, to investigate the formation and evolution of the coronal rain phenomenon. We perform the simulation in initially linear force-free magnetic fields which host chromospheric, transition region, and coronal plasma, with turbulent heating localized on their footpoints. Due to thermal instability, condensations start to occur at the loop top, and rebound shocks are generated by the siphon inflows. Condensations fragment into smaller blobs moving downwards and as they hit the lower atmosphere, concurrent upflows are triggered. Larger clumps show us clear "coronal rain showers" as dark structures in synthetic EUV hot channels and bright blobs with cool cores in the 304 Å channel, well resembling real observations. Following coronal rain dynamics for more than 10 hours, we carry out a statistical study of all coronal rain blobs to quantify their widths, lengths, areas, velocity distributions, and other properties. The coronal rain shows us continuous heating-condensation cycles, as well as cycles in EUV emissions. Compared to previous studies adopting steady heating, the rain happens faster and in more erratic cycles. Although most blobs are falling downward, upward-moving blobs exist at basically every moment. We also track the movement of individual blobs to study their dynamics and the forces driving their movements. The blobs have a prominence-corona transition-region-like structure surrounding them, and their movements are dominated by the pressure evolution in the very dynamic loop system.

astro-ph.SR↗

arXiv2021

Quasi-Bayesian Dual Instrumental Variable Regression

Recent years have witnessed an upsurge of interest in employing flexible machine learning models for instrumental variable (IV) regression, but the development of uncertainty quantification methodology is still lacking. In this work we present a novel quasi-Bayesian procedure for IV regression, building upon the recently developed kernelized IV models and the dual/minimax formulation of IV regression. We analyze the frequentist behavior of the proposed method, by establishing minimax optimal contraction rates in $L_2$ and Sobolev norms, and discussing the frequentist validity of credible balls. We further derive a scalable inference algorithm which can be extended to work with wide neural network models. Empirical evaluation shows that our method produces informative uncertainty estimates on complex high-dimensional problems.

stat.ML↗

arXiv2021

When hot meets cold: post-flare coronal rain

Most solar flares demonstrate a prolonged, hourlong post-flare (or gradual) phase, characterized by arcade-like, post-flare loops (PFLs) visible in many extreme ultraviolet (EUV) passbands. These coronal loops are filled with hot -- $\sim 30 \,\mathrm{MK}$ -- and dense plasma, evaporated from the chromosphere during the impulsive phase of the flare, and they very gradually recover to normal coronal density and temperature conditions. During this gradual cooling down to $\sim 1 \,\mathrm{MK}$ regimes, much cooler -- $\sim 0.01 \,\mathrm{MK}$ -- and denser coronal rain is frequently observed inside PFLs. Understanding PFL dynamics in this long-duration, gradual phase is crucial to the entire corona-chromosphere mass and energy cycle. Here we report a simulation in which a solar flare evolves from pre-flare, over impulsive phase all the way into its gradual phase, which successfully reproduces post-flare coronal rain. This rain results from catastrophic cooling caused by thermal instability, and we analyse the entire mass and energy budget evolution driving this sudden condensation phenomenon. We find that the runaway cooling and rain formation also induces the appearance of dark post-flare loop systems, as observed in EUV channels. We confirm and augment earlier observational findings, suggesting that thermal conduction and radiative losses alternately dominate the cooling of PFLs.

astro-ph.SR↗

arXiv2021

Design of a Flying Humanoid Robot Based on Thrust Vector Control

Achieving short-distance flight helps improve the efficiency of humanoid robots moving in complex environments (e.g., crossing large obstacles or reaching high places) for rapid emergency missions. This study proposes a design of a flying humanoid robot named Jet-HR2. The robot has 10 joints driven by brushless motors and harmonic drives for locomotion. To overcome the challenge of the stable-attitude takeoff in small thrust-to-weight conditions, the robot was designed based on the concept of thrust vectoring. The propulsion system consists of four ducted fans, that is, two fixed on the waist of the robot and the other two mounted on the feet, for thrust vector control. The thrust vector is controlled by adjusting the attitude of the foot during the flight. A simplified model and control strategies are proposed to solve the problem of attitude instability caused by mass errors and joint position errors during takeoff. The experimental results show that the robot's spin and dive behaviors during takeoff were effectively suppressed by controlling the thrust vector of the ducted fan on the foot. The robot successfully achieved takeoff at a thrust-to-weight ratio of 1.17 (17 kg / 20 kg) and maintained a stable attitude, reaching a takeoff height of over 1000 mm.

cs.RO↗

arXiv2021

Retrieval and Localization with Observation Constraints

Accurate visual re-localization is very critical to many artificial intelligence applications, such as augmented reality, virtual reality, robotics and autonomous driving. To accomplish this task, we propose an integrated visual re-localization method called RLOCS by combining image retrieval, semantic consistency and geometry verification to achieve accurate estimations. The localization pipeline is designed as a coarse-to-fine paradigm. In the retrieval part, we cascade the architecture of ResNet101-GeM-ArcFace and employ DBSCAN followed by spatial verification to obtain a better initial coarse pose. We design a module called observation constraints, which combines geometry information and semantic consistency for filtering outliers. Comprehensive experiments are conducted on open datasets, including retrieval on R-Oxford5k and R-Paris6k, semantic segmentation on Cityscapes, localization on Aachen Day-Night and InLoc. By creatively modifying separate modules in the total pipeline, our method achieves many performance improvements on the challenging localization benchmarks.

cs.CV↗

arXiv2021

Formation and Characteristics of Filament Threads in Double-Dipped Magnetic Flux Tubes

As one of the main formation mechanisms of solar filament formation, the chromospheric evaporation-coronal condensation model has been confirmed by numerical simulations to explain the formation of filament threads very well in flux tubes with single dips. However, coronal magnetic extrapolations indicated that some magnetic field lines might possess more than one dip. It is expected that the formation process would be significantly different in this case compared to a single-dipped magnetic flux tube. In this paper, based on the evaporation-condensation model, we study filament thread formation in double-dipped magnetic flux tubes by numerical simulations. We find that only with particular combinations of magnetic configuration and heating, e.g., concentrated localized heating and a long magnetic flux tube with deep dips, can two threads form and persist in a double-dipped magnetic flux tube. Comparing our parametric survey with observations, we conclude that such magnetically connected threads due to multiple dips are more likely to exist in quiescent filaments than in active-region filaments. Moreover, we find that these threads are usually shorter than independently trapped threads, which might be one of the reasons why quiescent filaments have short threads. These characteristics of magnetically connected threads could also explain barbs and vertical threads in quiescent filaments.

astro-ph.SR↗

arXiv2021

Communication-Efficient Federated Learning with Compensated Overlap-FedAvg

Petabytes of data are generated each day by emerging Internet of Things (IoT), but only few of them can be finally collected and used for Machine Learning (ML) purposes due to the apprehension of data & privacy leakage, which seriously retarding ML's growth. To alleviate this problem, Federated learning is proposed to perform model training by multiple clients' combined data without the dataset sharing within the cluster. Nevertheless, federated learning introduces massive communication overhead as the synchronized data in each epoch is of the same size as the model, and thereby leading to a low communication efficiency. Consequently, variant methods mainly focusing on the communication rounds reduction and data compression are proposed to reduce the communication overhead of federated learning. In this paper, we propose Overlap-FedAvg, a framework that parallels the model training phase with model uploading & downloading phase, so that the latter phase can be totally covered by the former phase. Compared to vanilla FedAvg, Overlap-FedAvg is further developed with a hierarchical computing strategy, a data compensation mechanism and a nesterov accelerated gradients~(NAG) algorithm. Besides, Overlap-FedAvg is orthogonal to many other compression methods so that they can be applied together to maximize the utilization of the cluster. Furthermore, the theoretical analysis is provided to prove the convergence of the proposed Overlap-FedAvg framework. Extensive experiments on both conventional and recurrent tasks with multiple models and datasets also demonstrate that the proposed Overlap-FedAvg framework substantially boosts the federated learning process.

cs.LG↗

arXiv2021

LANA: Towards Personalized Deep Knowledge Tracing Through Distinguishable Interactive Sequences

In educational applications, Knowledge Tracing (KT), the problem of accurately predicting students' responses to future questions by summarizing their knowledge states, has been widely studied for decades as it is considered a fundamental task towards adaptive online learning. Among all the proposed KT methods, Deep Knowledge Tracing (DKT) and its variants are by far the most effective ones due to the high flexibility of the neural network. However, DKT often ignores the inherent differences between students (e.g. memory skills, reasoning skills, ...), averaging the performances of all students, leading to the lack of personalization, and therefore was considered insufficient for adaptive learning. To alleviate this problem, in this paper, we proposed Leveled Attentive KNowledge TrAcing (LANA), which firstly uses a novel student-related features extractor (SRFE) to distill students' unique inherent properties from their respective interactive sequences. Secondly, the pivot module was utilized to dynamically reconstruct the decoder of the neural network on attention of the extracted features, successfully distinguishing the performance between students over time. Moreover, inspired by Item Response Theory (IRT), the interpretable Rasch model was used to cluster students by their ability levels, and thereby utilizing leveled learning to assign different encoders to different groups of students. With pivot module reconstructed the decoder for individual students and leveled learning specialized encoders for groups, personalized DKT was achieved. Extensive experiments conducted on two real-world large-scale datasets demonstrated that our proposed LANA improves the AUC score by at least 1.00% (i.e. EdNet 1.46% and RAIEd2020 1.00%), substantially surpassing the other State-Of-The-Art KT methods.

cs.CY↗

arXiv2020

Limits of PageRank-based ranking methods in sports data

While PageRank has been extensively used to rank sport tournament participants (teams or individuals), its superiority over simpler ranking methods has been never clearly demonstrated. We use sports results from 18 major leagues to calibrate a state-of-art model for synthetic sports results. Model data are then used to assess the ranking performance of PageRank in a controlled setting. We find that PageRank outperforms the benchmark ranking by the number of wins only when a small fraction of all games have been played. Increased randomness in the data, such as intrinsic randomness of outcomes or advantage of home teams, further reduces the range of PageRank's superiority. We propose a new PageRank variant which outperforms PageRank in all evaluated settings, yet shares its sensitivity to increased randomness in the data. Our main findings are confirmed by evaluating the ranking algorithms on real data. Our work demonstrates the danger of using novel metrics and algorithms without considering their limits of applicability.

cs.SI↗

arXiv2020

HPSGD: Hierarchical Parallel SGD With Stale Gradients Featuring

While distributed training significantly speeds up the training process of the deep neural network (DNN), the utilization of the cluster is relatively low due to the time-consuming data synchronizing between workers. To alleviate this problem, a novel Hierarchical Parallel SGD (HPSGD) strategy is proposed based on the observation that the data synchronization phase can be paralleled with the local training phase (i.e., Feed-forward and back-propagation). Furthermore, an improved model updating method is unitized to remedy the introduced stale gradients problem, which commits updates to the replica (i.e., a temporary model that has the same parameters as the global model) and then merges the average changes to the global model. Extensive experiments are conducted to demonstrate that the proposed HPSGD approach substantially boosts the distributed DNN training, reduces the disturbance of the stale gradients and achieves better accuracy in given fixed wall-time.

cs.LG↗

arXiv2020

Nonparametric Score Estimators

Estimating the score, i.e., the gradient of log density function, from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models that involve flexible yet intractable densities. Kernel estimators based on Stein's methods or score matching have shown promise, however their theoretical properties and relationships have not been fully-understood. We provide a unifying view of these estimators under the framework of regularized nonparametric regression. It allows us to analyse existing estimators and construct new ones with desirable properties by choosing different hypothesis spaces and regularizers. A unified convergence analysis is provided for such estimators. Finally, we propose score estimators based on iterative regularization that enjoy computational benefits from curl-free kernels and fast convergence.

stat.ML↗

arXiv2020

Visual Localization Using Semantic Segmentation and Depth Prediction

In this paper, we propose a monocular visual localization pipeline leveraging semantic and depth cues. We apply semantic consistency evaluation to rank the image retrieval results and a practical clustering technique to reject estimation outliers. In addition, we demonstrate a substantial performance boost achieved with a combination of multiple feature extractors. Furthermore, by using depth prediction with a deep neural network, we show that a significant amount of falsely matched keypoints are identified and eliminated. The proposed pipeline outperforms most of the existing approaches at the Long-Term Visual Localization benchmark 2020.

cs.CV↗