arXiv subjects

Jun Liu

Publications and source records attributed to Jun Liu.

At least 541 records · Page 30Linked to original sources

arXiv2023

Robustly Complete Finite-State Abstractions for Control Synthesis of Stochastic Systems

The essential step of abstraction-based control synthesis for nonlinear systems to satisfy a given specification is to obtain a finite-state abstraction of the original systems. The complexity of the abstraction is usually the dominating factor that determines the efficiency of the algorithm. For the control synthesis of discrete-time nonlinear stochastic systems modelled by nonlinear stochastic difference equations, recent literature has demonstrated the soundness of abstractions in preserving robust probabilistic satisfaction of ω-regular lineartime properties. However, unnecessary transitions exist within the abstractions, which are difficult to quantify, and the completeness of abstraction-based control synthesis in the stochastic setting remains an open theoretical question. In this paper, we address this fundamental question from the topological view of metrizable space of probability measures, and propose constructive finite-state abstractions for control synthesis of probabilistic linear temporal specifications. Such abstractions are both sound and approximately complete. That is, given a concrete discrete-time stochastic system and an arbitrarily small L1-perturbation of this system, there exists a family of finite-state controlled Markov chains that both abstracts the concrete system and is abstracted by the slightly perturbed system. In other words, given an arbitrarily small prescribed precision, an abstraction always exists to decide whether a control strategy exists for the concrete system to satisfy the probabilistic specification.

eess.SY↗

arXiv2023

A nearly optimal explicitly-sparse representation for oscillatory kernels with curvelet-like functions

A nearly optimal explicitly-sparse representation for oscillatory kernels is presented in this work by developing a curvelet based method. Multilevel curvelet-like functions are constructed as the transform of the original nodal basis. Then the system matrix in a new non-standard form is derived with respect to the curvelet basis, which would be nearly optimally sparse due to the directional low rank property of the oscillatory kernel. Its sparsity is further enhanced via a-posteriori compression. Finally its nearly optimial log-linear computational complexity with controllable accuracy is demonstrated with numerical results. This explicitly-sparse representation is expected to lay ground to future work related to fast direct solvers and effective preconditioners for high frequency problems. It may also be viewed as the generalization of wavelet based methods to high frequency cases, and used as a new wideband fast algorithm for wave problems.

math.NA↗

arXiv2023

Almost Sure Saddle Avoidance of Stochastic Gradient Methods without the Bounded Gradient Assumption

We prove that various stochastic gradient descent methods, including the stochastic gradient descent (SGD), stochastic heavy-ball (SHB), and stochastic Nesterov's accelerated gradient (SNAG) methods, almost surely avoid any strict saddle manifold. To the best of our knowledge, this is the first time such results are obtained for SHB and SNAG methods. Moreover, our analysis expands upon previous studies on SGD by removing the need for bounded gradients of the objective function and uniformly bounded noise. Instead, we introduce a more practical local boundedness assumption for the noisy gradient, which is naturally satisfied in empirical risk minimization problems typically seen in training of neural networks.

cs.LG↗

arXiv2023

The Event Horizon Telescope Image of the Quasar NRAO 530

We report on the observations of the quasar NRAO 530 with the Event Horizon Telescope (EHT) on 2017 April 5-7, when NRAO 530 was used as a calibrator for the EHT observations of Sagittarius A*. At z=0.902 this is the most distant object imaged by the EHT so far. We reconstruct the first images of the source at 230 GHz, at an unprecedented angular resolution of $\sim$ 20 $μ$as, both in total intensity and in linear polarization. We do not detect source variability, allowing us to represent the whole data set with static images. The images reveal a bright feature located on the southern end of the jet, which we associate with the core. The feature is linearly polarized, with a fractional polarization of $\sim$5-8% and has a sub-structure consisting of two components. Their observed brightness temperature suggests that the energy density of the jet is dominated by the magnetic field. The jet extends over 60 $μ$as along a position angle PA$\sim -$28$^\circ$. It includes two features with orthogonal directions of polarization (electric vector position angle, EVPA), parallel and perpendicular to the jet axis, consistent with a helical structure of the magnetic field in the jet. The outermost feature has a particularly high degree of linear polarization, suggestive of a nearly uniform magnetic field. Future EHT observations will probe the variability of the jet structure on $μ$as scales, while simultaneous multi-wavelength monitoring will provide insight into the high energy emission origin.

astro-ph.HE↗

arXiv2023

Bridging Physics-Informed Neural Networks with Reinforcement Learning: Hamilton-Jacobi-Bellman Proximal Policy Optimization (HJBPPO)

This paper introduces the Hamilton-Jacobi-Bellman Proximal Policy Optimization (HJBPPO) algorithm into reinforcement learning. The Hamilton-Jacobi-Bellman (HJB) equation is used in control theory to evaluate the optimality of the value function. Our work combines the HJB equation with reinforcement learning in continuous state and action spaces to improve the training of the value network. We treat the value network as a Physics-Informed Neural Network (PINN) to solve for the HJB equation by computing its derivatives with respect to its inputs exactly. The Proximal Policy Optimization (PPO)-Clipped algorithm is improvised with this implementation as it uses a value network to compute the objective function for its policy network. The HJBPPO algorithm shows an improved performance compared to PPO on the MuJoCo environments.

cs.LG↗

arXiv2023

Modeling Uncertain Feature Representation for Domain Generalization

Though deep neural networks have achieved impressive success on various vision tasks, obvious performance degradation still exists when models are tested in out-of-distribution scenarios. In addressing this limitation, we ponder that the feature statistics (mean and standard deviation), which carry the domain characteristics of the training data, can be properly manipulated to improve the generalization ability of deep learning models. Existing methods commonly consider feature statistics as deterministic values measured from the learned features and do not explicitly model the uncertain statistics discrepancy caused by potential domain shifts during testing. In this paper, we improve the network generalization ability by modeling domain shifts with uncertainty (DSU), i.e., characterizing the feature statistics as uncertain distributions during training. Specifically, we hypothesize that the feature statistic, after considering the potential uncertainties, follows a multivariate Gaussian distribution. During inference, we propose an instance-wise adaptation strategy that can adaptively deal with the unforeseeable shift and further enhance the generalization ability of the trained model with negligible additional cost. We also conduct theoretical analysis on the aspects of generalization error bound and the implicit regularization effect, showing the efficacy of our method. Extensive experiments demonstrate that our method consistently improves the network generalization ability on multiple vision tasks, including image classification, semantic segmentation, instance retrieval, and pose estimation. Our methods are simple yet effective and can be readily integrated into networks without additional trainable parameters or loss constraints. Code will be released in https://github.com/lixiaotong97/DSU.

cs.CV↗

arXiv2023

The state-of-the-art 3D anisotropic intracranial hemorrhage segmentation on non-contrast head CT: The INSTANCE challenge

Automatic intracranial hemorrhage segmentation in 3D non-contrast head CT (NCCT) scans is significant in clinical practice. Existing hemorrhage segmentation methods usually ignores the anisotropic nature of the NCCT, and are evaluated on different in-house datasets with distinct metrics, making it highly challenging to improve segmentation performance and perform objective comparisons among different methods. The INSTANCE 2022 was a grand challenge held in conjunction with the 2022 International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI). It is intended to resolve the above-mentioned problems and promote the development of both intracranial hemorrhage segmentation and anisotropic data processing. The INSTANCE released a training set of 100 cases with ground-truth and a validation set with 30 cases without ground-truth labels that were available to the participants. A held-out testing set with 70 cases is utilized for the final evaluation and ranking. The methods from different participants are ranked based on four metrics, including Dice Similarity Coefficient (DSC), Hausdorff Distance (HD), Relative Volume Difference (RVD) and Normalized Surface Dice (NSD). A total of 13 teams submitted distinct solutions to resolve the challenges, making several baseline models, pre-processing strategies and anisotropic data processing techniques available to future researchers. The winner method achieved an average DSC of 0.6925, demonstrating a significant growth over our proposed baseline method. To the best of our knowledge, the proposed INSTANCE challenge releases the first intracranial hemorrhage segmentation benchmark, and is also the first challenge that intended to resolve the anisotropic problem in 3D medical image segmentation, which provides new alternatives in these research fields.

eess.IV↗

arXiv2023

The grand picture behind Jensen's inequality

Let $I$ and $J$ be two intervals, and let $f, g: I \rightarrow \mathbb{R}$. If for any points $a$ and $b$ in $I$ and any positive numbers $p$ and $q$ such that $p + q = 1$, we have \begin{align} \nonumber p f(a) + q f(b) + g(pa + qb) \in J, \end{align} then for any points $x_{1}, \ldots, x_{n}$ in $I$ and any positive numbers $λ_{1}, \ldots, λ_{n}$ such that $\sum_{i=1}^{n}λ_{i} = 1$, we have \begin{align} \nonumber \sum_{i=1}^{n}λ_{i} f(x_{i}) + g\left( \sum_{i=1}^{n}λ_{i}x_{i} \right) \in J. \end{align} If we take $g = -f$ and $J = [0, +\infty)$, then the Jensen's inequality. The conclusion is only a short glimpse of the grand picture behind Jensen's inequality shows in this paper.

math.GM↗

arXiv2023

Mind Reasoning Manners: Enhancing Type Perception for Generalized Zero-shot Logical Reasoning over Text

Logical reasoning task involves diverse types of complex reasoning over text, based on the form of multiple-choice question answering. Given the context, question and a set of options as the input, previous methods achieve superior performances on the full-data setting. However, the current benchmark dataset has the ideal assumption that the reasoning type distribution on the train split is close to the test split, which is inconsistent with many real application scenarios. To address it, there remain two problems to be studied: (1) How is the zero-shot capability of the models (train on seen types and test on unseen types)? (2) How to enhance the perception of reasoning types for the models? For problem 1, we propose a new benchmark for generalized zero-shot logical reasoning, named ZsLR. It includes six splits based on the three type sampling strategies. For problem 2, a type-aware model TaCo is proposed. It utilizes both the heuristic input reconstruction and the contrastive learning to improve the type perception in the global representation. Extensive experiments on both the zero-shot and full-data settings prove the superiority of TaCo over the state-of-the-art methods. Also, we experiment and verify the generalization capability of TaCo on other logical reasoning dataset.

cs.AI↗

arXiv2022

GPTR: Gestalt-Perception Transformer for Diagram Object Detection

Diagram object detection is the key basis of practical applications such as textbook question answering. Because the diagram mainly consists of simple lines and color blocks, its visual features are sparser than those of natural images. In addition, diagrams usually express diverse knowledge, in which there are many low-frequency object categories in diagrams. These lead to the fact that traditional data-driven detection model is not suitable for diagrams. In this work, we propose a gestalt-perception transformer model for diagram object detection, which is based on an encoder-decoder architecture. Gestalt perception contains a series of laws to explain human perception, that the human visual system tends to perceive patches in an image that are similar, close or connected without abrupt directional changes as a perceptual whole object. Inspired by these thoughts, we build a gestalt-perception graph in transformer encoder, which is composed of diagram patches as nodes and the relationships between patches as edges. This graph aims to group these patches into objects via laws of similarity, proximity, and smoothness implied in these edges, so that the meaningful objects can be effectively detected. The experimental results demonstrate that the proposed GPTR achieves the best results in the diagram object detection task. Our model also obtains comparable results over the competitors in natural image object detection.

cs.CV↗

arXiv2022

Compact On-Chip crystalline Resonator Integration with Etching Tapered Fiber Waveguide

Whispering-gallery mode crystalline resonators currently maintain the best quality factor (Q) record, however, compact on-chip packaging is still a challenge although various coupling architectures have been developed. Here, a chemical etching method is proposed to fabricate a miniaturized tapered fiber waveguide on silicon substrate. The Marangoni effect is implemented to reduce the surface roughness of the cone region. The optical loss of 0.1 dB/mm is obtained, and the Q of on-chip crystalline resonator exceeds 108. Additionally, TEC is implanted in the package to actively customize the temperature, and the temperature response of 18 pm/K is consistent with the theoretical calculation.

physics.optics↗

arXiv2022

HDNet: A Hierarchically Decoupled Network for Crowd Counting

Recently, density map regression-based methods have dominated in crowd counting owing to their excellent fitting ability on density distribution. However, further improvement tends to saturate mainly because of the confusing background noise and the large density variation. In this paper, we propose a Hierarchically Decoupled Network (HDNet) to solve the above two problems within a unified framework. Specifically, a background classification sub-task is decomposed from the density map prediction task, which is then assigned to a Density Decoupling Module (DDM) to exploit its highly discriminative ability. For the remaining foreground prediction sub-task, it is further hierarchically decomposed to several density-specific sub-tasks by the DDM, which are then solved by the regression-based experts in a Foreground Density Estimation Module (FDEM). Although the proposed strategy effectively reduces the hypothesis space so as to relieve the optimization for those task-specific experts, the high correlation of these sub-tasks are ignored. Therefore, we introduce three types of interaction strategies to unify the whole framework, which are Feature Interaction, Gradient Interaction, and Scale Interaction. Integrated with the above spirits, HDNet achieves state-of-the-art performance on several popular counting benchmarks.

cs.CV↗

arXiv2022

Global Meets Local: Effective Multi-Label Image Classification via Category-Aware Weak Supervision

Multi-label image classification, which can be categorized into label-dependency and region-based methods, is a challenging problem due to the complex underlying object layouts. Although region-based methods are less likely to encounter issues with model generalizability than label-dependency methods, they often generate hundreds of meaningless or noisy proposals with non-discriminative information, and the contextual dependency among the localized regions is often ignored or over-simplified. This paper builds a unified framework to perform effective noisy-proposal suppression and to interact between global and local features for robust feature learning. Specifically, we propose category-aware weak supervision to concentrate on non-existent categories so as to provide deterministic information for local feature learning, restricting the local branch to focus on more high-quality regions of interest. Moreover, we develop a cross-granularity attention module to explore the complementary information between global and local features, which can build the high-order feature correlation containing not only global-to-local, but also local-to-local relations. Both advantages guarantee a boost in the performance of the whole network. Extensive experiments on two large-scale datasets (MS-COCO and VOC 2007) demonstrate that our framework achieves superior performance over state-of-the-art methods.

cs.CV↗

arXiv2022

High sensitivity magnetic field sensor via sandwich type PDMS resonator

The sandwich structure as the core layer of PDMS resonator is proposed for single-axis magnetic sensor with high sensitivity. The small Young's modulus of flexible material corresponds to larger variation, resulting in a highly sensitive magnetic response. The sandwich structure pre-set magnetic field provides directional sensing feature. The experimental results show that the redshift sensitivity of 1.08 nm/mT and the blueshift sensitivity of 1.12 nm/mT in the unshielded environment, which is attributed to slight variation in PDMS Young's modulus. At 1.4 kHz, a minimum detectable magnetic field of 0.96 nT Hz-1/2 is realized.

physics.optics↗

arXiv2022

Lyapunov-Barrier Characterization of Robust Reach-Avoid-Stay Specifications for Hybrid Systems

Stability, reachability, and safety are crucial properties of dynamical systems. While verification and control synthesis of reach-avoid-stay objectives can be effectively handled by abstraction-based formal methods, such approaches can be computationally expensive due to the use of state-space discretization. In contrast, Lyapunov methods qualitatively characterize stability and safety properties without any state-space discretization. Recent work on converse Lyapunov-barrier theorems also demonstrates an approximate completeness or verifying reach-avoid-stay specifications of systems modelled by nonlinear differential equations. In this paper, based on the topology of hybrid arcs, we extend the Lyapunov-barrier characterization to more general hybrid systems described by differential and difference inclusions. We show that Lyapunov-barrier functions are not only sufficient to guarantee reach-avoid-stay specifications for well-posed hybrid systems, but also necessary for arbitrarily slightly perturbed systems under mild conditions. Numerical examples are provided to illustrate the main results.

math.DS↗

arXiv2022

Rethinking the Metric in Few-shot Learning: From an Adaptive Multi-Distance Perspective

Few-shot learning problem focuses on recognizing unseen classes given a few labeled images. In recent effort, more attention is paid to fine-grained feature embedding, ignoring the relationship among different distance metrics. In this paper, for the first time, we investigate the contributions of different distance metrics, and propose an adaptive fusion scheme, bringing significant improvements in few-shot classification. We start from a naive baseline of confidence summation and demonstrate the necessity of exploiting the complementary property of different distance metrics. By finding the competition problem among them, built upon the baseline, we propose an Adaptive Metrics Module (AMM) to decouple metrics fusion into metric-prediction fusion and metric-losses fusion. The former encourages mutual complementary, while the latter alleviates metric competition via multi-task collaborative learning. Based on AMM, we design a few-shot classification framework AMTNet, including the AMM and the Global Adaptive Loss (GAL), to jointly optimize the few-shot task and auxiliary self-supervised task, making the embedding features more robust. In the experiment, the proposed AMM achieves 2% higher performance than the naive metrics fusion module, and our AMTNet outperforms the state-of-the-arts on multiple benchmark datasets.

cs.CV↗

arXiv2022

Neural Lyapunov Control of Unknown Nonlinear Systems with Stability Guarantees

Learning for control of dynamical systems with formal guarantees remains a challenging task. This paper proposes a learning framework to simultaneously stabilize an unknown nonlinear system with a neural controller and learn a neural Lyapunov function to certify a region of attraction (ROA) for the closed-loop system. The algorithmic structure consists of two neural networks and a satisfiability modulo theories (SMT) solver. The first neural network is responsible for learning the unknown dynamics. The second neural network aims to identify a valid Lyapunov function and a provably stabilizing nonlinear controller. The SMT solver then verifies that the candidate Lyapunov function indeed satisfies the Lyapunov conditions. We provide theoretical guarantees of the proposed learning framework in terms of the closed-loop stability for the unknown nonlinear system. We illustrate the effectiveness of the approach with a set of numerical experiments.

eess.SY↗

arXiv2022

Improving the Reliability for Confidence Estimation

Confidence estimation, a task that aims to evaluate the trustworthiness of the model's prediction output during deployment, has received lots of research attention recently, due to its importance for the safe deployment of deep models. Previous works have outlined two important qualities that a reliable confidence estimation model should possess, i.e., the ability to perform well under label imbalance and the ability to handle various out-of-distribution data inputs. In this work, we propose a meta-learning framework that can simultaneously improve upon both qualities in a confidence estimation model. Specifically, we first construct virtual training and testing sets with some intentionally designed distribution differences between them. Our framework then uses the constructed sets to train the confidence estimation model through a virtual training and testing scheme leading it to learn knowledge that generalizes to diverse distributions. We show the effectiveness of our framework on both monocular depth estimation and image classification.

cs.CV↗