arXiv subjects

Xiaojie Li

Publications and source records attributed to Xiaojie Li.

At least 55 records · Page 3Linked to original sources

arXiv2024

GenView: Enhancing View Quality with Pretrained Generative Model for Self-Supervised Learning

Self-supervised learning has achieved remarkable success in acquiring high-quality representations from unlabeled data. The widely adopted contrastive learning framework aims to learn invariant representations by minimizing the distance between positive views originating from the same image. However, existing techniques to construct positive views highly rely on manual transformations, resulting in limited diversity and potentially false positive pairs. To tackle these challenges, we present GenView, a controllable framework that augments the diversity of positive views leveraging the power of pretrained generative models while preserving semantics. We develop an adaptive view generation method that dynamically adjusts the noise level in sampling to ensure the preservation of essential semantic meaning while introducing variability. Additionally, we introduce a quality-driven contrastive loss, which assesses the quality of positive pairs by considering both foreground similarity and background diversity. This loss prioritizes the high-quality positive pairs we construct while reducing the influence of low-quality pairs, thereby mitigating potential semantic inconsistencies introduced by generative models and aggressive data augmentation. Thanks to the improved positive view quality and the quality-driven contrastive loss, GenView significantly improves self-supervised learning across various tasks. For instance, GenView improves MoCov2 performance by 2.5%/2.2% on ImageNet linear/semi-supervised classification. Moreover, GenView even performs much better than naively augmenting the ImageNet dataset with Laion400M or ImageNet21K. Code: https://github.com/xiaojieli0903/genview.

cs.CV↗

arXiv2024

Preview-based Category Contrastive Learning for Knowledge Distillation

Knowledge distillation is a mainstream algorithm in model compression by transferring knowledge from the larger model (teacher) to the smaller model (student) to improve the performance of student. Despite many efforts, existing methods mainly investigate the consistency between instance-level feature representation or prediction, which neglects the category-level information and the difficulty of each sample, leading to undesirable performance. To address these issues, we propose a novel preview-based category contrastive learning method for knowledge distillation (PCKD). It first distills the structural knowledge of both instance-level feature correspondence and the relation between instance features and category centers in a contrastive learning fashion, which can explicitly optimize the category representation and explore the distinct correlation between representations of instances and categories, contributing to discriminative category centers and better classification results. Besides, we introduce a novel preview strategy to dynamically determine how much the student should learn from each sample according to their difficulty. Different from existing methods that treat all samples equally and curriculum learning that simply filters out hard samples, our method assigns a small weight for hard instances as a preview to better guide the student training. Extensive experiments on several challenging datasets, including CIFAR-100 and ImageNet, demonstrate the superiority over state-of-the-art methods.

cs.CV↗

arXiv2024

RadioGAT: A Joint Model-based and Data-driven Framework for Multi-band Radiomap Reconstruction via Graph Attention Networks

Multi-band radiomap reconstruction (MB-RMR) is a key component in wireless communications for tasks such as spectrum management and network planning. However, traditional machine-learning-based MB-RMR methods, which rely heavily on simulated data or complete structured ground truth, face significant deployment challenges. These challenges stem from the differences between simulated and actual data, as well as the scarcity of real-world measurements. To address these challenges, our study presents RadioGAT, a novel framework based on Graph Attention Network (GAT) tailored for MB-RMR within a single area, eliminating the need for multi-region datasets. RadioGAT innovatively merges model-based spatial-spectral correlation encoding with data-driven radiomap generalization, thus minimizing the reliance on extensive data sources. The framework begins by transforming sparse multi-band data into a graph structure through an innovative encoding strategy that leverages radio propagation models to capture the spatial-spectral correlation inherent in the data. This graph-based representation not only simplifies data handling but also enables tailored label sampling during training, significantly enhancing the framework's adaptability for deployment. Subsequently, The GAT is employed to generalize the radiomap information across various frequency bands. Extensive experiments using raytracing datasets based on real-world environments have demonstrated RadioGAT's enhanced accuracy in supervised learning settings and its robustness in semi-supervised scenarios. These results underscore RadioGAT's effectiveness and practicality for MB-RMR in environments with limited data availability.

eess.SP↗

arXiv2024

Tuning-Free Visual Customization via View Iterative Self-Attention Control

Fine-Tuning Diffusion Models enable a wide range of personalized generation and editing applications on diverse visual modalities. While Low-Rank Adaptation (LoRA) accelerates the fine-tuning process, it still requires multiple reference images and time-consuming training, which constrains its scalability for large-scale and real-time applications. In this paper, we propose \textit{View Iterative Self-Attention Control (VisCtrl)} to tackle this challenge. Specifically, VisCtrl is a training-free method that injects the appearance and structure of a user-specified subject into another subject in the target image, unlike previous approaches that require fine-tuning the model. Initially, we obtain the initial noise for both the reference and target images through DDIM inversion. Then, during the denoising phase, features from the reference image are injected into the target image via the self-attention mechanism. Notably, by iteratively performing this feature injection process, we ensure that the reference image features are gradually integrated into the target image. This approach results in consistent and harmonious editing with only one reference image in a few denoising steps. Moreover, benefiting from our plug-and-play architecture design and the proposed Feature Gradual Sampling strategy for multi-view editing, our method can be easily extended to edit in complex visual domains. Extensive experiments show the efficacy of VisCtrl across a spectrum of tasks, including personalized editing of images, videos, and 3D scenes.

cs.CV↗

arXiv2024

Approximating arrival costs in distributed moving horizon estimation: A recursive method

In this paper, we present a new approach to distributed moving horizon estimation for constrained nonlinear processes. The method involves approximating the arrival costs of local estimators through a recursive framework. First, distributed full-information estimation for linear unconstrained systems is presented, which serves as the foundation for deriving the analytical expression of the arrival costs for the local estimators. Subsequently, we develop a recursive arrival cost design for linear distributed moving horizon estimation. Sufficient conditions are derived to ensure the stability of the estimation error for constrained linear systems. Next, we extend the arrival cost design derived for linear systems to account for nonlinear systems, and a partition-based constrained distributed moving horizon estimation algorithm for nonlinear systems is formulated. A benchmark chemical process is used to illustrate the effectiveness and superiority of the proposed method.

eess.SY↗

arXiv2024

Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation

Relieving the reliance of neural network training on a global back-propagation (BP) has emerged as a notable research topic due to the biological implausibility and huge memory consumption caused by BP. Among the existing solutions, local learning optimizes gradient-isolated modules of a neural network with local errors and has been proved to be effective even on large-scale datasets. However, the reconciliation among local errors has never been investigated. In this paper, we first theoretically study non-greedy layer-wise training and show that the convergence cannot be assured when the local gradient in a module w.r.t. its input is not reconciled with the local gradient in the previous module w.r.t. its output. Inspired by the theoretical result, we further propose a local training strategy that successively regularizes the gradient reconciliation between neighboring modules without breaking gradient isolation or introducing any learnable parameters. Our method can be integrated into both local-BP and BP-free settings. In experiments, we achieve significant performance improvements compared to previous methods. Particularly, our method for CNN and Transformer architectures on ImageNet is able to attain a competitive performance with global BP, saving more than 40% memory consumption.

cs.LG↗

arXiv2024

Iterative distributed moving horizon estimation of linear systems with penalties on both system disturbances and noise

In this paper, partition-based distributed state estimation of general linear systems is considered. A distributed moving horizon state estimation scheme is developed via decomposing the entire system model into subsystem models and partitioning the global objective function of centralized moving horizon estimation (MHE) into local objective functions. The subsystem estimators of the distributed scheme that are required to be executed iteratively within each sampling period are designed based on MHE. Two distributed MHE algorithms are proposed to handle the unconstrained case and the case when hard constraints on states and disturbances, respectively. Sufficient conditions on the convergence of the estimates and the stability of the estimation error dynamics for the entire system are derived for both cases. A benchmark reactor-separator process example is introduced to illustrate the proposed distributed state estimation approach.

eess.SY↗

arXiv2024

Partition-based distributed extended Kalman filter for large-scale nonlinear processes with application to chemical and wastewater treatment processes

In this paper, we address a partition-based distributed state estimation problem for large-scale general nonlinear processes by proposing a Kalman-based approach. First, we formulate a linear full-information estimation design within a distributed framework as the basis for developing our approach. Second, the analytical solution to the local optimization problems associated with the formulated distributed full-information design is established, in the form of a recursive distributed Kalman filter algorithm. Then, the linear distributed Kalman filter is extended to the nonlinear context by incorporating successive linearization of nonlinear subsystem models, and the proposed distributed extended Kalman filter approach is formulated. We conduct rigorous analysis and prove the stability of the estimation error dynamics provided by the proposed method for general nonlinear processes consisting of interconnected subsystems. A chemical process example is used to illustrate the effectiveness of the proposed method and to justify the validity of the theoretical findings. In addition, the proposed method is applied to a wastewater treatment process for estimating the full state of the process with 145 state variables.

eess.SY↗

arXiv2024

Data-driven parallel Koopman subsystem modeling and distributed moving horizon state estimation for large-scale nonlinear processes

In this work, we consider a state estimation problem for large-scale nonlinear processes in the absence of first-principles process models. By exploiting process operation data, both process modeling and state estimation design are addressed within a distributed framework. By leveraging the Koopman operator concept, a parallel subsystem modeling approach is proposed to establish interactive linear subsystem process models in higher-dimensional subspaces, each of which correlates with the original nonlinear subspace of the corresponding process subsystem via a nonlinear mapping. The data-driven linear subsystem models can be used to collaboratively characterize and predict the dynamical behaviors of the entire nonlinear process. Based on the established subsystem models, local state estimators that can explicitly handle process operation constraints are designed using moving horizon estimation. The local estimators are integrated via information exchange to form a distributed estimation scheme, which provides estimates of the unmeasured/unmeasurable state variables of the original nonlinear process in a linear manner. The proposed framework is applied to a chemical process and an agro-hydrological process to illustrate its effectiveness and applicability. Good open-loop predictability of the linear subsystem models is confirmed, and accurate estimates of the process states are obtained without requiring a first-principles process model.

eess.SY↗

arXiv2023

Improving Hand Recognition in Uncontrolled and Uncooperative Environments using Multiple Spatial Transformers and Loss Functions

The prevalence of smartphone and consumer camera has led to more evidence in the form of digital images, which are mostly taken in uncontrolled and uncooperative environments. In these images, criminals likely hide or cover their faces while their hands are observable in some cases, creating a challenging use case for forensic investigation. Many existing hand-based recognition methods perform well for hand images collected in controlled environments with user cooperation. However, their performance deteriorates significantly in uncontrolled and uncooperative environments. A recent work has exposed the potential of hand recognition in these environments. However, only the palmar regions were considered, and the recognition performance is still far from satisfactory. To improve the recognition accuracy, an algorithm integrating a multi-spatial transformer network (MSTN) and multiple loss functions is proposed to fully utilize information in full hand images. MSTN is firstly employed to localize the palms and fingers and estimate the alignment parameters. Then, the aligned images are further fed into pretrained convolutional neural networks, where features are extracted. Finally, a training scheme with multiple loss functions is used to train the network end-to-end. To demonstrate the effectiveness of the proposed algorithm, the trained model is evaluated on NTU-PI-v1 database and six benchmark databases from different domains. Experimental results show that the proposed algorithm performs significantly better than the existing methods in these uncontrolled and uncooperative environments and has good generalization capabilities to samples from different domains.

cs.CV↗

arXiv2022

Cross-Modality Domain Adaptation for Freespace Detection: A Simple yet Effective Baseline

As one of the fundamental functions of autonomous driving system, freespace detection aims at classifying each pixel of the image captured by the camera as drivable or non-drivable. Current works of freespace detection heavily rely on large amount of densely labeled training data for accuracy and robustness, which is time-consuming and laborious to collect and annotate. To the best of our knowledge, we are the first work to explore unsupervised domain adaptation for freespace detection to alleviate the data limitation problem with synthetic data. We develop a cross-modality domain adaptation framework which exploits both RGB images and surface normal maps generated from depth images. A Collaborative Cross Guidance (CCG) module is proposed to leverage the context information of one modality to guide the other modality in a cross manner, thus realizing inter-modality intra-domain complement. To better bridge the domain gap between source domain (synthetic data) and target domain (real-world data), we also propose a Selective Feature Alignment (SFA) module which only aligns the features of consistent foreground area between the two domains, thus realizing inter-domain intra-modality adaptation. Extensive experiments are conducted by adapting three different synthetic datasets to one real-world dataset for freespace detection respectively. Our method performs closely to fully supervised freespace detection methods (93.08 v.s. 97.50 F1 score) and outperforms other general unsupervised domain adaptation methods for semantic segmentation with large margins, which shows the promising potential of domain adaptation for freespace detection.

cs.CV↗

arXiv2022

HEAD: HEtero-Assists Distillation for Heterogeneous Object Detectors

Conventional knowledge distillation (KD) methods for object detection mainly concentrate on homogeneous teacher-student detectors. However, the design of a lightweight detector for deployment is often significantly different from a high-capacity detector. Thus, we investigate KD among heterogeneous teacher-student pairs for a wide application. We observe that the core difficulty for heterogeneous KD (hetero-KD) is the significant semantic gap between the backbone features of heterogeneous detectors due to the different optimization manners. Conventional homogeneous KD (homo-KD) methods suffer from such a gap and are hard to directly obtain satisfactory performance for hetero-KD. In this paper, we propose the HEtero-Assists Distillation (HEAD) framework, leveraging heterogeneous detection heads as assistants to guide the optimization of the student detector to reduce this gap. In HEAD, the assistant is an additional detection head with the architecture homogeneous to the teacher head attached to the student backbone. Thus, a hetero-KD is transformed into a homo-KD, allowing efficient knowledge transfer from the teacher to the student. Moreover, we extend HEAD into a Teacher-Free HEAD (TF-HEAD) framework when a well-trained teacher detector is unavailable. Our method has achieved significant improvement compared to current detection KD methods. For example, on the MS-COCO dataset, TF-HEAD helps R18 RetinaNet achieve 33.9 mAP (+2.2), while HEAD further pushes the limit to 36.2 mAP (+4.5).

cs.CV↗

arXiv2022

Single-beam double-pass miniaturized atomic magnetometer for bio-magnetic imaging systems

Miniaturized atomic magnetometers, particularly spin-exchange relaxation-free atomic magnetometers, have been emerging in clinical imaging applications such magnetocardiography and magnetoencephalography. Miniaturization, portability, and low cost are primary development targets for bio-magnetic imaging technologies, as well as high sensitivity and spatial and time resolution. In this paper, we propose a low-cost solution for a bio-magnetic imaging system based on atomic magnetometers, in which one laser source is used for a multi-channel atomic magnetometer sensor array. A novel design is demonstrated for a miniaturized spin-exchange relaxation-free atomic magnetometer, consisting of a single-beam double-pass configuration based on an optical fiber circulator. The effects of temperature and laser power on the zero-field magnetic resonance line-width are characterized, and the experimental results show that the present design achieves better performance, than a traditional single-beam single-pass configuration. The noise power spectrum shows that the closed-loop miniaturized atomic magnetometer reaches a sensitivity of approximately 120 fT/Hz$^{1/2}$ at a bandwidth of 10 Hz. This design is especially suitable for atomic magnetometers operating in arrays as a basic building element for low-cost bio-magnetic imaging systems.

physics.atom-ph↗

arXiv2022

Temporal-Relational Hypergraph Tri-Attention Networks for Stock Trend Prediction

Predicting the future price trends of stocks is a challenging yet intriguing problem given its critical role to help investors make profitable decisions. In this paper, we present a collaborative temporal-relational modeling framework for end-to-end stock trend prediction. The temporal dynamics of stocks is firstly captured with an attention-based recurrent neural network. Then, different from existing studies relying on the pairwise correlations between stocks, we argue that stocks are naturally connected as a collective group, and introduce the hypergraph structures to jointly characterize the stock group-wise relationships of industry-belonging and fund-holding. A novel hypergraph tri-attention network (HGTAN) is proposed to augment the hypergraph convolutional networks with a hierarchical organization of intra-hyperedge, inter-hyperedge, and inter-hypergraph attention modules. In this manner, HGTAN adaptively determines the importance of nodes, hyperedges, and hypergraphs during the information propagation among stocks, so that the potential synergies between stock movements can be fully exploited. Extensive experiments on real-world data demonstrate the effectiveness of our approach. Also, the results of investment simulation show that our approach can achieve a more desirable risk-adjusted return. The data and codes of our work have been released at https://github.com/lixiaojieff/HGTAN.

q-fin.ST↗

arXiv2021

Face Deblurring Based on Separable Normalization and Adaptive Denormalization

Face deblurring aims to restore a clear face image from a blurred input image with more explicit structure and facial details. However, most conventional image and face deblurring methods focus on the whole generated image resolution without consideration of special face part texture and generally produce unsufficient details. Considering that faces and backgrounds have different distribution information, in this study, we designed an effective face deblurring network based on separable normalization and adaptive denormalization (SNADNet). First, We fine-tuned the face parsing network to obtain an accurate face structure. Then, we divided the face parsing feature into face foreground and background. Moreover, we constructed a new feature adaptive denormalization to regularize fafcial structures as a condition of the auxiliary to generate more harmonious and undistorted face structure. In addition, we proposed a texture extractor and multi-patch discriminator to enhance the generated facial texture information. Experimental results on both CelebA and CelebA-HQ datasets demonstrate that the proposed face deblurring network restores face structure with more facial details and performs favorably against state-of-the-art methods in terms of structured similarity indexing method (SSIM), peak signal-to-noise ratio (PSNR), Frechet inception distance (FID) and L1, and qualitative comparisons.

cs.CV↗

arXiv2020

Domain Embedded Multi-model Generative Adversarial Networks for Image-based Face Inpainting

Prior knowledge of face shape and structure plays an important role in face inpainting. However, traditional face inpainting methods mainly focus on the generated image resolution of the missing portion without consideration of the special particularities of the human face explicitly and generally produce discordant facial parts. To solve this problem, we present a domain embedded multi-model generative adversarial model for inpainting of face images with large cropped regions. We firstly represent only face regions using the latent variable as the domain knowledge and combine it with the non-face parts textures to generate high-quality face images with plausible contents. Two adversarial discriminators are finally used to judge whether the generated distribution is close to the real distribution or not. It can not only synthesize novel image structures but also explicitly utilize the embedded face domain knowledge to generate better predictions with consistency on structures and appearance. Experiments on both CelebA and CelebA-HQ face datasets demonstrate that our proposed approach achieved state-of-the-art performance and generates higher quality inpainting results than existing ones.

cs.CV↗

arXiv2019

Detector-in-Detector: Multi-Level Analysis for Human-Parts

Vision-based person, hand or face detection approaches have achieved incredible success in recent years with the development of deep convolutional neural network (CNN). In this paper, we take the inherent correlation between the body and body parts into account and propose a new framework to boost up the detection performance of the multi-level objects. In particular, we adopt a region-based object detection structure with two carefully designed detectors to separately pay attention to the human body and body parts in a coarse-to-fine manner, which we call Detector-in-Detector network (DID-Net). The first detector is designed to detect human body, hand, and face. The second detector, based on the body detection results of the first detector, mainly focus on the detection of small hand and face inside each body. The framework is trained in an end-to-end way by optimizing a multi-task loss. Due to the lack of human body, face and hand detection dataset, we have collected and labeled a new large dataset named Human-Parts with 14,962 images and 106,879 annotations. Experiments show that our method can achieve excellent performance on Human-Parts.

cs.CV↗

arXiv2018

Understanding the mechanical properties of reduced activation steels

Reduced activation ferritic/martensitic (RAFM) steels are structural materials with potential application in Generation-IV fission and fusion reactors. We use density-functional theory to scrutinize the micro-mechanical properties of the main alloy phases of three RAFM steels based on the body-centered cubic FeCrWVMn solid solution. We assess the lattice parameters and elastic properties of ferromagnetic $α$-Fe and Fe$_{91}$Cr$_{9}$, which are the main building blocks of the RAFM steels, and present a detailed analysis of the calculated alloying effects of V, Cr, Mn, and W on the mechanical properties of Fe$_{91}$Cr$_{9}$. The composition dependence of the elastic parameters is decomposed into electronic and volumetric contributions and studied for alloying levels that cover the typical intervals in RAFM steels. A linear superposition of the individual solute effects on the properties of Fe$_{91}$Cr$_{9}$ is shown to provide an excellent approximation for the \emph{ab initio} values obtained for the RAFM steels. The intrinsic ductility is evaluated through Rice's phenomenological theory using the surface and unstable stacking fault energies, and the predictions are contrasted with those obtained by empirical criteria. Alloying with V or W is found to enhance the ductility, whereas additional Cr or Mn turns the RAFM base alloys more brittle.

cond-mat.mtrl-sci↗