arXiv subjects

Jiachen Li

Publications and source records attributed to Jiachen Li.

At least 181 records · Page 10Linked to original sources

arXiv2023

Neighborhood Attention Transformer

We present Neighborhood Attention (NA), the first efficient and scalable sliding-window attention mechanism for vision. NA is a pixel-wise operation, localizing self attention (SA) to the nearest neighboring pixels, and therefore enjoys a linear time and space complexity compared to the quadratic complexity of SA. The sliding-window pattern allows NA's receptive field to grow without needing extra pixel shifts, and preserves translational equivariance, unlike Swin Transformer's Window Self Attention (WSA). We develop NATTEN (Neighborhood Attention Extension), a Python package with efficient C++ and CUDA kernels, which allows NA to run up to 40% faster than Swin's WSA while using up to 25% less memory. We further present Neighborhood Attention Transformer (NAT), a new hierarchical transformer design based on NA that boosts image classification and downstream vision performance. Experimental results on NAT are competitive; NAT-Tiny reaches 83.2% top-1 accuracy on ImageNet, 51.4% mAP on MS-COCO and 48.4% mIoU on ADE20K, which is 1.9% ImageNet accuracy, 1.0% COCO mAP, and 2.6% ADE20K mIoU improvement over a Swin model with similar size. To support more research based on sliding-window attention, we open source our project and release our checkpoints at: https://github.com/SHI-Labs/Neighborhood-Attention-Transformer .

cs.CV↗

arXiv2023

Linear Scaling Calculations of Excitation Energies with Active-Space Particle-Particle Random Phase Approximation

We developed an efficient active-space particle-particle random phase approximation (ppRPA) approach to calculate accurate charge-neutral excitation energies of molecular systems. The active-space ppRPA approach constrains both indexes in particle and hole pairs in the ppRPA matrix, which only selects frontier orbitals with dominant contributions to low-lying excitation energies. It employs the truncation in both orbital indexes in the particle-particle and the hole-hole spaces. The resulting matrix, the eigenvalues of which are excitation energies, has a dimension that is independent of the size of the systems. The computational effort for the excitation energy calculation, therefore, scales linearly with system size and is negligible compared with the ground-state calculation of the (N-2)-electron system, where N is the electron number of the molecule. With the active space consisting of 30 occupied and 30 virtual orbitals, the active-space ppRPA approach predicts excitation energies of valence, charge-transfer, Rydberg, double and diradical excitations with the mean absolute errors (MAEs) smaller than 0.03 eV compared with the full-space ppRPA results. As a side product, we also applied the active-space ppRPA approach in the renormalized singles (RS) T-matrix approach. Combining the non-interacting pair approximation that approximates the contribution to the self-energy outside the active space, the active-space $G_{\text{RS}}T_{\text{RS}}$@PBE approach predicts accurate absolute and relative core-level binding energies with the MAE around 1.58 eV and 0.3 eV, respectively. The developed linear scaling calculation of excitation energies is promising for applications to large and complex systems.

physics.chem-ph↗

arXiv2023

Shedding Light on Rechargeable Na/Cl$_2$ Battery

Advancing new ideas of rechargeable batteries represents an important path to meeting the ever increasing energy storage needs. Recently we showed rechargeable sodium/chlorine (Na/Cl$_2$) (or lithium/chlorine Li/Cl$_2$) batteries that used a Na (or Li) metal negative electrode, a microporous amorphous carbon nanosphere (aCNS) positive electrode and an electrolyte containing dissolved AlCl$_3$ and fluoride additives in thionyl chloride (SOCl$_2$). The main battery redox reaction involved conversion between NaCl and Cl$_2$ trapped in the carbon positive electrode, delivering a cyclable capacity of up to 1200 mAh g$^{-1}$ (based on positive electrode mass) at a ~ 3.5 V discharge voltage. Here, we discovered by X-ray photoelectron spectroscopy (XPS) that upon charging a Na/Cl$_2$ battery, chlorination of carbon in the positive electrode occurred to form C-Cl accompanied by molecular Cl$_2$ infiltrating the porous aCNS, consistent with Cl$_2$ probed by mass spectrometry. Synchrotron X-ray diffraction observed the development of graphitic ordering in the initially amorphous aCNS under battery charging when the carbon matrix was oxidized/chlorinated and infiltrated with Cl$_2$. The C-Cl, Cl$_2$ species and graphitic ordering were reversible upon discharge, accompanied by NaCl formation. The results revealed redox conversion between NaCl and Cl$_2$, reversible graphitic ordering/amorphourization of carbon through battery charge/discharge, and for the first time probed trapped Cl$_2$ in porous carbon by XPS.

physics.app-ph↗

arXiv2023

CoNIC Challenge: Pushing the Frontiers of Nuclear Detection, Segmentation, Classification and Counting

Nuclear detection, segmentation and morphometric profiling are essential in helping us further understand the relationship between histology and patient outcome. To drive innovation in this area, we setup a community-wide challenge using the largest available dataset of its kind to assess nuclear segmentation and cellular composition. Our challenge, named CoNIC, stimulated the development of reproducible algorithms for cellular recognition with real-time result inspection on public leaderboards. We conducted an extensive post-challenge analysis based on the top-performing models using 1,658 whole-slide images of colon tissue. With around 700 million detected nuclei per model, associated features were used for dysplasia grading and survival analysis, where we demonstrated that the challenge's improvement over the previous state-of-the-art led to significant boosts in downstream performance. Our findings also suggest that eosinophils and neutrophils play an important role in the tumour microevironment. We release challenge models and WSI-level results to foster the development of further methods for biomarker discovery.

cs.CV↗

arXiv2023

Causal Balancing for Domain Generalization

While machine learning models rapidly advance the state-of-the-art on various real-world tasks, out-of-domain (OOD) generalization remains a challenging problem given the vulnerability of these models to spurious correlations. We propose a balanced mini-batch sampling strategy to transform a biased data distribution into a spurious-free balanced distribution, based on the invariance of the underlying causal mechanisms for the data generation process. We argue that the Bayes optimal classifiers trained on such balanced distribution are minimax optimal across a diverse enough environment space. We also provide an identifiability guarantee of the latent variable model of the proposed data generation process, when utilizing enough train environments. Experiments are conducted on DomainBed, demonstrating empirically that our method obtains the best performance across 20 baselines reported on the benchmark.

cs.LG↗

arXiv2023

Interaction Modeling with Multiplex Attention

Modeling multi-agent systems requires understanding how agents interact. Such systems are often difficult to model because they can involve a variety of types of interactions that layer together to drive rich social behavioral dynamics. Here we introduce a method for accurately modeling multi-agent systems. We present Interaction Modeling with Multiplex Attention (IMMA), a forward prediction model that uses a multiplex latent graph to represent multiple independent types of interactions and attention to account for relations of different strengths. We also introduce Progressive Layer Training, a training strategy for this architecture. We show that our approach outperforms state-of-the-art models in trajectory forecasting and relation inference, spanning three multi-agent scenarios: social navigation, cooperative task achievement, and team sports. We further demonstrate that our approach can improve zero-shot generalization and allows us to probe how different interactions impact agent behavior.

cs.LG↗

arXiv2022

OneFormer: One Transformer to Rule Universal Image Segmentation

Universal Image Segmentation is not a new concept. Past attempts to unify image segmentation in the last decades include scene parsing, panoptic segmentation, and, more recently, new panoptic architectures. However, such panoptic architectures do not truly unify image segmentation because they need to be trained individually on the semantic, instance, or panoptic segmentation to achieve the best performance. Ideally, a truly universal framework should be trained only once and achieve SOTA performance across all three image segmentation tasks. To that end, we propose OneFormer, a universal image segmentation framework that unifies segmentation with a multi-task train-once design. We first propose a task-conditioned joint training strategy that enables training on ground truths of each domain (semantic, instance, and panoptic segmentation) within a single multi-task training process. Secondly, we introduce a task token to condition our model on the task at hand, making our model task-dynamic to support multi-task training and inference. Thirdly, we propose using a query-text contrastive loss during training to establish better inter-task and inter-class distinctions. Notably, our single OneFormer model outperforms specialized Mask2Former models across all three segmentation tasks on ADE20k, CityScapes, and COCO, despite the latter being trained on each of the three tasks individually with three times the resources. With new ConvNeXt and DiNAT backbones, we observe even more performance improvement. We believe OneFormer is a significant step towards making image segmentation more universal and accessible. To support further research, we open-source our code and models at https://github.com/SHI-Labs/OneFormer

cs.CV↗

arXiv2022

VMFormer: End-to-End Video Matting with Transformer

Video matting aims to predict the alpha mattes for each frame from a given input video sequence. Recent solutions to video matting have been dominated by deep convolutional neural networks (CNN) for the past few years, which have become the de-facto standard for both academia and industry. However, they have inbuilt inductive bias of locality and do not capture global characteristics of an image due to the CNN-based architectures. They also lack long-range temporal modeling considering computational costs when dealing with feature maps of multiple frames. In this paper, we propose VMFormer: a transformer-based end-to-end method for video matting. It makes predictions on alpha mattes of each frame from learnable queries given a video input sequence. Specifically, it leverages self-attention layers to build global integration of feature sequences with short-range temporal modeling on successive frames. We further apply queries to learn global representations through cross-attention in the transformer decoder with long-range temporal modeling upon all queries. In the prediction stage, both queries and corresponding feature maps are used to make the final prediction of alpha matte. Experiments show that VMFormer outperforms previous CNN-based video matting methods on the composited benchmarks. To our best knowledge, it is the first end-to-end video matting solution built upon a full vision transformer with predictions on the learnable queries. The project is open-sourced at https://chrisjuniorli.github.io/project/VMFormer/

cs.CV↗

arXiv2022

Off-policy Reinforcement Learning with Optimistic Exploration and Distribution Correction

Improving the sample efficiency of reinforcement learning algorithms requires effective exploration. Following the principle of $\textit{optimism in the face of uncertainty}$ (OFU), we train a separate exploration policy to maximize the approximate upper confidence bound of the critics in an off-policy actor-critic framework. However, this introduces extra differences between the replay buffer and the target policy regarding their stationary state-action distributions. To mitigate the off-policy-ness, we adapt the recently introduced DICE framework to learn a distribution correction ratio for off-policy RL training. In particular, we correct the training distribution for both policies and critics. Empirically, we evaluate our proposed method in several challenging continuous control tasks and show superior performance compared to state-of-the-art methods. We also conduct extensive ablation studies to demonstrate the effectiveness and rationality of the proposed method.

cs.LG↗

arXiv2022

Transformer Based Multi-Grained Features for Unsupervised Person Re-Identification

Multi-grained features extracted from convolutional neural networks (CNNs) have demonstrated their strong discrimination ability in supervised person re-identification (Re-ID) tasks. Inspired by them, this work investigates the way of extracting multi-grained features from a pure transformer network to address the unsupervised Re-ID problem that is label-free but much more challenging. To this end, we build a dual-branch network architecture based upon a modified Vision Transformer (ViT). The local tokens output in each branch are reshaped and then uniformly partitioned into multiple stripes to generate part-level features, while the global tokens of two branches are averaged to produce a global feature. Further, based upon offline-online associated camera-aware proxies (O2CAP) that is a top-performing unsupervised Re-ID method, we define offline and online contrastive learning losses with respect to both global and part-level features to conduct unsupervised learning. Extensive experiments on three person Re-ID datasets show that the proposed method outperforms state-of-the-art unsupervised methods by a considerable margin, greatly mitigating the gap to supervised counterparts. Code will be available soon at https://github.com/RikoLi/WACV23-workshop-TMGF.

cs.CV↗

arXiv2022

Point-to-Box Network for Accurate Object Detection via Single Point Supervision

Object detection using single point supervision has received increasing attention over the years. However, the performance gap between point supervised object detection (PSOD) and bounding box supervised detection remains large. In this paper, we attribute such a large performance gap to the failure of generating high-quality proposal bags which are crucial for multiple instance learning (MIL). To address this problem, we introduce a lightweight alternative to the off-the-shelf proposal (OTSP) method and thereby create the Point-to-Box Network (P2BNet), which can construct an inter-objects balanced proposal bag by generating proposals in an anchor-like way. By fully investigating the accurate position information, P2BNet further constructs an instance-level bag, avoiding the mixture of multiple objects. Finally, a coarse-to-fine policy in a cascade fashion is utilized to improve the IoU between proposals and ground-truth (GT). Benefiting from these strategies, P2BNet is able to produce high-quality instance-level bags for object detection. P2BNet improves the mean average precision (AP) by more than 50% relative to the previous best PSOD method on the MS COCO dataset. It also demonstrates the great potential to bridge the performance gap between point supervised and bounding-box supervised detectors. The code will be released at github.com/ucas-vg/P2BNet.

cs.CV↗

arXiv2022

Combining Renormalized Singles $GW$ Methods with the Bethe-Salpeter Equation for Accurate Neutral Excitation Energies

We apply the renormalized singles (RS) Green's function in the Bethe-Salpeter equation (BSE)/$GW$ approach to predict accurate neutral excitation energies of molecular systems. The BSE calculations are performed on top of the $G_{\text{RS}}W_{\text{RS}}$ method, which uses the RS Green's function also for the computation of the screened Coulomb interaction $W$. We show that the BSE/$G_{\text{RS}}W_{\text{RS}}$ approach significantly outperforms BSE/$G_0W_0$ for predicting excitation energies of valence, Rydberg and charge transfer (CT) excitations by benchmarking the Truhlar-Gagliardi set, Stein CT set and an atomic Rydberg test set. For the Truhlar-Gagliardi test set, BSE/$G_{\text{RS}}W_{\text{RS}}$ provides comparable accuracy to time-dependent density functional theory (TDDFT) and is slightly better than BSE starting from eigenvalue self-consistent $GW$ (ev$GW$). For the Stein CT test set, BSE/$G_{\text{RS}}W_{\text{RS}}$ significantly outperforms BSE/$G_0W_0$ and TDDFT with the accuracy comparable to BSE/ev$GW$. We also show that BSE/$G_{\text{RS}}W_{\text{RS}}$ predicts Rydberg excitation energies of atomic systems well. Besides the excellent accuracy, BSE/$G_{\text{RS}}W_{\text{RS}}$ largely eliminates the dependence on the choice of the density functional approximation. This work demonstrates that the BSE/$G_{\text{RS}}W_{\text{RS}}$ approach is accurate and efficient for predicting excitation energies for a broad range of systems, which expands the applicability of the BSE/$GW$ approach.

physics.chem-ph↗

arXiv2022

Learning Physical Dynamics with Subequivariant Graph Neural Networks

Graph Neural Networks (GNNs) have become a prevailing tool for learning physical dynamics. However, they still encounter several challenges: 1) Physical laws abide by symmetry, which is a vital inductive bias accounting for model generalization and should be incorporated into the model design. Existing simulators either consider insufficient symmetry, or enforce excessive equivariance in practice when symmetry is partially broken by gravity. 2) Objects in the physical world possess diverse shapes, sizes, and properties, which should be appropriately processed by the model. To tackle these difficulties, we propose a novel backbone, Subequivariant Graph Neural Network, which 1) relaxes equivariance to subequivariance by considering external fields like gravity, where the universal approximation ability holds theoretically; 2) introduces a new subequivariant object-aware message passing for learning physical interactions between multiple objects of various shapes in the particle-based representation; 3) operates in a hierarchical fashion, allowing for modeling long-range and complex interactions. Our model achieves on average over 3% enhancement in contact prediction accuracy across 8 scenarios on Physion and 2X lower rollout MSE on RigidFall compared with state-of-the-art GNN simulators, while exhibiting strong generalization and data efficiency.

cs.LG↗

arXiv2022

DRAMA: Joint Risk Localization and Captioning in Driving

Considering the functionality of situational awareness in safety-critical automation systems, the perception of risk in driving scenes and its explainability is of particular importance for autonomous and cooperative driving. Toward this goal, this paper proposes a new research direction of joint risk localization in driving scenes and its risk explanation as a natural language description. Due to the lack of standard benchmarks, we collected a large-scale dataset, DRAMA (Driving Risk Assessment Mechanism with A captioning module), which consists of 17,785 interactive driving scenarios collected in Tokyo, Japan. Our DRAMA dataset accommodates video- and object-level questions on driving risks with associated important objects to achieve the goal of visual captioning as a free-form language description utilizing closed and open-ended responses for multi-level questions, which can be used to evaluate a range of visual captioning capabilities in driving scenarios. We make this data available to the community for further research. Using DRAMA, we explore multiple facets of joint risk localization and captioning in interactive driving scenarios. In particular, we benchmark various multi-task prediction architectures and provide a detailed analysis of joint risk localization and risk captioning. The data set is available at https://usa.honda-ri.com/drama

cs.CV↗

arXiv2022

Offline-Online Associated Camera-Aware Proxies for Unsupervised Person Re-identification

Recently, unsupervised person re-identification (Re-ID) has received increasing research attention due to its potential for label-free applications. A promising way to address unsupervised Re-ID is clustering-based, which generates pseudo labels by clustering and uses the pseudo labels to train a Re-ID model iteratively. However, most clustering-based methods take each cluster as a pseudo identity class, neglecting the intra-cluster variance mainly caused by the change of cameras. To address this issue, we propose to split each single cluster into multiple proxies according to camera views. The camera-aware proxies explicitly capture local structures within clusters, by which the intra-ID variance and inter-ID similarity can be better tackled. Assisted with the camera-aware proxies, we design two proxy-level contrastive learning losses that are, respectively, based on offline and online association results. The offline association directly associates proxies according to the clustering and splitting results, while the online strategy dynamically associates proxies in terms of up-to-date features to reduce the noise caused by the delayed update of pseudo labels. The combination of two losses enables us to train a desirable Re-ID model. Extensive experiments on three person Re-ID datasets and one vehicle Re-ID dataset show that our proposed approach demonstrates competitive performance with state-of-the-art methods. Code will be available at: https://github.com/Terminator8758/O2CAP.

cs.CV↗

arXiv2022

Dynamics-Aware Spatiotemporal Occupancy Prediction in Urban Environments

Detection and segmentation of moving obstacles, along with prediction of the future occupancy states of the local environment, are essential for autonomous vehicles to proactively make safe and informed decisions. In this paper, we propose a framework that integrates the two capabilities together using deep neural network architectures. Our method first detects and segments moving objects in the scene, and uses this information to predict the spatiotemporal evolution of the environment around autonomous vehicles. To address the problem of direct integration of both static-dynamic object segmentation and environment prediction models, we propose using occupancy-based environment representations across the whole framework. Our method is validated on the real-world Waymo Open Dataset and demonstrates higher prediction accuracy than baseline methods.

cs.RO↗

arXiv2022

EvolveHypergraph: Group-Aware Dynamic Relational Reasoning for Trajectory Prediction

While the modeling of pair-wise relations has been widely studied in multi-agent interacting systems, its ability to capture higher-level and larger-scale group-wise activities is limited. In this paper, we propose a group-aware relational reasoning approach (named EvolveHypergraph) with explicit inference of the underlying dynamically evolving relational structures, and we demonstrate its effectiveness for multi-agent trajectory prediction. In addition to the edges between a pair of nodes (i.e., agents), we propose to infer hyperedges that adaptively connect multiple nodes to enable group-aware relational reasoning in an unsupervised manner without fixing the number of hyperedges. The proposed approach infers the dynamically evolving relation graphs and hypergraphs over time to capture the evolution of relations, which are used by the trajectory predictor to obtain future states. Moreover, we propose to regularize the smoothness of the relation evolution and the sparsity of the inferred graphs or hypergraphs, which effectively improves training stability and enhances the explainability of inferred relations. The proposed approach is validated on both synthetic crowd simulations and multiple real-world benchmark datasets. Our approach infers explainable, reasonable group-aware relations and achieves state-of-the-art performance in long-term prediction.

cs.CV↗

arXiv2022

Renormalized Singles with Correlation in $GW$ Green's Function Theory for Accurate Quasiparticle Energies

We apply the renormalized singles with correlation (RSc) Green's function in the $GW$ approximation to calculate accurate quasiparticle (QP) energies and orbitals. The RSc Green's function includes all orders of singles contributions from the associated density functional approximation (DFA) and treats higher order contributions in a perturbative manner. The $G_{\text{RSc}}W_{\text{RSc}}$ method uses the RSc Green's function as the new starting point and calculates the screened interaction with the RSc Green's function. The $G_{\text{RSc}}W_0$ methods fixes the screened interaction at the DFA level. For the calculations of ionization potentials in the GW100 set, $G_{\text{RSc}}W_{\text{RSc}}$ significantly reduces the dependence on the starting point of DFAs used and provides accurate results with the mean absolute error (MAE) of $0.34$ eV comparable to ev$GW$. For the calculations of core-level binding energies in the CORE65 set, $G_{\text{RSc}}W_{\text{RSc}}$ slightly overestimates the results because of underscreening, but $G_{\text{RSc}}W_0$ with GGA functionals provides the optimal accuracy of $0.40$ eV MAE comparable to ev$GW_0$. We also show that $G_{\text{RSc}}W_{\text{RSc}}$ predicts accurate dipole moments of small molecules. These two methods, $G_{\text{RSc}}W_{\text{RSc}}$ and $G_{\text{RSc}}W_0$, are computationally much favorable than any flavor of self-consistent $GW$ methods. Thus, the RSc approach is promising for making $GW$ and other Green's function methods efficient and robust.

physics.chem-ph↗