arXiv subjects

Xinyu Liu

Publications and source records attributed to Xinyu Liu.

At least 217 records · Page 12Linked to original sources

arXiv2023

A photo-mechanical coupling theory for photoisomerization hydrogel considering the distribution state of molecular chains

Owing to the possibility of controlling its specific mechanical behaviors up taken by irradiated by light at particular wavelengths, the photoisomerization hydrogels have a broad range of potential applications. A theory connecting the optical excitation to mechanical behavior is essential to precisely control the photo-mechanical behaviors of the hydrogel. In this work, a photo-mechanical coupling theory is developed to describe the photo-mechanical responses of photoisomerization hydrogels within the framework of finite deformation continuum thermodynamics. In the model, the deformation gradient is decomposed into two parts to effectively model the light induced deformation and the elastic one. To consider the effect of the optical excitation on mechanical behaviors, we first investigate the transporting mechanism of light in hydrogel, as well as the photochemical reaction process; and we then explore the disturbance of light irradiation on the equilibrium of the thermodynamic system of hydrogel, as well as the relationship of conformational entropy of hydrogel network with the photochemical reaction; finally, based on the entropy elasticity theory, we propose a new free energy function of the photosensitive hydrogel to consider the effect of molecular chain distribution evolution on the stiffness of the hydrogel network. With the implementation of the proposed model, we study the photo-mechanical behaviors and mechanical properties of photoisomerization hydrogels. The present research is helpful for understanding the multi-field coupling behaviors of the photosensitive hydrogel, and then providing guidelines for the application of photoisomerization hydrogel.

physics.chem-ph↗

arXiv2023

EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention

Vision transformers have shown great success due to their high model capabilities. However, their remarkable performance is accompanied by heavy computation costs, which makes them unsuitable for real-time applications. In this paper, we propose a family of high-speed vision transformers named EfficientViT. We find that the speed of existing transformer models is commonly bounded by memory inefficient operations, especially the tensor reshaping and element-wise functions in MHSA. Therefore, we design a new building block with a sandwich layout, i.e., using a single memory-bound MHSA between efficient FFN layers, which improves memory efficiency while enhancing channel communication. Moreover, we discover that the attention maps share high similarities across heads, leading to computational redundancy. To address this, we present a cascaded group attention module feeding attention heads with different splits of the full feature, which not only saves computation cost but also improves attention diversity. Comprehensive experiments demonstrate EfficientViT outperforms existing efficient models, striking a good trade-off between speed and accuracy. For instance, our EfficientViT-M5 surpasses MobileNetV3-Large by 1.9% in accuracy, while getting 40.4% and 45.2% higher throughput on Nvidia V100 GPU and Intel Xeon CPU, respectively. Compared to the recent efficient model MobileViT-XXS, EfficientViT-M2 achieves 1.8% superior accuracy, while running 5.8x/3.7x faster on the GPU/CPU, and 7.4x faster when converted to ONNX format. Code and models are available at https://github.com/microsoft/Cream/tree/main/EfficientViT.

cs.CV↗

arXiv2023

An efficient and robust SAV based algorithm for discrete gradient systems arising from optimizations

We propose in this paper a new minimization algorithm based on a slightly modified version of the scalar auxiliary variable (SAV) approach coupled with a relaxation step and an adaptive strategy. It enjoys several distinct advantages over popular gradient based methods: (i) it is unconditionally energy diminishing with a modified energy which is intrinsically related to the original energy, thus no parameter tuning is needed for stability; (ii) it allows the use of large step-sizes which can effectively accelerate the convergence rate. We also present a convergence analysis for some SAV based algorithms, which include our new algorithm without the relaxation step as a special case. We apply our new algorithm to several illustrative and benchmark problems, and compare its performance with several popular gradient based methods. The numerical results indicate that the new algorithm is very robust, and its adaptive version usually converges significantly faster than those popular gradient descent based methods.

math.NA↗

arXiv2023

Instruction-ViT: Multi-Modal Prompts for Instruction Learning in ViT

Prompts have been proven to play a crucial role in large language models, and in recent years, vision models have also been using prompts to improve scalability for multiple downstream tasks. In this paper, we focus on adapting prompt design based on instruction tuning into a visual transformer model for image classification which we called Instruction-ViT. The key idea is to implement multi-modal prompts (text or image prompt) related to category information to guide the fine-tuning of the model. Based on the experiments of several image captionining tasks, the performance and domain adaptability were improved. Our work provided an innovative strategy to fuse multi-modal prompts with better performance and faster adaptability for visual classification models.

cs.CV↗

arXiv2023

Delving into Shape-aware Zero-shot Semantic Segmentation

Thanks to the impressive progress of large-scale vision-language pretraining, recent recognition models can classify arbitrary objects in a zero-shot and open-set manner, with a surprisingly high accuracy. However, translating this success to semantic segmentation is not trivial, because this dense prediction task requires not only accurate semantic understanding but also fine shape delineation and existing vision-language models are trained with image-level language descriptions. To bridge this gap, we pursue \textbf{shape-aware} zero-shot semantic segmentation in this study. Inspired by classical spectral methods in the image segmentation literature, we propose to leverage the eigen vectors of Laplacian matrices constructed with self-supervised pixel-wise features to promote shape-awareness. Despite that this simple and effective technique does not make use of the masks of seen classes at all, we demonstrate that it out-performs a state-of-the-art shape-aware formulation that aligns ground truth and predicted edges during training. We also delve into the performance gains achieved on different datasets using different backbones and draw several interesting and conclusive observations: the benefits of promoting shape-awareness highly relates to mask compactness and language embedding locality. Finally, our method sets new state-of-the-art performance for zero-shot semantic segmentation on both Pascal and COCO, with significant margins. Code and models will be accessed at https://github.com/Liuxinyv/SAZS.

cs.CV↗

arXiv2023

Learning for Vehicle-to-Vehicle Cooperative Perception under Lossy Communication

Deep learning has been widely used in the perception (e.g., 3D object detection) of intelligent vehicle driving. Due to the beneficial Vehicle-to-Vehicle (V2V) communication, the deep learning based features from other agents can be shared to the ego vehicle so as to improve the perception of the ego vehicle. It is named as Cooperative Perception in the V2V research, whose algorithms have been dramatically advanced recently. However, all the existing cooperative perception algorithms assume the ideal V2V communication without considering the possible lossy shared features because of the Lossy Communication (LC) which is common in the complex real-world driving scenarios. In this paper, we first study the side effect (e.g., detection performance drop) by the lossy communication in the V2V Cooperative Perception, and then we propose a novel intermediate LC-aware feature fusion method to relieve the side effect of lossy communication by a LC-aware Repair Network (LCRN) and enhance the interaction between the ego vehicle and other vehicles by a specially designed V2V Attention Module (V2VAM) including intra-vehicle attention of ego vehicle and uncertainty-aware inter-vehicle attention. The extensive experiment on the public cooperative perception dataset OPV2V (based on digital-twin CARLA simulator) demonstrates that the proposed method is quite effective for the cooperative point cloud based 3D object detection under lossy V2V communication.

cs.CV↗

arXiv2023

Protecting quantum entanglement between error-corrected logical qubits

Entanglement represents one of the most important conceptual advances in physics during the last century and is also one of the most essential resources in quantum information science. However, entanglement is fragile and its potential advantages in applications are hindered by decoherence in practice. Here, we experimentally realize entangled logical qubits (ELQ) with a bosonic quantum module by encoding quantum information into spatially separated microwave modes. The entanglement is protected by repetitive quantum error correction, and the coherence time of the purified ELQ via error detection is improved by 45$\%$ compared with the unprotected ELQ and exceeds that of the entangled physical qubits. In addition, violation of the Bell inequality by logical qubits is demonstrated for the first time with the measured Bell signal B=2.250$\pm$0.019 after purification, surpassing the classical bound by 13 standard deviations. The protected ELQ could be applied in future explorations of quantum foundations and applications of quantum networks.

quant-ph↗

arXiv2023

ADAPT: Action-aware Driving Caption Transformer

End-to-end autonomous driving has great potential in the transportation industry. However, the lack of transparency and interpretability of the automatic decision-making process hinders its industrial adoption in practice. There have been some early attempts to use attention maps or cost volume for better model explainability which is difficult for ordinary passengers to understand. To bridge the gap, we propose an end-to-end transformer-based architecture, ADAPT (Action-aware Driving cAPtion Transformer), which provides user-friendly natural language narrations and reasoning for each decision making step of autonomous vehicular control and action. ADAPT jointly trains both the driving caption task and the vehicular control prediction task, through a shared video representation. Experiments on BDD-X (Berkeley DeepDrive eXplanation) dataset demonstrate state-of-the-art performance of the ADAPT framework on both automatic metrics and human evaluation. To illustrate the feasibility of the proposed framework in real-world applications, we build a novel deployable system that takes raw car videos as input and outputs the action narrations and reasoning in real time. The code, models and data are available at https://github.com/jxbbb/ADAPT.

cs.CV↗

arXiv2022

Artificial Intelligence Security Competition (AISC)

The security of artificial intelligence (AI) is an important research area towards safe, reliable, and trustworthy AI systems. To accelerate the research on AI security, the Artificial Intelligence Security Competition (AISC) was organized by the Zhongguancun Laboratory, China Industrial Control Systems Cyber Emergency Response Team, Institute for Artificial Intelligence, Tsinghua University, and RealAI as part of the Zhongguancun International Frontier Technology Innovation Competition (https://www.zgc-aisc.com/en). The competition consists of three tracks, including Deepfake Security Competition, Autonomous Driving Security Competition, and Face Recognition Security Competition. This report will introduce the competition rules of these three tracks and the solutions of top-ranking teams in each track.

cs.CR↗

arXiv2022

Discovery of a high-temperature antiferromagnetic state and transport signatures of exchange interactions in a Bi2Se3/EuSe heterostructure

Spatial confinement of electronic topological surface states (TSS) in topological insulators poses a formidable challenge because TSS are protected by time-reversal symmetry. In previous works formation of a gap in the electronic spectrum of TSS has been successfully demonstrated in topological insulator/magnetic material heterostructures, where ferromagnetic exchange interactions locally lifts the time-reversal symmetry. Here we report an experimental evidence of exchange interaction between a topological insulator Bi2Se3 and a magnetic insulator EuSe. Spin-polarized neutron reflectometry reveals a reduction of the in-plane magnetic susceptibility within a 2 nm interfacial layer of EuSe, and the combination of SQUID magnetometry and Hall measurements points to the formation of an antiferromagnetic layer with at least five-fold enhancement of Néel's temperature. Abrupt resistance changes in high magnetic fields indicate interfacial exchange coupling that affects transport in a TSS. High temperature local control of TSS with zero net magnetization unlocks new opportunities for the design of electronic, spintronic and quantum computation devices, ranging from quantization of Hall conductance in zero fields to spatial localization of non-Abelian excitations in superconducting topological qubits.

cond-mat.mes-hall↗

arXiv2022

End-to-end joint optimization of metasurface and image processing for compact snapshot hyperspectral imaging

Traditional snapshot hyperspectral imaging systems generally require multiple refractive-optics-based elements to modulate light, resulting in bulky framework. In pursuit of a more compact form factor, a metasurface-based snapshot hyperspectral imaging system, which achieves joint optimization of metasurface and image processing, is proposed in this paper. The unprecedented light manipulation capabilities of metasurfaces are used in conjunction with neural networks to encode and decode light fields for better hyperspectral imaging. Specifically, the extremely strong dispersion of metasurfaces is exploited to distinguish spectral information, and a neural network based on spectral priors is applied for hyperspectral image reconstruction. By constructing a fully differentiable model of metasurface-based hyperspectral imaging, the front-end metasurface phase distribution and the back-end recovery network parameters can be jointly optimized. This method achieves high-quality hyperspectral reconstruction results numerically, outperforming separation optimization methods. The proposed system holds great potential for miniaturization and portability of hyperspectral imaging systems.

physics.optics↗

arXiv2022

WE model: A Machine Learning Model Based on Data-Driven Movie Derivatives Market Prediction

The mature development and the extension of the industry chain make the income structure of the film industry. The income of the traditional film industry depends on the box office and also includes movie merchandising, advertisement, home entertainment, book sales etc. Movie merchandising can even become more profitable than the box office. Therefore, market analysis and forecasting methods for multi-feature merchandising of multi-type films are particularly important. Traditional market research is time-consuming and labour-intensive, and its practical value is restricted. Due to the limited research method, more effective predictive analysis technology needs to be formed. With the rapid development of machine learning and big data, a large number of machine learning algorithms for predictive regression and classification recognition have been proposed and widely used in product design and industry analysis. This paper proposes a high-precision movie merchandising prediction model based on machine learning technology: WE model. This model integrates three machine learning algorithms to accurately predict the movie merchandising market. The WE model learns the relationship between the movie merchandising market and movie features by analyzing the main feature information of movies. After testing, the accuracy rate of prediction and evaluation in the merchandising market reaches 72.5%, and it has achieved a strong market control effect.

cs.CY↗

arXiv2022

Towards Robust Adaptive Object Detection under Noisy Annotations

Domain Adaptive Object Detection (DAOD) models a joint distribution of images and labels from an annotated source domain and learns a domain-invariant transformation to estimate the target labels with the given target domain images. Existing methods assume that the source domain labels are completely clean, yet large-scale datasets often contain error-prone annotations due to instance ambiguity, which may lead to a biased source distribution and severely degrade the performance of the domain adaptive detector de facto. In this paper, we represent the first effort to formulate noisy DAOD and propose a Noise Latent Transferability Exploration (NLTE) framework to address this issue. It is featured with 1) Potential Instance Mining (PIM), which leverages eligible proposals to recapture the miss-annotated instances from the background; 2) Morphable Graph Relation Module (MGRM), which models the adaptation feasibility and transition probability of noisy samples with relation matrices; 3) Entropy-Aware Gradient Reconcilement (EAGR), which incorporates the semantic information into the discrimination process and enforces the gradients provided by noisy and clean samples to be consistent towards learning domain-invariant representations. A thorough evaluation on benchmark DAOD datasets with noisy source annotations validates the effectiveness of NLTE. In particular, NLTE improves the mAP by 8.4\% under 60\% corrupted annotations and even approaches the ideal upper bound of training on a clean source dataset.

cs.CV↗

arXiv2022

SIGMA: Semantic-complete Graph Matching for Domain Adaptive Object Detection

Domain Adaptive Object Detection (DAOD) leverages a labeled domain to learn an object detector generalizing to a novel domain free of annotations. Recent advances align class-conditional distributions by narrowing down cross-domain prototypes (class centers). Though great success,they ignore the significant within-class variance and the domain-mismatched semantics within the training batch, leading to a sub-optimal adaptation. To overcome these challenges, we propose a novel SemantIc-complete Graph MAtching (SIGMA) framework for DAOD, which completes mismatched semantics and reformulates the adaptation with graph matching. Specifically, we design a Graph-embedded Semantic Completion module (GSC) that completes mismatched semantics through generating hallucination graph nodes in missing categories. Then, we establish cross-image graphs to model class-conditional distributions and learn a graph-guided memory bank for better semantic completion in turn. After representing the source and target data as graphs, we reformulate the adaptation as a graph matching problem, i.e., finding well-matched node pairs across graphs to reduce the domain gap, which is solved with a novel Bipartite Graph Matching adaptor (BGM). In a nutshell, we utilize graph nodes to establish semantic-aware node affinity and leverage graph edges as quadratic constraints in a structure-aware matching loss, achieving fine-grained adaptation with a node-to-node graph matching. Extensive experiments verify that SIGMA outperforms existing works significantly. Our code is available at https://github.com/CityU-AIM-Group/SIGMA.

cs.CV↗

arXiv2022

Generalizing to New Domains by Mapping Natural Language to Lifted LTL

Recent work on using natural language to specify commands to robots has grounded that language to LTL. However, mapping natural language task specifications to LTL task specifications using language models require probability distributions over finite vocabulary. Existing state-of-the-art methods have extended this finite vocabulary to include unseen terms from the input sequence to improve output generalization. However, novel out-of-vocabulary atomic propositions cannot be generated using these methods. To overcome this, we introduce an intermediate contextual query representation which can be learned from single positive task specification examples, associating a contextual query with an LTL template. We demonstrate that this intermediate representation allows for generalization over unseen object references, assuming accurate groundings are available. We compare our method of mapping natural language task specifications to intermediate contextual queries against state-of-the-art CopyNet models capable of translating natural language to LTL, by evaluating whether correct LTL for manipulation and navigation task specifications can be output, and show that our method outperforms the CopyNet model on unseen object references. We demonstrate that the grounded LTL our method outputs can be used for planning in a simulated OO-MDP environment. Finally, we discuss some common failure modes encountered when translating natural language task specifications to grounded LTL.

cs.CL↗

arXiv2022

Neural Architecture Searching for Facial Attributes-based Depression Recognition

Recent studies show that depression can be partially reflected from human facial attributes. Since facial attributes have various data structure and carry different information, existing approaches fail to specifically consider the optimal way to extract depression-related features from each of them, as well as investigates the best fusion strategy. In this paper, we propose to extend Neural Architecture Search (NAS) technique for designing an optimal model for multiple facial attributes-based depression recognition, which can be efficiently and robustly implemented in a small dataset. Our approach first conducts a warmer up step to the feature extractor of each facial attribute, aiming to largely reduce the search space and providing customized architecture, where each feature extractor can be either a Convolution Neural Networks (CNN) or Graph Neural Networks (GNN). Then, we conduct an end-to-end architecture search for all feature extractors and the fusion network, allowing the complementary depression cues to be optimally combined with less redundancy. The experimental results on AVEC 2016 dataset show that the model explored by our approach achieves breakthrough performance with 27\% and 30\% RMSE and MAE improvements over the existing state-of-the-art. In light of these findings, this paper provides solid evidences and a strong baseline for applying NAS to time-series data-based mental health analysis.

cs.CV↗

arXiv2021

Observation of coexisting weak localization and superconducting fluctuations in strained Sn1-xInxTe thin films

Topological superconductors have attracted tremendous excitement as they are predicted to host Majorana zero modes that can be utilized for topological quantum computing. Candidate topological superconductor Sn1-xInxTe thin films (0<x<0.3) grown by molecular beam epitaxy and strained in the (111) plane are shown to host three coexisting quantum effects: localization, antilocalization and superconducting fluctuations above the critical temperature Tc. An analysis of the normal state magnetoresistance reveals these effects. Weak localization is consistently observed in superconducting samples, indicating that superconductivity originates dominantly from trivial valence band states that may be strongly spin-orbit split. A large enhancement of the conductivity is observed above Tc, indicating that quantum coherent quasiparticle effects coexist with superconducting fluctuations. Our results motivate a re-examination of the debated pairing symmetry of this material when subjected to quantum confinement and lattice strain.

cond-mat.supr-con↗

arXiv2021

Fermi level tuning and band alignment in Mn doped InAs/GaSb

InAs/GaSb hosts a broken gap band alignment that has been shown to generate helical topological edge states. Upon the introduction of Mn into the structure, it has been predicted to host a quantized anomalous Hall effect. Here, we show that dilute Mn doping on InAs in InAs/GaSb, allows a tuning of the Fermi level, the introduction of paramagnetism, but also has a non-trivial impact on the band alignment of the system. The measurement of Shubnikov-de-Haas oscillations, cyclotron resonance, and a non-linear Hall effect in Mn-doped samples indicate the coexistence of a high mobility two-dimensional electron gas and a hole gas. Conversely, in undoped InAs/GaSb, pure-n-type transport is observed. We hypothesize that Mn acceptor levels can pin the Fermi energy near the valence band edge of InAs, far from the interface, which introduces a strong band bending to preserve the band offset at the InAs/GaSb interface. The realization of the QAHE in this structure will thus require a careful control of the band alignment to preserve topological insulating character.

cond-mat.mes-hall↗