arXiv subjects

Xingyu Liao

Publications and source records attributed to Xingyu Liao.

18 recordsLinked to original sources

arXiv2025

Serving Large Language Models on Huawei CloudMatrix384

The rapid evolution of large language models (LLMs), driven by growing parameter scales, adoption of mixture-of-experts (MoE) architectures, and expanding context lengths, imposes unprecedented demands on AI infrastructure. Traditional AI clusters face limitations in compute intensity, memory bandwidth, inter-chip communication, and latency, compounded by variable workloads and strict service-level objectives. Addressing these issues requires fundamentally redesigned hardware-software integration. This paper introduces Huawei CloudMatrix, a next-generation AI datacenter architecture, realized in the production-grade CloudMatrix384 supernode. It integrates 384 Ascend 910 NPUs and 192 Kunpeng CPUs interconnected via an ultra-high-bandwidth Unified Bus (UB) network, enabling direct all-to-all communication and dynamic pooling of resources. These features optimize performance for communication-intensive operations, such as large-scale MoE expert parallelism and distributed key-value cache access. To fully leverage CloudMatrix384, we propose CloudMatrix-Infer, an advanced LLM serving solution incorporating three core innovations: a peer-to-peer serving architecture that independently scales prefill, decode, and caching; a large-scale expert parallelism strategy supporting EP320 via efficient UB-based token dispatch; and hardware-aware optimizations including specialized operators, microbatch-based pipelining, and INT8 quantization. Evaluation with the DeepSeek-R1 model shows CloudMatrix-Infer achieves state-of-the-art efficiency: prefill throughput of 6,688 tokens/s per NPU and decode throughput of 1,943 tokens/s per NPU (<50 ms TPOT). It effectively balances throughput and latency, sustaining 538 tokens/s per NPU even under stringent 15 ms latency constraints, while INT8 quantization maintains model accuracy across benchmarks.

cs.DC↗

arXiv2023

Absence of mixed valency for Pr in pristine and hole-doped PrNiO$_2$

Infinite-layer nickelates ($R$NiO$_2$) exhibit some distinct differences as compared to cuprate superconductors, leading to a debate concerning the role of rare-earth ions ($R$=La, Pr, Nd) in the low-energy many-body physics. Although rare-earth $4f$ orbitals are typically treated as inert `core' electrons in studies, this approximation has been questioned. An active participation of $4f$ states is most likely for PrNiO$_2$ based on an analogy to cuprates where Pr cuprates differ significantly from other cuprates. Here, we adopt density functional plus dynamical mean field theory (DFT+DMFT) to investigate the role of Pr $4f$ orbitals and more generally the correlated electronic structure of PrNiO$_2$ and its hole-doped variant. We find that the Pr $4f$ states are insulating and show no evidence for either a Kondo resonance or Zhang-Rice singlet formation as they do not have any hybridization channels near the Fermi energy. The biggest effects of hole doping are to shift the Pr $5d$ and $4f$ states further away from the Fermi energy while enhancing the Ni $3d$ - O $2p$ hybridization, thus reducing correlation effects as the O $2p$ states get closer to the Fermi energy. We again find no evidence for either Kondo or Zhang-Rice physics for the $4f$ states upon hole doping. We conclude by commenting on implications for other reduced valence nickelates.

cond-mat.str-el↗

arXiv2023

PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentation

Comprehensive modeling of the surrounding 3D world is key to the success of autonomous driving. However, existing perception tasks like object detection, road structure segmentation, depth & elevation estimation, and open-set object localization each only focus on a small facet of the holistic 3D scene understanding task. This divide-and-conquer strategy simplifies the algorithm development procedure at the cost of losing an end-to-end unified solution to the problem. In this work, we address this limitation by studying camera-based 3D panoptic segmentation, aiming to achieve a unified occupancy representation for camera-only 3D scene understanding. To achieve this, we introduce a novel method called PanoOcc, which utilizes voxel queries to aggregate spatiotemporal information from multi-frame and multi-view images in a coarse-to-fine scheme, integrating feature learning and scene representation into a unified occupancy representation. We have conducted extensive ablation studies to verify the effectiveness and efficiency of the proposed method. Our approach achieves new state-of-the-art results for camera-based semantic segmentation and panoptic segmentation on the nuScenes dataset. Furthermore, our method can be easily extended to dense occupancy prediction and has shown promising performance on the Occ3D benchmark. The code will be released at https://github.com/Robertwyq/PanoOcc.

cs.CV↗

arXiv2023

detrex: Benchmarking Detection Transformers

The DEtection TRansformer (DETR) algorithm has received considerable attention in the research community and is gradually emerging as a mainstream approach for object detection and other perception tasks. However, the current field lacks a unified and comprehensive benchmark specifically tailored for DETR-based models. To address this issue, we develop a unified, highly modular, and lightweight codebase called detrex, which supports a majority of the mainstream DETR-based instance recognition algorithms, covering various fundamental tasks, including object detection, segmentation, and pose estimation. We conduct extensive experiments under detrex and perform a comprehensive benchmark for DETR-based models. Moreover, we enhance the performance of detection transformers through the refinement of training hyper-parameters, providing strong baselines for supported algorithms.We hope that detrex could offer research communities a standardized and unified platform to evaluate and compare different DETR-based models while fostering a deeper understanding and driving advancements in DETR-based instance recognition. Our code is available at https://github.com/IDEA-Research/detrex. The project is currently being actively developed. We encourage the community to use detrex codebase for further development and contributions.

cs.CV↗

arXiv2023

SkinGPT-4: An Interactive Dermatology Diagnostic System with Visual Large Language Model

Skin and subcutaneous diseases rank high among the leading contributors to the global burden of nonfatal diseases, impacting a considerable portion of the population. Nonetheless, the field of dermatology diagnosis faces three significant hurdles. Firstly, there is a shortage of dermatologists accessible to diagnose patients, particularly in rural regions. Secondly, accurately interpreting skin disease images poses a considerable challenge. Lastly, generating patient-friendly diagnostic reports is usually a time-consuming and labor-intensive task for dermatologists. To tackle these challenges, we present SkinGPT-4, which is the world's first interactive dermatology diagnostic system powered by an advanced visual large language model. SkinGPT-4 leverages a fine-tuned version of MiniGPT-4, trained on an extensive collection of skin disease images (comprising 52,929 publicly available and proprietary images) along with clinical concepts and doctors' notes. We designed a two-step training process to allow SkinGPT to express medical features in skin disease images with natural language and make accurate diagnoses of the types of skin diseases. With SkinGPT-4, users could upload their own skin photos for diagnosis, and the system could autonomously evaluate the images, identifies the characteristics and categories of the skin conditions, performs in-depth analysis, and provides interactive treatment recommendations. Meanwhile, SkinGPT-4's local deployment capability and commitment to user privacy also render it an appealing choice for patients in search of a dependable and precise diagnosis of their skin ailments. To demonstrate the robustness of SkinGPT-4, we conducted quantitative evaluations on 150 real-life cases, which were independently reviewed by certified dermatologists, and showed that SkinGPT-4 could provide accurate diagnoses of skin diseases.

eess.IV↗

arXiv2023

Introducing Depth into Transformer-based 3D Object Detection

In this paper, we present DAT, a Depth-Aware Transformer framework designed for camera-based 3D detection. Our model is based on observing two major issues in existing methods: large depth translation errors and duplicate predictions along depth axes. To mitigate these issues, we propose two key solutions within DAT. To address the first issue, we introduce a Depth-Aware Spatial Cross-Attention (DA-SCA) module that incorporates depth information into spatial cross-attention when lifting image features to 3D space. To address the second issue, we introduce an auxiliary learning task called Depth-aware Negative Suppression loss. First, based on their reference points, we organize features as a Bird's-Eye-View (BEV) feature map. Then, we sample positive and negative features along each object ray that connects an object and a camera and train the model to distinguish between them. The proposed DA-SCA and DNS methods effectively alleviate these two problems. We show that DAT is a versatile method that enhances the performance of all three popular models, BEVFormer, DETR3D, and PETR. Our evaluation on BEVFormer demonstrates that DAT achieves a significant improvement of +2.8 NDS on nuScenes val under the same settings. Moreover, when using pre-trained VoVNet-99 as the backbone, DAT achieves strong results of 60.0 NDS and 51.5 mAP on nuScenes test. Our code will be soon.

cs.CV↗

arXiv2023

Audit to Forget: A Unified Method to Revoke Patients' Private Data in Intelligent Healthcare

Revoking personal private data is one of the basic human rights, which has already been sheltered by several privacy-preserving laws in many countries. However, with the development of data science, machine learning and deep learning techniques, this right is usually neglected or violated as more and more patients' data are being collected and used for model training, especially in intelligent healthcare, thus making intelligent healthcare a sector where technology must meet the law, regulations, and privacy principles to ensure that the innovation is for the common good. In order to secure patients' right to be forgotten, we proposed a novel solution by using auditing to guide the forgetting process, where auditing means determining whether a dataset has been used to train the model and forgetting requires the information of a query dataset to be forgotten from the target model. We unified these two tasks by introducing a new approach called knowledge purification. To implement our solution, we developed AFS, a unified open-source software, which is able to evaluate and revoke patients' private data from pre-trained deep learning models. We demonstrated the generality of AFS by applying it to four tasks on different datasets with various data sizes and architectures of deep learning networks. The software is publicly available at \url{https://github.com/JoshuaChou2018/AFS}.

cs.LG↗

arXiv2022

Effects of the surface termination and oxygen vacancy positions and on LaNiO$_{3}$ ultra-thin films: First-principles study

While ultra-thin layers of the LaNiO$_3$ film exhibit a remarkable metal-insulator transition as the film thickness becomes smaller than a few unit cell (u.c.), the formation of oxygen vacancies and their effects on the correlated electronic structure have been rarely studied. Here, we investigate the effects of the surface termination and the oxygen vacancy position on the electronic properties and vacancy energetics of LaNiO$_3$ ultra-thin films using density functional theory plus U (DFT+U). We find that oxygen vacancies can be easily formed in the Ni layers with the NiO$_2$ terminated surface (0.5 u.c. and 1.5 u.c. thickness) compared to the structures with the LaO terminated surface and the in-plane vacancy is energetically favored than the out-of-plane vacancy. When two vacancy sites are allowed, the Ni square plane geometry is energetically more stable in most cases as two oxygen vacancies tend to stay near a Ni ion. The in-plane vacancy of the NiO$_2$ terminated structure is favored since the released charge due to the oxygen vacancy can be easily accommodated in the $d_{x^2-y^2}$ orbital, which is less occupied than the $d_{z^2}$ orbital. Remarkably, the oxygen vacancy structure containing the Ni square-plane geometry becomes an insulating state in DFT+U with a sizable band gap of 1.2eV because the large crystal field splitting between $d_{z^2}$ and $d_{x^2-y^2}$ orbitals in the square-plane favors an insulating state and the Mott insulating state is induced in other Ni sites due to strong electronic correlations.

cond-mat.mtrl-sci↗

arXiv2021

Semi-Supervised Domain Generalizable Person Re-Identification

Existing person re-identification (re-id) methods are stuck when deployed to a new unseen scenario despite the success in cross-camera person matching. Recent efforts have been substantially devoted to domain adaptive person re-id where extensive unlabeled data in the new scenario are utilized in a transductive learning manner. However, for each scenario, it is required to first collect enough data and then train such a domain adaptive re-id model, thus restricting their practical application. Instead, we aim to explore multiple labeled datasets to learn generalized domain-invariant representations for person re-id, which is expected universally effective for each new-coming re-id scenario. To pursue practicability in real-world systems, we collect all the person re-id datasets (20 datasets) in this field and select the three most frequently used datasets (i.e., Market1501, DukeMTMC, and MSMT17) as unseen target domains. In addition, we develop DataHunter that collects over 300K+ weak annotated images named YouTube-Human from YouTube street-view videos, which joins 17 remaining full labeled datasets to form multiple source domains. On such a large and challenging benchmark called FastHuman (~440K+ labeled images), we further propose a simple yet effective Semi-Supervised Knowledge Distillation (SSKD) framework. SSKD effectively exploits the weakly annotated data by assigning soft pseudo labels to YouTube-Human to improve models' generalization ability. Experiments on several protocols verify the effectiveness of the proposed SSKD framework on domain generalizable person re-id, which is even comparable to supervised learning on the target domains. Lastly, but most importantly, we hope the proposed benchmark FastHuman could bring the next development of domain generalizable person re-id algorithms.

cs.CV↗

arXiv2021

Learning Instance-level Spatial-Temporal Patterns for Person Re-identification

Person re-identification (Re-ID) aims to match pedestrians under dis-joint cameras. Most Re-ID methods formulate it as visual representation learning and image search, and its accuracy is consequently affected greatly by the search space. Spatial-temporal information has been proven to be efficient to filter irrelevant negative samples and significantly improve Re-ID accuracy. However, existing spatial-temporal person Re-ID methods are still rough and do not exploit spatial-temporal information sufficiently. In this paper, we propose a novel Instance-level and Spatial-Temporal Disentangled Re-ID method (InSTD), to improve Re-ID accuracy. In our proposed framework, personalized information such as moving direction is explicitly considered to further narrow down the search space. Besides, the spatial-temporal transferring probability is disentangled from joint distribution to marginal distribution, so that outliers can also be well modeled. Abundant experimental analyses are presented, which demonstrates the superiority and provides more insights into our method. The proposed method achieves mAP of 90.8% on Market-1501 and 89.1% on DukeMTMC-reID, improving from the baseline 82.2% and 72.7%, respectively. Besides, in order to provide a better benchmark for person re-identification, we release a cleaned data list of DukeMTMC-reID with this paper: https://github.com/RenMin1991/cleaned-DukeMTMC-reID/

cs.CV↗

arXiv2021

Oxygen vacancy induced site-selective mott transition in lanio3

While defects such as oxygen vacancies in correlated materials can modify their electronic properties dramatically, understanding the microscopic origin of electronic correlations in materials with defects has been elusive. Lanthanum nickelate with oxygen vacancies, LaNiO$_{3-x}$, exhibits the metal-to-insulator transition as the oxygen vacancy level $x$ increases from the stoichiometric LaNiO$_3$. In particular, LaNiO$_{2.5}$ exhibits a paramagnetic insulating phase, also stabilizing an antiferromagnetic state below $T_N\simeq152$K. Here, we study the electronic structure and energetics of LaNiO$_{3-x}$ using first-principles. We find that LaNiO$_{2.5}$ stabilizes a vacancy-ordered structure with an insulating ground state and the nature of the insulating phase is a "site-selective" paramagnetic Mott state as obtained using density functional theory plus dynamical mean field theory (DFT+DMFT). The Ni octahedron site develops a Mott insulating state with strong correlations as the Ni $e_g$ orbital is half-filled while the Ni square-planar site with apical oxygen vacancies becomes a band insulator. Our oxygen vacancy results can not be explained by the pure change of the Ni oxidation state alone within the rigid band shift approximation. Our DFT+DMFT density of states explains that the peak splitting of unoccupied states in LaNiO$_{3-x}$ measured by the experimental X-ray absorption spectra originates from two nonequivalent Ni ions in the vacancy-ordered structure.

cond-mat.str-el↗

arXiv2020

Black Re-ID: A Head-shoulder Descriptor for the Challenging Problem of Person Re-Identification

Person re-identification (Re-ID) aims at retrieving an input person image from a set of images captured by multiple cameras. Although recent Re-ID methods have made great success, most of them extract features in terms of the attributes of clothing (e.g., color, texture). However, it is common for people to wear black clothes or be captured by surveillance systems in low light illumination, in which cases the attributes of the clothing are severely missing. We call this problem the Black Re-ID problem. To solve this problem, rather than relying on the clothing information, we propose to exploit head-shoulder features to assist person Re-ID. The head-shoulder adaptive attention network (HAA) is proposed to learn the head-shoulder feature and an innovative ensemble method is designed to enhance the generalization of our model. Given the input person image, the ensemble method would focus on the head-shoulder feature by assigning a larger weight if the individual insides the image is in black clothing. Due to the lack of a suitable benchmark dataset for studying the Black Re-ID problem, we also contribute the first Black-reID dataset, which contains 1274 identities in training set. Extensive evaluations on the Black-reID, Market1501 and DukeMTMC-reID datasets show that our model achieves the best result compared with the state-of-the-art Re-ID methods on both Black and conventional Re-ID problems. Furthermore, our method is also proved to be effective in dealing with person Re-ID in similar clothing. Our code and dataset are avaliable on https://github.com/xbq1994/.

cs.CV↗

arXiv2020

FastReID: A Pytorch Toolbox for General Instance Re-identification

General Instance Re-identification is a very important task in the computer vision, which can be widely used in many practical applications, such as person/vehicle re-identification, face recognition, wildlife protection, commodity tracing, and snapshop, etc.. To meet the increasing application demand for general instance re-identification, we present FastReID as a widely used software system in JD AI Research. In FastReID, highly modular and extensible design makes it easy for the researcher to achieve new research ideas. Friendly manageable system configuration and engineering deployment functions allow practitioners to quickly deploy models into productions. We have implemented some state-of-the-art projects, including person re-id, partial re-id, cross-domain re-id and vehicle re-id, and plan to release these pre-trained models on multiple benchmark datasets. FastReID is by far the most general and high-performance toolbox that supports single and multiple GPU servers, you can reproduce our project results very easily and are very welcome to use it, the code and models are available at https://github.com/JDAI-CV/fast-reid.

cs.CV↗

arXiv2020

DMFTwDFT: An open-source code combining Dynamical Mean Field Theory with various Density Functional Theory packages

Dynamical Mean Field Theory (DMFT) is a successful method to compute the electronic structure of strongly correlated materials, especially when it is combined with density functional theory (DFT). Here, we present an open-source computational package (and a library) combining DMFT with various DFT codes interfaced through the Wannier90 package. The correlated subspace is expanded as a linear combination of Wannier functions introduced in the DMFT approach as local orbitals. In particular, we provide a library mode for computing the DMFT density matrix. This library can be linked and then internally called from any DFT package, assuming that a set of localized orbitals can be generated in the correlated subspace. The existence of this library allows developers of other DFT codes to interface with our package and achieve the charge-self-consistency within DFT+DMFT loops. To test and check our implementation, we computed the density of states and the band structure of well-known correlated materials, namely LaNiO3, SrVO3, and NiO. The obtained results are compared to those obtained from other DFT+DMFT implementations.

cond-mat.str-el↗

arXiv2020

A Strong Baseline and Batch Normalization Neck for Deep Person Re-identification

This study explores a simple but strong baseline for person re-identification (ReID). Person ReID with deep neural networks has progressed and achieved high performance in recent years. However, many state-of-the-art methods design complex network structures and concatenate multi-branch features. In the literature, some effective training tricks briefly appear in several papers or source codes. The present study collects and evaluates these effective training tricks in person ReID. By combining these tricks, the model achieves 94.5% rank-1 and 85.9% mean average precision on Market1501 with only using the global features of ResNet50. The performance surpasses all existing global- and part-based baselines in person ReID. We propose a novel neck structure named as batch normalization neck (BNNeck). BNNeck adds a batch normalization layer after global pooling layer to separate metric and classification losses into two different feature spaces because we observe they are inconsistent in one embedding space. Extended experiments show that BNNeck can boost the baseline, and our baseline can improve the performance of existing state-of-the-art methods. Our codes and models are available at: https://github.com/michuanhaohao/reid-strong-baseline.

cs.CV↗

arXiv2019

Video-based Person Re-identification via 3D Convolutional Networks and Non-local Attention

Video-based person re-identification (ReID) is a challenging problem, where some video tracks of people across non-overlapping cameras are available for matching. Feature aggregation from a video track is a key step for video-based person ReID. Many existing methods tackle this problem by average/maximum temporal pooling or RNNs with attention. However, these methods cannot deal with temporal dependency and spatial misalignment problems at the same time. We are inspired by video action recognition that involves the identification of different actions from video tracks. Firstly, we use 3D convolutions on video volume, instead of using 2D convolutions across frames, to extract spatial and temporal features simultaneously. Secondly, we use a non-local block to tackle the misalignment problem and capture spatial-temporal long-range dependencies. As a result, the network can learn useful spatial-temporal information as a weighted sum of the features in all space and temporal positions in the input feature map. Experimental results on three datasets show that our framework outperforms state-of-the-art approaches by a large margin on multiple metrics.

cs.CV↗

arXiv2019

Bag of Tricks and A Strong Baseline for Deep Person Re-identification

This paper explores a simple and efficient baseline for person re-identification (ReID). Person re-identification (ReID) with deep neural networks has made progress and achieved high performance in recent years. However, many state-of-the-arts methods design complex network structure and concatenate multi-branch features. In the literature, some effective training tricks are briefly appeared in several papers or source codes. This paper will collect and evaluate these effective training tricks in person ReID. By combining these tricks together, the model achieves 94.5% rank-1 and 85.9% mAP on Market1501 with only using global features. Our codes and models are available at https://github.com/michuanhaohao/reid-strong-baseline.

cs.CV↗

arXiv2019

Foreground-aware Pyramid Reconstruction for Alignment-free Occluded Person Re-identification

Re-identifying a person across multiple disjoint camera views is important for intelligent video surveillance, smart retailing and many other applications. However, existing person re-identification (ReID) methods are challenged by the ubiquitous occlusion over persons and suffer from performance degradation. This paper proposes a novel occlusion-robust and alignment-free model for occluded person ReID and extends its application to realistic and crowded scenarios. The proposed model first leverages the full convolution network (FCN) and pyramid pooling to extract spatial pyramid features. Then an alignment-free matching approach, namely Foreground-aware Pyramid Reconstruction (FPR), is developed to accurately compute matching scores between occluded persons, despite their different scales and sizes. FPR uses the error from robust reconstruction over spatial pyramid features to measure similarities between two persons. More importantly, we design an occlusion-sensitive foreground probability generator that focuses more on clean human body parts to refine the similarity computation with less contamination from occlusion. The FPR is easily embedded into any end-to-end person ReID models. The effectiveness of the proposed method is clearly demonstrated by the experimental results (Rank-1 accuracy) on three occluded person datasets: Partial REID (78.30\%), Partial iLIDS (68.08\%) and Occluded REID (81.00\%); and three benchmark person datasets: Market1501 (95.42\%), DukeMTMC (88.64\%) and CUHK03 (76.08\%)

cs.CV↗