arXiv subjects

Wei Zhao

Publications and source records attributed to Wei Zhao.

At least 271 records · Page 15Linked to original sources

arXiv2023

Voxelized 3D Feature Aggregation for Multiview Detection

Multi-view detection incorporates multiple camera views to alleviate occlusion in crowded scenes, where the state-of-the-art approaches adopt homography transformations to project multi-view features to the ground plane. However, we find that these 2D transformations do not take into account the object's height, and with this neglection features along the vertical direction of same object are likely not projected onto the same ground plane point, leading to impure ground-plane features. To solve this problem, we propose VFA, voxelized 3D feature aggregation, for feature transformation and aggregation in multi-view detection. Specifically, we voxelize the 3D space, project the voxels onto each camera view, and associate 2D features with these projected voxels. This allows us to identify and then aggregate 2D features along the same vertical line, alleviating projection distortions to a large extent. Additionally, because different kinds of objects (human vs. cattle) have different shapes on the ground plane, we introduce the oriented Gaussian encoding to match such shapes, leading to increased accuracy and efficiency. We perform experiments on multiview 2D detection and multiview 3D detection problems. Results on four datasets (including a newly introduced MultiviewC dataset) show that our system is very competitive compared with the state-of-the-art approaches. %Our code and data will be open-sourced.Code and MultiviewC are released at https://github.com/Robert-Mar/VFA.

cs.CV↗

arXiv2022

Less is More: Surgical Phase Recognition from Timestamp Supervision

Surgical phase recognition is a fundamental task in computer-assisted surgery systems. Most existing works are under the supervision of expensive and time-consuming full annotations, which require the surgeons to repeat watching videos to find the precise start and end time for a surgical phase. In this paper, we introduce timestamp supervision for surgical phase recognition to train the models with timestamp annotations, where the surgeons are asked to identify only a single timestamp within the temporal boundary of a phase. This annotation can significantly reduce the manual annotation cost compared to the full annotations. To make full use of such timestamp supervisions, we propose a novel method called uncertainty-aware temporal diffusion (UATD) to generate trustworthy pseudo labels for training. Our proposed UATD is motivated by the property of surgical videos, i.e., the phases are long events consisting of consecutive frames. To be specific, UATD diffuses the single labelled timestamp to its corresponding high confident ( i.e., low uncertainty) neighbour frames in an iterative way. Our study uncovers unique insights of surgical phase recognition with timestamp supervisions: 1) timestamp annotation can reduce 74% annotation time compared with the full annotation, and surgeons tend to annotate those timestamps near the middle of phases; 2) extensive experiments demonstrate that our method can achieve competitive results compared with full supervision methods, while reducing manual annotation cost; 3) less is more in surgical phase recognition, i.e., less but discriminative pseudo labels outperform full but containing ambiguous frames; 4) the proposed UATD can be used as a plug and play method to clean ambiguous labels near boundaries between phases, and improve the performance of the current surgical phase recognition methods.

cs.CV↗

arXiv2022

Joint nnU-Net and Radiomics Approaches for Segmentation and Prognosis of Head and Neck Cancers with PET/CT images

Automatic segmentation of head and neck cancer (HNC) tumors and lymph nodes plays a crucial role in the optimization treatment strategy and prognosis analysis. This study aims to employ nnU-Net for automatic segmentation and radiomics for recurrence-free survival (RFS) prediction using pretreatment PET/CT images in multi-center HNC cohort. A multi-center HNC dataset with 883 patients (524 patients for training, 359 for testing) was provided in HECKTOR 2022. A bounding box of the extended oropharyngeal region was retrieved for each patient with fixed size of 224 x 224 x 224 $mm^{3}$. Then 3D nnU-Net architecture was adopted to automatic segmentation of primary tumor and lymph nodes synchronously.Based on predicted segmentation, ten conventional features and 346 standardized radiomics features were extracted for each patient. Three prognostic models were constructed containing conventional and radiomics features alone, and their combinations by multivariate CoxPH modelling. The statistical harmonization method, ComBat, was explored towards reducing multicenter variations. Dice score and C-index were used as evaluation metrics for segmentation and prognosis task, respectively. For segmentation task, we achieved mean dice score around 0.701 for primary tumor and lymph nodes by 3D nnU-Net. For prognostic task, conventional and radiomics models obtained the C-index of 0.658 and 0.645 in the test set, respectively, while the combined model did not improve the prognostic performance with the C-index of 0.648.

eess.IV↗

arXiv2022

A SVD-based Dynamic Harmonic Phasor Estimator with Improved Suppression of Out-of-Band Interference

The diffusion of nonlinear loads and power electronic devices in power systems deteriorates the signal environment and increases the difficulty of measuring harmonic phasors. Considering accurate harmonic phasor information is necessary to deal with harmonic-related issues, this paper focuses on realizing accurate dynamic harmonic phasor estimation when the signal is contaminated by certain interharmonic tones, i.e. the out-of-band interference (OBI). Specifically, this work introduces the singular value decomposition into the Taylor-Fourier transform algorithm, deriving a general decomposition form for harmonic phasor filters, and yielding a set of adjustable parameters. Then these parameters are configured to minimize the negative influence of OBI on the dynamic harmonic phasor filters. Based on the recommended parameter values, the optimized harmonic estimator is obtained, and then tested under signals distorted by OBI, harmonic, noise, and in worse cases, frequency deviation, and some dynamic conditions. Test results show the proposal achieves good harmonic phasor estimation even under multiple interferences and dynamic scenarios, and has a much weaker dependence on OBI tones compared to conventional approaches.

eess.SP↗

arXiv2022

STCGAT: A Spatio-temporal Causal Graph Attention Network for traffic flow prediction in Intelligent Transportation Systems

Air pollution and carbon emissions caused by modern transportation are closely related to global climate change. With the help of next-generation information technology such as Internet of Things (IoT) and Artificial Intelligence (AI), accurate traffic flow prediction can effectively solve problems such as traffic congestion and mitigate environmental pollution and climate change. It further promotes the development of Intelligent Transportation Systems (ITS) and smart cities. However, the strong spatial and temporal correlation of traffic data makes the task of accurate traffic forecasting a significant challenge. Existing methods are usually based on graph neural networks using predefined spatial adjacency graphs of traffic networks to model spatial dependencies, ignoring the dynamic correlation of relationships between road nodes. In addition, they usually use independent Spatio-temporal components to capture Spatio-temporal dependencies and do not effectively model global Spatio-temporal dependencies. This paper proposes a new Spatio-temporal Causal Graph Attention Network (STCGAT) for traffic prediction to address the above challenges. In STCGAT, we use a node embedding approach that can adaptively generate spatial adjacency subgraphs at each time step without a priori geographic knowledge and fine-grained modeling of the topology of dynamically generated graphs for different time steps. Meanwhile, we propose an efficient causal temporal correlation component that contains node adaptive learning, graph convolution, and local and global causal temporal convolution modules to learn local and global Spatio-temporal dependencies jointly. Extensive experiments on four real, large traffic datasets show that our model consistently outperforms all baseline models.

cs.LG↗

arXiv2022

Multi-Spatio-temporal Fusion Graph Recurrent Network for Traffic forecasting

Traffic forecasting is essential for the traffic construction of smart cities in the new era. However, traffic data's complex spatial and temporal dependencies make traffic forecasting extremely challenging. Most existing traffic forecasting methods rely on the predefined adjacency matrix to model the Spatio-temporal dependencies. Nevertheless, the road traffic state is highly real-time, so the adjacency matrix should change dynamically with time. This article presents a new Multi-Spatio-temporal Fusion Graph Recurrent Network (MSTFGRN) to address the issues above. The network proposes a data-driven weighted adjacency matrix generation method to compensate for real-time spatial dependencies not reflected by the predefined adjacency matrix. It also efficiently learns hidden Spatio-temporal dependencies by performing a new two-way Spatio-temporal fusion operation on parallel Spatio-temporal relations at different moments. Finally, global Spatio-temporal dependencies are captured simultaneously by integrating a global attention mechanism into the Spatio-temporal fusion module. Extensive trials on four large-scale, real-world traffic datasets demonstrate that our method achieves state-of-the-art performance compared to alternative baselines.

cs.LG↗

arXiv2022

Constrained Density Matching and Modeling for Cross-lingual Alignment of Contextualized Representations

Multilingual representations pre-trained with monolingual data exhibit considerably unequal task performances across languages. Previous studies address this challenge with resource-intensive contextualized alignment, which assumes the availability of large parallel data, thereby leaving under-represented language communities behind. In this work, we attribute the data hungriness of previous alignment techniques to two limitations: (i) the inability to sufficiently leverage data and (ii) these techniques are not trained properly. To address these issues, we introduce supervised and unsupervised density-based approaches named Real-NVP and GAN-Real-NVP, driven by Normalizing Flow, to perform alignment, both dissecting the alignment of multilingual subspaces into density matching and density modeling. We complement these approaches with our validation criteria in order to guide the training process. Our experiments encompass 16 alignments, including our approaches, evaluated across 6 language pairs, synthetic data and 5 NLP tasks. We demonstrate the effectiveness of our approaches in the scenarios of limited and no parallel data. First, our supervised approach trained on 20k parallel data (sentences) mostly surpasses Joint-Align and InfoXLM trained on over 100k parallel sentences. Second, parallel data can be removed without sacrificing performance when integrating our unsupervised approach in our bootstrapping procedure, which is theoretically motivated to enforce equality of multilingual subspaces. Moreover, we demonstrate the advantages of validation criteria over validation data for guiding supervised training.

cs.CL↗

arXiv2022

Entropy Induced Pruning Framework for Convolutional Neural Networks

Structured pruning techniques have achieved great compression performance on convolutional neural networks for image classification task. However, the majority of existing methods are weight-oriented, and their pruning results may be unsatisfactory when the original model is trained poorly. That is, a fully-trained model is required to provide useful weight information. This may be time-consuming, and the pruning results are sensitive to the updating process of model parameters. In this paper, we propose a metric named Average Filter Information Entropy (AFIE) to measure the importance of each filter. It is calculated by three major steps, i.e., low-rank decomposition of the "input-output" matrix of each convolutional layer, normalization of the obtained eigenvalues, and calculation of filter importance based on information entropy. By leveraging the proposed AFIE, the proposed framework is able to yield a stable importance evaluation of each filter no matter whether the original model is trained fully. We implement our AFIE based on AlexNet, VGG-16, and ResNet-50, and test them on MNIST, CIFAR-10, and ImageNet, respectively. The experimental results are encouraging. We surprisingly observe that for our methods, even when the original model is only trained with one epoch, the importance evaluation of each filter keeps identical to the results when the model is fully-trained. This indicates that the proposed pruning strategy can perform effectively at the beginning stage of the training process for the original model.

cs.CV↗

arXiv2022

SBPF: Sensitiveness Based Pruning Framework For Convolutional Neural Network On Image Classification

Pruning techniques are used comprehensively to compress convolutional neural networks (CNNs) on image classification. However, the majority of pruning methods require a well pre-trained model to provide useful supporting parameters, such as C1-norm, BatchNorm value and gradient information, which may lead to inconsistency of filter evaluation if the parameters of the pre-trained model are not well optimized. Therefore, we propose a sensitiveness based method to evaluate the importance of each layer from the perspective of inference accuracy by adding extra damage for the original model. Because the performance of the accuracy is determined by the distribution of parameters across all layers rather than individual parameter, the sensitiveness based method will be robust to update of parameters. Namely, we can obtain similar importance evaluation of each convolutional layer between the imperfect-trained and fully trained models. For VGG-16 on CIFAR-10, even when the original model is only trained with 50 epochs, we can get same evaluation of layer importance as the results when the model is trained fully. Then we will remove filters proportional from each layer by the quantified sensitiveness. Our sensitiveness based pruning framework is verified efficiently on VGG-16, a customized Conv-4 and ResNet-18 with CIFAR-10, MNIST and CIFAR-100, respectively.

cs.CV↗

arXiv2022

Learning Quality-aware Dynamic Memory for Video Object Segmentation

Recently, several spatial-temporal memory-based methods have verified that storing intermediate frames and their masks as memory are helpful to segment target objects in videos. However, they mainly focus on better matching between the current frame and the memory frames without explicitly paying attention to the quality of the memory. Therefore, frames with poor segmentation masks are prone to be memorized, which leads to a segmentation mask error accumulation problem and further affect the segmentation performance. In addition, the linear increase of memory frames with the growth of frame number also limits the ability of the models to handle long videos. To this end, we propose a Quality-aware Dynamic Memory Network (QDMN) to evaluate the segmentation quality of each frame, allowing the memory bank to selectively store accurately segmented frames to prevent the error accumulation problem. Then, we combine the segmentation quality with temporal consistency to dynamically update the memory bank to improve the practicability of the models. Without any bells and whistles, our QDMN achieves new state-of-the-art performance on both DAVIS and YouTube-VOS benchmarks. Moreover, extensive experiments demonstrate that the proposed Quality Assessment Module (QAM) can be applied to memory-based methods as generic plugins and significantly improves performance. Our source code is available at https://github.com/workforai/QDMN.

cs.CV↗

arXiv2022

A revisit of generalized scaling of forced turbulence through flux analysis

In this investigation, we theoretically studied the transports of kinetic energy and scalar variance in turbulence driven by a scalar-based volume force in $M{\nabla}^β s'$ form associated with scalar fluctuations $s'$ in wavenumber space relies on flux conservation equation. The equation has one real solution and two complex solutions, which lead to four different cascade processes, including inertial subrange (constant fluxes of kinetic energy and scalar variance), CEF subrange (quasi-constant flux of kinetic energy), CSF subrange (quasi-constant flux of scalar variance), and a new subrange with both non-constant fluxes of kinetic energy and scalar variance in addition to dissipation subrange. $β$ controls the cascade processes and the scaling exponents. For the real solution, in the CEF subrange, $ξ_u$ is always -5/3, while $ξ_s=-(6β+1)/3$. In the CSF subrange, $ξ_u=(4β-11)/5$ and $ξ_s=-(2β+7)/5$ which are both consistent with the theory of Zhao and Wang (2021). Relying on $β$, the transport of kinetic energy and scalar variance can be distinguished as four cases. (1) When $β<3/2$ (except $β=2/3$), the CEF and CSF subranges are coexisted, with the former located on the lower wavenumber side of the latter. At $β=2/3$, a new inertial subrange with both $ξ_u$ and $ξ_s$ equal to -5/3 is present. (2) When $3/2<=β<2$, only the CEF subrange is predicted. (3) At $β=2$, special and singular exponents of $ξ_u=-1$, $ξ_s=-3$, $λ_u=1$, and $λ_s=-1$ can be found, if $MN=1$. Otherwise, a CSF subrange is predicted. (4) When $2<β<=4$, only the CSF subrange is predicted. Thus, a complete transport picture of both kinetic energy and scalar variance has been established for the type of forced turbulence.

physics.flu-dyn↗

arXiv2022

Onset of nonlinear electroosmotic flow under AC electric field

Nonlinearity of electroosmotic flows (EOFs) is ubiquitous and plays a crucial role in the mass and energy transfer in ion transport, specimen mixing, electrochemistry reaction, and electric energy storage and utilizing. When and how the transition from a linear regime to a nonlinear one is essential for understanding, prohibiting or utilizing nonlinear EOF. However, suffers the lacking of reliable experimental instruments with high spatial and temporal resolutions, the investigation of the onset of nonlinear EOF still stays in theory. Herein, we experimentally studied the velocity fluctuations of EOFs driven by AC electric field via ultra-sensitive fluorescent blinking tricks. The linear and nonlinear AC EOFs are successfully identified from both the time trace and energy spectra of velocity fluctuations. The critical electric field ($E_{A,C}$) separating the two statuses is determined and is discovered by defining a generalized scaling law with respect to the convection velocity ($U$) and AC frequency ($f_f$) as $E_{A,C}$~${f_f}^{0.48-0.027U}$. The universal control parameters are determined with surprising accuracy for governing the status of AC EOFs. We hope the current investigation could be essential in the development of both theory and applications of nonlinear EOF.

physics.flu-dyn↗

arXiv2022

Investigating the environmental dependence of ultralight scalar dark matter with atom interferometers

We study the environmental dependence of ultralight scalar dark matter (DM) with linear interactions to the standard model particles. The solution to the DM field turns out to be a sum of the cosmic harmonic oscillation term and the local exponential fluctuation term. The amplitude of the first term depends on the local DM density and the mass of the DM field. The second term is induced by the local distribution of matter, such as the Earth. And it depends not only on the mass of the Earth, but also the density of the Earth. Then, we compute the phase shift induced by the DM field in atom interferometers (AIs), through solving the trajectories of atoms. Especially, the AI signal for the violation of weak equivalence principle (WEP) caused by the DM field is calculated. Depending on the values of the DM coupling parameters, contributions to the WEP violation from the first and second terms of the DM field can be either comparable or one larger than the other. Finally, we give some constraints to DM coupling parameters using results from the terrestrial atomic WEP tests.

physics.atom-ph↗

arXiv2022

Mixing and flow transition in an optimized electrokinetic turbulent micromixer

Micromixer is a key element in lab on a chip for broad applications in the analysis and measurement of chemistry and engineering. Previous investigations reported electrokinetic (EK) turbulence could be realized in a Y-type micromixer with a cross-sectional dimension of 100 $μ$m order. Although the ultrafast turbulent mixing can be generated at a bulk flow Reynolds number of O(1), the micromixer has not been optimized. In this investigation, we systematically investigated the influence of electric field intensity, AC frequency, electric conductivity ratio, and channel width at the entrance on the mixing effect and transition electric Rayleigh number in the "Y" type electrokinetic micromixer. It is found the optimal mixing is realized in a 350 $μ$m wide micromixer, under 100 kHz and 1.14*10^5 V/m AC electric field, with an electric conductivity ratio of 1:3000. Under the conditions, a maximum degree of mixedness of 0.93 can be achieved at 84 $μ$m from the entrance and 100 ms. A further investigation of the critical electric field and the critical electric Rayleigh number indicates the most unstable condition of EK flow instability is inconsistent with that of the optimal mixing in EK turbulence. To predict the evolution of EK flow under high $Ra_{e}$, it is necessary to apply a computational turbulence model, instead of linear instability analysis.

physics.flu-dyn↗

arXiv2022

Towards Explainable Evaluation Metrics for Natural Language Generation

Unlike classical lexical overlap metrics such as BLEU, most current evaluation metrics (such as BERTScore or MoverScore) are based on black-box language models such as BERT or XLM-R. They often achieve strong correlations with human judgments, but recent research indicates that the lower-quality classical metrics remain dominant, one of the potential reasons being that their decision processes are transparent. To foster more widespread acceptance of the novel high-quality metrics, explainability thus becomes crucial. In this concept paper, we identify key properties and propose key goals of explainable machine translation evaluation metrics. We also provide a synthesizing overview over recent approaches for explainable machine translation metrics and discuss how they relate to those goals and properties. Further, we conduct own novel experiments, which (among others) find that current adversarial NLP techniques are unsuitable for automatically identifying limitations of high-quality black-box evaluation metrics, as they are not meaning-preserving. Finally, we provide a vision of future approaches to explainable evaluation metrics and their evaluation. We hope that our work can help catalyze and guide future research on explainable evaluation metrics and, mediately, also contribute to better and more transparent text generation systems.

cs.CL↗

arXiv2022

SimKGC: Simple Contrastive Knowledge Graph Completion with Pre-trained Language Models

Knowledge graph completion (KGC) aims to reason over known facts and infer the missing links. Text-based methods such as KGBERT (Yao et al., 2019) learn entity representations from natural language descriptions, and have the potential for inductive KGC. However, the performance of text-based methods still largely lag behind graph embedding-based methods like TransE (Bordes et al., 2013) and RotatE (Sun et al., 2019b). In this paper, we identify that the key issue is efficient contrastive learning. To improve the learning efficiency, we introduce three types of negatives: in-batch negatives, pre-batch negatives, and self-negatives which act as a simple form of hard negatives. Combined with InfoNCE loss, our proposed model SimKGC can substantially outperform embedding-based methods on several benchmark datasets. In terms of mean reciprocal rank (MRR), we advance the state-of-the-art by +19% on WN18RR, +6.8% on the Wikidata5M transductive setting, and +22% on the Wikidata5M inductive setting. Thorough analyses are conducted to gain insights into each component. Our code is available at https://github.com/intfloat/SimKGC .

cs.CL↗

arXiv2022

Ultralight scalar dark matter detection with ZAIGA

ZAIGA is a proposed underground long-baseline atom interferometer (AI) facility, aiming for experimental research on gravitation and related problems. In this paper, we study the possibility of detecting the ultralight scalar dark matter (DM) with ZAIGA. According to a popular scalar DM model, the DM field contains a background oscillation term and a local exponential fluctuation term. In order to calculate the proposed constraints on DM coupling parameters, we need to first compute the DM signals in ZAIGA. For the case of two AIs vertically separated by 300 meters, the DM-induced differential phase consists of three contributions, coming from the DM-induced changes in atomic internal energy levels, atomic masses and the gravitational acceleration. For the case of two AIs horizontally separated by several kilometers, the signal comes from the DM-induced changes in atomic internal energy levels. With the current and future technical parameters of ZAIGA, we then obtain the proposed constraints on five DM coupling parameters. It turns out that our proposed constraints could be several orders of magnitude better than the ones set by the MICROSCOPE space mission.

physics.atom-ph↗

arXiv2022

What Do They Capture? -- A Structural Analysis of Pre-Trained Language Models for Source Code

Recently, many pre-trained language models for source code have been proposed to model the context of code and serve as a basis for downstream code intelligence tasks such as code completion, code search, and code summarization. These models leverage masked pre-training and Transformer and have achieved promising results. However, currently there is still little progress regarding interpretability of existing pre-trained code models. It is not clear why these models work and what feature correlations they can capture. In this paper, we conduct a thorough structural analysis aiming to provide an interpretation of pre-trained language models for source code (e.g., CodeBERT, and GraphCodeBERT) from three distinctive perspectives: (1) attention analysis, (2) probing on the word embedding, and (3) syntax tree induction. Through comprehensive analysis, this paper reveals several insightful findings that may inspire future studies: (1) Attention aligns strongly with the syntax structure of code. (2) Pre-training language models of code can preserve the syntax structure of code in the intermediate representations of each Transformer layer. (3) The pre-trained models of code have the ability of inducing syntax trees of code. Theses findings suggest that it may be helpful to incorporate the syntax structure of code into the process of pre-training for better code representations.

cs.SE↗