arXiv subjects

Yue Cao

Publications and source records attributed to Yue Cao.

At least 145 records · Page 8Linked to original sources

arXiv2021

The DR21(OH) Trident -- Resolving the Massive Ridge into Three Entangled Fibers As the Initial Condition of Cluster Formation

DR21(OH) ridge, the central part of a high-mass star and cluster forming hub-filament system, is resolved spatially and kinematically into three nearly parallel fibers (f1, f2, and f3) with a roughly north-south orientation, using the observations of molecular transitions of H$^{13}$CO$^+$ (1-0), N$_2$H$^+$ (1-0), and NH$_2$D (1$_{1,1}$-1$_{0,1}$) with the Combined Array for Research in Millimeter Astronomy. These fibers are all mildly supersonic ($σ_{\rm V}$ about 2 times the sound speed), having lengths around 2 pc and widths about 0.1 pc, and they entangle and conjoin in the south where the most active high-mass star formation takes place. They all have line masses 1 - 2 orders of magnitude higher than their low-mass counterparts and are gravitationally unstable both radially and axially. However, only f1 exhibits high-mass star formation all the way along the fiber, yet f2 and f3 show no signs of significant star formation in their northern parts. A large velocity gradient increasing from north to south is seen in f3, and can be well reproduced with a model of free-fall motion toward the most massive and active dense core in the region, which corroborates the global collapse of the ridge and suggests that the disruptive effects of the tidal forces may explain the inefficiency of star formation in f2 and f3. On larger scales, some of the lower-density, peripheral filaments are likely to be the outer extensions of the fibers, and provide hints on the origin of the ridge.

astro-ph.GA↗

arXiv2021

Surveys of Clumps, Cores, and Condensations in the Cygnus X: II. Radio Properties of the Massive Dense Cores

We have carried out a high-sensitivity and high-resolution radio continuum study towards a sample of 47 massive dense cores (MDCs) in the Cygnus X star-forming complex using the Karl G. Jansky Very Large Array, aiming to detect and characterize the radio emission associated with star-forming activities down to ~0.01 pc scales. We have detected 64 radio sources within or closely around the full width at half-maximum (FWHM) of the MDCs, of which 37 are reported for the first time. The majority of the detected radio sources are associated with dust condensations embedded within the MDCs, and they are mostly weak and compact. We are able to build spectral energy distributions for 8 sources. Two of them indicate non-thermal emission and the other six indicate thermal free-free emission. We have determined that most of the radio sources are ionized jets or winds originating from massive young stellar objects, whereas only a few sources are likely to be ultra-compact HII regions. Further quantitative analyses indicate that the radio luminosity of the detected radio sources increases along the evolution path of the MDCs.

astro-ph.GA↗

arXiv2021

ALMA observations of NGC 6334S. II. Subsonic and Transonic Narrow Filaments in a High-mass Star Formation Cloud

We present a study of narrow filaments toward a massive infrared dark cloud, NGC 6334S, using the Atacama Large Millimeter/submillimeter Array (ALMA). Thirteen gas filaments are identified using the H$^{13}$CO$^{+}$ line, while a single continuum filament is revealed by the continuum emission. The filaments present a compact radial distribution with a median filament width of $\sim$0.04 pc narrower than the previously proposed `quasi-universal' 0.1~pc filament width. The higher spatial resolution observations and higher-density gas tracer tend to identify even narrower and lower mass filaments. The filament widths are roughly twice the size of embedded cores. The gas filaments are largely supported by thermal motions. The nonthermal motions are predominantly subsonic and transonic in both identified gas filaments and embedded cores, which may imply that stars are likely born in environments of low turbulence. A fraction of embedded objects show a narrower velocity dispersion compared with their corresponding natal filaments, which may indicate that the turbulent dissipation is taking place in these embedded cores. The physical properties (mass, mass per unit length, gas kinematics, and width) of gas filaments are analogous to those of narrow filaments found in low- to high-mass star-forming regions. The more evolved sources are found to be farther away from the filaments, a situation that may have resulted from the relative motions between the YSOs and their natal filaments.

astro-ph.GA↗

arXiv2021

Bootstrap Your Object Detector via Mixed Training

We introduce MixTraining, a new training paradigm for object detection that can improve the performance of existing detectors for free. MixTraining enhances data augmentation by utilizing augmentations of different strengths while excluding the strong augmentations of certain training samples that may be detrimental to training. In addition, it addresses localization noise and missing labels in human annotations by incorporating pseudo boxes that can compensate for these errors. Both of these MixTraining capabilities are made possible through bootstrapping on the detector, which can be used to predict the difficulty of training on a strong augmentation, as well as to generate reliable pseudo boxes thanks to the robustness of neural networks to labeling error. MixTraining is found to bring consistent improvements across various detectors on the COCO dataset. In particular, the performance of Faster R-CNN \cite{ren2015faster} with a ResNet-50 \cite{he2016deep} backbone is improved from 41.7 mAP to 44.0 mAP, and the accuracy of Cascade-RCNN \cite{cai2018cascade} with a Swin-Small \cite{liu2021swin} backbone is raised from 50.9 mAP to 52.8 mAP. The code and models will be made publicly available at \url{https://github.com/MendelXu/MixTraining}.

cs.CV↗

arXiv2021

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

This paper presents a new vision Transformer, called Swin Transformer, that capably serves as a general-purpose backbone for computer vision. Challenges in adapting Transformer from language to vision arise from differences between the two domains, such as large variations in the scale of visual entities and the high resolution of pixels in images compared to words in text. To address these differences, we propose a hierarchical Transformer whose representation is computed with \textbf{S}hifted \textbf{win}dows. The shifted windowing scheme brings greater efficiency by limiting self-attention computation to non-overlapping local windows while also allowing for cross-window connection. This hierarchical architecture has the flexibility to model at various scales and has linear computational complexity with respect to image size. These qualities of Swin Transformer make it compatible with a broad range of vision tasks, including image classification (87.3 top-1 accuracy on ImageNet-1K) and dense prediction tasks such as object detection (58.7 box AP and 51.1 mask AP on COCO test-dev) and semantic segmentation (53.5 mIoU on ADE20K val). Its performance surpasses the previous state-of-the-art by a large margin of +2.7 box AP and +2.6 mask AP on COCO, and +3.2 mIoU on ADE20K, demonstrating the potential of Transformer-based models as vision backbones. The hierarchical design and the shifted window approach also prove beneficial for all-MLP architectures. The code and models are publicly available at~\url{https://github.com/microsoft/Swin-Transformer}.

cs.CV↗

arXiv2021

The discovery of the largest gas filament in our Galaxy, or a new spiral arm?

Using the Five-hundred-meter Aperture Spherical radio Telescope (FAST), we detect a giant HI filamentary structure in the sky region of 307$.\!\!^{\circ}$7 $<$ $α$ $<$ 311$.\!\!^{\circ}$0 and 40$.\!\!^{\circ}$9 $<$ $δ$ $<$ 43$.\!\!^{\circ}$4. The structure has a velocity range of $-$170 km s$^{-1}$ to $-$130 km s$^{-1}$, and a mean velocity of $-$150 km s$^{-1}$, putting it to a Galactocentric distance of 22 kpc. The HI structure has a length of 1.1 kpc, which appears to be so far the furthest and largest giant filament in the Galaxy and we name it Cattail. Its mass is calculated to be 6.5 $\times$ 10$^4$ M$_{\odot}$ and the linear mass density is 60 M$_{\odot}$ pc$^{-1}$. Its width is 207 pc, corresponding to an aspect ratio of 5:1. Cattail possesses a small velocity gradient (0.02 km s$^{-1}$ pc$^{-1}$) along its major axis. Together with the HI4PI data, we find that Cattail could have an even larger length, up to 5 kpc. We also identify another new elongated structure to be the extension into the Galactic first quadrant of the Outer Scutum-Centaurus (OSC) arm, and Cattail appears to be located far behind the OSC. The question about how such a huge filament is produced at the extreme Galactic location remains open. Alternatively, Cattail might be part of a new arm beyond the OSC, though it is puzzling that the structure does not fully follow the warp of the Galactic disk.

astro-ph.GA↗

arXiv2021

Core mass function of a single giant molecular cloud complex with ~10^4 cores

Similarity in shape between the initial mass function (IMF) and the core mass functions (CMFs) in star-forming regions prompts the idea that the IMF originates from the CMF through a self-similar core-to-star mass mapping process. To accurately determine the shape of the CMF, we create a sample of 8,431 cores with the dust continuum maps of the Cygnus X giant molecular cloud complex, and design a procedure for deriving the CMF considering the mass uncertainty, binning uncertainty, sample incompleteness, and the statistical errors. The resultant CMF coincides well with the IMF for core masses from a few $M_{\odot}$ to the highest masses of 1300 $M_{\odot}$ with a power-law of ${\rm d}N/{\rm d}M\propto M^{-2.30\pm0.04}$, but does not present an obvious flattened turnover in the low-mass range as the IMF does. More detailed inspection reveals that the slope of the CMF steepens with increasing mass. Given the numerous high-mass star-forming activities of Cygnus X, this is in stark contrast with the existing top-heavy CMFs found in high-mass star-forming clumps. We also find that the similarity between the IMF and the mass function of cloud structures is not unique at core scales, but can be seen for cloud structures of up to several pc scales. Finally, our SMA observations toward a subset of the cores do not present evidence for the self-similar mapping. The latter two results indicate that the shape of the IMF may not be directly inherited from the CMF.

astro-ph.GA↗

arXiv2021

Fold2Seq: A Joint Sequence(1D)-Fold(3D) Embedding-based Generative Model for Protein Design

Designing novel protein sequences for a desired 3D topological fold is a fundamental yet non-trivial task in protein engineering. Challenges exist due to the complex sequence--fold relationship, as well as the difficulties to capture the diversity of the sequences (therefore structures and functions) within a fold. To overcome these challenges, we propose Fold2Seq, a novel transformer-based generative framework for designing protein sequences conditioned on a specific target fold. To model the complex sequence--structure relationship, Fold2Seq jointly learns a sequence embedding using a transformer and a fold embedding from the density of secondary structural elements in 3D voxels. On test sets with single, high-resolution and complete structure inputs for individual folds, our experiments demonstrate improved or comparable performance of Fold2Seq in terms of speed, coverage, and reliability for sequence design, when compared to existing state-of-the-art methods that include data-driven deep generative models and physics-based RosettaDesign. The unique advantages of fold-based Fold2Seq, in comparison to a structure-based deep model and RosettaDesign, become more evident on three additional real-world challenges originating from low-quality, incomplete, or ambiguous input structures. Source code and data are available at https://github.com/IBM/fold2seq.

cs.LG↗

arXiv2021

Video Swin Transformer

The vision community is witnessing a modeling shift from CNNs to Transformers, where pure Transformer architectures have attained top accuracy on the major video recognition benchmarks. These video models are all built on Transformer layers that globally connect patches across the spatial and temporal dimensions. In this paper, we instead advocate an inductive bias of locality in video Transformers, which leads to a better speed-accuracy trade-off compared to previous approaches which compute self-attention globally even with spatial-temporal factorization. The locality of the proposed video architecture is realized by adapting the Swin Transformer designed for the image domain, while continuing to leverage the power of pre-trained image models. Our approach achieves state-of-the-art accuracy on a broad range of video recognition benchmarks, including on action recognition (84.9 top-1 accuracy on Kinetics-400 and 86.1 top-1 accuracy on Kinetics-600 with ~20x less pre-training data and ~3x smaller model size) and temporal modeling (69.6 top-1 accuracy on Something-Something v2). The code and models will be made publicly available at https://github.com/SwinTransformer/Video-Swin-Transformer.

cs.CV↗

arXiv2021

Self-Supervised Learning with Swin Transformers

We are witnessing a modeling shift from CNN to Transformers in computer vision. In this work, we present a self-supervised learning approach called MoBY, with Vision Transformers as its backbone architecture. The approach basically has no new inventions, which is combined from MoCo v2 and BYOL and tuned to achieve reasonably high accuracy on ImageNet-1K linear evaluation: 72.8% and 75.0% top-1 accuracy using DeiT-S and Swin-T, respectively, by 300-epoch training. The performance is slightly better than recent works of MoCo v3 and DINO which adopt DeiT as the backbone, but with much lighter tricks. More importantly, the general-purpose Swin Transformer backbone enables us to also evaluate the learnt representations on downstream tasks such as object detection and semantic segmentation, in contrast to a few recent approaches built on ViT/DeiT which only report linear evaluation results on ImageNet-1K due to ViT/DeiT not tamed for these dense prediction tasks. We hope our results can facilitate more comprehensive evaluation of self-supervised learning methods designed for Transformer architectures. Our code and models are available at https://github.com/SwinTransformer/Transformer-SSL, which will be continually enriched.

cs.CV↗

arXiv2021

Group-Free 3D Object Detection via Transformers

Recently, directly detecting 3D objects from 3D point clouds has received increasing attention. To extract object representation from an irregular point cloud, existing methods usually take a point grouping step to assign the points to an object candidate so that a PointNet-like network could be used to derive object features from the grouped points. However, the inaccurate point assignments caused by the hand-crafted grouping scheme decrease the performance of 3D object detection. In this paper, we present a simple yet effective method for directly detecting 3D objects from the 3D point cloud. Instead of grouping local points to each object candidate, our method computes the feature of an object from all the points in the point cloud with the help of an attention mechanism in the Transformers \cite{vaswani2017attention}, where the contribution of each point is automatically learned in the network training. With an improved attention stacking scheme, our method fuses object features in different stages and generates more accurate object detection results. With few bells and whistles, the proposed method achieves state-of-the-art 3D object detection performance on two widely used benchmarks, ScanNet V2 and SUN RGB-D. The code and models are publicly available at \url{https://github.com/zeliu98/Group-Free-3D}

cs.CV↗

arXiv2021

Cross-Iteration Batch Normalization

A well-known issue of Batch Normalization is its significantly reduced effectiveness in the case of small mini-batch sizes. When a mini-batch contains few examples, the statistics upon which the normalization is defined cannot be reliably estimated from it during a training iteration. To address this problem, we present Cross-Iteration Batch Normalization (CBN), in which examples from multiple recent iterations are jointly utilized to enhance estimation quality. A challenge of computing statistics over multiple iterations is that the network activations from different iterations are not comparable to each other due to changes in network weights. We thus compensate for the network weight changes via a proposed technique based on Taylor polynomials, so that the statistics can be accurately estimated and batch normalization can be effectively applied. On object detection and image classification with small mini-batch sizes, CBN is found to outperform the original batch normalization and a direct calculation of statistics over previous iterations without the proposed compensation technique. Code is available at https://github.com/Howal/Cross-iterationBatchNorm .

cs.LG↗

arXiv2021

Pseudo-ISP: Learning Pseudo In-camera Signal Processing Pipeline from A Color Image Denoiser

The success of deep denoisers on real-world color photographs usually relies on the modeling of sensor noise and in-camera signal processing (ISP) pipeline. Performance drop will inevitably happen when the sensor and ISP pipeline of test images are different from those for training the deep denoisers (i.e., noise discrepancy). In this paper, we present an unpaired learning scheme to adapt a color image denoiser for handling test images with noise discrepancy. We consider a practical training setting, i.e., a pre-trained denoiser, a set of test noisy images, and an unpaired set of clean images. To begin with, the pre-trained denoiser is used to generate the pseudo clean images for the test images. Pseudo-ISP is then suggested to jointly learn the pseudo ISP pipeline and signal-dependent rawRGB noise model using the pairs of test and pseudo clean images. We further apply the learned pseudo ISP and rawRGB noise model to clean color images to synthesize realistic noisy images for denoiser adaption. Pseudo-ISP is effective in synthesizing realistic noisy sRGB images, and improved denoising performance can be achieved by alternating between Pseudo-ISP training and denoiser adaption. Experiments show that our Pseudo-ISP not only can boost simple Gaussian blurring-based denoiser to achieve competitive performance against CBDNet, but also is effective in improving state-of-the-art deep denoisers, e.g., CBDNet and RIDNet.

cs.CV↗

arXiv2021

Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning

Contrastive learning methods for unsupervised visual representation learning have reached remarkable levels of transfer performance. We argue that the power of contrastive learning has yet to be fully unleashed, as current methods are trained only on instance-level pretext tasks, leading to representations that may be sub-optimal for downstream tasks requiring dense pixel predictions. In this paper, we introduce pixel-level pretext tasks for learning dense feature representations. The first task directly applies contrastive learning at the pixel level. We additionally propose a pixel-to-propagation consistency task that produces better results, even surpassing the state-of-the-art approaches by a large margin. Specifically, it achieves 60.2 AP, 41.4 / 40.5 mAP and 77.2 mIoU when transferred to Pascal VOC object detection (C4), COCO object detection (FPN / C4) and Cityscapes semantic segmentation using a ResNet-50 backbone network, which are 2.6 AP, 0.8 / 1.0 mAP and 1.0 mIoU better than the previous best methods built on instance-level contrastive learning. Moreover, the pixel-level pretext tasks are found to be effective for pre-training not only regular backbone networks but also head networks used for dense downstream tasks, and are complementary to instance-level contrastive methods. These results demonstrate the strong potential of defining pretext tasks at the pixel level, and suggest a new path forward in unsupervised visual representation learning. Code is available at \url{https://github.com/zdaxie/PixPro}.

cs.CV↗

arXiv2021

ParaSCI: A Large Scientific Paraphrase Dataset for Longer Paraphrase Generation

We propose ParaSCI, the first large-scale paraphrase dataset in the scientific field, including 33,981 paraphrase pairs from ACL (ParaSCI-ACL) and 316,063 pairs from arXiv (ParaSCI-arXiv). Digging into characteristics and common patterns of scientific papers, we construct this dataset though intra-paper and inter-paper methods, such as collecting citations to the same paper or aggregating definitions by scientific terms. To take advantage of sentences paraphrased partially, we put up PDBERT as a general paraphrase discovering method. The major advantages of paraphrases in ParaSCI lie in the prominent length and textual diversity, which is complementary to existing paraphrase datasets. ParaSCI obtains satisfactory results on human evaluation and downstream tasks, especially long paraphrase generation.

cs.CL↗

arXiv2020

Global Context Networks

The Non-Local Network (NLNet) presents a pioneering approach for capturing long-range dependencies within an image, via aggregating query-specific global context to each query position. However, through a rigorous empirical analysis, we have found that the global contexts modeled by the non-local network are almost the same for different query positions. In this paper, we take advantage of this finding to create a simplified network based on a query-independent formulation, which maintains the accuracy of NLNet but with significantly less computation. We further replace the one-layer transformation function of the non-local block by a two-layer bottleneck, which further reduces the parameter number considerably. The resulting network element, called the global context (GC) block, effectively models global context in a lightweight manner, allowing it to be applied at multiple layers of a backbone network to form a global context network (GCNet). Experiments show that GCNet generally outperforms NLNet on major benchmarks for various recognition tasks. The code and network configurations are available at https://github.com/xvjiarui/GCNet.

cs.CV↗

arXiv2020

Ultrafast ferromagnetic fluctuations preceding magnetoelastic first-order transitions

First-order magnetic transitions are of both fundamental and technological interest given that a number of emergent phases and functionalities are thereby created. Of particular interest are giant magnetocaloric effects, which are attributed to first-order magnetic transitions and have attracted broad attention for solid-state refrigeration applications. While the conventional wisdom is that atomic lattices play an important role in first-order magnetic transitions, a coherent microscopic description of the lattice and spin degrees of freedom is still lacking. Here, we study the magnetic phase transition dynamics on the intermetallic LaFe13-xSix, which is one of the most classical giant magnetocaloric systems, in both frequency and time domains utilizing neutron scattering and ultrafast X-ray diffraction. We have observed a strong magnetic diffuse scattering in the paramagnetic state preceding the first-order magnetic transition, corresponding to picosecond ferromagnetic fluctuations. Upon photon-excitation, the ferromagnetic state is completely suppressed in 0.9 ps and recovered in 20 ps. The ultrafast dynamics suggest that the magnetic degree of freedom dominates this magnetoelastic transition and ferromagnetic fluctuations might be universally relevant for this kind of compounds.

cond-mat.mtrl-sci↗

arXiv2020

DR 21 South Filament: a Parsec-sized Dense Gas Accretion Flow onto the DR 21 Massive Young Cluster

DR21 south filament (DR21SF) is a unique component of the giant network of filamentary molecular clouds in the north region of Cygnus X complex. Unlike the highly fragmented and star-forming active environment it resides, DR21SF exhibits a coherent profile in the column density map with very few star formation signposts, even though the previously reported linear density of the filament is an order of magnitude higher than the thermal stable threshold. We derive the size (3.6~pc by 0.13~pc), temperature (10 to 15~K), and mass (1048~\textit{M$_\odot$}) of DR21SF from Shanghai 65 m TianMa Radio Telescope (TMRT) observations of NH$_3$ (1, 1) and (2, 2) inversion lines in conjunction with the column density map from our previous work. Star-forming sites are identified along the filament where gas temperature excesses. We find clear gradients in radial velocity and intrinsic line-width along the spine of the filament. The gradients can be well interpreted with a scenario of an accretion flow feeding DR 21 at a mass transfer rate of $1.1 \times 10^{-3}$~\textit{M$_\odot$} yr$^{-1}$. Based on the analysis of its kinematic temperature, intrinsic line-width and mass distribution, we conclude that DR21SF is in an overall trans-critical status, which indicates an early evolutionary stage.

astro-ph.GA↗