arXiv subjects

Zhiqiang Shen

Publications and source records attributed to Zhiqiang Shen.

At least 181 records · Page 10Linked to original sources

arXiv2021

FAST discovery of an extremely radio-faint millisecond pulsar from the Fermi-LAT unassociated source 3FGL J0318.1+0252

High sensitivity radio searches of unassociated $γ$-ray sources have proven to be an effective way of finding new pulsars. Using the Five-hundred-meter Aperture Spherical radio Telescope (FAST) during its commissioning phase, we have carried out a number of targeted deep searches of \textit{Fermi} Large Area Telescope (LAT) $γ$-ray sources. On Feb. 27$^{th}$, 2018 we discovered an isolated millisecond pulsar (MSP), PSR J0318+0253, coincident with the unassociated $γ$-ray source 3FGL J0318.1+0252. PSR J0318+0253 has a spin period of $5.19$ milliseconds, a dispersion measure (DM) of $26$ pc cm$^{-3}$ corresponding to a DM distance of about $1.3$ kpc, and a period-averaged flux density of $\sim$11 $\pm$ 2 $μ$Jy at L-band (1.05-1.45 GHz). Among all high energy MSPs, PSR J0318+0253 is the faintest ever detected in radio bands, by a factor of at least $\sim$4 in terms of L-band fluxes. With the aid of the radio ephemeris, an analysis of 9.6 years of \textit{Fermi}-LAT data revealed that PSR J0318+0253 also displays strong $γ$-ray pulsations. Follow-up observations carried out by both Arecibo and FAST suggest a likely spectral turn-over around 350 MHz. This is the first result from the collaboration between FAST and the \textit{Fermi}-LAT teams as well as the first confirmed new MSP discovery by FAST, raising hopes for the detection of many more MSPs. Such discoveries will make a significant contribution to our understanding of the neutron star zoo while potentially contributing to the future detection of gravitational waves, via pulsar timing array (PTA) experiments.

astro-ph.HE↗

arXiv2021

Propionamide (C2H5CONH2): The largest peptide-like molecule in space

Peptide bonds, as the molecular bridges that connect amino acids, are crucial to the formation of proteins. Searches and studies of molecules with embedded peptide-like bonds are thus important for the understanding of protein formation in space. Here we report the first tentative detection of propionamide (C2H5CONH2), the largest peptide-like molecule detected in space toward Sagittarius B2(N1) at a position called N1E that is slightly offset from the continuum peak. A new laboratory measurements of the propionamide spectrum were carried out in the 9-461 GHz, which provide good opportunity to check directly for the transition frequencies of detected interstellar lines of propionamide. Our observing result indicates that propionamide emission comes from the warm, compact cores in Sagittarius B2, in which massive protostellars are forming. The column density of propionamide toward Sgr B2(N1E) was derived to be 1.5\times 10^{16} cm^-2, which is three fifths of that of acetamide, and one nineteenth of that of formamide. This detection suggests that large peptide-like molecules can form and survive during star-forming process and may form more complex molecules in the interstellar medium. The detection of propionamide bodes well for the presence of polypeptides, as well as other complex prebiotic molecules in the interstellar medium.

astro-ph.GA↗

arXiv2021

Spatial correlations of entangled polymer dynamics

The spatial correlations of entangled polymer dynamics are examined by molecular dynamics simulations and neutron spin-echo spectroscopy. Due to the soft nature of topological constraints, the initial spatial decays of intermediate scattering functions of entangled chains are, to the first approximation, surprisingly similar to those of an unentangled system in the functional forms. However, entanglements reveal themselves as a long tail in the reciprocal-space correlations, implying a weak but persistent dynamic localization in real space. Comparison with a number of existing theoretical models of entangled polymers suggests that they cannot fully describe the spatial correlations revealed by simulations and experiments. In particular, the strict one-dimensional diffusion idea of the original tube model is shown to be flawed. The dynamic spatial correlation analysis demonstrated in this work provides a useful tool for interrogating the dynamics of entangled polymers. Lastly, the failure of the investigated models to even qualitatively predict the spatial correlations of collective single-chain density fluctuations points to a possible critical role of incompressibility in polymer melt dynamics.

cond-mat.soft↗

arXiv2021

Deformation measurement by single spherical near-field intensity measurement for large reflector antenna

This paper presents a new method to obtain the deformation distribution on the main reflector of an antenna only by measuring the electric intensity on a spherical surface with the focal point as the center of the sphere, regardless of phase. Combining the differential geometry theory with geometric optics method, this paper has derived a deformation-intensity equation to relate the surface deformation to the intensity distribution of a spherical near-field directly. Based on the Finite difference method (FDM) and Gauss-Seidel iteration, deformation has been calculated from intensity simulated by GO and PO method, respectively, with relatively small errors, which prove the effectiveness of the equation proposed in this paper. By means of this method , it is possible to measure the deformation only by scanning the electric intensity of a single hemispherical near-field whose area is only about $1/15$ of the aperture. And the measurement only needs a plane wave at any frequency as the incident wave, which means that both the signals from the outer space satellite and the far-field artificial beacon could be used as the sources. The scanning can be realized no matter what attitude and elevation angle the antenna is in because the size and angle of the hemisphere are changeable.

astro-ph.IM↗

arXiv2021

S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibration

Previous studies dominantly target at self-supervised learning on real-valued networks and have achieved many promising results. However, on the more challenging binary neural networks (BNNs), this task has not yet been fully explored in the community. In this paper, we focus on this more difficult scenario: learning networks where both weights and activations are binary, meanwhile, without any human annotated labels. We observe that the commonly used contrastive objective is not satisfying on BNNs for competitive accuracy, since the backbone network contains relatively limited capacity and representation ability. Hence instead of directly applying existing self-supervised methods, which cause a severe decline in performance, we present a novel guided learning paradigm from real-valued to distill binary networks on the final prediction distribution, to minimize the loss and obtain desirable accuracy. Our proposed method can boost the simple contrastive learning baseline by an absolute gain of 5.5~15% on BNNs. We further reveal that it is difficult for BNNs to recover the similar predictive distributions as real-valued models when training without labels. Thus, how to calibrate them is key to address the degradation in performance. Extensive experiments are conducted on the large-scale ImageNet and downstream datasets. Our method achieves substantial improvement over the simple contrastive learning baseline, and is even comparable to many mainstream supervised BNN methods. Code is available at https://github.com/szq0214/S2-BNN.

cs.CV↗

arXiv2021

How Do Adam and Training Strategies Help BNNs Optimization?

The best performing Binary Neural Networks (BNNs) are usually attained using Adam optimization and its multi-step training variants. However, to the best of our knowledge, few studies explore the fundamental reasons why Adam is superior to other optimizers like SGD for BNN optimization or provide analytical explanations that support specific training strategies. To address this, in this paper we first investigate the trajectories of gradients and weights in BNNs during the training process. We show the regularization effect of second-order momentum in Adam is crucial to revitalize the weights that are dead due to the activation saturation in BNNs. We find that Adam, through its adaptive learning rate strategy, is better equipped to handle the rugged loss surface of BNNs and reaches a better optimum with higher generalization ability. Furthermore, we inspect the intriguing role of the real-valued weights in binary networks, and reveal the effect of weight decay on the stability and sluggishness of BNN optimization. Through extensive experiments and analysis, we derive a simple training scheme, building on existing Adam-based optimization, which achieves 70.5% top-1 accuracy on the ImageNet dataset using the same architecture as the state-of-the-art ReActNet while achieving 1.1% higher accuracy. Code and models are available at https://github.com/liuzechun/AdamBNN.

cs.LG↗

arXiv2021

COTR: Convolution in Transformer Network for End to End Polyp Detection

Purpose: Colorectal cancer (CRC) is the second most common cause of cancer mortality worldwide. Colonoscopy is a widely used technique for colon screening and polyp lesions diagnosis. Nevertheless, manual screening using colonoscopy suffers from a substantial miss rate of polyps and is an overwhelming burden for endoscopists. Computer-aided diagnosis (CAD) for polyp detection has the potential to reduce human error and human burden. However, current polyp detection methods based on object detection framework need many handcrafted pre-processing and post-processing operations or user guidance that require domain-specific knowledge. Methods: In this paper, we propose a convolution in transformer (COTR) network for end-to-end polyp detection. Motivated by the detection transformer (DETR), COTR is constituted by a CNN for feature extraction, transformer encoder layers interleaved with convolutional layers for feature encoding and recalibration, transformer decoder layers for object querying, and a feed-forward network for detection prediction. Considering the slow convergence of DETR, COTR embeds convolution layers into transformer encoder for feature reconstruction and convergence acceleration. Results: Experimental results on two public polyp datasets show that COTR achieved 91.49\% precision, 82.69% sensitivity, and 86.87% F1-score on the ETIS-LARIB, and 91.67% precision, 93.54% sensitivity, and 92.60% F1-score on the CVC-ColonDB. Conclusion: This study proposed an end to end detection method based on detection transformer for colorectal polyp detection. Experimental results on ETIS-LARIB and CVC-ColonDB dataset demonstrated that the proposed model achieved comparable performance against state-of-the-art methods.

cs.CV↗

arXiv2021

An 86-GHz search for Pulsars in the Galactic Center with the Atacama Large Millimeter/submillimeter Array

We report on the first pulsar and transient survey of the Galactic Center (GC) with the Atacama Large Millimeter/submillimeter Array (ALMA). The observations were conducted during the Global Millimeter VLBI Array campaign in 2017 and 2018. We carry out searches using timeseries of both total intensity and other polarization components in the form of Stokes parameters. We incorporate acceleration and its derivative in the pulsar search, and also search in segments of the entire observation to compensate for potential orbital motion of the pulsar. While no new pulsar is found, our observations yield the polarization profile of the GC magnetar PSR J1745-2900 at mm-wavelength for the first time, which turns out to be nearly 100 % linearly polarized. Additionally, we estimate the survey sensitivity placed by both system and red noise, and evaluate its capability of finding pulsars in orbital motion with either Sgr A* or a binary companion. We show that the survey is sensitive to only the most luminous pulsars in the known population, and future observations with ALMA in Band-1 will deliver significantly deeper survey sensitivity on the GC pulsar population.

astro-ph.HE↗

arXiv2021

"BNN - BN = ?": Training Binary Neural Networks without Batch Normalization

Batch normalization (BN) is a key facilitator and considered essential for state-of-the-art binary neural networks (BNN). However, the BN layer is costly to calculate and is typically implemented with non-binary parameters, leaving a hurdle for the efficient implementation of BNN training. It also introduces undesirable dependence between samples within each batch. Inspired by the latest advance on Batch Normalization Free (BN-Free) training, we extend their framework to training BNNs, and for the first time demonstrate that BNs can be completed removed from BNN training and inference regimes. By plugging in and customizing techniques including adaptive gradient clipping, scale weight standardization, and specialized bottleneck block, a BN-free BNN is capable of maintaining competitive accuracy compared to its BN-based counterpart. Extensive experiments validate the effectiveness of our proposal across diverse BNN backbones and datasets. For example, after removing BNs from the state-of-the-art ReActNets, it can still be trained with our proposed methodology to achieve 92.08%, 68.34%, and 68.0% accuracy on CIFAR-10, CIFAR-100, and ImageNet respectively, with marginal performance drop (0.23%~0.44% on CIFAR and 1.40% on ImageNet). Codes and pre-trained models are available at: https://github.com/VITA-Group/BNN_NoBN.

cs.LG↗

arXiv2021

Is Label Smoothing Truly Incompatible with Knowledge Distillation: An Empirical Study

This work aims to empirically clarify a recently discovered perspective that label smoothing is incompatible with knowledge distillation. We begin by introducing the motivation behind on how this incompatibility is raised, i.e., label smoothing erases relative information between teacher logits. We provide a novel connection on how label smoothing affects distributions of semantically similar and dissimilar classes. Then we propose a metric to quantitatively measure the degree of erased information in sample's representation. After that, we study its one-sidedness and imperfection of the incompatibility view through massive analyses, visualizations and comprehensive experiments on Image Classification, Binary Networks, and Neural Machine Translation. Finally, we broadly discuss several circumstances wherein label smoothing will indeed lose its effectiveness. Project page: http://zhiqiangshen.com/projects/LS_and_KD/index.html.

cs.LG↗

arXiv2021

Unsupervised Disentanglement of Linear-Encoded Facial Semantics

We propose a method to disentangle linear-encoded facial semantics from StyleGAN without external supervision. The method derives from linear regression and sparse representation learning concepts to make the disentangled latent representations easily interpreted as well. We start by coupling StyleGAN with a stabilized 3D deformable facial reconstruction method to decompose single-view GAN generations into multiple semantics. Latent representations are then extracted to capture interpretable facial semantics. In this work, we make it possible to get rid of labels for disentangling meaningful facial semantics. Also, we demonstrate that the guided extrapolation along the disentangled representations can help with data augmentation, which sheds light on handling unbalanced data. Finally, we provide an analysis of our learned localized facial representations and illustrate that the semantic information is encoded, which surprisingly complies with human intuition. The overall unsupervised design brings more flexibility to representation learning in the wild.

cs.CV↗

arXiv2021

The TMRT K Band Observations towards 26 Infrared Dark Clouds: NH$_{3}$, CCS, and HC$_{3}$N

We present one of the first Shanghai Tian Ma Radio Telescope (TMRT) K Band observations towards a sample of 26 infrared dark clouds (IRDCs). We observed the (1,1), (2,2), (3,3), and (4,4) transitions of NH$_{3}$ together with CCS (2$_{1}$-1$_{0}$) and HC$_{3}$N $J\,$=2-1, simultaneously. The survey dramatically increases the existing CCS-detected IRDC sample from 8 to 23, enabling a better statistical study of the ratios of carbon-chain molecules (CCM) to N-bearing molecules in IRDCs. With the newly developed hyperfine group ratio (HFGR) method of fitting NH$_{3}$ inversion lines, we found the gas temperature to be between 10 and 18 K. The column density ratios of CCS to NH$_{3}$ for most of the IRDCs are less than 10$^{-2}$, distinguishing IRDCs from low-mass star-forming regions. We carried out chemical evolution simulations based on a three-phase chemical model NAUTILUS. Our measurements of the column density ratios between CCM and NH$_{3}$ are consistent with chemical evolutionary ages of $\lesssim$10$^{5}$ yr in the models. Comparisons of the data and chemical models suggest that CCS, HC$_{3}$N, and NH$_{3}$ are sensitive to the chemical evolutionary stages of the sources.

astro-ph.GA↗

arXiv2021

MEAL V2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet without Tricks

We introduce a simple yet effective distillation framework that is able to boost the vanilla ResNet-50 to 80%+ Top-1 accuracy on ImageNet without tricks. We construct such a framework through analyzing the problems in the existing classification system and simplify the base method ensemble knowledge distillation via discriminators by: (1) adopting the similarity loss and discriminator only on the final outputs and (2) using the average of softmax probabilities from all teacher ensembles as the stronger supervision. Intriguingly, three novel perspectives are presented for distillation: (1) weight decay can be weakened or even completely removed since the soft label also has a regularization effect; (2) using a good initialization for students is critical; and (3) one-hot/hard label is not necessary in the distillation process if the weights are well initialized. We show that such a straight-forward framework can achieve state-of-the-art results without involving any commonly-used techniques, such as architecture modification; outside training data beyond ImageNet; autoaug/randaug; cosine learning rate; mixup/cutmix training; label smoothing; etc. Our method obtains 80.67% top-1 accuracy on ImageNet using a single crop-size of 224x224 with vanilla ResNet-50, outperforming the previous state-of-the-arts by a significant margin under the same network structure. Our result can be regarded as a strong baseline using knowledge distillation, and to our best knowledge, this is also the first method that is able to boost vanilla ResNet-50 to surpass 80% on ImageNet without architecture modification or additional training data. On smaller ResNet-18, our distillation framework consistently improves from 69.76% to 73.19%, which shows tremendous practical values in real-world applications. Our code and models are available at: https://github.com/szq0214/MEAL-V2.

cs.CV↗

arXiv2021

Semantic Relation Reasoning for Shot-Stable Few-Shot Object Detection

Few-shot object detection is an imperative and long-lasting problem due to the inherent long-tail distribution of real-world data. Its performance is largely affected by the data scarcity of novel classes. But the semantic relation between the novel classes and the base classes is constant regardless of the data availability. In this work, we investigate utilizing this semantic relation together with the visual information and introduce explicit relation reasoning into the learning of novel object detection. Specifically, we represent each class concept by a semantic embedding learned from a large corpus of text. The detector is trained to project the image representations of objects into this embedding space. We also identify the problems of trivially using the raw embeddings with a heuristic knowledge graph and propose to augment the embeddings with a dynamic relation graph. As a result, our few-shot detector, termed SRR-FSD, is robust and stable to the variation of shots of novel objects. Experiments show that SRR-FSD can achieve competitive results at higher shots, and more importantly, a significantly better performance given both lower explicit and implicit shots. The benchmark protocol with implicit shots removed from the pretrained classification dataset can serve as a more realistic setting for future research.

cs.CV↗

arXiv2021

The Changing Look Blazar B2 1420+32

Blazars are active galactic nuclei with their relativistic jets pointing toward the observer, with two major sub-classes, the flat spectrum radio quasars and BL Lac objects. We present multi-wavelength photometric and spectroscopic monitoring observations of the blazar, B2 1420+32, focusing on its outbursts in 2018-2020. Multi-epoch spectra show that the blazar exhibited large scale spectral variability in both its continuum and line emission, accompanied by dramatic gamma-ray and optical variability by factors of up to 40 and 15, respectively, on week to month timescales. Over the last decade, the gamma-ray and optical fluxes increased by factors of 1500 and 100, respectively. B2 1420+32 was an FSRQ with broad emission lines in 1995. Following a series of flares starting in 2018, it transitioned between BL Lac and FSRQ states multiple times, with the emergence of a strong Fe pseudo continuum. Two spectra also contain components that can be modeled as single-temperature black bodies of 12,000 and 5,200 K. Such a collection of "changing look" features has never been observed previously in a blazar. We measure gamma-ray-optical and the inter-band optical lags implying emission region separations of less than 800 and 130 gravitational radii respectively. Since most emission line flux variations, except the Fe continuum, are within a factor of 2-3, the transitions between FSRQ and BL Lac classifications are mainly caused by the continuum variability. The large Fe continuum flux increase suggests the occurrence of dust sublimation releasing more Fe ions in the central engine and an energy transfer from the relativistic jet to sub-relativistic emission components.

astro-ph.HE↗

arXiv2021

Interpretative Computer-aided Lung Cancer Diagnosis: from Radiology Analysis to Malignancy Evaluation

Background and Objective:Computer-aided diagnosis (CAD) systems promote diagnosis effectiveness and alleviate pressure of radiologists. A CAD system for lung cancer diagnosis includes nodule candidate detection and nodule malignancy evaluation. Recently, deep learning-based pulmonary nodule detection has reached satisfactory performance ready for clinical application. However, deep learning-based nodule malignancy evaluation depends on heuristic inference from low-dose computed tomography volume to malignant probability, which lacks clinical cognition. Methods:In this paper, we propose a joint radiology analysis and malignancy evaluation network (R2MNet) to evaluate the pulmonary nodule malignancy via radiology characteristics analysis. Radiological features are extracted as channel descriptor to highlight specific regions of the input volume that are critical for nodule malignancy evaluation. In addition, for model explanations, we propose channel-dependent activation mapping to visualize the features and shed light on the decision process of deep neural network. Results:Experimental results on the LIDC-IDRI dataset demonstrate that the proposed method achieved area under curve of 96.27% on nodule radiology analysis and AUC of 97.52% on nodule malignancy evaluation. In addition, explanations of CDAM features proved that the shape and density of nodule regions were two critical factors that influence a nodule to be inferred as malignant, which conforms with the diagnosis cognition of experienced radiologists. Conclusion:Incorporating radiology analysis with nodule malignant evaluation, the network inference process conforms to the diagnostic procedure of radiologists and increases the confidence of evaluation results. Besides, model interpretation with CDAM features shed light on the regions which DNNs focus on when they estimate nodule malignancy probabilities.

eess.IV↗

arXiv2021

Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning

The goal of few-shot learning is to learn a classifier that can recognize unseen classes from limited support data with labels. A common practice for this task is to train a model on the base set first and then transfer to novel classes through fine-tuning (Here fine-tuning procedure is defined as transferring knowledge from base to novel data, i.e. learning to transfer in few-shot scenario.) or meta-learning. However, as the base classes have no overlap to the novel set, simply transferring whole knowledge from base data is not an optimal solution since some knowledge in the base model may be biased or even harmful to the novel class. In this paper, we propose to transfer partial knowledge by freezing or fine-tuning particular layer(s) in the base model. Specifically, layers will be imposed different learning rates if they are chosen to be fine-tuned, to control the extent of preserved transferability. To determine which layers to be recast and what values of learning rates for them, we introduce an evolutionary search based method that is efficient to simultaneously locate the target layers and determine their individual learning rates. We conduct extensive experiments on CUB and mini-ImageNet to demonstrate the effectiveness of our proposed method. It achieves the state-of-the-art performance on both meta-learning and non-meta based frameworks. Furthermore, we extend our method to the conventional pre-training + fine-tuning paradigm and obtain consistent improvement.

cs.CV↗

arXiv2020

DR 21 South Filament: a Parsec-sized Dense Gas Accretion Flow onto the DR 21 Massive Young Cluster

DR21 south filament (DR21SF) is a unique component of the giant network of filamentary molecular clouds in the north region of Cygnus X complex. Unlike the highly fragmented and star-forming active environment it resides, DR21SF exhibits a coherent profile in the column density map with very few star formation signposts, even though the previously reported linear density of the filament is an order of magnitude higher than the thermal stable threshold. We derive the size (3.6~pc by 0.13~pc), temperature (10 to 15~K), and mass (1048~\textit{M$_\odot$}) of DR21SF from Shanghai 65 m TianMa Radio Telescope (TMRT) observations of NH$_3$ (1, 1) and (2, 2) inversion lines in conjunction with the column density map from our previous work. Star-forming sites are identified along the filament where gas temperature excesses. We find clear gradients in radial velocity and intrinsic line-width along the spine of the filament. The gradients can be well interpreted with a scenario of an accretion flow feeding DR 21 at a mass transfer rate of $1.1 \times 10^{-3}$~\textit{M$_\odot$} yr$^{-1}$. Based on the analysis of its kinematic temperature, intrinsic line-width and mass distribution, we conclude that DR21SF is in an overall trans-critical status, which indicates an early evolutionary stage.

astro-ph.GA↗