arXiv subjects

Song Wang

Publications and source records attributed to Song Wang.

At least 343 records · Page 19Linked to original sources

arXiv2022

A Benchmark of Video-Based Clothes-Changing Person Re-Identification

Person re-identification (Re-ID) is a classical computer vision task and has achieved great progress so far. Recently, long-term Re-ID with clothes-changing has attracted increasing attention. However, existing methods mainly focus on image-based setting, where richer temporal information is overlooked. In this paper, we focus on the relatively new yet practical problem of clothes-changing video-based person re-identification (CCVReID), which is less studied. We systematically study this problem by simultaneously considering the challenge of the clothes inconsistency issue and the temporal information contained in the video sequence for the person Re-ID problem. Based on this, we develop a two-branch confidence-aware re-ranking framework for handling the CCVReID problem. The proposed framework integrates two branches that consider both the classical appearance features and cloth-free gait features through a confidence-guided re-ranking strategy. This method provides the baseline method for further studies. Also, we build two new benchmark datasets for CCVReID problem, including a large-scale synthetic video dataset and a real-world one, both containing human sequences with various clothing changes. We will release the benchmark and code in this work to the public.

cs.CV↗

arXiv2022

Panoramic Human Activity Recognition

To obtain a more comprehensive activity understanding for a crowded scene, in this paper, we propose a new problem of panoramic human activity recognition (PAR), which aims to simultaneous achieve the individual action, social group activity, and global activity recognition. This is a challenging yet practical problem in real-world applications. For this problem, we develop a novel hierarchical graph neural network to progressively represent and model the multi-granularity human activities and mutual social relations for a crowd of people. We further build a benchmark to evaluate the proposed method and other existing related methods. Experimental results verify the rationality of the proposed PAR problem, the effectiveness of our method and the usefulness of the benchmark. We will release the source code and benchmark to the public for promoting the study on this problem.

cs.CV↗

arXiv2022

Self-supervised Social Relation Representation for Human Group Detection

Human group detection, which splits crowd of people into groups, is an important step for video-based human social activity analysis. The core of human group detection is the human social relation representation and division.In this paper, we propose a new two-stage multi-head framework for human group detection. In the first stage, we propose a human behavior simulator head to learn the social relation feature embedding, which is self-supervisely trained by leveraging the socially grounded multi-person behavior relationship. In the second stage, based on the social relation embedding, we develop a self-attention inspired network for human group detection. Remarkable performance on two state-of-the-art large-scale benchmarks, i.e., PANDA and JRDB-Group, verifies the effectiveness of the proposed framework. Benefiting from the self-supervised social relation embedding, our method can provide promising results with very few (labeled) training data. We will release the source code to the public.

cs.CV↗

arXiv2022

DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation

Dialog response generation in open domain is an important research topic where the main challenge is to generate relevant and diverse responses. In this paper, we propose a new dialog pre-training framework called DialogVED, which introduces continuous latent variables into the enhanced encoder-decoder pre-training framework to increase the relevance and diversity of responses. With the help of a large dialog corpus (Reddit), we pre-train the model using the following 4 tasks adopted in language models (LMs) and variational autoencoders (VAEs): 1) masked language model; 2) response generation; 3) bag-of-words prediction; and 4) KL divergence reduction. We also add additional parameters to model the turn structure in dialogs to improve the performance of the pre-trained model. We conduct experiments on PersonaChat, DailyDialog, and DSTC7-AVSD benchmarks for response generation. Experimental results show that our model achieves the new state-of-the-art results on all these datasets.

cs.CL↗

arXiv2022

Stellar chromospheric activities revealed from the LAMOST-K2 time-domain survey

By using the LAMOST time-domain survey data, we study stellar activities based on the $\rm{H_α}$ lines for about 2000 stars in four $K$2 plates. Two indices, $R_{\rm{Hα}}^{'}$ and $R_{\rm{Hα}}^{+}$, are computed from LAMOST spectra, the former of which is derived by excluding the photospheric contributions to the $\rm{H_α}$ lines, while the latter is derived by further subtracting the non-dynamo driven chromospheric emission. Meanwhile, the periodicity and variation amplitudes are computed from \emph{K2} light curves. Both the $R_{\rm{Hα}}^{'}$-Ro relation and $R_{\rm{Hα}}^{+}$-Ro relation show complicated profiles in the non-saturated decay region. Hot stars show flatter slopes and higher activity level than cool stars, and the behaviour is more notable in the $R_{\rm{Hα}}^{+}$-$R_{o}$ relation. This is consistent with recent studies using other activity proxies, including $L_{\rm{x}}/L_{\rm{bol}}$, $R_{\rm{HK}}^{'}$ and amplitudes of optical light curves. % This may suggest different kinds of stars follow different power laws in the decay region. Most of our targets have multiple observations, and some of them exhibit significant variability of ${\rm{Hα}}$ emissions, which may cause the large scatters shown in the decay region. We find three targets exhibiting positive correlation in rotational phase, possibly indicating that their optical light curves are dominated by hot faculae rather than cool starspots.

astro-ph.SR↗

arXiv2022

Graph Few-shot Learning with Task-specific Structures

Graph few-shot learning is of great importance among various graph learning tasks. Under the few-shot scenario, models are often required to conduct classification given limited labeled samples. Existing graph few-shot learning methods typically leverage Graph Neural Networks (GNNs) and perform classification across a series of meta-tasks. Nevertheless, these methods generally rely on the original graph (i.e., the graph that the meta-task is sampled from) to learn node representations. Consequently, the graph structure used in each meta-task is identical. Since the class sets are different across meta-tasks, node representations should be learned in a task-specific manner to promote classification performance. Therefore, to adaptively learn node representations across meta-tasks, we propose a novel framework that learns a task-specific structure for each meta-task. To handle the variety of nodes across meta-tasks, we extract relevant nodes and learn task-specific structures based on node influence and mutual information. In this way, we can learn node representations with the task-specific structure tailored for each meta-task. We further conduct extensive experiments on five node classification datasets under both single- and multiple-graph settings to validate the superiority of our framework over the state-of-the-art baselines. Our code is provided at https://github.com/SongW-SW/GLITTER.

cs.LG↗

arXiv2022

G2NetPL: Generic Game-Theoretic Network for Partial-Label Image Classification

Multi-label image classification aims to predict all possible labels in an image. It is usually formulated as a partial-label learning problem, since it could be expensive in practice to annotate all the labels in every training image. Existing works on partial-label learning focus on the case where each training image is labeled with only a subset of its positive/negative labels. To effectively address partial-label classification, this paper proposes an end-to-end Generic Game-theoretic Network (G2NetPL) for partial-label learning, which can be applied to most partial-label settings, including a very challenging, but annotation-efficient case where only a subset of the training images are labeled, each with only one positive label, while the rest of the training images remain unlabeled. In G2NetPL, each unobserved label is associated with a soft pseudo label, which, together with the network, formulates a two-player non-zero-sum non-cooperative game. The objective of the network is to minimize the loss function with given pseudo labels, while the pseudo labels will seek convergence to 1 (positive) or 0 (negative) with a penalty of deviating from the predicted labels determined by the network. In addition, we introduce a confidence-aware scheduler into the loss of the network to adaptively perform easy-to-hard learning for different labels. Extensive experiments demonstrate that our proposed G2NetPL outperforms many state-of-the-art multi-label classification methods under various partial-label settings on three different datasets.

cs.CV↗

arXiv2022

Are We Building on the Rock? On the Importance of Data Preprocessing for Code Summarization

Code summarization, the task of generating useful comments given the code, has long been of interest. Most of the existing code summarization models are trained and validated on widely-used code comment benchmark datasets. However, little is known about the quality of the benchmark datasets built from real-world projects. Are the benchmark datasets as good as expected? To bridge the gap, we conduct a systematic research to assess and improve the quality of four benchmark datasets widely used for code summarization tasks. First, we propose an automated code-comment cleaning tool that can accurately detect noisy data caused by inappropriate data preprocessing operations from existing benchmark datasets. Then, we apply the tool to further assess the data quality of the four benchmark datasets, based on the detected noises. Finally, we conduct comparative experiments to investigate the impact of noisy data on the performance of code summarization models. The results show that these data preprocessing noises widely exist in all four benchmark datasets, and removing these noisy data leads to a significant improvement on the performance of code summarization. We believe that the findings and insights will enable a better understanding of data quality in code summarization tasks, and pave the way for relevant research and practice.

cs.SE↗

arXiv2022

Discovery of one neutron star candidate from radial velocity monitoring

We report the discovery of one possible neutron star binary ($P_{\rm orb} =$ 0.8666 day) by using the LAMOST low-resolution spectroscopic data. The visible companion is a late A-type dwarf ($T_{\rm eff} = 7900 \pm 200$ K; log$g$ $=$ 4.3$\pm$0.2; $M =$ 1.7$\pm$0.1 M$_{\odot}$; $R\ =\ 1.7\pm0.2$ R$_{\odot}$), at a distance of 1.11$\pm0.03$ kpc. No double-lined feature can be seen from the GTC/HORuS high-resolution spectra, thus the radial velocity variation indicates an invisible object hiding in the binary. The system's optical light curves show clear ellipsoidal variability, suggesting that the visible companion is tidal distorted. By fitting the multi-band light curves with the ELC and WD codes, we constrain the mass of the invisible star to be 1.1--1.3 M$_{\odot}$. Spectral disentangling shows no additional component with optical absorption spectra, supporting the system contains one compact object. No X-ray or UV emission are detected in the ROSAT archive observations. Therefore, we suspect the invisible object is more likely a neutron star rather than a white dwarf. Our finding suggests the ability of LAMOST spectroscopic survey to discover X-ray quiescent compact objects.

astro-ph.HE↗

arXiv2022

Probable Dormant Neutron Star in a Short-Period Binary System

We have identified 2XMM J125556.57+565846.4, at a distance of 600 pc, as a binary system consisting of a normal star and a probable dormant neutron star. Optical spectra exhibit a slightly evolved F-type single star, displaying periodic Doppler shifts with a 2.76-day Keplerian circular orbit, with no indication of light from a secondary component. Optical and UV photometry reveal ellipsoidal variations with half the orbital period, due to the tidal deformation of the F star. The mass of the unseen companion is constrained to the range $1.1$--$2.1\,M_{\odot}$ at $3σ$ confidence, with the median of the mass distribution at $1.4\,M_{\odot}$, the typical mass of known neutron stars. A main-sequence star cannot masquerade as the dark companion. The distribution of possible companion masses still allows for the possibility of a very massive white dwarf. The companion itself could also be a close pair consisting of a white dwarf and an M star, or two white dwarfs, although the binary evolution that would lead to such a close triple system is unlikely. Similar ambiguities regarding the certain identification of a dormant neutron star are bound to affect most future discoveries of this type of non-interacting system. If the system indeed contains a dormant neutron star, it will become, in the future, a bright X-ray source and afterwards might even host a millisecond pulsar.

astro-ph.SR↗

arXiv2022

A long-period pre-ELM system discovered from LAMOST medium-resolution survey

We present LAMOST~J041920.07+072545.4 (hereafter J0419), a close binary consisting of a bloated extremely low mass pre-white dwarf (pre-ELM WD) and a compact object with an orbital period of 0.607189~days. The large-amplitude ellipsoidal variations and the evident Balmer and He~I emission lines suggest a filled Roche lobe and ongoing mass transfer. No outburst events were detected in the 15 years of monitoring of J0419, indicating a very low mass transfer rate. The temperature of the pre-ELM, $T_\mathrm{eff} = 5793_{-133}^{+124}\,\rm K$, is cooler than the known ELMs, but hotter than most CV donors. Combining the mean density within the Roche lobe and the radius constrained from our SED fitting, we obtain the mass of the pre-ELM, $M_1 = 0.176\pm 0.014\,M_\odot$. The joint fitting of light and radial velocity curves yields an inclination angle of $i = 66.5_{-1.7}^{+1.4}$ degrees, corresponding to the compact object mass of $M_2 = 1.09\pm 0.05\,M_\odot$. The very bloated pre-ELM has a smaller surface gravity ($\log g = 3.9\pm 0.01$, $R_1 = 0.78 \pm 0.02\,R_\odot$) than the known ELMs or pre-ELMs. The temperature and the luminosity ($L_\mathrm{bol} = 0.62_{-0.10}^{+0.11}\,L_\odot$) of J0419 are close to the main sequence, which makes the selection of such systems through the HR diagram inefficient. Based on the evolutionary model, the relatively long period and small $\log g$ indicate that J0419 could be close to the "bifurcation period" in the orbit evolution, which makes J0419 to be a unique source to connect ELM/pre-ELM WD systems, wide binaries and cataclysmic variables.

astro-ph.SR↗

arXiv2022

A White Dwarf-Main Sequence Binary Unveiled by Time-Domain Observations from LAMOST and TESS

We report a single-lined white dwarf-main sequence binary system, LAMOST J172900.17+652952.8, which is discovered by LAMOST's medium resolution time-domain surveys. The radial velocity semi-amplitude and orbital period of the optical visible star are measured by using the Palomar 200-inch telescope follow-up observations and the light curves from TESS. Thus the mass function of the invisible candidate white dwarf is derived, $f(M_{\rm{2}}) = 0.120\,\pm\,0.003\,M_{\odot}$. The mass of the visible star is measured based on the spectral energy distribution fitting, $M_{\mathrm{1}}$ = $0.81^{+0.07}_{-0.06}\,M_{\odot}$. Hence, the mass of its invisible companion is $M_{\rm{2}}\,\gtrsim\,0.63\,M_{\odot}$. The companion ought to be a compact object rather than a main-sequence star owing to the mass ratio $q = M_{\rm{2}} / M_{\rm 1} \gtrsim 0.78$ and the single-lined spectra. The compact object is likely to be a white dwarf except for small inclination angle, $i\,\lesssim\,40^{\circ}$. By using the GALEX NUV flux, the effective temperature of the white dwarf candidate is constrained as $T_{\rm eff}^{\rm WD}\,\lesssim\,12000-13500$ K. It is difficult to detect white dwarfs which are outshone by their bright companions via single-epoch optical spectroscopic surveys. Therefore, the optical time-domain surveys can play an important role in unveiling invisible white dwarfs and other compact objects in binaries.

astro-ph.SR↗

arXiv2022

A dynamically discovered and characterized non-accreting neutron star -- M dwarf binary candidate

Optical time-domain surveys can unveil and characterize exciting but less-explored non-accreting and/or non-beaming neutron stars (NS) in binaries. Here we report the discovery of such a NS candidate using the LAMOST spectroscopic survey. The candidate, designated LAMOST J112306.9+400736 (hereafter J1123), is in a single-lined spectroscopic binary containing an optically visible M star. The star's large radial velocity variation and ellipsoidal variations indicate a relatively massive unseen companion. Utilizing follow-up spectroscopy from the Palomar 200-inch telescope and high-precision photometry from TESS, we measure a companion mass of $1.24_{-0.03}^{+0.03}~M_{\odot}$. Main-sequence stars with this mass are ruled out, leaving a NS or a massive white dwarf (WD). Although a massive WD cannot be ruled out, the lack of UV excess radiation from the companion supports the NS hypothesis. Deep radio observations with FAST yielded no detections of either pulsed or persistent emission. J1123 is not detected in numerous X-ray and gamma-ray surveys. These non-detections suggest that the NS candidate is not presently accreting and pulsing. Our work exemplifies the capability of discovering compact objects in non-accreting close binaries by synergizing the optical time-domain spectroscopy and high-cadence photometry.

astro-ph.SR↗

arXiv2022

View-Invariant Skeleton-based Action Recognition via Global-Local Contrastive Learning

Skeleton-based human action recognition has been drawing more interest recently due to its low sensitivity to appearance changes and the accessibility of more skeleton data. However, even the 3D skeletons captured in practice are still sensitive to the viewpoint and direction gave the occlusion of different human-body joints and the errors in human joint localization. Such view variance of skeleton data may significantly affect the performance of action recognition. To address this issue, we propose in this paper a new view-invariant representation learning approach, without any manual action labeling, for skeleton-based human action recognition. Specifically, we leverage the multi-view skeleton data simultaneously taken for the same person in the network training, by maximizing the mutual information between the representations extracted from different views, and then propose a global-local contrastive loss to model the multi-scale co-occurrence relationships in both spatial and temporal domains. Extensive experimental results show that the proposed method is robust to the view difference of the input skeleton data and significantly boosts the performance of unsupervised skeleton-based human action methods, resulting in new state-of-the-art accuracies on two challenging multi-view benchmarks of PKUMMD and NTU RGB+D.

cs.CV↗

arXiv2022

Meta-RangeSeg: LiDAR Sequence Semantic Segmentation Using Multiple Feature Aggregation

LiDAR sensor is essential to the perception system in autonomous vehicles and intelligent robots. To fulfill the real-time requirements in real-world applications, it is necessary to efficiently segment the LiDAR scans. Most of previous approaches directly project 3D point cloud onto the 2D spherical range image so that they can make use of the efficient 2D convolutional operations for image segmentation. Although having achieved the encouraging results, the neighborhood information is not well-preserved in the spherical projection. Moreover, the temporal information is not taken into consideration in the single scan segmentation task. To tackle these problems, we propose a novel approach to semantic segmentation for LiDAR sequences named Meta-RangeSeg, where a new range residual image representation is introduced to capture the spatial-temporal information. Specifically, Meta-Kernel is employed to extract the meta features, which reduces the inconsistency between the 2D range image coordinates input and 3D Cartesian coordinates output. An efficient U-Net backbone is used to obtain the multi-scale features. Furthermore, Feature Aggregation Module (FAM) strengthens the role of range channel and aggregates features at different levels. We have conducted extensive experiments for performance evaluation on SemanticKITTI and SemanticPOSS. The promising results show that our proposed Meta-RangeSeg method is more efficient and effective than the existing approaches. Our full implementation is publicly available at https://github.com/songw-zju/Meta-RangeSeg .

cs.CV↗

arXiv2022

Automatic Comment Generation via Multi-Pass Deliberation

Deliberation is a common and natural behavior in human daily life. For example, when writing papers or articles, we usually first write drafts, and then iteratively polish them until satisfied. In light of such a human cognitive process, we propose DECOM, which is a multi-pass deliberation framework for automatic comment generation. DECOM consists of multiple Deliberation Models and one Evaluation Model. Given a code snippet, we first extract keywords from the code and retrieve a similar code fragment from a pre-defined corpus. Then, we treat the comment of the retrieved code as the initial draft and input it with the code and keywords into DECOM to start the iterative deliberation process. At each deliberation, the deliberation model polishes the draft and generates a new comment. The evaluation model measures the quality of the newly generated comment to determine whether to end the iterative process or not. When the iterative process is terminated, the best-generated comment will be selected as the target comment. Our approach is evaluated on two real-world datasets in Java (87K) and Python (108K), and experiment results show that our approach outperforms the state-of-the-art baselines. A human evaluation study also confirms the comments generated by DECOM tend to be more readable, informative, and useful.

cs.SE↗

arXiv2022

Long-Tailed Classification of Thorax Diseases on Chest X-Ray: A New Benchmark Study

Imaging exams, such as chest radiography, will yield a small set of common findings and a much larger set of uncommon findings. While a trained radiologist can learn the visual presentation of rare conditions by studying a few representative examples, teaching a machine to learn from such a "long-tailed" distribution is much more difficult, as standard methods would be easily biased toward the most frequent classes. In this paper, we present a comprehensive benchmark study of the long-tailed learning problem in the specific domain of thorax diseases on chest X-rays. We focus on learning from naturally distributed chest X-ray data, optimizing classification accuracy over not only the common "head" classes, but also the rare yet critical "tail" classes. To accomplish this, we introduce a challenging new long-tailed chest X-ray benchmark to facilitate research on developing long-tailed learning methods for medical image classification. The benchmark consists of two chest X-ray datasets for 19- and 20-way thorax disease classification, containing classes with as many as 53,000 and as few as 7 labeled training images. We evaluate both standard and state-of-the-art long-tailed learning methods on this new benchmark, analyzing which aspects of these methods are most beneficial for long-tailed medical image classification and summarizing insights for future algorithm design. The datasets, trained models, and code are available at https://github.com/VITA-Group/LongTailCXR.

cs.CV↗

arXiv2022

PLMCL: Partial-Label Momentum Curriculum Learning for Multi-Label Image Classification

Multi-label image classification aims to predict all possible labels in an image. It is usually formulated as a partial-label learning problem, given the fact that it could be expensive in practice to annotate all labels in every training image. Existing works on partial-label learning focus on the case where each training image is annotated with only a subset of its labels. A special case is to annotate only one positive label in each training image. To further relieve the annotation burden and enhance the performance of the classifier, this paper proposes a new partial-label setting in which only a subset of the training images are labeled, each with only one positive label, while the rest of the training images remain unlabeled. To handle this new setting, we propose an end-to-end deep network, PLMCL (Partial Label Momentum Curriculum Learning), that can learn to produce confident pseudo labels for both partially-labeled and unlabeled training images. The novel momentum-based law updates soft pseudo labels on each training image with the consideration of the updating velocity of pseudo labels, which help avoid trapping to low-confidence local minimum, especially at the early stage of training in lack of both observed labels and confidence on pseudo labels. In addition, we present a confidence-aware scheduler to adaptively perform easy-to-hard learning for different labels. Extensive experiments demonstrate that our proposed PLMCL outperforms many state-of-the-art multi-label classification methods under various partial-label settings on three different datasets.

cs.CV↗