arXiv subjects

Jing Jin

Publications and source records attributed to Jing Jin.

At least 73 records · Page 4Linked to original sources

arXiv2023

Content-aware Warping for View Synthesis

Existing image-based rendering methods usually adopt depth-based image warping operation to synthesize novel views. In this paper, we reason the essential limitations of the traditional warping operation to be the limited neighborhood and only distance-based interpolation weights. To this end, we propose content-aware warping, which adaptively learns the interpolation weights for pixels of a relatively large neighborhood from their contextual information via a lightweight neural network. Based on this learnable warping module, we propose a new end-to-end learning-based framework for novel view synthesis from a set of input source views, in which two additional modules, namely confidence-based blending and feature-assistant spatial refinement, are naturally proposed to handle the occlusion issue and capture the spatial correlation among pixels of the synthesized view, respectively. Besides, we also propose a weight-smoothness loss term to regularize the network. Experimental results on light field datasets with wide baselines and multi-view datasets show that the proposed method significantly outperforms state-of-the-art methods both quantitatively and visually. The source code will be publicly available at https://github.com/MantangGuo/CW4VS.

cs.CV↗

arXiv2022

Experimental Performance Evaluation of Cell-free Massive MIMO Systems Using COTS RRU with OTA Reciprocity Calibration and Phase Synchronization

Downlink coherent multiuser transmission is an essential technique for cell-free massive multiple-input multiple output (MIMO) systems, and the availability of channel state information (CSI) at the transmitter is a basic requirement. To avoid CSI feedback in a time-division duplex system, the uplink channel parameters should be calibrated to obtain the downlink CSI due to the radio frequency circuit mismatch of the transceiver. In this paper, a design of a reference signal for over-the-air reciprocity calibration is proposed. The frequency domain generated reference signals can make full use of the flexible frame structure of the fifth generation (5G) new radio, which can be completely transparent to commercial off-the-shelf (COTS) remote radio units (RRUs) and commercial user equipments. To further obtain the calibration of multiple RRUs, an interleaved RRU grouping with a genetic algorithm is proposed, and an averaged Argos calibration algorithm is also presented. We develop a cell-free massive MIMO prototype system with COTS RRUs, demonstrate the statistical characteristics of the calibration error and the effectiveness of the calibration algorithm, and evaluate the impact of the calibration delay on the different cooperative transmission schemes.

cs.IT↗

arXiv2022

Tensor Decomposition based Personalized Federated Learning

Federated learning (FL) is a new distributed machine learning framework that can achieve reliably collaborative training without collecting users' private data. However, due to FL's frequent communication and average aggregation strategy, they experience challenges scaling to statistical diversity data and large-scale models. In this paper, we propose a personalized FL framework, named Tensor Decomposition based Personalized Federated learning (TDPFed), in which we design a novel tensorized local model with tensorized linear layers and convolutional layers to reduce the communication cost. TDPFed uses a bi-level loss function to decouple personalized model optimization from the global model learning by controlling the gap between the personalized model and the tensorized local model. Moreover, an effective distributed learning strategy and two different model aggregation strategies are well designed for the proposed TDPFed framework. Theoretical convergence analysis and thorough experiments demonstrate that our proposed TDPFed framework achieves state-of-the-art performance while reducing the communication cost.

cs.LG↗

arXiv2022

An Insight-HXMT view of the mHz quasi-regular modulation phenomenon in the black hole X-ray binary 4U 1630-47

Here we report the spectral-timing results of the black hole X-ray binary 4U 1630-47 during its 2021 outburst using observations from the Hard X-ray Modulation Telescope. Type-C quasi-periodic oscillations (QPOs) in 1.6--4.2 Hz and quasi-regular modulation (QRM) near 60 mHz are detected during the outburst. The mHz QRM has a fractional rms of 10%--16% in the 8--35 keV energy band with a Q factor (frequency/width) of 2--4. Benefiting from the broad energy band of hxmt, we study the energy dependence of the 60 mHz QRM in 1--100 keV for the first time. We find that the fractional rms of the mHz QRM increases with photon energy, while the time lags of the mHz QRM are soft and decrease with photon energy. Fast recurrence of the mHz QRM, in a timescale of less than one hour, has been observed during the outburst. During this period, the corresponding energy spectra moderately change when the source transitions from the QRM state to the non-QRM state. The QRM phenomena also shows a dependence with the accretion rate. We suggest that the QRM could be caused by an unknown accretion instability aroused from the corona.

astro-ph.HE↗

arXiv2022

3D Face Parsing via Surface Parameterization and 2D Semantic Segmentation Network

Face parsing assigns pixel-wise semantic labels as the face representation for computers, which is the fundamental part of many advanced face technologies. Compared with 2D face parsing, 3D face parsing shows more potential to achieve better performance and further application, but it is still challenging due to 3D mesh data computation. Recent works introduced different methods for 3D surface segmentation, while the performance is still limited. In this paper, we propose a method based on the "3D-2D-3D" strategy to accomplish 3D face parsing. The topological disk-like 2D face image containing spatial and textural information is transformed from the sampled 3D face data through the face parameterization algorithm, and a specific 2D network called CPFNet is proposed to achieve the semantic segmentation of the 2D parameterized face data with multi-scale technologies and feature aggregation. The 2D semantic result is then inversely re-mapped to 3D face data, which finally achieves the 3D face parsing. Experimental results show that both CPFNet and the "3D-2D-3D" strategy accomplish high-quality 3D face parsing and outperform state-of-the-art 2D networks as well as 3D methods in both qualitative and quantitative comparisons.

cs.CV↗

arXiv2022

Forecasts on CMB lensing observations with AliCPT-1

AliCPT-1 is the first Chinese CMB experiment aiming for high precision measurement of Cosmic Microwave Background B-mode polarization. The telescope, currently under deployment in Tibet, will observe in two frequency bands centered at 90 and 150 GHz. We forecast the CMB lensing reconstruction, lensing-galaxy as well as lensing-CIB (Cosmic Infrared Background) cross correlation signal-to-noise ratio (SNR) for AliCPT-1. We consider two stages with different integrated observation time, namely "4 module*yr" (first stage) and "48 module*yr" (final stage). For lensing reconstruction, we use three different quadratic estimators, namely temperature-only, polarization-only and minimum-variance estimators, using curved sky geometry. We take into account the impact of inhomogeneous hit counts as well as of the mean-field bias due to incomplete sky coverage. In the first stage, our results show that the 150 GHz channel is able to measure the lensing signal at $15σ$ significance with the minimum-variance estimator. In the final stage, the measurement significance will increase to $31σ$. We also combine the two frequency data in the harmonic domain to optimize the SNR. Our result show that the coadding procedure can significantly reduce the reconstruction bias in the multiple range l>800. Thanks to the high quality of the polarization data in the final stage of AliCPT-1, the EB estimator will dominate the lensing reconstruction in this stage. We also estimate the SNR of cross-correlations between AliCPT-1 CMB lensing and other tracers of the large scale structure of the universe. For its cross-correlation with DESI galaxies/quasars, we report the cross-correlation SNR = 10-20 for the 4 redshift bins at 0.05<z<2.1. In the first stage, the total SNR is about $32$. In the final stage, the lensing-galaxy cross-correlation can reach SNR=52.

astro-ph.CO↗

arXiv2022

Quasi-periodic oscillations of the X-ray burst from the magnetar SGR J1935+2154 and associated with the fast radio burst FRB 200428

The origin(s) and mechanism(s) of fast radio bursts (FRBs), which are short radio pulses from cosmological distances, have remained a major puzzle since their discovery. We report a strong Quasi-Periodic Oscillation(QPO) of 40 Hz in the X-ray burst from the magnetar SGR J1935+2154 and associated with FRB 200428, significantly detected with the Hard X-ray Modulation Telescope (Insight-HXMT) and also hinted by the Konus-Wind data. QPOs from magnetar bursts have only been rarely detected; our 3.4 sigma (p-value is 2.9e-4) detection of the QPO reported here reveals the strongest QPO signal observed from magnetars (except in some very rare giant flares), making this X-ray burst unique among magnetar bursts. The two X-ray spikes coinciding with the two FRB pulses are also among the peaks of the QPO. Our results suggest that at least some FRBs are related to strong oscillation processes of neutron stars. We also show that we may overestimate the significance of the QPO signal and underestimate the errors of QPO parameters if QPO exists only in a fraction of the time series of a X-ray burst which we use to calculate the Leahy-normalized periodogram.

astro-ph.HE↗

arXiv2022

Using calibrator to improve robustness in Machine Reading Comprehension

Machine Reading Comprehension(MRC) has achieved a remarkable result since some powerful models, such as BERT, are proposed. However, these models are not robust enough and vulnerable to adversarial input perturbation and generalization examples. Some works tried to improve the performance on specific types of data by adding some related examples into training data while it leads to degradation on the original dataset, because the shift of data distribution makes the answer ranking based on the softmax probability of model unreliable. In this paper, we propose a method to improve the robustness by using a calibrator as the post-hoc reranker, which is implemented based on XGBoost model. The calibrator combines both manual features and representation learning features to rerank candidate results. Experimental results on adversarial datasets show that our model can achieve performance improvement by more than 10\% and also make improvement on the original and generalization datasets.

cs.CL↗

arXiv2022

Occlusion-aware Unsupervised Learning of Depth from 4-D Light Fields

Depth estimation is a fundamental issue in 4-D light field processing and analysis. Although recent supervised learning-based light field depth estimation methods have significantly improved the accuracy and efficiency of traditional optimization-based ones, these methods rely on the training over light field data with ground-truth depth maps which are challenging to obtain or even unavailable for real-world light field data. Besides, due to the inevitable gap (or domain difference) between real-world and synthetic data, they may suffer from serious performance degradation when generalizing the models trained with synthetic data to real-world data. By contrast, we propose an unsupervised learning-based method, which does not require ground-truth depth as supervision during training. Specifically, based on the basic knowledge of the unique geometry structure of light field data, we present an occlusion-aware strategy to improve the accuracy on occlusion areas, in which we explore the angular coherence among subsets of the light field views to estimate initial depth maps, and utilize a constrained unsupervised loss to learn their corresponding reliability for final depth prediction. Additionally, we adopt a multi-scale network with a weighted smoothness loss to handle the textureless areas. Experimental results on synthetic data show that our method can significantly shrink the performance gap between the previous unsupervised method and supervised ones, and produce depth maps with comparable accuracy to traditional methods with obviously reduced computational cost. Moreover, experiments on real-world datasets show that our method can avoid the domain shift problem presented in supervised methods, demonstrating the great potential of our method.

cs.CV↗

arXiv2021

Deep Selective Combinatorial Embedding and Consistency Regularization for Light Field Super-resolution

Light field (LF) images acquired by hand-held devices usually suffer from low spatial resolution as the limited detector resolution has to be shared with the angular dimension. LF spatial super-resolution (SR) thus becomes an indispensable part of the LF camera processing pipeline. The high-dimensionality characteristic and complex geometrical structure of LF images make the problem more challenging than traditional single-image SR. The performance of existing methods is still limited as they fail to thoroughly explore the coherence among LF sub-aperture images (SAIs) and are insufficient in accurately preserving the scene's parallax structure. To tackle this challenge, we propose a novel learning-based LF spatial SR framework. Specifically, each SAI of an LF image is first coarsely and individually super-resolved by exploring the complementary information among SAIs with selective combinatorial geometry embedding. To achieve efficient and effective selection of the complementary information, we propose two novel sub-modules conducted hierarchically: the patch selector provides an option of retrieving similar image patches based on offline disparity estimation to handle large-disparity correlations; and the SAI selector adaptively and flexibly selects the most informative SAIs to improve the embedding efficiency. To preserve the parallax structure among the reconstructed SAIs, we subsequently append a consistency regularization network trained over a structure-aware loss function to refine the parallax relationships over the coarse estimation. In addition, we extend the proposed method to irregular LF data. To the best of our knowledge, this is the first learning-based SR method for irregular LF data. Experimental results over both synthetic and real-world LF datasets demonstrate the significant advantage of our approach over state-of-the-art methods.

eess.IV↗

arXiv2021

DSSL: Deep Surroundings-person Separation Learning for Text-based Person Retrieval

Many previous methods on text-based person retrieval tasks are devoted to learning a latent common space mapping, with the purpose of extracting modality-invariant features from both visual and textual modality. Nevertheless, due to the complexity of high-dimensional data, the unconstrained mapping paradigms are not able to properly catch discriminative clues about the corresponding person while drop the misaligned information. Intuitively, the information contained in visual data can be divided into person information (PI) and surroundings information (SI), which are mutually exclusive from each other. To this end, we propose a novel Deep Surroundings-person Separation Learning (DSSL) model in this paper to effectively extract and match person information, and hence achieve a superior retrieval accuracy. A surroundings-person separation and fusion mechanism plays the key role to realize an accurate and effective surroundings-person separation under a mutually exclusion constraint. In order to adequately utilize multi-modal and multi-granular information for a higher retrieval accuracy, five diverse alignment paradigms are adopted. Extensive experiments are carried out to evaluate the proposed DSSL on CUHK-PEDES, which is currently the only accessible dataset for text-base person retrieval task. DSSL achieves the state-of-the-art performance on CUHK-PEDES. To properly evaluate our proposed DSSL in the real scenarios, a Real Scenarios Text-based Person Reidentification (RSTPReid) dataset is constructed to benefit future research on text-based person retrieval, which will be publicly available.

cs.CV↗

arXiv2021

Learning Dynamic Interpolation for Extremely Sparse Light Fields with Wide Baselines

In this paper, we tackle the problem of dense light field (LF) reconstruction from sparsely-sampled ones with wide baselines and propose a learnable model, namely dynamic interpolation, to replace the commonly-used geometry warping operation. Specifically, with the estimated geometric relation between input views, we first construct a lightweight neural network to dynamically learn weights for interpolating neighbouring pixels from input views to synthesize each pixel of novel views independently. In contrast to the fixed and content-independent weights employed in the geometry warping operation, the learned interpolation weights implicitly incorporate the correspondences between the source and novel views and adapt to different image content information. Then, we recover the spatial correlation between the independently synthesized pixels of each novel view by referring to that of input views using a geometry-based spatial refinement module. We also constrain the angular correlation between the novel views through a disparity-oriented LF structure loss. Experimental results on LF datasets with wide baselines show that the reconstructed LFs achieve much higher PSNR/SSIM and preserve the LF parallax structure better than state-of-the-art methods. The source code is publicly available at https://github.com/MantangGuo/DI4SLF.

cs.CV↗

arXiv2021

BinaryBERT: Pushing the Limit of BERT Quantization

The rapid development of large pre-trained language models has greatly increased the demand for model compression techniques, among which quantization is a popular solution. In this paper, we propose BinaryBERT, which pushes BERT quantization to the limit by weight binarization. We find that a binary BERT is hard to be trained directly than a ternary counterpart due to its complex and irregular loss landscape. Therefore, we propose ternary weight splitting, which initializes BinaryBERT by equivalently splitting from a half-sized ternary network. The binary model thus inherits the good performance of the ternary one, and can be further enhanced by fine-tuning the new architecture after splitting. Empirical results show that our BinaryBERT has only a slight performance drop compared with the full-precision model while being 24x smaller, achieving the state-of-the-art compression results on the GLUE and SQuAD benchmarks.

cs.CL↗

arXiv2021

Insight-HXMT observations of jet-like corona in a black hole X-ray binary MAXI J1820+070

A black hole X-ray binary produces hard X-ray radiation from its corona and disk when the accreting matter heats up. During an outburst, the disk and corona co-evolves with each other. However, such an evolution is still unclear in both its geometry and dynamics. Here we report the unusual decrease of the reflection fraction in MAXI J1820+070, which is the ratio of the coronal intensity illuminating the disk to the coronal intensity reaching the observer, as the corona is observed to contrast during the decay phase. We postulate a jet-like corona model, in which the corona can be understood as a standing shock where the material flowing through. In this dynamical scenario, the decrease of the reflection fraction is a signature of the corona's bulk velocity. Our findings suggest that as the corona is observed to get closer to the black hole, the coronal material might be outflowing faster.

astro-ph.HE↗

arXiv2021

A Method for Pulsar Searching: Combining a Two-dimensional Autocorrelation Profile Map and a Deep Convolutional Neural Network

In pulsar astronomy, detecting effective pulsar signals among numerous pulsar candidates is an important research topic. Starting from space X-ray pulsar signals, the two-dimensional autocorrelation profile map (2D-APM) feature modelling method, which utilizes epoch folding of the autocorrelation function of X-ray signals and expands the time-domain information of the periodic axis, is proposed. A uniform setting criterion regarding the time resolution of the periodic axis addresses pulsar signals without any prior information. Compared with the traditional profile, the model has a strong anti-noise ability, a greater abundance of information and consistent characteristics. The new feature is simulated with double Gaussian components, and the characteristic distribution of the model is revealed to be closely related to the distance between the double peaks of the profile. Next, a deep convolutional neural network (DCNN) is built, named Inception-ResNet. According to the order of the peak separation and number of arriving photons, 30 data sets based on the Poisson process are simulated to construct the training set, and the observation data of PSRs B0531+21, B0540-69 and B1509-58 from the Rossi X-ray Timing Explorer (RXTE) are selected to generate the test set. The number of training sets and the number of test sets are 30,000 and 5,400, respectively. After achieving convergence stability, more than 99$\%$ of the pulsar signals are recognized, and more than 99$\%$ of the interference is successfully rejected, which verifies the high degree of agreement between the network and the feature model and the high potential of the proposed method in searching for pulsars.

astro-ph.IM↗

arXiv2021

Performance Comparison between Reconfigurable Intelligent Surface and Relays: Theoretical Methods and a Perspective from Operator

Reconfigurable intelligent surface (RIS) is an emerging technique employing metasurface to reflect the signal from the source node to the destination node without consuming any energy. Not only the spectral efficiency but also the energy efficiency can be improved through RIS. Essentially, RIS can be considered as a passive relay between the source and destination node. On the other hand, a relay node in a traditional relay network has to be active, which indicates that it will consume energy when it is relaying the signal or information between the source and destination nodes. In this paper, we compare the performances between RIS and active relay for a general multiple-input multiple-output (MIMO) system. To make the comparison fair and comprehensive, both the performances of RIS and active relay are optimized with best-effort. In terms of the RIS, transmit beamforming and reflecting coefficient at the RIS are jointly optimized so as to maximize the end-to-end throughput. Although the optimization problem is non-convex, it is transformed equivalently to a weighted mean-square error (MSE) minimization problem and an alternating optimization problem is proposed, which can ensure the convergence to a stationary point. In terms of active relay, both half duplex relay (HDR) and full duplex relay (FDR) are considered. End-to-end throughput is maximized via an alternating optimization method. Numerical results are presented to demonstrate the effectiveness of the proposed algorithm. Finally, comparisons between RIS and relays are investigated from the perspective of system model, performance, deployment and controlling method.

cs.IT↗

arXiv2021

KDLSQ-BERT: A Quantized Bert Combining Knowledge Distillation with Learned Step Size Quantization

Recently, transformer-based language models such as BERT have shown tremendous performance improvement for a range of natural language processing tasks. However, these language models usually are computation expensive and memory intensive during inference. As a result, it is difficult to deploy them on resource-restricted devices. To improve the inference performance, as well as reduce the model size while maintaining the model accuracy, we propose a novel quantization method named KDLSQ-BERT that combines knowledge distillation (KD) with learned step size quantization (LSQ) for language model quantization. The main idea of our method is that the KD technique is leveraged to transfer the knowledge from a "teacher" model to a "student" model when exploiting LSQ to quantize that "student" model during the quantization training process. Extensive experiment results on GLUE benchmark and SQuAD demonstrate that our proposed KDLSQ-BERT not only performs effectively when doing different bit (e.g. 2-bit $\sim$ 8-bit) quantization, but also outperforms the existing BERT quantization methods, and even achieves comparable performance as the full-precision base-line model while obtaining 14.9x compression ratio. Our code will be public available.

cs.CL↗

arXiv2020

Deep Coarse-to-fine Dense Light Field Reconstruction with Flexible Sampling and Geometry-aware Fusion

A densely-sampled light field (LF) is highly desirable in various applications, such as 3-D reconstruction, post-capture refocusing and virtual reality. However, it is costly to acquire such data. Although many computational methods have been proposed to reconstruct a densely-sampled LF from a sparsely-sampled one, they still suffer from either low reconstruction quality, low computational efficiency, or the restriction on the regularity of the sampling pattern. To this end, we propose a novel learning-based method, which accepts sparsely-sampled LFs with irregular structures, and produces densely-sampled LFs with arbitrary angular resolution accurately and efficiently. We also propose a simple yet effective method for optimizing the sampling pattern. Our proposed method, an end-to-end trainable network, reconstructs a densely-sampled LF in a coarse-to-fine manner. Specifically, the coarse sub-aperture image (SAI) synthesis module first explores the scene geometry from an unstructured sparsely-sampled LF and leverages it to independently synthesize novel SAIs, in which a confidence-based blending strategy is proposed to fuse the information from different input SAIs, giving an intermediate densely-sampled LF. Then, the efficient LF refinement module learns the angular relationship within the intermediate result to recover the LF parallax structure. Comprehensive experimental evaluations demonstrate the superiority of our method on both real-world and synthetic LF images when compared with state-of-the-art methods. In addition, we illustrate the benefits and advantages of the proposed approach when applied in various LF-based applications, including image-based rendering and depth estimation enhancement.

eess.IV↗