arXiv subjects

Cong Ma

Publications and source records attributed to Cong Ma.

At least 73 records · Page 4Linked to original sources

arXiv2021

Spectral Methods for Data Science: A Statistical Perspective

Spectral methods have emerged as a simple yet surprisingly effective approach for extracting information from massive, noisy and incomplete data. In a nutshell, spectral methods refer to a collection of algorithms built upon the eigenvalues (resp. singular values) and eigenvectors (resp. singular vectors) of some properly designed matrices constructed from data. A diverse array of applications have been found in machine learning, data science, and signal processing. Due to their simplicity and effectiveness, spectral methods are not only used as a stand-alone estimator, but also frequently employed to initialize other more sophisticated algorithms to improve performance. While the studies of spectral methods can be traced back to classical matrix perturbation theory and methods of moments, the past decade has witnessed tremendous theoretical advances in demystifying their efficacy through the lens of statistical modeling, with the aid of non-asymptotic random matrix theory. This monograph aims to present a systematic, comprehensive, yet accessible introduction to spectral methods from a modern statistical perspective, highlighting their algorithmic implications in diverse large-scale applications. In particular, our exposition gravitates around several central questions that span various applications: how to characterize the sample efficiency of spectral methods in reaching a target level of statistical accuracy, and how to assess their stability in the face of random noise, missing data, and adversarial corruptions? In addition to conventional $\ell_2$ perturbation analysis, we present a systematic $\ell_{\infty}$ and $\ell_{2,\infty}$ perturbation theory for eigenspace and singular subspaces, which has only recently become available owing to a powerful "leave-one-out" analysis framework.

stat.ML↗

arXiv2021

Design and evaluation of a FPGA-ADC prototype for the PET detector based on LYSO Crystals and SiPM arrays

The aim of this study is to design and evaluate a simple free running Analog-Digital Converter (ADC) based on the Field Programmable Gate Array (FPGA) device to accomplish the energy and position readout of the silicon photomultiplier (SiPM) array for application as PET scanners. This simple FPGA-ADC based on a carry chain Time-Digital Converter (TDC) implemented on a Kintex-7 FPGA consists of only one off-chip resistor so it has greater advantages in improving system integration and reducing cost than commercial chips. In this paper, a FPGA-ADC based front-end electronics prototype is presented, and both the design principle and implementation considerations are discussed. Experiments were performed using an 8 x 8 (crystal size: 4 x 4 x 15 mm3 ) and a 12 x 12 (crystal size: 2.65 x2.65 x 15 mm3 ) segmented LYSO crystals coupled with an 8 x 8 SiPM (Jseries, from ON Semiconductor) array which is under 22Na point source excitation. Initial test results indicate that the energy resolution of the two detectors after correction is around 13.2% and 13.5 % at 511 keV, and the profiles of the flood histograms show a clear visualization of the discrete scintillator element. All measurements were carried out at room temperature (~25 degree), without additional cooling

physics.ins-det↗

arXiv2021

TANet++: Triple Attention Network with Filtered Pointcloud on 3D Detection

TANet is one of state-of-the-art 3D object detection method on KITTI and JRDB benchmark, the network contains a Triple Attention module and Coarse-to-Fine Regression module to improve the robustness and accuracy of 3D Detection. However, since the original input data (point clouds) contains a lot of noise during collecting the data, which will further affect the training of the model. For example, the object is far from the robot, the sensor is difficult to obtain enough pointcloud. If the objects only contains few point clouds, and the samples are fed into model with the normal samples together during training, the detector will be difficult to distinguish the individual with few pointcloud belong to object or background. In this paper, we propose TANet++ to improve the performance on 3D Detection, which adopt a novel training strategy on training the TANet. In order to reduce the negative impact by the weak samples, the training strategy previously filtered the training data, and then the TANet++ is trained by the rest of data. The experimental results shows that AP score of TANet++ is 8.98 higher than TANet on JRDB benchmark.

cs.CV↗

arXiv2021

Accelerating Ill-Conditioned Low-Rank Matrix Estimation via Scaled Gradient Descent

Low-rank matrix estimation is a canonical problem that finds numerous applications in signal processing, machine learning and imaging science. A popular approach in practice is to factorize the matrix into two compact low-rank factors, and then optimize these factors directly via simple iterative methods such as gradient descent and alternating minimization. Despite nonconvexity, recent literatures have shown that these simple heuristics in fact achieve linear convergence when initialized properly for a growing number of problems of interest. However, upon closer examination, existing approaches can still be computationally expensive especially for ill-conditioned matrices: the convergence rate of gradient descent depends linearly on the condition number of the low-rank matrix, while the per-iteration cost of alternating minimization is often prohibitive for large matrices. The goal of this paper is to set forth a competitive algorithmic approach dubbed Scaled Gradient Descent (ScaledGD) which can be viewed as pre-conditioned or diagonally-scaled gradient descent, where the pre-conditioners are adaptive and iteration-varying with a minimal computational overhead. With tailored variants for low-rank matrix sensing, robust principal component analysis and matrix completion, we theoretically show that ScaledGD achieves the best of both worlds: it converges linearly at a rate independent of the condition number of the low-rank matrix similar as alternating minimization, while maintaining the low per-iteration cost of gradient descent. Our analysis is also applicable to general loss functions that are restricted strongly convex and smooth over low-rank matrices. To the best of our knowledge, ScaledGD is the first algorithm that provably has such properties over a wide range of low-rank matrix estimation tasks.

cs.LG↗

arXiv2021

Low-Rank Matrix Recovery with Scaled Subgradient Methods: Fast and Robust Convergence Without the Condition Number

Many problems in data science can be treated as estimating a low-rank matrix from highly incomplete, sometimes even corrupted, observations. One popular approach is to resort to matrix factorization, where the low-rank matrix factors are optimized via first-order methods over a smooth loss function, such as the residual sum of squares. While tremendous progresses have been made in recent years, the natural smooth formulation suffers from two sources of ill-conditioning, where the iteration complexity of gradient descent scales poorly both with the dimension as well as the condition number of the low-rank matrix. Moreover, the smooth formulation is not robust to corruptions. In this paper, we propose scaled subgradient methods to minimize a family of nonsmooth and nonconvex formulations -- in particular, the residual sum of absolute errors -- which is guaranteed to converge at a fast rate that is almost dimension-free and independent of the condition number, even in the presence of corruptions. We illustrate the effectiveness of our approach when the observation operator satisfies certain mixed-norm restricted isometry properties, and derive state-of-the-art performance guarantees for a variety of problems such as robust low-rank matrix sensing and quadratic sampling.

cs.LG↗

arXiv2021

Learning Mixtures of Low-Rank Models

We study the problem of learning mixtures of low-rank models, i.e. reconstructing multiple low-rank matrices from unlabelled linear measurements of each. This problem enriches two widely studied settings -- low-rank matrix sensing and mixed linear regression -- by bringing latent variables (i.e. unknown labels) and structural priors (i.e. low-rank structures) into consideration. To cope with the non-convexity issues arising from unlabelled heterogeneous data and low-complexity structure, we develop a three-stage meta-algorithm that is guaranteed to recover the unknown matrices with near-optimal sample and computational complexities under Gaussian designs. In addition, the proposed algorithm is provably stable against random noise. We complement the theoretical studies with empirical evidence that confirms the efficacy of our algorithm.

stat.ML↗

arXiv2021

Bridging Convex and Nonconvex Optimization in Robust PCA: Noise, Outliers, and Missing Data

This paper delivers improved theoretical guarantees for the convex programming approach in low-rank matrix estimation, in the presence of (1) random noise, (2) gross sparse outliers, and (3) missing data. This problem, often dubbed as robust principal component analysis (robust PCA), finds applications in various domains. Despite the wide applicability of convex relaxation, the available statistical support (particularly the stability analysis vis-à-vis random noise) remains highly suboptimal, which we strengthen in this paper. When the unknown matrix is well-conditioned, incoherent, and of constant rank, we demonstrate that a principled convex program achieves near-optimal statistical accuracy, in terms of both the Euclidean loss and the $\ell_{\infty}$ loss. All of this happens even when nearly a constant fraction of observations are corrupted by outliers with arbitrary magnitudes. The key analysis idea lies in bridging the convex program in use and an auxiliary nonconvex optimization algorithm, and hence the title of this paper.

stat.ML↗

arXiv2021

Minimax Off-Policy Evaluation for Multi-Armed Bandits

We study the problem of off-policy evaluation in the multi-armed bandit model with bounded rewards, and develop minimax rate-optimal procedures under three settings. First, when the behavior policy is known, we show that the Switch estimator, a method that alternates between the plug-in and importance sampling estimators, is minimax rate-optimal for all sample sizes. Second, when the behavior policy is unknown, we analyze performance in terms of the competitive ratio, thereby revealing a fundamental gap between the settings of known and unknown behavior policies. When the behavior policy is unknown, any estimator must have mean-squared error larger -- relative to the oracle estimator equipped with the knowledge of the behavior policy -- by a multiplicative factor proportional to the support size of the target policy. Moreover, we demonstrate that the plug-in approach achieves this worst-case competitive ratio up to a logarithmic factor. Third, we initiate the study of the partial knowledge setting in which it is assumed that the minimum probability taken by the behavior policy is known. We show that the plug-in estimator is optimal for relatively large values of the minimum probability, but is sub-optimal when the minimum probability is low. In order to remedy this gap, we propose a new estimator based on approximation by Chebyshev polynomials that provably achieves the optimal estimation error. Numerical experiments on both simulated and real data corroborate our theoretical findings.

stat.ML↗

arXiv2021

Beyond Procrustes: Balancing-Free Gradient Descent for Asymmetric Low-Rank Matrix Sensing

Low-rank matrix estimation plays a central role in various applications across science and engineering. Recently, nonconvex formulations based on matrix factorization are provably solved by simple gradient descent algorithms with strong computational and statistical guarantees. However, when the low-rank matrices are asymmetric, existing approaches rely on adding a regularization term to balance the scale of the two matrix factors which in practice can be removed safely without hurting the performance when initialized via the spectral method. In this paper, we provide a theoretical justification to this for the matrix sensing problem, which aims to recover a low-rank matrix from a small number of linear measurements. As long as the measurement ensemble satisfies the restricted isometry property, gradient descent -- in conjunction with spectral initialization -- converges linearly without the need of explicitly promoting balancedness of the factors; in fact, the factors stay balanced automatically throughout the execution of the algorithm. Our analysis is based on analyzing the evolution of a new distance metric that directly accounts for the ambiguity due to invertible transforms, and might be of independent interest.

eess.SP↗

arXiv2020

A high resolution multi-phase clock Time-Digital Convertor implemented on Kintex-7 FPGA

Time-digital Converter (TDC) aims to measure the arrival time of the leading edge of the pulse signal. Our recent work presented a high resolution multi-phase TDC based on the Kintex-7 Field Programmable Gate Array (FPGA) device. A simple I/O tile based circular input buffer is employed to oscillate the input signal periodically, and then a multi-phase TDC based on ISERDES core with a 625 ps bin size is used to accomplish the multiple measurements for getting higher resolution performance. In this paper, the design concept, architecture, as well as kernel implementation considerations are all discussed. To evaluate the TDC's performance, we built a verification system based on Kintex-7 FPGA. Initial test results indicate that the TDC's effective bin size is successfully reduced from 625 ps to 78.125 ps, and the measured dual-channel time resolution is better than 35 ps RMS.

physics.ins-det↗

arXiv2020

Microwave Photonic Imaging Radar with a Millimeter-level Resolution

Microwave photonic radars enable fast or even real-time high-resolution imaging thanks to its broad bandwidth. Nevertheless, the frequency range of the radars usually overlaps with other existed radio-frequency (RF) applications, and only a centimeter-level imaging resolution has been reported, making them insufficient for civilian applications. Here, we propose a microwave photonic imaging radar with a millimeter-level resolution by introducing a frequency-stepped chirp signal based on an optical frequency shifting loop. As compared with the conventional linear-frequency modulated (LFM) signal, the frequency-stepped chirp signal can bring the system excellent capability of anti-interference. In an experiment, a frequency-stepped chirp signal with a total bandwidth of 18.2 GHz (16.9 to 35.1 GHz) is generated. Postprocessing the radar echo, radar imaging with a two-dimensional imaging resolution of ~8.5 mm$\times$~8.3 mm is achieved. An auto-regressive algorithm is used to reconstruct the disturbed signal when a frequency interference exists, and the high-resolution imaging is sustained.

eess.SP↗

arXiv2019

Inference and Uncertainty Quantification for Noisy Matrix Completion

Noisy matrix completion aims at estimating a low-rank matrix given only partial and corrupted entries. Despite substantial progress in designing efficient estimation algorithms, it remains largely unclear how to assess the uncertainty of the obtained estimates and how to perform statistical inference on the unknown matrix (e.g.~constructing a valid and short confidence interval for an unseen entry). This paper takes a step towards inference and uncertainty quantification for noisy matrix completion. We develop a simple procedure to compensate for the bias of the widely used convex and nonconvex estimators. The resulting de-biased estimators admit nearly precise non-asymptotic distributional characterizations, which in turn enable optimal construction of confidence intervals\,/\,regions for, say, the missing entries and the low-rank factors. Our inferential procedures do not rely on sample splitting, thus avoiding unnecessary loss of data efficiency. As a byproduct, we obtain a sharp characterization of the estimation accuracy of our de-biased estimators, which, to the best of our knowledge, are the first tractable algorithms that provably achieve full statistical efficiency (including the preconstant). The analysis herein is built upon the intimate link between convex and nonconvex optimization --- an appealing feature recently discovered by \cite{chen2019noisy}.

stat.ML↗

arXiv2019

Multi-wavelength properties of radio and machine-learning identified counterparts to submillimeter sources in S2COSMOS

We identify multi-wavelength counterparts to 1,147 submillimeter sources from the S2COSMOS SCUBA-2 survey of the COSMOS field by employing a recently developed radio$+$machine-learning method trained on a large sample of ALMA-identified submillimeter galaxies (SMGs), including 260 SMGs identified in the AS2COSMOS pilot survey. In total, we identify 1,222 optical/near-infrared(NIR)/radio counterparts to the 897 S2COSMOS submillimeter sources with S$_{850}$>1.6mJy, yielding an overall identification rate of ($78\pm9$)%. We find that ($22\pm5$)% of S2COSMOS sources have multiple identified counterparts. We estimate that roughly 27% of these multiple counterparts within the same SCUBA-2 error circles very likely arise from physically associated galaxies rather than line-of-sight projections by chance. The photometric redshift of our radio$+$machine-learning identified SMGs ranges from z=0.2 to 5.7 and peaks at $z=2.3\pm0.1$. The AGN fraction of our sample is ($19\pm4$)%, which is consistent with that of ALMA SMGs in the literature. Comparing with radio/NIR-detected field galaxy population in the COSMOS field, our radio+machine-learning identified counterparts of SMGs have the highest star-formation rates and stellar masses. These characteristics suggest that our identified counterparts of S2COSMOS sources are a representative sample of SMGs at z<3. We employ our machine-learning technique to the whole COSMOS field and identified 6,877 potential SMGs, most of which are expected to have submillimeter emission fainter than the confusion limit of our S2COSMOS surveys (S$_{850}$<1.5mJy). We study the clustering properties of SMGs based on this statistically large sample, finding that they reside in high-mass dark matter halos ($(1.2\pm0.3)\times10^{13}\,h^{-1}\,\rm M_{\odot}$), which suggests that SMGs may be the progenitors of massive ellipticals we see in the local Universe.

astro-ph.GA↗

arXiv2019

Noisy Matrix Completion: Understanding Statistical Guarantees for Convex Relaxation via Nonconvex Optimization

This paper studies noisy low-rank matrix completion: given partial and noisy entries of a large low-rank matrix, the goal is to estimate the underlying matrix faithfully and efficiently. Arguably one of the most popular paradigms to tackle this problem is convex relaxation, which achieves remarkable efficacy in practice. However, the theoretical support of this approach is still far from optimal in the noisy setting, falling short of explaining its empirical success. We make progress towards demystifying the practical efficacy of convex relaxation vis-à-vis random noise. When the rank and the condition number of the unknown matrix are bounded by a constant, we demonstrate that the convex programming approach achieves near-optimal estimation errors --- in terms of the Euclidean loss, the entrywise loss, and the spectral norm loss --- for a wide range of noise levels. All of this is enabled by bridging convex relaxation with the nonconvex Burer-Monteiro approach, a seemingly distinct algorithmic paradigm that is provably robust against noise. More specifically, we show that an approximate critical point of the nonconvex formulation serves as an extremely tight approximation of the convex solution, thus allowing us to transfer the desired statistical guarantees of the nonconvex approach to its convex counterpart.

stat.ML↗

arXiv2019

A Time-to-Digital Converter-based Correction Method for Charge Measurement through Area Integration

A high-precision charge measurement can be achieved by the area integration of a digitized quasi-Gaussian signal after the signal passes through the shaper and analog-to-digital converter (ADC). The charge measurement contains an error due to the uncertainty of the first sampled point of a signal waveform. To reduce the error, we employ a time-to-digital converter (TDC) to measure the uncertainty precisely, and we design correction algorithms to improve the resolution of the charge measurement. This work includes analysis and simulations of the proposed algorithms and implementation of them in an FPGA device. Besides, the tests are also conducted to evaluate the performance of the correction method. Test results indicate that the resolution of the charge measurement is successfully improved from 0.231% to 0.126% by using a signal from the shaping circuit (with the amplitude of 2 V, and leading and trailing edges of about 80 ns and 280 ns, respectively) digitized at the sampling rate of 62.5 Msps.

physics.ins-det↗

arXiv2019

Prototype of the Readout Electronics for WCDA in LHAASO

In the Large High Altitude Air Shower Observatory (LHAASO), the Water Cherenkov Detector Array (WCDA) is one of the key parts. The WCDA consists of 3600 Photomultiplier Tubes (PMTs) scattered in a 90000 m2 area, and both high precision time and charge measurements are required over a large dynamic range from 1 to 4000 Photo Electrons (P.E.). To achieve time measurement precision better than 500 ps RMS, high quality clock distribution and automatic phase compensation are needed among the 400 Front End Electronics (FEE) modules. To simplify the readout electronics architecture, clock, data, and commands are transferred simultaneously over 400-meter fibers, while high speed data transfer interface is implemented based on TCP/IP protocol. Design and testing of the readout electronics prototype for WCDA is presented in this paper. Test results indicate that a charge resolution better than 10% RMS @ 1 P.E. and 1% RMS @ 4000 P.E., and a time resolution better than 300 ps RMS are successfully achieved over the whole dynamic range, beyond the application requirement.

physics.ins-det↗

arXiv2019

Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval, Matrix Completion, and Blind Deconvolution

Recent years have seen a flurry of activities in designing provably efficient nonconvex procedures for solving statistical estimation problems. Due to the highly nonconvex nature of the empirical loss, state-of-the-art procedures often require proper regularization (e.g. trimming, regularized cost, projection) in order to guarantee fast convergence. For vanilla procedures such as gradient descent, however, prior theory either recommends highly conservative learning rates to avoid overshooting, or completely lacks performance guarantees. This paper uncovers a striking phenomenon in nonconvex optimization: even in the absence of explicit regularization, gradient descent enforces proper regularization implicitly under various statistical models. In fact, gradient descent follows a trajectory staying within a basin that enjoys nice geometry, consisting of points incoherent with the sampling mechanism. This "implicit regularization" feature allows gradient descent to proceed in a far more aggressive fashion without overshooting, which in turn results in substantial computational savings. Focusing on three fundamental statistical estimation problems, i.e. phase retrieval, low-rank matrix completion, and blind deconvolution, we establish that gradient descent achieves near-optimal statistical and computational guarantees without explicit regularization. In particular, by marrying statistical modeling with generic optimization theory, we develop a general recipe for analyzing the trajectories of iterative algorithms via a leave-one-out perturbation argument. As a byproduct, for noisy matrix completion, we demonstrate that gradient descent achieves near-optimal error control --- measured entrywise and by the spectral norm --- which might be of independent interest.

cs.LG↗

arXiv2019

Precise Clock Synchronization in the Readout Electronics of WCDA in LHAASO

The Water Cherenkov Detector Array (WCDA) is one of the key parts in the Large High Altitude Air Shower Observatory (LHAASO). In the WCDA, 3600 Photomultiplier Tubes (PMTs) and the Front End Electronics (FEEs) are scattered within a 90000 m2 area, while a time measurement resolution better than 0.5 ns is required in the readout electronics. To achieve such time measurement precision, high quality clock distribution and synchronization among the 400 FEEs (each FEE for 9 PMTs readout) is required. To simplify the electronics system architecture, data, commands, and clock are transmitted simultaneously through fibers over a 400-meter distance between FEEs and the Clock and Data Transfer Modules (CDTMs). In this article, we propose a new method based on the White Rabbit (WR) to achieve completely automatic clock phase alignment between different FEEs. The original WR is enhanced to overcome the clock delay fluctuations due to ambient temperature variations. This paper presents the general scheme, the design of prototype electronics, and initial test results. These indicate that a clock synchronization precision better than 50 ps is achieved over 1 km fibers, which is well beyond the application requirement.

physics.ins-det↗