arXiv subjects

Ruobing Shen

Publications and source records attributed to Ruobing Shen.

5 recordsLinked to original sources

arXiv2022

Connectivity-constrained Interactive Panoptic Segmentation

We address interactive panoptic annotation, where one segment all object and stuff regions in an image. We investigate two graph-based segmentation algorithms that both enforce connectivity of each region, with a notable class-aware Integer Linear Programming (ILP) formulation that ensures global optimum. Both algorithms can take RGB, or utilize the feature maps from any DCNN, whether trained on the target dataset or not, as input. We then propose an interactive, scribble-based annotation framework.

cs.CV↗

arXiv2020

Learning Discontinuous Piecewise Affine Fitting Functions using Mixed Integer Programming for Segmentation and Denoising

Piecewise affine functions are widely used to approximate nonlinear and discontinuous functions. However, most, if not all existing models only deal with fitting continuous functions. In this paper, we investigate the problem of fitting a discontinuous piecewise affine function to given data that lie in an orthogonal grid, where no restriction on the partition is enforced (i.e., its geometric shape can be nonconvex). This is useful for segmentation and denoising when data corresponding to images. We propose a novel Mixed Integer Program (MIP) formulation for the piecewise affine fitting problem, where binary variables determine the location of break-points. To obtain consistent partitions (i.e. image segmentation), we include multi-cut constraints in the formulation. Since the resulting problem is $\mathcal{NP}$-hard, two techniques are introduced to improve the computation. One is to add facet-defining inequalities to the formulation and the other to provide initial integer solutions using a special heuristic algorithm. We conduct extensive experiments by some synthetic images as well as real depth images, and the results demonstrate the feasibility of our model.

math.OC↗

arXiv2018

An ILP Solver for Multi-label MRFs with Connectivity Constraints

Integer Linear Programming (ILP) formulations of Markov random fields (MRFs) models with global connectivity priors were investigated previously in computer vision, e.g., \cite{globalinter,globalconn}. In these works, only Linear Programing (LP) relaxations \cite{globalinter,globalconn} or simplified versions \cite{graphcutbase} of the problem were solved. This paper investigates the ILP of multi-label MRF with exact connectivity priors via a branch-and-cut method, which provably finds globally optimal solutions. The method enforces connectivity priors iteratively by a cutting plane method, and provides feasible solutions with a guarantee on sub-optimality even if we terminate it earlier. The proposed ILP can be applied as a post-processing method on top of any existing multi-label segmentation approach. As it provides globally optimal solution, it can be used off-line to generate ground-truth labeling, which serves as quality check for any fast on-line algorithm. Furthermore, it can be used to generate ground-truth proposals for weakly supervised segmentation. We demonstrate the power and usefulness of our model by several experiments on the BSDS500 and PASCAL image dataset, as well as on medical images with trained probability maps.

cs.CV↗

arXiv2018

Discrete Potts Model for Generating Superpixels on Noisy Images

Many computer vision applications, such as object recognition and segmentation, increasingly build on superpixels. However, there have been so far few superpixel algorithms that systematically deal with noisy images. We propose to first decompose the image into equal-sized rectangular patches, which also sets the maximum superpixel size. Within each patch, a Potts model for simultaneous segmentation and denoising is applied, that guarantees connected and non-overlapping superpixels and also produces a denoised image. The corresponding optimization problem is formulated as a mixed integer linear program (MILP), and solved by a commercial solver. Extensive experiments on the BSDS500 dataset images with noises are compared with other state-of-the-art superpixel methods. Our method achieves the best result in terms of a combined score (OP) composed of the under-segmentation error, boundary recall and compactness.

cs.CV↗

arXiv2018

A First Derivative Potts Model for Segmentation and Denoising Using ILP

Unsupervised image segmentation and denoising are two fundamental tasks in image processing. Usually, graph based models such as multicut are used for segmentation and variational models are employed for denoising. Our approach addresses both problems at the same time. We propose a novel ILP formulation of the first derivative Potts model with the $\ell_1$ data term, where binary variables are introduced to deal with the $\ell_0$ norm of the regularization term. The ILP is then solved by a standard off-the-shelf MIP solver. Numerical experiments are compared with the multicut problem.

cs.CV↗