arXiv subjects

Zhijun Zhang

Publications and source records attributed to Zhijun Zhang.

At least 37 records · Page 2Linked to original sources

arXiv2023

UniBrain: Unify Image Reconstruction and Captioning All in One Diffusion Model from Human Brain Activity

Image reconstruction and captioning from brain activity evoked by visual stimuli allow researchers to further understand the connection between the human brain and the visual perception system. While deep generative models have recently been employed in this field, reconstructing realistic captions and images with both low-level details and high semantic fidelity is still a challenging problem. In this work, we propose UniBrain: Unify Image Reconstruction and Captioning All in One Diffusion Model from Human Brain Activity. For the first time, we unify image reconstruction and captioning from visual-evoked functional magnetic resonance imaging (fMRI) through a latent diffusion model termed Versatile Diffusion. Specifically, we transform fMRI voxels into text and image latent for low-level information and guide the backward diffusion process through fMRI-based image and text conditions derived from CLIP to generate realistic captions and images. UniBrain outperforms current methods both qualitatively and quantitatively in terms of image reconstruction and reports image captioning results for the first time on the Natural Scenes Dataset (NSD) dataset. Moreover, the ablation experiments and functional region-of-interest (ROI) analysis further exhibit the superiority of UniBrain and provide comprehensive insight for visual-evoked brain decoding.

cs.CV↗

arXiv2023

Base Placement Optimization for Coverage Mobile Manipulation Tasks

Base placement optimization (BPO) is a fundamental capability for mobile manipulation and has been researched for decades. However, it is still very challenging for some reasons. First, compared with humans, current robots are extremely inflexible, and therefore have higher requirements on the accuracy of base placements (BPs). Second, the BP and task constraints are coupled with each other. The optimal BP depends on the task constraints, and in BP will affect task constraints in turn. More tricky is that some task constraints are flexible and non-deterministic. Third, except for fulfilling tasks, some other performance metrics such as optimal energy consumption and minimal execution time need to be considered, which makes the BPO problem even more complicated. In this paper, a Scale-like disc (SLD) representation of the workspace is used to decouple task constraints and BPs. To evaluate reachability and return optimal working pose over SLDs, a reachability map (RM) is constructed offline. In order to optimize the objectives of coverage, manipulability, and time cost simultaneously, this paper formulates the BPO as a multi-objective optimization problem (MOOP). Among them, the time optimal objective is modeled as a traveling salesman problem (TSP), which is more in line with the actual situation. The evolutionary method is used to solve the MOOP. Besides, to ensure the validity and optimality of the solution, collision detection is performed on the candidate BPs, and solutions from BPO are further fine-tuned according to the specific given task. Finally, the proposed method is used to solve a real-world toilet coverage cleaning task. Experiments show that the optimized BPs can significantly improve the coverage and efficiency of the task.

cs.RO↗

arXiv2022

Rounds vs Communication Tradeoffs for Maximal Independent Sets

We consider the problem of finding a maximal independent set (MIS) in the shared blackboard communication model with vertex-partitioned inputs. There are $n$ players corresponding to vertices of an undirected graph, and each player sees the edges incident on its vertex -- this way, each edge is known by both its endpoints and is thus shared by two players. The players communicate in simultaneous rounds by posting their messages on a shared blackboard visible to all players, with the goal of computing an MIS of the graph. While the MIS problem is well studied in other distributed models, and while shared blackboard is, perhaps, the simplest broadcast model, lower bounds for our problem were only known against one-round protocols. We present a lower bound on the round-communication tradeoff for computing an MIS in this model. Specifically, we show that when $r$ rounds of interaction are allowed, at least one player needs to communicate $Ω(n^{1/20^{r+1}})$ bits. In particular, with logarithmic bandwidth, finding an MIS requires $Ω(\log\log{n})$ rounds. This lower bound can be compared with the algorithm of Ghaffari, Gouleakis, Konrad, Mitrović, and Rubinfeld [PODC 2018] that solves MIS in $O(\log\log{n})$ rounds but with a logarithmic bandwidth for an average player. Additionally, our lower bound further extends to the closely related problem of maximal bipartite matching. To prove our results, we devise a new round elimination framework, which we call partial-input embedding, that may also be useful in future work for proving round-sensitive lower bounds in the presence of edge-sharing between players. Finally, we discuss several implications of our results to multi-round (adaptive) distributed sketching algorithms, broadcast congested clique, and to the welfare maximization problem in two-sided matching markets.

cs.DS↗

arXiv2022

High-Fidelity Variable-Rate Image Compression via Invertible Activation Transformation

Learning-based methods have effectively promoted the community of image compression. Meanwhile, variational autoencoder (VAE) based variable-rate approaches have recently gained much attention to avoid the usage of a set of different networks for various compression rates. Despite the remarkable performance that has been achieved, these approaches would be readily corrupted once multiple compression/decompression operations are executed, resulting in the fact that image quality would be tremendously dropped and strong artifacts would appear. Thus, we try to tackle the issue of high-fidelity fine variable-rate image compression and propose the Invertible Activation Transformation (IAT) module. We implement the IAT in a mathematical invertible manner on a single rate Invertible Neural Network (INN) based model and the quality level (QLevel) would be fed into the IAT to generate scaling and bias tensors. IAT and QLevel together give the image compression model the ability of fine variable-rate control while better maintaining the image fidelity. Extensive experiments demonstrate that the single rate image compression model equipped with our IAT module has the ability to achieve variable-rate control without any compromise. And our IAT-embedded model obtains comparable rate-distortion performance with recent learning-based image compression methods. Furthermore, our method outperforms the state-of-the-art variable-rate image compression method by a large margin, especially after multiple re-encodings.

eess.IV↗

arXiv2022

Category-Aware Transformer Network for Better Human-Object Interaction Detection

Human-Object Interactions (HOI) detection, which aims to localize a human and a relevant object while recognizing their interaction, is crucial for understanding a still image. Recently, transformer-based models have significantly advanced the progress of HOI detection. However, the capability of these models has not been fully explored since the Object Query of the model is always simply initialized as just zeros, which would affect the performance. In this paper, we try to study the issue of promoting transformer-based HOI detectors by initializing the Object Query with category-aware semantic information. To this end, we innovatively propose the Category-Aware Transformer Network (CATN). Specifically, the Object Query would be initialized via category priors represented by an external object detection model to yield better performance. Moreover, such category priors can be further used for enhancing the representation ability of features via the attention mechanism. We have firstly verified our idea via the Oracle experiment by initializing the Object Query with the groundtruth category information. And then extensive experiments have been conducted to show that a HOI detection model equipped with our idea outperforms the baseline by a large margin to achieve a new state-of-the-art result.

cs.CV↗

arXiv2022

Effective Actor-centric Human-object Interaction Detection

While Human-Object Interaction(HOI) Detection has achieved tremendous advances in recent, it still remains challenging due to complex interactions with multiple humans and objects occurring in images, which would inevitably lead to ambiguities. Most existing methods either generate all human-object pair candidates and infer their relationships by cropped local features successively in a two-stage manner, or directly predict interaction points in a one-stage procedure. However, the lack of spatial configurations or reasoning steps of two- or one- stage methods respectively limits their performance in such complex scenes. To avoid this ambiguity, we propose a novel actor-centric framework. The main ideas are that when inferring interactions: 1) the non-local features of the entire image guided by actor position are obtained to model the relationship between the actor and context, and then 2) we use an object branch to generate pixel-wise interaction area prediction, where the interaction area denotes the object central area. Moreover, we also use an actor branch to get interaction prediction of the actor and propose a novel composition strategy based on center-point indexing to generate the final HOI prediction. Thanks to the usage of the non-local features and the partly-coupled property of the human-objects composition strategy, our proposed framework can detect HOI more accurately especially for complex images. Extensive experimental results show that our method achieves the state-of-the-art on the challenging V-COCO and HICO-DET benchmarks and is more robust especially in multiple persons and/or objects scenes.

cs.CV↗

arXiv2021

Learning Oriented Remote Sensing Object Detection via Naive Geometric Computing

Detecting oriented objects along with estimating their rotation information is one crucial step for analyzing remote sensing images. Despite that many methods proposed recently have achieved remarkable performance, most of them directly learn to predict object directions under the supervision of only one (e.g. the rotation angle) or a few (e.g. several coordinates) groundtruth values individually. Oriented object detection would be more accurate and robust if extra constraints, with respect to proposal and rotation information regression, are adopted for joint supervision during training. To this end, we innovatively propose a mechanism that simultaneously learns the regression of horizontal proposals, oriented proposals, and rotation angles of objects in a consistent manner, via naive geometric computing, as one additional steady constraint (see Figure 1). An oriented center prior guided label assignment strategy is proposed for further enhancing the quality of proposals, yielding better performance. Extensive experiments demonstrate the model equipped with our idea significantly outperforms the baseline by a large margin to achieve a new state-of-the-art result without any extra computational burden during inference. Our proposed idea is simple and intuitive that can be readily implemented. Source codes and trained models are involved in supplementary files.

cs.CV↗

arXiv2020

Modification of Gesture-Determined-Dynamic Function with Consideration of Margins for Motion Planning of Humanoid Robots

The gesture-determined-dynamic function (GDDF) offers an effective way to handle the control problems of humanoid robots. Specifically, GDDF is utilized to constrain the movements of dual arms of humanoid robots and steer specific gestures to conduct demanding tasks under certain conditions. However, there is still a deficiency in this scheme. Through experiments, we found that the joints of the dual arms, which can be regarded as the redundant manipulators, could exceed their limits slightly at the joint angle level. The performance straightly depends on the parameters designed beforehand for the GDDF, which causes a lack of adaptability to the practical applications of this method. In this paper, a modified scheme of GDDF with consideration of margins (MGDDF) is proposed. This MGDDF scheme is based on quadratic programming (QP) framework, which is widely applied to solving the redundancy resolution problems of robot arms. Moreover, three margins are introduced in the proposed MGDDF scheme to avoid joint limits. With consideration of these margins, the joints of manipulators of the humanoid robots will not exceed their limits, and the potential damages which might be caused by exceeding limits will be completely avoided. Computer simulations conducted on MATLAB further verify the feasibility and superiority of the proposed MGDDF scheme.

cs.RO↗

arXiv2020

The Communication Complexity of Set Intersection and Multiple Equality Testing

In this paper we explore fundamental problems in randomized communication complexity such as computing Set Intersection on sets of size $k$ and Equality Testing between vectors of length $k$. Sağlam and Tardos and Brody et al. showed that for these types of problems, one can achieve optimal communication volume of $O(k)$ bits, with a randomized protocol that takes $O(\log^* k)$ rounds. Aside from rounds and communication volume, there is a \emph{third} parameter of interest, namely the \emph{error probability} $p_{\mathrm{err}}$. It is straightforward to show that protocols for Set Intersection or Equality Testing need to send $Ω(k + \log p_{\mathrm{err}}^{-1})$ bits. Is it possible to simultaneously achieve optimality in all three parameters, namely $O(k + \log p_{\mathrm{err}}^{-1})$ communication and $O(\log^* k)$ rounds? In this paper we prove that there is no universally optimal algorithm, and complement the existing round-communication tradeoffs with a new tradeoff between rounds, communication, and probability of error. In particular: 1. Any protocol for solving Multiple Equality Testing in $r$ rounds with failure probability $2^{-E}$ has communication volume $Ω(Ek^{1/r})$. 2. There exists a protocol for solving Multiple Equality Testing in $r + \log^*(k/E)$ rounds with $O(k + rEk^{1/r})$ communication, thereby essentially matching our lower bound and that of Sağlam and Tardos. Our original motivation for considering $p_{\mathrm{err}}$ as an independent parameter came from the problem of enumerating triangles in distributed ($\textsf{CONGEST}$) networks having maximum degree $Δ$. We prove that this problem can be solved in $O(Δ/\log n + \log\log Δ)$ time with high probability $1-1/\operatorname{poly}(n)$.

cs.IT↗

arXiv2020

Polar-Slotted ALOHA over Slot Erasure Channels

In this paper, we design a new polar slotted ALOHA (PSA) protocol over the slot erasure channels, which uses polar coding to construct the identical slot pattern (SP) assembles within each active user and base station. A theoretical analysis framework for the PSA is provided. First, by using the packet-oriented operation for the overlap packets when they conflict in a slot interval, we introduce the packet-based polarization transform and prove that this transform is independent of the packet's length. Second, guided by the packet-based polarization, an SP assignment (SPA) method with the variable slot erasure probability (SEP) and a SPA method with a fixed SEP value are designed for the PSA scheme. Then, a packet-oriented successive cancellation (pSC) and a pSC list (pSCL) decoding algorithm are developed. Simultaneously, the finite-slots throughput bounds and the asymptotic throughput for the pSC algorithm are analyzed. The simulation results show that the proposed PSA scheme can achieve an improved throughput with the pSC/SCL decoding algorithm over the traditional repetition slotted ALOHA scheme.

cs.IT↗

arXiv2019

A Gegenbauer Neural Network with Regularized Weights Direct Determination for Classification

Single-hidden layer feed forward neural networks (SLFNs) are widely used in pattern classification problems, but a huge bottleneck encountered is the slow speed and poor performance of the traditional iterative gradient-based learning algorithms. Although the famous extreme learning machine (ELM) has successfully addressed the problems of slow convergence, it still has computational robustness problems brought by input weights and biases randomly assigned. Thus, in order to overcome the aforementioned problems, in this paper, a novel type neural network based on Gegenbauer orthogonal polynomials, termed as GNN, is constructed and investigated. This model could overcome the computational robustness problems of ELM, while still has comparable structural simplicity and approximation capability. Based on this, we propose a regularized weights direct determination (R-WDD) based on equality-constrained optimization to determine the optimal output weights. The R-WDD tends to minimize the empirical risks and structural risks of the network, thus to lower the risk of over fitting and improve the generalization ability. This leads us to a the final GNN with R-WDD, which is a unified learning mechanism for binary and multi-class classification problems. Finally, as is verified in the various comparison experiments, GNN with R-WDD tends to have comparable (or even better) generalization performances, computational scalability and efficiency, and classification robustness, compared to least square support vector machine (LS-SVM), ELM with Gaussian kernel.

cs.LG↗

arXiv2018

RedNet: Residual Encoder-Decoder Network for indoor RGB-D Semantic Segmentation

Indoor semantic segmentation has always been a difficult task in computer vision. In this paper, we propose an RGB-D residual encoder-decoder architecture, named RedNet, for indoor RGB-D semantic segmentation. In RedNet, the residual module is applied to both the encoder and decoder as the basic building block, and the skip-connection is used to bypass the spatial feature between the encoder and decoder. In order to incorporate the depth information of the scene, a fusion structure is constructed, which makes inference on RGB image and depth image separately, and fuses their features over several layers. In order to efficiently optimize the network's parameters, we propose a `pyramid supervision' training scheme, which applies supervised learning over different layers in the decoder, to cope with the problem of gradients vanishing. Experiment results show that the proposed RedNet(ResNet-50) achieves a state-of-the-art mIoU accuracy of 47.8% on the SUN RGB-D benchmark dataset.

cs.CV↗

arXiv2018

Incorporating Depth into both CNN and CRF for Indoor Semantic Segmentation

To improve segmentation performance, a novel neural network architecture (termed DFCN-DCRF) is proposed, which combines an RGB-D fully convolutional neural network (DFCN) with a depth-sensitive fully-connected conditional random field (DCRF). First, a DFCN architecture which fuses depth information into the early layers and applies dilated convolution for later contextual reasoning is designed. Then, a depth-sensitive fully-connected conditional random field (DCRF) is proposed and combined with the previous DFCN to refine the preliminary result. Comparative experiments show that the proposed DFCN-DCRF has the best performance compared with most state-of-the-art methods.

cs.CV↗

arXiv2018

Understanding Molecular Dynamics with Stochastic Processes via Real or Virtual Dynamics

Molecular dynamics with the stochastic process provides a convenient way to compute structural and thermodynamic properties of chemical, biological, and materials systems. It is demonstrated that the virtual dynamics case that we proposed for the Langevin equation [J. Chem. Phys. 147, 184104 (2017)] in principle exists in other types of stochastic thermostats as well. The recommended middle scheme [J. Chem. Phys. 147, 034109 (2017)] of the Andersen thermostat is investigated as an example. As shown by both analytic and numerical results, while the real and virtual dynamics cases approach the same plateau of the characteristic correlation time in the high collision frequency limit, the accuracy and efficiency of sampling are relatively insensitive to the value of the collision frequency in a broad range. After we compare the behaviors of the Andersen thermostat to those of Langevin dynamics, a heuristic schematic representation is proposed for understanding efficient stochastic thermostatting processes with molecular dynamics.

cond-mat.stat-mech↗

arXiv2017

Stationary state distribution and efficiency analysis of the Langevin equation via real or virtual dynamics

Langevin dynamics has become a popular tool to simulate the Boltzmann equilibrium distribution. When the repartition of the Langevin equation involves the exact realization of the Ornstein-Uhlenbeck noise, in addition to the conventional density evolution, there exists another type of discrete evolution that may not correspond to a continuous, real dynamical counterpart. This virtual dynamics case is also able to produce the desired stationary distribution. Different types of repartition lead to different numerical schemes, of which the accuracy and efficiency are investigated through studying the harmonic oscillator potential, an analytical solvable model. By analyzing the asymptotic distribution and characteristic correlation time that are derived by either directly solving the discrete equations of motion or using the related phase space propagators, it is shown that the optimal friction coefficient resulting in the minimal characteristic correlation time depends on the time interval chosen in the numerical implementation. When the recommended "middle" scheme is employed, both analytical and numerical results demonstrate that for a good numerical performance in efficiency as well as accuracy, one may choose a friction coefficient in a wide range from around the optimal value to the high friction limit.

physics.chem-ph↗

arXiv2017

Weighted Low-rank Tensor Recovery for Hyperspectral Image Restoration

Hyperspectral imaging, providing abundant spatial and spectral information simultaneously, has attracted a lot of interest in recent years. Unfortunately, due to the hardware limitations, the hyperspectral image (HSI) is vulnerable to various degradations, such noises (random noise, HSI denoising), blurs (Gaussian and uniform blur, HSI deblurring), and down-sampled (both spectral and spatial downsample, HSI super-resolution). Previous HSI restoration methods are designed for one specific task only. Besides, most of them start from the 1-D vector or 2-D matrix models and cannot fully exploit the structurally spectral-spatial correlation in 3-D HSI. To overcome these limitations, in this work, we propose a unified low-rank tensor recovery model for comprehensive HSI restoration tasks, in which non-local similarity between spectral-spatial cubic and spectral correlation are simultaneously captured by 3-order tensors. Further, to improve the capability and flexibility, we formulate it as a weighted low-rank tensor recovery (WLRTR) model by treating the singular values differently, and study its analytical solution. We also consider the exclusive stripe noise in HSI as the gross error by extending WLRTR to robust principal component analysis (WLRTR-RPCA). Extensive experiments demonstrate the proposed WLRTR models consistently outperform state-of-the-arts in typical low level vision HSI tasks, including denoising, destriping, deblurring and super-resolution.

cs.CV↗

arXiv2017

A unified thermostat scheme for efficient configurational sampling for classical/quantum canonical ensembles via molecular dynamics

We show a unified second-order scheme for constructing simple, robust and accurate algorithms for typical thermostats for configurational sampling for the canonical ensemble. When Langevin dynamics is used, the scheme leads to the BAOAB algorithm that has been recently investigated. We show that the scheme is also useful for other types of thermostat, such as the Andersen thermostat and Nosé-Hoover chain. Two 1-dimensional models and three typical realistic molecular systems that range from the gas phase, clusters, to the condensed phase are used in numerical examples for demonstration. Accuracy may be increased by an order of magnitude for estimating coordinate-dependent properties in molecular dynamics (when the same time interval is used), irrespective of which type of thermostat is applied. The scheme is especially useful for path integral molecular dynamics, because it consistently improves the efficiency for evaluating all thermodynamic properties for any type of thermostat.

physics.chem-ph↗

arXiv2017

Enhanced betatron radiation by steering a low-energy-spread electron beam in a deflected laser-driven plasma wiggler

Laser wakefield accelerators (LWFA) hold great potential to produce high-quality high-energy electron beams (e beams) and simultaneously bright x-ray sources via betatron radiation, which are very promising for pump-probe study in ultrafast science. However, in order to obtain a high-quality e beam, electron injection and acceleration should be carefully manipulated, where a large oscillation amplitude has to be avoided and thus the emitted x-ray yield is limited. Here, we report a new scheme to experimentally enhance betatron radiation significantly both in photon yield and photon energy by separating electron injection and acceleration from manipulation of the e-beam transverse oscillation in the wake via introducing a slanted thin plasma refraction slab. Particle-in-cell simulations indicate that the e-beam transverse oscillation amplitude can be increased by more than 10 folds, after being steered into the deflected laser-driven wakefield due to refraction at the slab's boundaries. Spectral broadening of the x-rays can be suppressed owing to the small variation in the peak energy of the low-energy-spread e beam in a plasma wiggler regime. We demonstrate that the high-quality e-beam generation, refracting and wiggling can act as a whole to realize the concurrence of monoenergetic e beam and bright x-rays in a compact LWFA.

physics.plasm-ph↗