arXiv subjects

Jing Zhang

Publications and source records attributed to Jing Zhang.

At least 541 records · Page 30Linked to original sources

arXiv2023

LaViP:Language-Grounded Visual Prompts

We introduce a language-grounded visual prompting method to adapt the visual encoder of vision-language models for downstream tasks. By capitalizing on language integration, we devise a parameter-efficient strategy to adjust the input of the visual encoder, eliminating the need to modify or add to the model's parameters. Due to this design choice, our algorithm can operate even in black-box scenarios, showcasing adaptability in situations where access to the model's parameters is constrained. We will empirically demonstrate that, compared to prior art, grounding visual prompts with language enhances both the accuracy and speed of adaptation. Moreover, our algorithm excels in base-to-novel class generalization, overcoming limitations of visual prompting and exhibiting the capacity to generalize beyond seen classes. We thoroughly assess and evaluate our method across a variety of image recognition datasets, such as EuroSAT, UCF101, DTD, and CLEVR, spanning different learning situations, including few-shot learning, base-to-novel class generalization, and transfer learning.

cs.CV↗

arXiv2023

From heavy rain removal to detail restoration: A faster and better network

The profound accumulation of precipitation during intense rainfall events can markedly degrade the quality of images, leading to the erosion of textural details. Despite the improvements observed in existing learning-based methods specialized for heavy rain removal, it is discerned that a significant proportion of these methods tend to overlook the precise reconstruction of the intricate details. In this work, we introduce a simple dual-stage progressive enhancement network, denoted as DPENet, aiming to achieve effective deraining while preserving the structural accuracy of rain-free images. This approach comprises two key modules, a rain streaks removal network (R$^2$Net) focusing on accurate rain removal, and a details reconstruction network (DRNet) designed to recover the textural details of rain-free images. Firstly, we introduce a dilated dense residual block (DDRB) within R$^2$Net, enabling the aggregation of high-level and low-level features. Secondly, an enhanced residual pixel-wise attention block (ERPAB) is integrated into DRNet to facilitate the incorporation of contextual information. To further enhance the fidelity of our approach, we employ a comprehensive loss function that accentuates both the marginal and regional accuracy of rain-free images. Extensive experiments conducted on publicly available benchmarks demonstrates the noteworthy efficiency and effectiveness of our proposed DPENet. The source code and pre-trained models are currently available at \url{https://github.com/chdwyb/DPENet}.

cs.CV↗

arXiv2023

GLT-T++: Global-Local Transformer for 3D Siamese Tracking with Ranking Loss

Siamese trackers based on 3D region proposal network (RPN) have shown remarkable success with deep Hough voting. However, using a single seed point feature as the cue for voting fails to produce high-quality 3D proposals. Additionally, the equal treatment of seed points in the voting process, regardless of their significance, exacerbates this limitation. To address these challenges, we propose a novel transformer-based voting scheme to generate better proposals. Specifically, a global-local transformer (GLT) module is devised to integrate object- and patch-aware geometric priors into seed point features, resulting in robust and accurate cues for offset learning of seed points. To train the GLT module, we introduce an importance prediction branch that learns the potential importance weights of seed points as a training constraint. Incorporating this transformer-based voting scheme into 3D RPN, a novel Siamese method dubbed GLT-T is developed for 3D single object tracking on point clouds. Moreover, we identify that the highest-scored proposal in the Siamese paradigm may not be the most accurate proposal, which limits tracking performance. Towards this concern, we approach the binary score prediction task as a ranking problem, and design a target-aware ranking loss and a localization-aware ranking loss to produce accurate ranking of proposals. With the ranking losses, we further present GLT-T++, an enhanced version of GLT-T. Extensive experiments on multiple benchmarks demonstrate that our GLT-T and GLT-T++ outperform state-of-the-art methods in terms of tracking accuracy while maintaining a real-time inference speed. The source code will be made available at https://github.com/haooozi/GLT-T.

cs.CV↗

arXiv2023

ViTPose++: Vision Transformer for Generic Body Pose Estimation

In this paper, we show the surprisingly good properties of plain vision transformers for body pose estimation from various aspects, namely simplicity in model structure, scalability in model size, flexibility in training paradigm, and transferability of knowledge between models, through a simple baseline model dubbed ViTPose. Specifically, ViTPose employs the plain and non-hierarchical vision transformer as an encoder to encode features and a lightweight decoder to decode body keypoints in either a top-down or a bottom-up manner. It can be scaled up from about 20M to 1B parameters by taking advantage of the scalable model capacity and high parallelism of the vision transformer, setting a new Pareto front for throughput and performance. Besides, ViTPose is very flexible regarding the attention type, input resolution, and pre-training and fine-tuning strategy. Based on the flexibility, a novel ViTPose+ model is proposed to deal with heterogeneous body keypoint categories in different types of body pose estimation tasks via knowledge factorization, i.e., adopting task-agnostic and task-specific feed-forward networks in the transformer. We also empirically demonstrate that the knowledge of large ViTPose models can be easily transferred to small ones via a simple knowledge token. Experimental results show that our ViTPose model outperforms representative methods on the challenging MS COCO Human Keypoint Detection benchmark at both top-down and bottom-up settings. Furthermore, our ViTPose+ model achieves state-of-the-art performance simultaneously on a series of body pose estimation tasks, including MS COCO, AI Challenger, OCHuman, MPII for human keypoint detection, COCO-Wholebody for whole-body keypoint detection, as well as AP-10K and APT-36K for animal keypoint detection, without sacrificing inference speed.

cs.CV↗

arXiv2023

Encoder-minimal and Decoder-minimal Framework for Remote Sensing Image Dehazing

Haze obscures remote sensing images, hindering valuable information extraction. To this end, we propose RSHazeNet, an encoder-minimal and decoder-minimal framework for efficient remote sensing image dehazing. Specifically, regarding the process of merging features within the same level, we develop an innovative module called intra-level transposed fusion module (ITFM). This module employs adaptive transposed self-attention to capture comprehensive context-aware information, facilitating the robust context-aware feature fusion. Meanwhile, we present a cross-level multi-view interaction module (CMIM) to enable effective interactions between features from various levels, mitigating the loss of information due to the repeated sampling operations. In addition, we propose a multi-view progressive extraction block (MPEB) that partitions the features into four distinct components and employs convolution with varying kernel sizes, groups, and dilation factors to facilitate view-progressive feature learning. Extensive experiments demonstrate the superiority of our proposed RSHazeNet. We release the source code and all pre-trained models at \url{https://github.com/chdwyb/RSHazeNet}.

cs.CV↗

arXiv2023

Gradient-Based Markov Chain Monte Carlo for MIMO Detection

Accurately detecting symbols transmitted over multiple-input multiple-output (MIMO) wireless channels is crucial in realizing the benefits of MIMO techniques. However, optimal MIMO detection is associated with a complexity that grows exponentially with the MIMO dimensions and quickly becomes impractical. Recently, stochastic sampling-based Bayesian inference techniques, such as Markov chain Monte Carlo (MCMC), have been combined with the gradient descent (GD) method to provide a promising framework for MIMO detection. In this work, we propose to efficiently approach optimal detection by exploring the discrete search space via MCMC random walk accelerated by Nesterov's gradient method. Nesterov's GD guides MCMC to make efficient searches without the computationally expensive matrix inversion and line search. Our proposed method operates using multiple GDs per random walk, achieving sufficient descent towards important regions of the search space before adding random perturbations, guaranteeing high sampling efficiency. To provide augmented exploration, extra samples are derived through the trajectory of Nesterov's GD by simple operations, effectively supplementing the sample list for statistical inference and boosting the overall MIMO detection performance. Furthermore, we design an early stopping tactic to terminate unnecessary further searches, remarkably reducing the complexity. Simulation results and complexity analysis reveal that the proposed method achieves exceptional performance in both uncoded and coded MIMO systems, adapts to realistic channel models, and scales well to large MIMO dimensions.

eess.SP↗

arXiv2023

Semi-supervised Salient Object Detection with Effective Confidence Estimation

The success of existing salient object detection models relies on a large pixel-wise labeled training dataset, which is time-consuming and expensive to obtain. We study semi-supervised salient object detection, with access to a small number of labeled samples and a large number of unlabeled samples. Specifically, we present a pseudo label based learn-ing framework with a Conditional Energy-based Model. We model the stochastic nature of human saliency labels using the stochastic latent variable of the Conditional Energy-based Model. It further enables generation of a high-quality pixel-wise uncertainty map, highlighting the reliability of corresponding pseudo label generated for the unlabeled sample. This minimises the contribution of low-certainty pseudo labels in optimising the model, preventing the error propagation. Experimental results show that the proposed strategy can effectively explore the contribution of unlabeled data. With only 1/16 labeled samples, our model achieves competitive performance compared with state-of-the-art fully-supervised models.

cs.CV↗

arXiv2023

Low-Complexity Joint Beamforming for RIS-Assisted MU-MISO Systems Based on Model-Driven Deep Learning

Reconfigurable intelligent surfaces (RIS) can improve signal propagation environments by adjusting the phase of the incident signal. However, optimizing the phase shifts jointly with the beamforming vector at the access point is challenging due to the non-convex objective function and constraints. In this study, we propose an algorithm based on weighted minimum mean square error optimization and power iteration to maximize the weighted sum rate (WSR) of a RIS-assisted downlink multi-user multiple-input single-output system. To further improve performance, a model-driven deep learning (DL) approach is designed, where trainable variables and graph neural networks are introduced to accelerate the convergence of the proposed algorithm. We also extend the proposed method to include beamforming with imperfect channel state information and derive a two-timescale stochastic optimization algorithm. Simulation results show that the proposed algorithm outperforms state-of-the-art algorithms in terms of complexity and WSR. Specifically, the model-driven DL approach has a runtime that is approximately 3% of the state-of-the-art algorithm to achieve the same performance. Additionally, the proposed algorithm with 2-bit phase shifters outperforms the compared algorithm with continuous phase shift.

eess.SP↗

arXiv2023

Decoding trust: A reinforcement learning perspective

Behavioral experiments on the trust game have shown that trust and trustworthiness are universal among human beings, contradicting the prediction by assuming \emph{Homo economicus} in orthodox Economics. This means some mechanism must be at work that favors their emergence. Most previous explanations however need to resort to some factors based upon imitative learning, a simple version of social learning. Here, we turn to the paradigm of reinforcement learning, where individuals update their strategies by evaluating the long-term return through accumulated experience. Specifically, we investigate the trust game with the Q-learning algorithm, where each participant is associated with two evolving Q-tables that guide one's decision making as trustor and trustee respectively. In the pairwise scenario, we reveal that high levels of trust and trustworthiness emerge when individuals appreciate both their historical experience and returns in the future. Mechanistically, the evolution of the Q-tables shows a crossover that resembles human's psychological changes. We also provide the phase diagram for the game parameters, where the boundary analysis is conducted. These findings are robust when the scenario is extended to a latticed population. Our results thus provide a natural explanation for the emergence of trust and trustworthiness without external factors involved. More importantly, the proposed paradigm shows the potential in deciphering many puzzles in human behaviors.

q-bio.PE↗

arXiv2023

Single-Chip Silicon Photonic Processor for Analog Optical and Microwave Signals

The explosion of data volume in communications, AI training, and cloud computing requires efficient data handling, which is typically stored as digital electrical information and transmitted as wireless radio frequency (RF) signals or light waves in optical fibres. Today's communications systems mostly treat the RF and optical signals separately, which results in unnecessary conversion losses and increased cost. In this work, we report the first fully on-chip signal processor for high-speed RF and optical signals based on a silicon photonic circuit. Our chip is capable of both generation and detection of analog electrical and optical signals, and can program a user-defined filter response in both domains. The single silicon photonic chip integrates all essential components like modulators, optical filters, and photodetectors, as well as tunable lasers enabled by transfer-printed Indium Phosphide (InP) optical amplifiers. The system's configuration is locally programmed through thermo-optic phase shifters and monitored by photodetectors. We demonstrate our chip's capabilities with different combinations of RF and optical signal processing functions, including optical and RF signal generation and filtering. This represents a key step towards compact microwave photonic systems for future wireless communication and sensing applications.

physics.optics↗

arXiv2023

Probabilistic and Semantic Descriptions of Image Manifolds and Their Applications

This paper begins with a description of methods for estimating image probability density functions that reflects the observation that such data is usually constrained to lie in restricted regions of the high-dimensional image space-not every pattern of pixels is an image. It is common to say that images lie on a lower-dimensional manifold in the high-dimensional space. However, it is not the case that all points on the manifold have an equal probability of being images. Images are unevenly distributed on the manifold, and our task is to devise ways to model this distribution as a probability distribution. We therefore consider popular generative models. For our purposes, generative/probabilistic models should have the properties of 1) sample generation: the possibility to sample from this distribution with the modelled density function, and 2) probability computation: given a previously unseen sample from the dataset of interest, one should be able to compute its probability, at least up to a normalising constant. To this end, we investigate the use of methods such as normalising flow and diffusion models. We then show how semantic interpretations are used to describe points on the manifold. To achieve this, we consider an emergent language framework that uses variational encoders for a disentangled representation of points that reside on a given manifold. Trajectories between points on a manifold can then be described as evolving semantic descriptions. We also show that such probabilistic descriptions (bounded) can be used to improve semantic consistency by constructing defences against adversarial attacks. We evaluate our methods with improved semantic robustness and OoD detection capability, explainable and editable semantic interpolation, and improved classification accuracy under patch attacks. We also discuss the limitation in diffusion models.

cs.CV↗

arXiv2023

More Ramsey theory for highly connected monochromatic subgraphs

An infinite graph is said to be highly connected if the induced subgraph on the complement of any set of vertices of smaller size is connected. We continue the study of weaker versions of Ramsey Theorem on uncountable cardinals asserting that if we color edges of the complete graph we can find a large highly connected monochromatic subgraph. In particular, several questions of Bergfalk, Hrušák and Shelah are answered by showing that assuming the consistency of suitable large cardinals the following are relatively consistent with $\mathsf{ZFC}$: $κ\to_{hc} (κ)^2_ω$ for every regular cardinal $κ\geq \aleph_2$ and $\neg\mathsf{CH}+ \aleph_2 \to_{hc} (\aleph_1)^2_ω$. Building on a work of Lambie-Hanson, we also show that $\aleph_2 \to_{hc} [\aleph_2]^2_{ω,2}$ is consistent with $\neg\mathsf{CH}$. To prove these results, we use the existence of ideals with strong combinatorial properties after collapsing suitable large cardinals.

math.LO↗

arXiv2023

PT-Tuning: Bridging the Gap between Time Series Masked Reconstruction and Forecasting via Prompt Token Tuning

Self-supervised learning has been actively studied in time series domain recently, especially for masked reconstruction. Most of these methods follow the "Pre-training + Fine-tuning" paradigm in which a new decoder replaces the pre-trained decoder to fit for a specific downstream task, leading to inconsistency of upstream and downstream tasks. In this paper, we first point out that the unification of task objectives and adaptation for task difficulty are critical for bridging the gap between time series masked reconstruction and forecasting. By reserving the pre-trained mask token during fine-tuning stage, the forecasting task can be taken as a special case of masked reconstruction, where the future values are masked and reconstructed based on history values. It guarantees the consistency of task objectives but there is still a gap in task difficulty. Because masked reconstruction can utilize contextual information while forecasting can only use historical information to reconstruct. To further mitigate the existed gap, we propose a simple yet effective prompt token tuning (PT-Tuning) paradigm, in which all pre-trained parameters are frozen and only a few trainable prompt tokens are added to extended mask tokens in element-wise manner. Extensive experiments on real-world datasets demonstrate the superiority of our proposed paradigm with state-of-the-art performance compared to representation learning and end-to-end supervised forecasting methods.

cs.LG↗

arXiv2023

Analytical Study of KOH Wet Etch Surface Passivation for III-Nitride Micropillars

III-Nitride micropillar structures show great promise for applications in micro light-emitting diodes and vertical power transistors due to their excellent scalability and outstanding electrical properties. Typically, III-Nitride micropillars are fabricated through a top-down approach using reactive ion etch which leads to roughened, non-vertical sidewalls that results in significant performance degradation. Thus, it is essential to remove this plasma etch induced surface damage. Here, we show that potassium hydroxide (KOH) acts as a crystallographic etchant for III-Nitride micropillars, preferentially exposing the vertical <1-100> m-plane, and effectively removing dry etch damage and reducing the structure diameter at up to 36.6 nm/min. Both KOH solution temperature and concentration have a dramatic effect on this wet etch progression. We found that a solution of 20% AZ400K (2% KOH) at 90 C is effective at producing smooth, highly vertical sidewalls with RMS surface roughness as low as 2.59 nm, ideal for high-performance electronic and optoelectronic devices.

physics.app-ph↗

arXiv2023

Multi-grained Hypergraph Interest Modeling for Conversational Recommendation

Conversational recommender system (CRS) interacts with users through multi-turn dialogues in natural language, which aims to provide high-quality recommendations for user's instant information need. Although great efforts have been made to develop effective CRS, most of them still focus on the contextual information from the current dialogue, usually suffering from the data scarcity issue. Therefore, we consider leveraging historical dialogue data to enrich the limited contexts of the current dialogue session. In this paper, we propose a novel multi-grained hypergraph interest modeling approach to capture user interest beneath intricate historical data from different perspectives. As the core idea, we employ hypergraph to represent complicated semantic relations underlying historical dialogues. In our approach, we first employ the hypergraph structure to model users' historical dialogue sessions and form a session-based hypergraph, which captures coarse-grained, session-level relations. Second, to alleviate the issue of data scarcity, we use an external knowledge graph and construct a knowledge-based hypergraph considering fine-grained, entity-level semantics. We further conduct multi-grained hypergraph convolution on the two kinds of hypergraphs, and utilize the enhanced representations to develop interest-aware CRS. Extensive experiments on two benchmarks ReDial and TG-ReDial validate the effectiveness of our approach on both recommendation and conversation tasks. Code is available at: https://github.com/RUCAIBox/MHIM.

cs.IR↗

arXiv2023

Network Intrusion Detection with Edge-Directed Graph Multi-Head Attention Networks

A network intrusion usually involves a number of network locations. Data flow (including the data generated by intrusion behaviors) among these locations (usually represented by IP addresses) naturally forms a graph. Thus, graph neural networks (GNNs) have been used in the construction of intrusion detection models in recent years since they have an excellent ability to capture graph topological features of intrusion data flow. However, existing GNN models treat node mean aggregation equally in node information aggregation. In reality, the correlations of nodes and their neighbors as well as the linked edges are different. Assigning higher weights to nodes and edges with high similarity can highlight the correlation among them, which will enhance the accuracy and expressiveness of the model. To this end, this paper proposes novel Edge-Directed Graph Multi-Head Attention Networks (EDGMAT) for network intrusion detection. The proposed EDGMAT model introduces a multi-head attention mechanism into the intrusion detection model. Additional weight learning is realized through the combination of a multi-head attention mechanism and edge features. Weighted aggregation makes better use of the relationship between different network traffic data. Experimental results on four recent NIDS benchmark datasets show that the performance of EDGMAT in terms of weighted F1-Score is significantly better than that of four state-of-the-art models in multi-class detection tasks.

cs.CR↗

arXiv2023

AniPixel: Towards Animatable Pixel-Aligned Human Avatar

Although human reconstruction typically results in human-specific avatars, recent 3D scene reconstruction techniques utilizing pixel-aligned features show promise in generalizing to new scenes. Applying these techniques to human avatar reconstruction can result in a volumetric avatar with generalizability but limited animatability due to rendering only being possible for static representations. In this paper, we propose AniPixel, a novel animatable and generalizable human avatar reconstruction method that leverages pixel-aligned features for body geometry prediction and RGB color blending. Technically, to align the canonical space with the target space and the observation space, we propose a bidirectional neural skinning field based on skeleton-driven deformation to establish the target-to-canonical and canonical-to-observation correspondences. Then, we disentangle the canonical body geometry into a normalized neutral-sized body and a subject-specific residual for better generalizability. As the geometry and appearance are closely related, we introduce pixel-aligned features to facilitate the body geometry prediction and detailed surface normals to reinforce the RGB color blending. We also devise a pose-dependent and view direction-related shading module to represent the local illumination variance. Experiments show that AniPixel renders comparable novel views while delivering better novel pose animation results than state-of-the-art methods.

cs.CV↗

arXiv2023

On the measure concentration of infinitely divisible distributions

Let ${\cal I}$ be the set of all infinitely divisible random variables\ with finite second moments, ${\cal I}_0=\{X\in{\cal I}:{\rm Var}(X)>0\}$, $P_{\cal I}=\inf_{X\in{\cal I}}P\{|X-E[X]|\le \sqrt{{\rm Var}(X)}\}$ and $P_{{\cal I}_0}=\inf_{X\in{\cal I}_0} P\{|X-E[X]|< \sqrt{{\rm Var}(X)}\}$. Firstly, we prove that $P_{\cal I}\ge P_{{\cal I}_0}>0$. Secondly, we find the exact values of $\inf_{X\in{\cal J}}P\{|X-E[X]|\le \sqrt{{\rm Var}(X)}\}$ and $\inf_{X\in\cal J} P\{|X-E[X]|< \sqrt{{\rm Var}(X)}\}$ for the cases that $\cal J$ is the set of all geometric random variables, symmetric geometric random variables, Poisson random variables and symmetric Poisson random variables, respectively. As a consequence, we obtain that $P_{\cal I}\le e^{-1}\sum_{k=0}^{\infty}\frac{1}{2^{2k}(k!)^2}\approx 0.46576$ and $P_{{\cal I}_0}\le e^{-1}\approx 0.36788$.

math.PR↗