Searcharxiv⌕ Search

arXiv subjects

Zhenwei Wang

Publications and source records attributed to Zhenwei Wang.

28 records · Page 2Linked to original sources

PM2: A New Prompting Multi-modal Model Paradigm for Few-shot Medical Image Classification

Few-shot learning has been successfully applied to medical image classification as only very few medical examples are available for training. Due to the challenging problem of limited number of annotated medical images, image representations should not be solely derived from a single image modality which is insufficient for characterizing concept classes. In this paper, we propose a new prompting multi-modal model paradigm on medical image classification based on multi-modal foundation models, called PM2. Besides image modality,PM2 introduces another supplementary text input, known as prompt, to further describe corresponding image or concept classes and facilitate few-shot learning across diverse modalities. To better explore the potential of prompt engineering, we empirically investigate five distinct prompt schemes under the new paradigm. Furthermore, linear probing in multi-modal models acts as a linear classification head taking as input only class token, which ignores completely merits of rich statistics inherent in high-level visual tokens. Thus, we alternatively perform a linear classification on feature distribution of visual tokens and class token simultaneously. To effectively mine such rich statistics, a global covariance pooling with efficient matrix power normalization is used to aggregate visual tokens. Then we study and combine two classification heads. One is shared for class token of image from vision encoder and prompt representation encoded by text encoder. The other is to classification on feature distribution of visual tokens from vision encoder. Extensive experiments on three medical datasets show that our PM2 significantly outperforms counterparts regardless of prompt schemes and achieves state-of-the-art performance.

cs.CV↗

GASE: Graph Attention Sampling with Edges Fusion for Solving Vehicle Routing Problems

Learning-based methods have become increasingly popular for solving vehicle routing problems due to their near-optimal performance and fast inference speed. Among them, the combination of deep reinforcement learning and graph representation allows for the abstraction of node topology structures and features in an encoder-decoder style. Such an approach makes it possible to solve routing problems end-to-end without needing complicated heuristic operators designed by domain experts. Existing research studies have been focusing on novel encoding and decoding structures via various neural network models to enhance the node embedding representation. Despite the sophisticated approaches applied, there is a noticeable lack of consideration for the graph-theoretic properties inherent to routing problems. Moreover, the potential ramifications of inter-nodal interactions on the decision-making efficacy of the models have not been adequately explored. To bridge this gap, we propose an adaptive Graph Attention Sampling with the Edges Fusion framework (GASE),where nodes' embedding is determined through attention calculation from certain highly correlated neighbourhoods and edges, utilizing a filtered adjacency matrix. In detail, the selections of particular neighbours and adjacency edges are led by a multi-head attention mechanism, contributing directly to the message passing and node embedding in graph attention sampling networks. Furthermore, we incorporate an adaptive actor-critic algorithm with policy improvements to expedite the training convergence. We then conduct comprehensive experiments against baseline methods on learning-based VRP tasks from different perspectives. Our proposed model outperforms the existing methods by 2.08\%-6.23\% and shows stronger generalization ability, achieving state-of-the-art performance on randomly generated instances and real-world datasets.

cs.LG↗

ThemeStation: Generating Theme-Aware 3D Assets from Few Exemplars

Real-world applications often require a large gallery of 3D assets that share a consistent theme. While remarkable advances have been made in general 3D content creation from text or image, synthesizing customized 3D assets following the shared theme of input 3D exemplars remains an open and challenging problem. In this work, we present ThemeStation, a novel approach for theme-aware 3D-to-3D generation. ThemeStation synthesizes customized 3D assets based on given few exemplars with two goals: 1) unity for generating 3D assets that thematically align with the given exemplars and 2) diversity for generating 3D assets with a high degree of variations. To this end, we design a two-stage framework that draws a concept image first, followed by a reference-informed 3D modeling stage. We propose a novel dual score distillation (DSD) loss to jointly leverage priors from both the input exemplars and the synthesized concept image. Extensive experiments and user studies confirm that ThemeStation surpasses prior works in producing diverse theme-aware 3D models with impressive quality. ThemeStation also enables various applications such as controllable 3D-to-3D generation.

cs.CV↗

Recasting Regional Lighting for Shadow Removal

Removing shadows requires an understanding of both lighting conditions and object textures in a scene. Existing methods typically learn pixel-level color mappings between shadow and non-shadow images, in which the joint modeling of lighting and object textures is implicit and inadequate. We observe that in a shadow region, the degradation degree of object textures depends on the local illumination, while simply enhancing the local illumination cannot fully recover the attenuated textures. Based on this observation, we propose to condition the restoration of attenuated textures on the corrected local lighting in the shadow region. Specifically, We first design a shadow-aware decomposition network to estimate the illumination and reflectance layers of shadow regions explicitly. We then propose a novel bilateral correction network to recast the lighting of shadow regions in the illumination layer via a novel local lighting correction module, and to restore the textures conditioned on the corrected illumination layer via a novel illumination-guided texture restoration module. We further annotate pixel-wise shadow masks for the public SRD dataset, which originally contains only image pairs. Experiments on three benchmarks show that our method outperforms existing state-of-the-art shadow removal methods.

cs.CV↗

Language-based Photo Color Adjustment for Graphic Designs

Adjusting the photo color to associate with some design elements is an essential way for a graphic design to effectively deliver its message and make it aesthetically pleasing. However, existing tools and previous works face a dilemma between the ease of use and level of expressiveness. To this end, we introduce an interactive language-based approach for photo recoloring, which provides an intuitive system that can assist both experts and novices on graphic design. Given a graphic design containing a photo that needs to be recolored, our model can predict the source colors and the target regions, and then recolor the target regions with the source colors based on the given language-based instruction. The multi-granularity of the instruction allows diverse user intentions. The proposed novel task faces several unique challenges, including: 1) color accuracy for recoloring with exactly the same color from the target design element as specified by the user; 2) multi-granularity instructions for parsing instructions correctly to generate a specific result or multiple plausible ones; and 3) locality for recoloring in semantically meaningful local regions to preserve original image semantics. To address these challenges, we propose a model called LangRecol with two main components: the language-based source color prediction module and the semantic-palette-based photo recoloring module. We also introduce an approach for generating a synthetic graphic design dataset with instructions to enable model training. We evaluate our model via extensive experiments and user studies. We also discuss several practical applications, showing the effectiveness and practicality of our approach. Code and data for this paper are at: https://zhenwwang.github.io/langrecol.

cs.CV↗

Double Resonant Tunable Second Harmonic Generation in Two-dimensional Layered Materials through Band Nesting

We proposed a mechanism to generate giant anisotropic second harmonic nonlinear response via double resonance effect, achieved through band nesting via electronic bandstructure engineering. The ideal band setup would be a triplet of nested bands separated by the fundamental resonance energy, $\hbarω$. We demonstrate theoretically that the proposed phenomenon can be realized in bilayer SnS by band tuning with perpendicular electrical bias, which maximizes the second harmonic susceptibility by several orders of magnitude. Moreover, the tunability of the polarization anisotropy can be useful for realizing novel polarization-sensitive devices.

physics.optics↗

Electronic properties and quasi-particle model of monolayer MoSi$_2$N$_4$

By a combined study with first-principles calculations and symmetry analysis, we theoretically investigate the electronic properties of monolayer MoSi$_2$N$_4$. While the spin-orbital coupling results in bands splitting, the horizontal mirror symmetry locks the spin polarization along z-direction. In addition, a three-band tight-binding model is constructed to describe the low-energy quasi-particle states of monolayer MoSi$_2$N$_4$, which can be generalized to strained MoSi$_2$N$_4$ and its derivatives. The calculations using the tight-binding model show an undamped $\sqrt{q}$-dependent plasmon mode that agrees well with the results of first-principles calculations. Our model can be extended to be suitable for future theoretical and numerical studies of low-energy properties in MoSi$_2$N$_4$ family materials. Furthermore, the study of electronic properties of monolayer MoSi$_2$N$_4$ paves a way for its applications in spintronics and plasmonics.

cond-mat.mtrl-sci↗

DIR-DBTnet: Deep iterative reconstruction network for 3D digital breast tomosynthesis imaging

Purpose: The goal of this study is to develop a novel deep learning (DL) based reconstruction framework to improve the digital breast tomosynthesis (DBT) imaging performance. Methods: In this work, the DIR-DBTnet is developed for DBT image reconstruction by unrolling the standard iterative reconstruction algorithm within the deep learning framework. In particular, such network learns the regularizer and the iteration parameters automatically through network training with a large amount of simulated DBT data. Afterwards, both numerical and experimental data are used to evaluate its performance. Quantitative metrics such as the artifact spread function (ASF), breast density, and the signal difference to noise ratio (SDNR) are used for image quality assessment. Results: For both numerical and experimental data, the proposed DIR-DBTnet generates reduced in-plane shadow artifacts and out-of-plane artifacts compared with the filtered back projection (FBP) and total variation (TV) methods. Quantitatively, the full width half maximum (FWHM) of the measured ASF curve from the numerical data is 33.4% and 19.7% smaller than those obtained with the FBP and TV methods, respectively; the breast density of the network reconstructed DBT images is more accurate and consistent with the ground truth. Conclusions: In conclusion, a deep iterative reconstruction network, DIR-DBTnet, has been proposed. Both qualitative and quantitative analyses of the numerical and experimental results show superior DBT imaging performance than the FBP and iterative algorithms.

physics.med-ph↗

Large Fermi-arc and robust Weyl semimetal phase in Ag$_2$S

Three-dimensional Dirac and Weyl semimetals have attracted widespread interest in condensed matter physics and material science. Here, based on first-principles calculations and symmetry analysis, we report that Ag$_2$S with P2$_1$2$_1$2$_1$ symmetry is a topological Dirac semimetal in the absence of spin-orbit coupling (SOC). Every Dirac points are composed of two Weyl points with the same chirality overlapping in the momentum space. After taking the SOC into consideration, each Dirac point is split into two Weyl points. The Weyl points with the opposite chirality are far from each other in the momentum space. Therefore, the Weyl pairs are not easy to annihilate and robust in the Ag$_2$S compound, which also lead to long Fermi-arcs on material surface. The robustness of the Weyl points against the strain is discussed.

cond-mat.mtrl-sci↗

Topological Phase Transition in Layered XIn$_2$P$_2$ (X = Ca, Sr)

Based on fully relativistic first-principles calculations, we studied the topological properties of layered XIn$_2$P$_2$ (X = Ca, Sr). Band inversion can be induced by strain without SOC, forming one nodal ring in the k$_z$ = 0 plane, which is protected by the coexistence of time-reversal and mirror-reflection symmetry. Including SOC, a substantial band gap is opened along the nodal line and the line-node semimetal would evolve into a topological insulator. These results reveal a category of materials showing quantum phase transition from trivial semiconductor and topologically nontrivial insulator by the tuneable elastic strain engineering. Our investigations provide a new perspective about the formation of topological line-node semimetal under stain.

cond-mat.mtrl-sci↗