arXiv subjects

Zhijun Tu

Publications and source records attributed to Zhijun Tu.

At least 37 records · Page 2Linked to original sources

arXiv2025

Autoregressive Image Generation with Vision Full-view Prompt

In autoregressive (AR) image generation, models based on the 'next-token prediction' paradigm of LLMs have shown comparable performance to diffusion models by reducing inductive biases. However, directly applying LLMs to complex image generation can struggle with reconstructing the image's structure and details, impacting the generation's accuracy and stability. Additionally, the 'next-token prediction' paradigm in the AR model does not align with the contextual scanning and logical reasoning processes involved in human visual perception, limiting effective image generation. Prompt engineering, as a key technique for guiding LLMs, leverages specifically designed prompts to improve model performance on complex natural language processing (NLP) tasks, enhancing accuracy and stability of generation while maintaining contextual coherence and logical consistency, similar to human reasoning. Inspired by prompt engineering from the field of NLP, we propose Vision Full-view prompt (VF prompt) to enhance autoregressive image generation. Specifically, we design specialized image-related VF prompts for AR image generation to simulate the process of human image creation. This enhances contextual logic ability by allowing the model to first perceive overall distribution information before generating the image, and improve generation stability by increasing the inference steps. Compared to the AR method without VF prompts, our method shows outstanding performance and achieves an approximate improvement of 20%.

cs.CV↗

arXiv2025

CBQ: Cross-Block Quantization for Large Language Models

Post-training quantization (PTQ) has played a key role in compressing large language models (LLMs) with ultra-low costs. However, existing PTQ methods only focus on handling the outliers within one layer or one block, which ignores the dependency of blocks and leads to severe performance degradation in low-bit settings. In this paper, we propose CBQ, a cross-block reconstruction-based PTQ method for LLMs. CBQ employs a cross-block dependency using a homologous reconstruction scheme, establishing long-range dependencies across multiple blocks to minimize error accumulation. Furthermore, CBQ incorporates a coarse-to-fine preprocessing (CFP) strategy for suppressing weight and activation outliers, coupled with an adaptive LoRA-Rounding technique for precise weight quantization. These innovations enable CBQ to not only handle extreme outliers effectively but also improve overall quantization accuracy. Extensive experiments show that CBQ achieves superior low-bit quantization (W4A4, W4A8, W2A16) and outperforms existing state-of-the-art methods across various LLMs and datasets. Notably, CBQ quantizes the 4-bit LLAMA1-65B model within only 4.3 hours on a single GPU, achieving a commendable tradeoff between performance and quantization efficiency.

cs.LG↗

arXiv2025

Three Brown Dwarfs Masquerading as High-Redshift Galaxies in JWST Observations

We report the spectroscopic identification of three brown dwarf candidates -- o005_s41280, o006_s00089, and o006_s35616 -- discovered in the RUBIES using James Webb Space Telescope (JWST) Near-Infrared Spectrograph (NIRSpec) PRISM/CLEAR spectroscopy. We fit these sources with multiple substellar atmosphere models and present the atmospheric parameters, including effective temperature ($T_\mathrm{eff}$), surface gravity, and other derived properties. The results suggest that o005_s41280 and o006_s35616, with $T_\mathrm{eff}$ in the ranges of 2100--2300 K and 1800--2000 K, are likely L dwarfs, while o006_s00089, with $T_\mathrm{eff} < 1000$ K, is consistent with a late T dwarf classification. The best-fit model spectra provide a reasonable match to the observed spectra. However, distinct residuals exist in the $Y$, $J$, and $H$ bands for the two L dwarf candidates, particularly for o006_s35616. Incorporating the extinction parameter into the fitting process can significantly reduce these residuals. The distance estimates indicate that these candidates are about 2 kpc away. The analysis of the color-color diagram using multiple JWST NIRcam photometry suggests that cooler T dwarfs, such as o006_s00089, overlap with little red dots (LRDs), while hotter L dwarfs, like o005_s41280 and o006_s35616, tend to contaminate the high-redshift galaxy cluster. These findings suggest a brown dwarf contamination rate of approximately 0.1% in extragalactic deep field surveys, with L dwarfs being more frequently detected than cooler T and Y dwarfs.

astro-ph.SR↗

arXiv2025

PyEMILI: A New Generation Computer-aided Spectral Line Identifier

Deep high-dispersion spectroscopy of Galactic photoionized gaseous nebulae, mainly planetary nebulae and HII regions, has revealed numerous emission lines. As a key step of spectral analysis, identification of emission lines hitherto has mostly been done manually, which is a tedious task, given that each line needs to be carefully checked against huge volumes of atomic transition/spectroscopic database to reach a reliable assignment of identity. Using Python, we have developed a line-identification code PyEMILI, which is a significant improvement over the Fortran-based package EMILI introduced ~20 years ago. In our new code PyEMILI, the major shortcomings in EMILI's line-identification technique have been amended. Moreover, the atomic transition database utilized by PyEMILI was adopted from Atomic Line List v3.00b4 but greatly supplemented with theoretical transition data from the literature. The effective recombination coefficients of the CII, OII, NII and NeII nebular lines are collected from the literature to form a subset of the atomic transition database to aid identification of faint optical recombination lines in the spectra of PNe and HII regions. PyEMILI is tested using the deep, high-dispersion spectra of two Galactic PNe, Hf2-2 and IC418, and gives better results of line identification than EMILI does. We also ran PyEMILI on the optical spectrum of a late-type [WC11] star UVQS J060819.93-715737.4 recently discovered in the Large Magellanic Cloud, and our results agree well with the previous manual identifications. The new identifier PyEMILI is applicable to not only emission-line nebulae but also emission stars, such as Wolf-Rayet stars.

astro-ph.SR↗

arXiv2024

U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformers

Diffusion Transformers (DiTs) introduce the transformer architecture to diffusion tasks for latent-space image generation. With an isotropic architecture that chains a series of transformer blocks, DiTs demonstrate competitive performance and good scalability; but meanwhile, the abandonment of U-Net by DiTs and their following improvements is worth rethinking. To this end, we conduct a simple toy experiment by comparing a U-Net architectured DiT with an isotropic one. It turns out that the U-Net architecture only gain a slight advantage amid the U-Net inductive bias, indicating potential redundancies within the U-Net-style DiT. Inspired by the discovery that U-Net backbone features are low-frequency-dominated, we perform token downsampling on the query-key-value tuple for self-attention that bring further improvements despite a considerable amount of reduction in computation. Based on self-attention with downsampled tokens, we propose a series of U-shaped DiTs (U-DiTs) in the paper and conduct extensive experiments to demonstrate the extraordinary performance of U-DiT models. The proposed U-DiT could outperform DiT-XL/2 with only 1/6 of its computation cost. Codes are available at https://github.com/YuchuanTian/U-DiT.

cs.CV↗

arXiv2024

Interstitial anionic electrons involved superconductivity and T-linear resistivity behavior in electride La3In

Electrides are unique materials because of the existence of interstitial anionic electrons (IAEs). Due to these loosely bound IAEs and their strong interaction with the framework of cations, electrides can host superconductivity with rather high Tc, especially under high pressure, as predicted in theory. However, the experimental observations of superconductivity in electrides are very rare, let alone the detailed studies on intrinsic properties of single crystals. Here, we report the superconducting and normal-state properties of electride La3In single crystals. La3In shows a type-II superconductivity with Tc ~ 9.4 K and a T-linear resistivity in a wide temperature range. Experimental measurements and theoretical calculations suggest that the relatively high Tc could be ascribed to the high density of states around the Fermi level caused by short flat bands along R-M direction and the strong electron-phonon coupling, partially derived from the IAEs. Meanwhile, the T-linear resistivity may reflect the significant electronic correlation effect in this material. These findings will shed light on understanding the role of IAEs in superconductivity and open a promising way to explore high-temperature superconductors in electrides.

cond-mat.supr-con↗

arXiv2024

Orbital-selective effect of spin reorientation on the Dirac fermions in a non-charge-ordered kagome ferromagnet Fe$_3$Ge

Kagome magnets provide a fascinating platform for the realization of correlated topological quantum phases under various magnetic ground states. However, the effect of the magnetic spin configurations on the characteristic electronic structure of the kagome lattice layer remains elusive. Here, utilizing angle-resolved photoemission spectroscopy and density functional theory calculations, we report the spectroscopic evidence for the spin-reorientation effect of a kagome ferromagnet Fe$_3$Ge, which is composed solely of kagome planes. As the Fe moments cant from the $c$ axis into the $ab$ plane upon cooling, the two kinds of kagome-derived Dirac fermions respond quite differently. The one with less-dispersive bands ($k_z$ $\sim$ 0) containing the $3d_{z^2}$ orbitals evolves from gapped into nearly gapless, while the other with linear dispersions ($k_z$ $\sim$ $π$) embracing the $3d_{xz}$/$3d_{yz}$ components remains intact, suggesting that the effect of spin reorientation on the Dirac fermions has an orbital selectivity. Moreover, we demonstrate that there is no signature of charge order formation in Fe$_3$Ge, contrasting with its sibling compound FeGe, a newly established charge-density-wave kagome magnet.

cond-mat.str-el↗

arXiv2024

Effective Diffusion Transformer Architecture for Image Super-Resolution

Recent advances indicate that diffusion models hold great promise in image super-resolution. While the latest methods are primarily based on latent diffusion models with convolutional neural networks, there are few attempts to explore transformers, which have demonstrated remarkable performance in image generation. In this work, we design an effective diffusion transformer for image super-resolution (DiT-SR) that achieves the visual quality of prior-based methods, but through a training-from-scratch manner. In practice, DiT-SR leverages an overall U-shaped architecture, and adopts a uniform isotropic design for all the transformer blocks across different stages. The former facilitates multi-scale hierarchical feature extraction, while the latter reallocates the computational resources to critical layers to further enhance performance. Moreover, we thoroughly analyze the limitation of the widely used AdaLN, and present a frequency-adaptive time-step conditioning module, enhancing the model's capacity to process distinct frequency information at different time steps. Extensive experiments demonstrate that DiT-SR outperforms the existing training-from-scratch diffusion-based SR methods significantly, and even beats some of the prior-based methods on pretrained Stable Diffusion, proving the superiority of diffusion transformer in image super-resolution.

cs.CV↗

arXiv2024

Physical Parameters and Properties of 20 Cold Brown Dwarfs in JWST

We present a comprehensive analysis of 20 T and Y dwarfs using spectroscopy from the NIRSpec CLEAR/PRISM and MIRI LRS instruments on the James Webb Space Telescope. To characterize the atmospheric parameters, we utilize two atmospheric model grids: the Sonora Elf Owl and ATMO2020++. The effective temperatures derived from the two models are relatively consistent, and metallicities are both close to solar values. However, significant discrepancies are found in other parameters, particularly in surface gravity, with the values obtained from the Sonora Elf Owl models typically being about 1 dex lower than those from the ATMO2020++ models. Further comparisons using the ATMO2020 models highlight that the adiabatic convective process introduced in the ATMO2020++ models has a significant impact on the determination of surface gravity. Using the fitted effective temperatures and absolute parallaxes from the literature, we derive radii for the brown dwarfs, which range from approximately 0.8 to 1.2 $R_{\mathrm{Jup}}$. The estimated masses and ages, derived using evolutionary tracks, indicate that most brown dwarfs in our sample have masses below 30 $M_{\mathrm{Jup}}$ and are younger than 6 Gyr. Specifically, Y dwarfs have masses ranging from 2 to 20 $M_{\mathrm{Jup}}$ and ages between 0.1 and 6.7 Gyr. In addition, We discuss the determination of atmospheric parameters using only NIRSpec or MIRI spectra. Comparisons with results from the combined spectra show that the effective temperatures and surface gravities derived solely from NIRSpec spectra are largely consistent with those obtained from the combined spectra.

astro-ph.SR↗

arXiv2024

One Step Diffusion-based Super-Resolution with Time-Aware Distillation

Diffusion-based image super-resolution (SR) methods have shown promise in reconstructing high-resolution images with fine details from low-resolution counterparts. However, these approaches typically require tens or even hundreds of iterative samplings, resulting in significant latency. Recently, techniques have been devised to enhance the sampling efficiency of diffusion-based SR models via knowledge distillation. Nonetheless, when aligning the knowledge of student and teacher models, these solutions either solely rely on pixel-level loss constraints or neglect the fact that diffusion models prioritize varying levels of information at different time steps. To accomplish effective and efficient image super-resolution, we propose a time-aware diffusion distillation method, named TAD-SR. Specifically, we introduce a novel score distillation strategy to align the data distribution between the outputs of the student and teacher models after minor noise perturbation. This distillation strategy enables the student network to concentrate more on the high-frequency details. Furthermore, to mitigate performance limitations stemming from distillation, we integrate a latent adversarial loss and devise a time-aware discriminator that leverages diffusion priors to effectively distinguish between real images and generated images. Extensive experiments conducted on synthetic and real-world datasets demonstrate that the proposed method achieves comparable or even superior performance compared to both previous state-of-the-art (SOTA) methods and the teacher model in just one sampling step. Codes are available at https://github.com/LearningHx/TAD-SR.

cs.CV↗

arXiv2024

LIPT: Latency-aware Image Processing Transformer

Transformer is leading a trend in the field of image processing. Despite the great success that existing lightweight image processing transformers have achieved, they are tailored to FLOPs or parameters reduction, rather than practical inference acceleration. In this paper, we present a latency-aware image processing transformer, termed LIPT. We devise the low-latency proportion LIPT block that substitutes memory-intensive operators with the combination of self-attention and convolutions to achieve practical speedup. Specifically, we propose a novel non-volatile sparse masking self-attention (NVSM-SA) that utilizes a pre-computing sparse mask to capture contextual information from a larger window with no extra computation overload. Besides, a high-frequency reparameterization module (HRM) is proposed to make LIPT block reparameterization friendly, which improves the model's detail reconstruction capability. Extensive experiments on multiple image processing tasks (e.g., image super-resolution (SR), JPEG artifact reduction, and image denoising) demonstrate the superiority of LIPT on both latency and PSNR. LIPT achieves real-time GPU inference with state-of-the-art performance on multiple image SR benchmarks.

cs.CV↗

arXiv2024

Data Upcycling Knowledge Distillation for Image Super-Resolution

Knowledge distillation (KD) compresses deep neural networks by transferring task-related knowledge from cumbersome pre-trained teacher models to compact student models. However, current KD methods for super-resolution (SR) networks overlook the nature of SR task that the outputs of the teacher model are noisy approximations to the ground-truth distribution of high-quality images (GT), which shades the teacher model's knowledge to result in limited KD effects. To utilize the teacher model beyond the GT upper-bound, we present the Data Upcycling Knowledge Distillation (DUKD), to transfer the teacher model's knowledge to the student model through the upcycled in-domain data derived from training data. Besides, we impose label consistency regularization to KD for SR by the paired invertible augmentations to improve the student model's performance and robustness. Comprehensive experiments demonstrate that the DUKD method significantly outperforms previous arts on several SR tasks.

cs.CV↗

arXiv2024

A Survey on Transformer Compression

Transformer plays a vital role in the realms of natural language processing (NLP) and computer vision (CV), specially for constructing large language models (LLM) and large vision models (LVM). Model compression methods reduce the memory and computational cost of Transformer, which is a necessary step to implement large language/vision models on practical devices. Given the unique architecture of Transformer, featuring alternative attention and feedforward neural network (FFN) modules, specific compression techniques are usually required. The efficiency of these compression methods is also paramount, as retraining large models on the entire training dataset is usually impractical. This survey provides a comprehensive review of recent compression methods, with a specific focus on their application to Transformer-based models. The compression methods are primarily categorized into pruning, quantization, knowledge distillation, and efficient architecture design (Mamba, RetNet, RWKV, etc.). In each category, we discuss compression methods for both language and vision tasks, highlighting common underlying principles. Finally, we delve into the relation between various compression methods, and discuss further directions in this domain.

cs.LG↗

arXiv2024

IPT-V2: Efficient Image Processing Transformer using Hierarchical Attentions

Recent advances have demonstrated the powerful capability of transformer architecture in image restoration. However, our analysis indicates that existing transformerbased methods can not establish both exact global and local dependencies simultaneously, which are much critical to restore the details and missing content of degraded images. To this end, we present an efficient image processing transformer architecture with hierarchical attentions, called IPTV2, adopting a focal context self-attention (FCSA) and a global grid self-attention (GGSA) to obtain adequate token interactions in local and global receptive fields. Specifically, FCSA applies the shifted window mechanism into the channel self-attention, helps capture the local context and mutual interaction across channels. And GGSA constructs long-range dependencies in the cross-window grid, aggregates global information in spatial dimension. Moreover, we introduce structural re-parameterization technique to feed-forward network to further improve the model capability. Extensive experiments demonstrate that our proposed IPT-V2 achieves state-of-the-art results on various image processing tasks, covering denoising, deblurring, deraining and obtains much better trade-off for performance and computational complexity than previous methods. Besides, we extend our method to image generation as latent diffusion backbone, and significantly outperforms DiTs.

cs.CV↗

arXiv2023

GenImage: A Million-Scale Benchmark for Detecting AI-Generated Image

The extraordinary ability of generative models to generate photographic images has intensified concerns about the spread of disinformation, thereby leading to the demand for detectors capable of distinguishing between AI-generated fake images and real images. However, the lack of large datasets containing images from the most advanced image generators poses an obstacle to the development of such detectors. In this paper, we introduce the GenImage dataset, which has the following advantages: 1) Plenty of Images, including over one million pairs of AI-generated fake images and collected real images. 2) Rich Image Content, encompassing a broad range of image classes. 3) State-of-the-art Generators, synthesizing images with advanced diffusion models and GANs. The aforementioned advantages allow the detectors trained on GenImage to undergo a thorough evaluation and demonstrate strong applicability to diverse images. We conduct a comprehensive analysis of the dataset and propose two tasks for evaluating the detection method in resembling real-world scenarios. The cross-generator image classification task measures the performance of a detector trained on one generator when tested on the others. The degraded image classification task assesses the capability of the detectors in handling degraded images such as low-resolution, blurred, and compressed images. With the GenImage dataset, researchers can effectively expedite the development and evaluation of superior AI-generated image detectors in comparison to prevailing methodologies.

cs.CV↗

arXiv2022

AdaBin: Improving Binary Neural Networks with Adaptive Binary Sets

This paper studies the Binary Neural Networks (BNNs) in which weights and activations are both binarized into 1-bit values, thus greatly reducing the memory usage and computational complexity. Since the modern deep neural networks are of sophisticated design with complex architecture for the accuracy reason, the diversity on distributions of weights and activations is very high. Therefore, the conventional sign function cannot be well used for effectively binarizing full-precision values in BNNs. To this end, we present a simple yet effective approach called AdaBin to adaptively obtain the optimal binary sets $\{b_1, b_2\}$ ($b_1, b_2\in \mathbb{R}$) of weights and activations for each layer instead of a fixed set (\textit{i.e.}, $\{-1, +1\}$). In this way, the proposed method can better fit different distributions and increase the representation ability of binarized features. In practice, we use the center position and distance of 1-bit values to define a new binary quantization function. For the weights, we propose an equalization method to align the symmetrical center of binary distribution to real-valued distribution, and minimize the Kullback-Leibler divergence of them. Meanwhile, we introduce a gradient-based optimization method to get these two parameters for activations, which are jointly trained in an end-to-end manner. Experimental results on benchmark models and datasets demonstrate that the proposed AdaBin is able to achieve state-of-the-art performance. For instance, we obtain a 66.4% Top-1 accuracy on the ImageNet using ResNet-18 architecture, and a 69.4 mAP on PASCAL VOC using SSD300. The PyTorch code is available at \url{https://github.com/huawei-noah/Efficient-Computing/tree/master/BinaryNetworks/AdaBin} and the MindSpore code is available at \url{https://gitee.com/mindspore/models/tree/master/research/cv/AdaBin}.

cs.CV↗

arXiv2022

Emergent nematicity and intrinsic vs. extrinsic electronic scattering processes in the kagome metal CsV$_3$Sb$_5$

Fermi surface fluctuations and lattice instabilities in the 2D metallic kagome superconductor CsV$_3$Sb$_5$ are elucidated via polarization-resolved Raman spectroscopy. The presence of a weak electronic continuum in high-quality samples marks the cross-over into the charge-density-wave (CDW) ordered phase, while impurity-rich samples promote strong defect-induced electronic scattering processes that affect the coherence of the CDW phase. CDW-induced phonon anomalies appear below $T_{\mathrm{CDW}}$, with emergent $C2$ symmetry for one of the CDW amplitude modes, alluding to nematicity. In conjunction with symmetry-breaking lattice distortions, a kink-like hardening of the A$_{1g}$ phonon energy at $T_{\mathrm{CDW}}$ signifies a concerted interplay of electronic correlations and electron-phonon coupling in the exotic CDW order.

cond-mat.str-el↗

arXiv2022

Microscopic evidence for anisotropic multigap superconductivity in the CsV$_3$Sb$_5$ kagome superconductor

The recently discovered kagome superconductor CsV$_3$Sb$_5$ ($T_c \simeq 2.5$ K) has been found to host charge order as well as a non-trivial band topology, encompassing multiple Dirac points and probable surface states. Such a complex and phenomenologically rich system is, therefore, an ideal playground for observing unusual electronic phases. Here, we report on microscopic studies of its anisotropic superconducting properties by means of transverse-field muon spin rotation ($μ$SR) experiments. The temperature dependences of the in-plane and out-of-plane components of the magnetic penetration depth $λ_{ab}^{-2}(T)$ and $λ_{c}^{-2}(T)$ indicate that the superconducting order parameter exhibits a two-gap ($s+s$)-wave symmetry, reflecting the multiple Fermi surfaces of CsV3Sb5. The multiband nature of its superconductivity is further validated by the different temperature dependences of the anisotropic magnetic penetration depth $γ_λ(T)$ and upper critical field $γ_{\rm B_{c2}}(T)$, both in close analogy with the well known two-gap superconductor MgB$_2$. Remarkably, the high value of the $T_c/λ^{-2}(0)$ ratio in both field orientations strongly suggests the unconventional nature of superconductivity. The relaxation rates obtained from zero field $μ$SR experiments do not show noticeable change across the superconducting transition, indicating that superconductivity does not break time reversal symmetry.

cond-mat.supr-con↗