arXiv subjects

Huajun Sun

Publications and source records attributed to Huajun Sun.

5 recordsLinked to original sources

arXiv2026

SwiftKV: An Edge-Oriented Attention Algorithm and Multi-Head Accelerator for Fast, Efficient LLM Decoding

Edge acceleration for large language models is crucial for their widespread application; however, achieving fast attention inference and efficient decoding on resource-constrained edge accelerators remains challenging. This paper presents SwiftKV Attention, a per-token pipelined, low-latency single-pass attention inference algorithm, where every (kt, vt) in the KV cache is processed exactly once in a uniform per-token pipeline without score materialization, blockwise softmax, or a second pass, thereby enabling fast execution on edge accelerators with a single hardware set and no resource-intensive parallelism. Furthermore, to address the limited support for multi-head LLM decoding in existing accelerators, we design the SwiftKV-MHA accelerator, which enables high precision attention and low precision GEMV on the same processor array, achieving fast and efficient multi-head parallel decoding. Experimental results show that, on the edge accelerator, the SwiftKV Attention algorithm achieves a 7.16* speedup over native attention and significantly outperforms other attention algorithms. SwiftKV-MHA further reduces attention latency by 13.48*; under the same settings, it improves generation speed by 17.4% and increases token efficiency by 1.98* compared with state-of-the-art works.

cs.AR↗

arXiv2025

Weighted Mean Frequencies: a handcraft Fourier feature for 4D Flow MRI segmentation

In recent decades, the use of 4D Flow MRI images has enabled the quantification of velocity fields within a volume of interest and along the cardiac cycle. However, the lack of resolution and the presence of noise in these biomarkers are significant issues. As indicated by recent studies, it appears that biomarkers such as wall shear stress are particularly impacted by the poor resolution of vessel segmentation. The Phase Contrast Magnetic Resonance Angiography (PC-MRA) is the state-of-the-art method to facilitate segmentation. The objective of this work is to introduce a new handcraft feature that provides a novel visualisation of 4D Flow MRI images, which is useful in the segmentation task. This feature, termed Weighted Mean Frequencies (WMF), is capable of revealing the region in three dimensions where a voxel has been passed by pulsatile flow. Indeed, this feature is representative of the hull of all pulsatile velocity voxels. The value of the feature under discussion is illustrated by two experiments. The experiments involved segmenting 4D Flow MRI images using optimal thresholding and deep learning methods. The results obtained demonstrate a substantial enhancement in terms of IoU and Dice, with a respective increase of 0.12 and 0.13 in comparison with the PC-MRA feature, as evidenced by the deep learning task. This feature has the potential to yield valuable insights that could inform future segmentation processes in other vascular regions, such as the heart or the brain.

eess.IV↗

arXiv2023

Impact of Zr substitution on the electronic structure of ferroelectric hafnia

$\mathrm{HfO_2}$-based dielectrics are promising for nanoscale ferroelectric applications, and the most favorable material within the family is Zr-substituted hafnia, i.e., $\mathrm{Hf_{1-x}Zr_xO_2}$ (HZO). The extent of Zr substitution can be great, and x is commonly set to 0.5. However, the band gap of $\mathrm{ZrO_2}$ is lower than $\mathrm{HfO_2}$, thus it is uncertain how the Zr content should influence the electronic band structure of HZO. A reduced band gap is detrimental to the cycling endurance as charge injection and dielectric breakdown would become easier. Another issue is regarding the comparison on the band gaps between $\mathrm{HfO_2}$/$\mathrm{ZrO_2}$ superlattices and HZO solid-state solutions. In this work we systematically investigated the electronic structures of $\mathrm{HfO_2}$, $\mathrm{ZrO_2}$ and HZO using self-energy corrected density functional theory. In particular, the conduction band minimum of $Pca2_1$-$\mathrm{HfO_2}$ is found to lie at an ordinary k-point on the Brillouin zone border, not related to any interlines between high-symmetry k-points. Moreover, the rule of HZO band gap variation with respect to x has been extracted. The physical mechanisms for the exponential reduction regime and linear decay regime have been revealed. The band gaps of $\mathrm{HfO_2}$/$\mathrm{ZrO_2}$ ferroelectric superlattices are investigated in a systematic manner, and the reason why the superlattice could possess a band gap lower than that of $\mathrm{ZrO_2}$ is revealed through comprehensive analysis.

cond-mat.mtrl-sci↗

arXiv2022

Designing wake-up free ferroelectric capacitors based on the $\mathrm{HfO_2/ZrO_2}$ superlattice structure

The wake-up phenomenon widely exists in hafnia-based ferroelectric capacitors, which causes device parameter variation over time. Crystallization at higher temperatures have been reported to be effective in eliminating wake-up, but high temperature may yield the monoclinic phase or generate high concentration oxygen vacancies. In this work, a unidirectional annealing method is proposed for the crystallization of $\mathrm{Hf_{0.5}Zr_{0.5}O_2}$ (HZO) superlattice ferroelectrics, which involves heating from the $\mathrm{Pt/ZrO_2}$ interface side. Nanoscale $\mathrm{ZrO_2}$ is selected to resist the formation of monoclinic phase, and the chemically inert Pt electrode can avoid the continuous generation of oxygen vacancies during annealing. It is demonstrated that $\mathrm{600^oC}$ annealing only leads to a moderate content of monoclinic phase in HZO, and the TiN/HZO/Pt capacitor exhibits wake-up free nature and a $2P_\mathrm{r}$ value of 27.4 $μ\mathrm{C/cm^2}$. On the other hand, heating from the $\mathrm{TiN/HfO_2}$ side, or using $\mathrm{500^oC}$ annealing temperature, both yield ferroelectric devices that require a wake-up process. The special configuration of $\mathrm{Pt/ZrO_2}$ is verified by comparative studies with several other superlattice structures and HZO solid-state solutions. It is discovered that heating from the $\mathrm{Pt/HfO_2}$ side at $\mathrm{600^oC}$ leads to high leakage current and a memristor behavior. The mechanisms of ferroelectric phase stabilization and memristor formation have been discussed. The unidirectional heating method can also be useful for other hafnia-based ferroelectric devices.

cond-mat.mtrl-sci↗

arXiv2022

Ferroelectricity in $\mathrm{HfO_2}$ from a chemical perspective

Ferroelectricity observed in thin film $\mathrm{HfO_2}$, either doped with Si, Al, etc. or in the $\mathrm{Hf_{0.5}Zr_{0.5}O_2}$ form, has gained great technical significance. However, the soft mode theory faces a difficulty in explaining the origin of such ferroelectricity. In this work, we propose that the 7 cation coordination number of $\mathrm{HfO_2/ZrO_2}$ lies at the heart of this ferroelectricity, which stems from the proper ionic radii of Hf/Zr compared with O. Among the numerous compounds with non-centrosymmetric nature, e.g., $mm2$ point group, $\mathrm{HfO_2}$ and $\mathrm{ZrO_2}$ are special in that they are close to the border of 7 and 8 cation coordination, such that the 8-coordination tetragonal intermediate phase could greatly reduce the switching barrier. Other 7-coordination candidates, including $\mathrm{SrI_2}$, TaON, YSBr and YOF are also studied in comparison to $\mathrm{HfO_2}$/$\mathrm{ZrO_2}$, and six switching paths are analyzed in detail for the $Pca2_1$ phase. A rule of preferred switching path in terms of ionic radii ratio and coordination number has been established. We also show the possible route from ferroelectric $Pca2_1$ phase to monoclinic $P2_1/c$ phase in $\mathrm{HfO_2}$, which is relevant to the fatigue phenomenon.

cond-mat.mtrl-sci↗