arXiv subjects

D. Mazur

Publications and source records attributed to D. Mazur.

4 recordsLinked to original sources

arXiv2024

TQCompressor: improving tensor decomposition methods in neural networks via permutations

We introduce TQCompressor, a novel method for neural network model compression with improved tensor decompositions. We explore the challenges posed by the computational and storage demands of pre-trained language models in NLP tasks and propose a permutation-based enhancement to Kronecker decomposition. This enhancement makes it possible to reduce loss in model expressivity which is usually associated with factorization. We demonstrate this method applied to the GPT-2$_{small}$. The result of the compression is TQCompressedGPT-2 model, featuring 81 mln. parameters compared to 124 mln. in the GPT-2$_{small}$. We make TQCompressedGPT-2 publicly available. We further enhance the performance of the TQCompressedGPT-2 through a training strategy involving multi-step knowledge distillation, using only a 3.1% of the OpenWebText. TQCompressedGPT-2 surpasses DistilGPT-2 and KnGPT-2 in comparative evaluations, marking an advancement in the efficient and effective deployment of models in resource-constrained environments.

cs.LG↗

arXiv2011

Endotaxial Si nanolines in Si(001):H

We present a detailed study of the structural and electronic properties of a self-assembled silicon nanoline embedded in the H-terminated silicon (001) surface, known as the Haiku stripe. The nanoline is a perfectly straight and defect free endotaxial structure of huge aspect ratio; it can grow micrometre long at a constant width of exactly four Si dimers (1.54nm). Another remarkable property is its capacity to be exposed to air without suffering any degradation. The nanoline grows independently of any step edges at tunable densities, from isolated nanolines to a dense array of nanolines. In addition to these unique structural characteristics, scanning tunnelling microscopy and density functional theory reveal a one-dimensional state confined along the Haiku core. This nanoline is a promising candidate for the long sought after electronic solid-state one-dimensional model system to explore the fascinating quantum properties emerging in such reduced dimensionality.

cond-mat.mes-hall↗

arXiv2010

One dimensional Si-in-Si(001) template for single-atom wire growth

Single atom metallic wires of arbitrary length are of immense technological and scientific interest. We describe a novel silicon-only template enabling the self-organised growth of isolated micrometer long surface and subsurface single-atom chains. It consists of a one dimensional, defect-free reconstruction - the Haiku core, here revealed for the first time in details - self-assembled on hydrogenated Si(001) terraces, independent of any step edges. We discuss the potential of this Si-in-Si template as an appealing alternative to vicinal surfaces for nanoscale patterning.

cond-mat.mes-hall↗

arXiv2007

Investigation of the Tunneling Spectra in HgBr2-Intercalated Bi-2212 Single Crystals below and above Tc

Interlayer tunneling spectroscopy measurements were performed on mesa arrays of Bi-2212 single crystals, intercalated with HgBr$_2$. Tunneling conductances were obtained over a wide temperature range to examine the spectral features, especially the behavior of the quasiparticle peaks corresponding to superconducting energy gaps (SGs). Experimental spectra showed that gap-like features are still present even for the temperatures far above the transition temperature, T$_c$. This evidence is consistent with the idea that the SG evolves into a pseudogap above T$_c$ for HgBr$_2$-intercalated Bi-2212 single crystals.

cond-mat.supr-con↗