arXiv subjects

Lingyu Kong

Publications and source records attributed to Lingyu Kong.

21 records · Page 2Linked to original sources

arXiv2023

Vary: Scaling up the Vision Vocabulary for Large Vision-Language Models

Modern Large Vision-Language Models (LVLMs) enjoy the same vision vocabulary -- CLIP, which can cover most common vision tasks. However, for some special vision task that needs dense and fine-grained vision perception, e.g., document-level OCR or chart understanding, especially in non-English scenarios, the CLIP-style vocabulary may encounter low efficiency in tokenizing the vision knowledge and even suffer out-of-vocabulary problem. Accordingly, we propose Vary, an efficient and effective method to scale up the vision vocabulary of LVLMs. The procedures of Vary are naturally divided into two folds: the generation and integration of a new vision vocabulary. In the first phase, we devise a vocabulary network along with a tiny decoder-only transformer to produce the desired vocabulary via autoregression. In the next, we scale up the vanilla vision vocabulary by merging the new one with the original one (CLIP), enabling the LVLMs can quickly garner new features. Compared to the popular BLIP-2, MiniGPT4, and LLaVA, Vary can maintain its vanilla capabilities while enjoying more excellent fine-grained perception and understanding ability. Specifically, Vary is competent in new document parsing features (OCR or markdown conversion) while achieving 78.2% ANLS in DocVQA and 36.2% in MMVet. Our code will be publicly available on the homepage.

cs.CV↗

arXiv2023

Overcoming the Size Limit of First Principles Molecular Dynamics Simulations with an In-Distribution Substructure Embedding Active Learner

Large-scale first principles molecular dynamics are crucial for simulating complex processes in chemical, biomedical, and materials sciences. However, the unfavorable time complexity with respect to system sizes leads to prohibitive computational costs when the simulation contains over a few hundred atoms in practice. We present an In-Distribution substructure Embedding Active Learner (IDEAL) to enable efficient simulation of large complex systems with quantum accuracy by maintaining a machine learning force field (MLFF) as an accurate surrogate to the first principles methods. By extracting high-uncertainty substructures into low-uncertainty atom environments, the active learner is allowed to concentrate on and learn from small substructures of interest rather than carrying out intractable quantum chemical computations on large structures. IDEAL is benchmarked on various systems and shows sub-linear complexity, accelerating the simulation thousands of times compared with conventional active learning and millions of times compared with pure first principles simulations. To demonstrate the capability of IDEAL in practical applications, we simulated a polycrystalline lithium system composed of one million atoms and the full ammonia formation process in a Haber-Bosch reaction on a 3-nm Iridium nanoparticle catalyst on a computing node comprising one single A100 GPU and 24 CPU cores.

cond-mat.mtrl-sci↗

arXiv2017

Ultralow thermal conductivity of single crystalline porous silicon nanowires

Porous materials provide a large surface to volume ratio, thereby providing a knob to alter fundamental properties in unprecedented ways. In thermal transport, porous nanomaterials can reduce thermal conductivity by not only enhancing phonon scattering from the boundaries of the pores and therefore decreasing the phonon mean free path, but also by reducing the phonon group velocity. Here we establish a structure-property relationship by measuring the porosity and thermal conductivity of individual electrolessly etched single crystalline silicon nanowires using a novel electron beam heating technique. Such porous silicon nanowires exhibit extremely low diffusive thermal conductivity (as low as 0.33 Wm-1K-1 at 300K for 43% porosity), even lower than that of amorphous silicon. The origin of such ultralow thermal conductivity is understood as a reduction in the phonon group velocity, experimentally verified by measuring the Young modulus, as well as the smallest structural size ever reported in crystalline Silicon (less than 5nm). Molecular dynamics simulations support the observation of a drastic reduction in thermal conductivity of silicon nanowires as a function of porosity. Such porous materials provide an intriguing platform to tune phonon transport, which can be useful in the design of functional materials towards electronics and nano-electromechanical systems.

cond-mat.mtrl-sci↗