arXiv subjects

Kai Liu

Publications and source records attributed to Kai Liu.

At least 163 records · Page 9Linked to original sources

arXiv2025

UniMoMo: Unified Generative Modeling of 3D Molecules for De Novo Binder Design

The design of target-specific molecules such as small molecules, peptides, and antibodies is vital for biological research and drug discovery. Existing generative methods are restricted to single-domain molecules, failing to address versatile therapeutic needs or utilize cross-domain transferability to enhance model performance. In this paper, we introduce Unified generative Modeling of 3D Molecules (UniMoMo), the first framework capable of designing binders of multiple molecular domains using a single model. In particular, UniMoMo unifies the representations of different molecules as graphs of blocks, where each block corresponds to either a standard amino acid or a molecular fragment. Subsequently, UniMoMo utilizes a geometric latent diffusion model for 3D molecular generation, featuring an iterative full-atom autoencoder to compress blocks into latent space points, followed by an E(3)-equivariant diffusion process. Extensive benchmarks across peptides, antibodies, and small molecules demonstrate the superiority of our unified framework over existing domain-specific models, highlighting the benefits of multi-domain training.

cs.LG↗

arXiv2025

Distributed Model Predictive Control Design for Multi-agent Systems via Bayesian Optimization

This paper introduces a new approach that leverages Multi-agent Bayesian Optimization (MABO) to design Distributed Model Predictive Control (DMPC) schemes for multi-agent systems. The primary objective is to learn optimal DMPC schemes even when local model predictive controllers rely on imperfect local models. The proposed method invokes a dual decomposition-based distributed optimization framework, incorporating an Alternating Direction Method of Multipliers (ADMM)-based MABO algorithm to enable coordinated learning of parameterized DMPC schemes. This enhances the closed-loop performance of local controllers, despite discrepancies between their models and the actual multi-agent system dynamics. In addition to the newly proposed algorithms, this work also provides rigorous proofs establishing the optimality and convergence of the underlying learning method. Finally, numerical examples are given to demonstrate the efficacy of the proposed MABO-based learning approach.

eess.SY↗

arXiv2025

Seed1.5-VL Technical Report

We present Seed1.5-VL, a vision-language foundation model designed to advance general-purpose multimodal understanding and reasoning. Seed1.5-VL is composed with a 532M-parameter vision encoder and a Mixture-of-Experts (MoE) LLM of 20B active parameters. Despite its relatively compact architecture, it delivers strong performance across a wide spectrum of public VLM benchmarks and internal evaluation suites, achieving the state-of-the-art performance on 38 out of 60 public benchmarks. Moreover, in agent-centric tasks such as GUI control and gameplay, Seed1.5-VL outperforms leading multimodal systems, including OpenAI CUA and Claude 3.7. Beyond visual and video understanding, it also demonstrates strong reasoning abilities, making it particularly effective for multimodal reasoning challenges such as visual puzzles. We believe these capabilities will empower broader applications across diverse tasks. In this report, we mainly provide a comprehensive review of our experiences in building Seed1.5-VL across model design, data construction, and training at various stages, hoping that this report can inspire further research. Seed1.5-VL is now accessible at https://www.volcengine.com/ (Volcano Engine Model ID: doubao-1-5-thinking-vision-pro-250428)

cs.CV↗

arXiv2025

Low-bit Model Quantization for Deep Neural Networks: A Survey

With unprecedented rapid development, deep neural networks (DNNs) have deeply influenced almost all fields. However, their heavy computation costs and model sizes are usually unacceptable in real-world deployment. Model quantization, an effective weight-lighting technique, has become an indispensable procedure in the whole deployment pipeline. The essence of quantization acceleration is the conversion from continuous floating-point numbers to discrete integer ones, which significantly speeds up the memory I/O and calculation, i.e., addition and multiplication. However, performance degradation also comes with the conversion because of the loss of precision. Therefore, it has become increasingly popular and critical to investigate how to perform the conversion and how to compensate for the information loss. This article surveys the recent five-year progress towards low-bit quantization on DNNs. We discuss and compare the state-of-the-art quantization methods and classify them into 8 main categories and 24 sub-categories according to their core techniques. Furthermore, we shed light on the potential research opportunities in the field of model quantization. A curated list of model quantization is provided at https://github.com/Kai-Liu001/Awesome-Model-Quantization.

cs.LG↗

arXiv2025

OmniScience: A Domain-Specialized LLM for Scientific Reasoning and Discovery

Large Language Models (LLMs) have demonstrated remarkable potential in advancing scientific knowledge and addressing complex challenges. In this work, we introduce OmniScience, a specialized large reasoning model for general science, developed through three key components: (1) domain adaptive pretraining on a carefully curated corpus of scientific literature, (2) instruction tuning on a specialized dataset to guide the model in following domain-specific tasks, and (3) reasoning-based knowledge distillation through fine-tuning to significantly enhance its ability to generate contextually relevant and logically sound responses. We demonstrate the versatility of OmniScience by developing a battery agent that efficiently ranks molecules as potential electrolyte solvents or additives. Comprehensive evaluations reveal that OmniScience is competitive with state-of-the-art large reasoning models on the GPQA Diamond and domain-specific battery benchmarks, while outperforming all public reasoning and non-reasoning models with similar parameter counts. We further demonstrate via ablation experiments that domain adaptive pretraining and reasoning-based knowledge distillation are critical to attain our performance levels, across benchmarks.

cs.AI↗

arXiv2025

A Thin Flexible Acoustic Transducer with piezoelectric-actuated microdomes for Underwater Communication

This paper presents a flexible thin-film underwater transducer based on a mesoporous PVDF membrane embedded with piezoelectrical-actuated microdomes. To enhance piezoelectric performance, ZnO nanoparticles were used as a sacrificial template to fabricate a sponge-like PVDF structure with increased \b{eta}-phase content and improved mechanical compliance. The device was modeled using finite element analysis and optimized through parametric studies of dome geometry, film thickness, and dome size. Acoustic performance was evaluated through underwater testing, demonstrating high SPL output and reliable data transmission even at low drive voltages. The proposed transducer offers a lightweight, low-cost, and energy-efficient solution for short-range underwater communication in next-generation Ocean IoT systems.

eess.SP↗

arXiv2025

Extremely strong spin-orbit coupling effect in light-element altermagnetic materials

Spin-orbit coupling is a key to realize many novel physical effects in condensed matter physics. Altermagnetic materials possess the duality of real-space antiferromagnetism and reciprocal-space ferromagnetism. It has not been explored that achieving strong spin-orbit coupling effect in light-element altermagnetic materials. In this work, based on symmetry analysis, the first-principles electronic structure calculations plus Dynamical Mean Field Theory, we demonstrate that there is strong spin-orbit coupling effect in light-element altermagnetic materials $\rm NiF_3$ and $\rm FeCO_3$, and then propose a mechanism for realizing such effective spin-orbit coupling. This mechanism reveals the cooperative effect of crystal symmetry, electron occupation, electronegativity, electron correlation, and intrinsic spin-orbit coupling. Our work provides an approach for searching light-element altermagnetic materials with an effective strong spin-orbit coupling.

cond-mat.mtrl-sci↗

arXiv2025

Output regulation for a reaction-diffusion system with input delay and unknown frequency

This study solves the output regulation problem for a reaction-diffusion system confronting concurrent input delay and fully unidentified disturbances (encompassing both unknown frequencies and amplitudes) across all channels. The principal innovation emerges from a novel adaptive control architecture that synergizes the modal decomposition technique with a dual-observer mechanism, enabling real-time concurrent estimation of unmeasurable system states and disturbances through a state observer and an adaptive disturbance estimator. Unlike existing approaches limited to either delay compensation or partial disturbance rejection, our methodology overcomes the technical barrier of coordinating these two requirements through a rigorously constructed tracking-error-based controller, achieving exponential convergence of system output to reference signals. Numerical simulations are presented to validate the effectiveness of the proposed output feedback control strategy.

math.OC↗

arXiv2025

NTIRE 2025 Challenge on Real-World Face Restoration: Methods and Results

This paper provides a review of the NTIRE 2025 challenge on real-world face restoration, highlighting the proposed solutions and the resulting outcomes. The challenge focuses on generating natural, realistic outputs while maintaining identity consistency. Its goal is to advance state-of-the-art solutions for perceptual quality and realism, without imposing constraints on computational resources or training data. The track of the challenge evaluates performance using a weighted image quality assessment (IQA) score and employs the AdaFace model as an identity checker. The competition attracted 141 registrants, with 13 teams submitting valid models, and ultimately, 10 teams achieved a valid score in the final ranking. This collaborative effort advances the performance of real-world face restoration while offering an in-depth overview of the latest trends in the field.

cs.CV↗

arXiv2025

Moiré enhanced flat band in rhombohedral graphene

The fractional quantum anomalous Hall effect (FQAHE) is a fascinating emergent quantum state characterized by fractionally charged excitations in the absence of magnetic field,which could arise from the intricate interplay between electron correlation, nontrivial topology and spontaneous time-reversal symmetry breaking. Recently, FQAHE has been realized in aligned rhombohedral pentalayer graphene on BN superlattice (aligned R5G/BN), where the topological flat band is modulated by the moiré potential. However, intriguingly, the FQAHE is observed only when electrons are pushed away from the moiré interface. The apparently opposite implications from these experimental observations, along with different theoretical models, have sparked intense debates regarding the role of the moiré potential. Unambiguous experimental observation of the topological flat band as well as moiré bands with energy and momentum resolved information is therefore critical to elucidate the underlying mechanism. Here by performing nanospot angle-resolved photoemission spectroscopy (NanoARPES) measurements, we directly reveal the topological flat band electronic structures of R5G, from which key hopping parameters essential for determining the fundamental electronic structure of rhombohedral graphene are extracted. Moreover, a comparison of electronic structures between aligned and non-aligned samples reveals that the moiré potential plays a pivotal role in enhancing the topological flat band in the aligned sample. Our study provides experimental guiding lines to narrow down the phase space of rhombohedral graphene, laying an important foundation for understanding exotic quantum phenomena in this emerging platform.

cond-mat.mes-hall↗

arXiv2025

Manipulating magnetization by orbital current from a light metal Ti

The orbital Hall effect, which does not rely on the spin-orbit coupling, has recently emerged as a promising mechanism for electrically manipulating magnetization in thin-film ferromagnets. Despite its potential, direct experimental observation of magnetization switching driven by orbital currents has been challenging, primarily because there is no direct exchange coupling between orbital angular momentum and local spin based magnetic moments. In this study, we present a compensated design to directly probe the contribution of orbital currents in the most promising light metal titanium (Ti), where symmetric layer structures allow zeroing out of the net spin current. By varying the thickness of the Ti layer in Ti(t)/Pt/Co/Pt/Co/Pt multilayers, we demonstrate the ability to control the magnetization switching polarity. We deduce the orbital charge conversion efficiency of the Ti layer to be approximately 0.17. These findings not only confirm the presence of the orbital Hall effect in Ti but also suggest that orbital currents may be promising candidates for developing energy-efficient magnetic devices with enhanced performance and scalability.

cond-mat.mtrl-sci↗

arXiv2025

A Modular Dataset to Demonstrate LLM Abstraction Capability

Large language models (LLMs) exhibit impressive capabilities but struggle with reasoning errors due to hallucinations and flawed logic. To investigate their internal representations of reasoning, we introduce ArrangementPuzzle, a novel puzzle dataset with structured solutions and automated stepwise correctness verification. We trained a classifier model on LLM activations on this dataset and found that it achieved over 80% accuracy in predicting reasoning correctness, implying that LLMs internally distinguish between correct and incorrect reasoning steps, with the strongest representations in middle-late Transformer layers. Further analysis reveals that LLMs encode abstract reasoning concepts within the middle activation layers of the transformer architecture, distinguishing logical from semantic equivalence. These findings provide insights into LLM reasoning mechanisms and contribute to improving AI reliability and interpretability, thereby offering the possibility to manipulate and refine LLM reasoning.

cs.AI↗

arXiv2025

Fast Critical Clearing Time Calculation for Power Systems with Synchronous and Asynchronous Generation

The increasing penetration of renewables is replacing traditional synchronous generation in modern power systems with low-inertia asynchronous converter-interfaced generators (CIGs). This penetration threatens the dynamic stability of the modern power system. To assess the latter, we resort to the critical clearing time (CCT) as a stability index, which is typically computed through a large number of time-domain simulations. This is especially true for CIG-embedded power systems, where the complexity of the model is further increased. To alleviate the computing burden, we developed a trajectory sensitivity-based method for assessing the CCT in power systems with synchronous and asynchronous generators. This allows us to obtain the CCT cost-effectively. The simulation results reveal the excellent performance of the proposed method.

eess.SY↗

arXiv2025

Crystal valley Hall effect

The time-reversal symmetry is thought to be a necessary condition for realizing valley Hall effect. If the time-reversal symmetry is broken, whether the valley Hall effect can be realized has not been explored. In this letter, based on symmetry analysis and the first-principles electronic structure calculations, we demonstrate that the vally Hall effect without time-reversal symmetry can be realized in two-dimensional altermagnetic materials Fe$_2$WSe$_4$ and Fe$_2$WS$_4$. Due to crystal symmetry required, the vally Hall effect without time-reversal symmetry is called crystal vally Hall effect. In addition, under uniaxial strain, both monolayer Fe$_2$WSe$_4$ and Fe$_2$WS$_4$ can realize piezomagnetic effect. Under biaxial compressive stress, both monolayer Fe$_2$WSe$_4$ and Fe$_2$WS$_4$ will transform from altermagnetic semiconductor phase to bipolarized topological Weyl semimetal phase. Our work not only provides a new direction for exploring the novel valley Hall effect, but also provides a good platform for exploring altermagnetic semiconductors and altermagnetic topological phase transitions.

cond-mat.mtrl-sci↗

arXiv2025

CondiQuant: Condition Number Based Low-Bit Quantization for Image Super-Resolution

Low-bit model quantization for image super-resolution (SR) is a longstanding task that is renowned for its surprising compression and acceleration ability. However, accuracy degradation is inevitable when compressing the full-precision (FP) model to ultra-low bit widths (2~4 bits). Experimentally, we observe that the degradation of quantization is mainly attributed to the quantization of activation instead of model weights. In numerical analysis, the condition number of weights could measure how much the output value can change for a small change in the input argument, inherently reflecting the quantization error. Therefore, we propose CondiQuant, a condition number based low-bit post-training quantization for image super-resolution. Specifically, we formulate the quantization error as the condition number of weight metrics. By decoupling the representation ability and the quantization sensitivity, we design an efficient proximal gradient descent algorithm to iteratively minimize the condition number and maintain the output still. With comprehensive experiments, we demonstrate that CondiQuant outperforms existing state-of-the-art post-training quantization methods in accuracy without computation overhead and gains the theoretically optimal compression ratio in model parameters. Our code and model are released at https://github.com/Kai-Liu001/CondiQuant.

cs.CV↗

arXiv2025

Interfacial superconductivity in the type-III heterostructure SnSe$_2$/PtTe$_2$

Interfacial superconductivity (IS) has been a topic of intense interest in the condensed matter physics, due to its unique properties and exotic photoelectrical performance. However, there are few reports about IS systems consisting of two insulators. Here, motivated by the emergence of an insulator-metal transition in the type-III heterostructure and the superconductivity in the some "special" two-dimensional (2D) semiconductors via the electron doping, we predict that 2D heterostructure SnSe$_2$/PtTe$_2$ is a model system for realizing the IS by using first-principles calculations. Our results show that due to the slight but crucial interlayer charge transfer, SnSe$_2$/PtTe$_2$ turns to be a type-III heterostructure with metallic properties and shows a superconducting transition with the critical temperature ($T_\text{c}$) of 3.73 K. Similar to the enhance electron-phonon coupling (EPC) in the electron doped SnSe$_2$ monolayer, the IS in the heterostructure SnSe$_2$/PtTe$_2$ mainly originates from the metallized SnSe$_2$ layer. Furthermore, we find that the superconductivity is sensitive to the tensile lattice strain, forming a dome-shaped superconducting phase diagram. Remarkably, at the 7\% tensile strain, the superconducting $T_\text{c}$ can increase more than twofold (8.80 K), resulting from the softened acoustic phonon at the M point and the enhanced EPC strength. Our study provides a concrete example for realizing IS in the type-III heterosturcture, which waits for future experimental verification.

cond-mat.supr-con↗

arXiv2025

PEARL: Parallel Speculative Decoding with Adaptive Draft Length

Speculative decoding (SD), where an extra draft model is employed to provide multiple draft tokens first, and then the original target model verifies these tokens in parallel, has shown great power for LLM inference acceleration. However, existing SD methods suffer from the mutual waiting problem, i.e., the target model gets stuck when the draft model is guessing tokens, and vice versa. This problem is directly incurred by the asynchronous execution of the draft model and the target model and is exacerbated due to the fixed draft length in speculative decoding. To address these challenges, we propose a conceptually simple, flexible, and general framework to boost speculative decoding, namely Parallel spEculative decoding with Adaptive dRaft Length (PEARL). Specifically, PEARL proposes pre-verify to verify the first draft token in advance during the drafting phase, and post-verify to generate more draft tokens during the verification phase. PEARL parallels the drafting phase and the verification phase via applying the two strategies, and achieves adaptive draft length for different scenarios, which effectively alleviates the mutual waiting problem. Experiments on various text generation benchmarks demonstrate the effectiveness of our PEARL, leading to a superior speed up performance up to 4.43$\times$ and 1.50$\times$, compared to auto-regressive decoding and vanilla speculative decoding, respectively. Our code is available at https://github.com/smart-lty/ParallelSpeculativeDecoding.

cs.CL↗

arXiv2025

Structure-aware Domain Knowledge Injection for Large Language Models

This paper introduces a pioneering methodology, termed StructTuning, to efficiently transform foundation Large Language Models (LLMs) into domain specialists. It significantly reduces the training corpus needs to a mere 5% while achieving an impressive 100% of traditional knowledge injection performance. Motivated by structured human education, we propose a novel two-stage strategy for knowledge injection and alignment: Structure-aware Continual Pre-Training (SCPT) and Structure-aware Supervised Fine-Tuning (SSFT). In the SCPT phase, we automatically extract the domain knowledge taxonomy and reorganize the training corpora, enabling LLMs to effectively link textual segments to targeted knowledge points within the taxonomy. In the SSFT phase, we explicitly prompt models to elucidate the underlying knowledge structure in their outputs, leveraging the structured domain insight to address practical problems. Our ultimate method was extensively evaluated across model architectures and scales on LongBench and MMedBench datasets, demonstrating superior performance against other knowledge injection methods. We also explored our method's scalability across different training corpus sizes, laying the foundation to enhance domain-specific LLMs with better data utilization.

cs.CL↗