arXiv subjects

Yujie Fang

Publications and source records attributed to Yujie Fang.

4 recordsLinked to original sources

arXiv2026

SRG: Score-based Relaxation-guided Generation for Mixed Integer Linear Programming

We propose Score-based Relaxation-guided Generation (SRG), a generative framework based on an approximate formulation of relaxation-guided stochastic differential equations (SDEs) for mixed-integer linear programming. SRG employs a Transformer-based score network that incorporates feasibility and optimality signals into score modeling, encouraging the learned generative model to place more probability mass on feasible, high-quality regions of the solution space. At inference time, SRG directly samples diverse candidate solutions from the learned score model without requiring any additional guidance module. These candidates are then used to construct compact trust-region subproblems for standard MILP solvers. Across multiple public benchmarks, SRG matches or improves upon the solution quality of the strongest learning-based baselines, with particularly strong gains in challenging candidate-generation settings. Moreover, SRG shows promising zero-shot transferability to unseen cross-scale and cross-problem instances, improving solver objectives and reducing search time in several cases through higher-quality initial candidates and compact trust-region search.

cs.LG↗

arXiv2026

HTMuon: Improving Muon via Heavy-Tailed Spectral Correction

Muon has recently shown promising results in LLM training. In this work, we study how to further improve Muon. We argue that Muon's orthogonalized update rule suppresses the emergence of heavy-tailed weight spectra and over-emphasizes the training along noise-dominated directions. Motivated by the Heavy-Tailed Self-Regularization (HT-SR) theory, we propose HTMuon. HTMuon preserves Muon's ability to capture parameter interdependencies while producing heavier-tailed updates and inducing heavier-tailed weight spectra. Experiments on LLM pretraining and image classification show that HTMuon consistently improves performance over state-of-the-art baselines and can also serve as a plug-in on top of existing Muon variants. For example, on LLaMA pretraining on the C4 dataset, HTMuon reduces perplexity by up to $0.98$ compared to Muon. We further theoretically show that HTMuon corresponds to steepest descent under the Schatten-$q$ norm constraint and provide convergence analysis in smooth non-convex settings. The implementation of HTMuon is available at https://github.com/TDCSZ327/HTmuon.

cs.LG↗

arXiv2024

Reassessing Layer Pruning in LLMs: New Insights and Methods

Although large language models (LLMs) have achieved remarkable success across various domains, their considerable scale necessitates substantial computational resources, posing significant challenges for deployment in resource-constrained environments. Layer pruning, as a simple yet effective compression method, removes layers of a model directly, reducing computational overhead. However, what are the best practices for layer pruning in LLMs? Are sophisticated layer selection metrics truly effective? Does the LoRA (Low-Rank Approximation) family, widely regarded as a leading method for pruned model fine-tuning, truly meet expectations when applied to post-pruning fine-tuning? To answer these questions, we dedicate thousands of GPU hours to benchmarking layer pruning in LLMs and gaining insights across multiple dimensions. Our results demonstrate that a simple approach, i.e., pruning the final 25\% of layers followed by fine-tuning the \texttt{lm\_head} and the remaining last three layer, yields remarkably strong performance. Following this guide, we prune Llama-3.1-8B-It and obtain a model that outperforms many popular LLMs of similar size, such as ChatGLM2-6B, Vicuna-7B-v1.5, Qwen1.5-7B and Baichuan2-7B. We release the optimal model weights on Huggingface, and the code is available on GitHub.

cs.LG↗

arXiv2024

Event-enhanced Passive Non-line-of-sight imaging for moving objects with Physical embedding

Non-line-of-sight (NLOS) imaging with intelligent sensors emerges as a novel technique in imaging and sensing occluded objects around corners. With the innovation of bio-inspired neuromorphic sensors, the applications of novel sensors in unconventional imaging tasks like NLOS imaging have shown promising prospects in intelligent perception, encompassing autonomous driving, medical endoscopy and other sensing scenarios. However, the most challenging point of sensors application in computational imaging is the inverse problem established between sensors acquisition and reconstructions. Traditional physical retrieval methods with certain sensors applications usually result in poor reconstruction due to the highly ill-posedness, particularly in moving object imaging. Thanks to the development of neural networks, data-driven methods have greatly improved its accuracy, however, heavy reliance on data volume has put great pressure on data collection and dataset fabrication. To the best of our knowledge, we firstly propose a sensor-dominated restoration prototype termed "event enhanced passive NLOS imaging prototype for moving objects with physical embedding" (EPNP), which illustrated the application of dynamic vision sensors in NLOS imaging. EPNP induces an event camera for feature extraction of dynamic diffusion spot and leverages simulation dataset to pre-train the physical embedded model before fine-tuning with limited real-shot data. The proposed EPNP prototype is verified by simulation and real-world experiments, while the comparisons of data paradigms also validate the superiority of event-based sensor applications in passive NLOS imaging for moving objects and perspectives in advanced imaging techniques.

physics.optics↗