arXiv subjects

Lin Ma

Publications and source records attributed to Lin Ma.

At least 109 records · Page 6Linked to original sources

arXiv2025

Dynamic Backtracking in GFlowNets: Enhancing Decision Steps with Reward-Dependent Adjustment Mechanisms

Generative Flow Networks (GFlowNets or GFNs) are probabilistic models predicated on Markov flows, and they employ specific amortization algorithms to learn stochastic policies that generate compositional substances including biomolecules, chemical materials, etc. With a strong ability to generate high-performance biochemical molecules, GFNs accelerate the discovery of scientific substances, effectively overcoming the time-consuming, labor-intensive, and costly shortcomings of conventional material discovery methods. However, previous studies rarely focus on accumulating exploratory experience by adjusting generative structures, which leads to disorientation in complex sampling spaces. Efforts to address this issue, such as LS-GFN, are limited to local greedy searches and lack broader global adjustments. This paper introduces a novel variant of GFNs, the Dynamic Backtracking GFN (DB-GFN), which improves the adaptability of decision-making steps through a reward-based dynamic backtracking mechanism. DB-GFN allows backtracking during the network construction process according to the current state's reward value, thereby correcting disadvantageous decisions and exploring alternative pathways during the exploration process. When applied to generative tasks involving biochemical molecules and genetic material sequences, DB-GFN outperforms GFN models such as LS-GFN and GTB, as well as traditional reinforcement learning methods, in sample quality, sample exploration quantity, and training convergence speed. Additionally, owing to its orthogonal nature, DB-GFN shows great potential in future improvements of GFNs, and it can be integrated with other strategies to achieve higher search performance.

cs.LG↗

arXiv2025

Subwavelength plasmonic antennas based on asymmetric split-ring-resonators for high near-field enhancements

As for plasmonic antenna structures that generate localized near-field enhancement, the most effective current implementations are based on electric dipole resonance modes, but this approach also imposes limitations on their further optimization. Here we introduce an ASRR structure whose ASR mode enables differential charge distribution across both sides of the split. Through asymmetric regulation, charges at one end can become highly localized, thereby achieving efficient near-field enhancement. The formation of this structure was initially driven by a hybrid computational framework integrating evolutionary optimization with residual neural networks, and subsequently simplified into an ASRR prototype using the Occam's Razor principle. The ASRR dimer structure can achieve an electric field intensity enhancement over 6.5 times larger than a traditional nanorod dimer, while maintaining a compact size (<1/3 the working wavelength). The ASRR configuration also demonstrates superior Purcell factor and fluorescence enhancement. These results can find applications in surface-enhanced spectroscopy, nonlinear optics, and quantum light-matter interactions.

physics.optics↗

arXiv2025

Graph-Dynamics correspondence in metallic glass-forming liquids

Theoretical challenges in understanding the nature of glass and the glass transition remain significant open questions in statistical and condensed matter physics. As a prototypical example of complex physical systems, glasses and the vitrification process have been central research topics, consistently attracting broad scientific interest. This focus has driven extensive studies on phenomena such as aging, non-exponential relaxation, dynamic anomalies, glass-forming ability, and the mechanical response of glasses under stress. Recent advances in computational and experimental techniques have enabled rigorous testing of theoretical models, shedding new light on glassy behavior. However, the intrinsic complexity of glass and the glass transition that lies in their physics, which spans multiple length and time scales, makes the system challenging to characterize. In this review, we emphasize the need to move beyond conventional approaches and propose a topological perspective as a promising alternative to address these challenges. Specifically, our findings reveal that the diversity in particle relaxation behavior is statistically linked to a global topological feature of the transient network structures formed by the particles in a given liquid. This direction offers opportunities to uncover novel phenomena that could fundamentally reshape our understanding of glassy materials.

cond-mat.dis-nn↗

arXiv2025

Spatially-resolved Galactic HII regions observed by LAMOST medium-Resolution Spectroscopic Survey of Nebulae (LAMOST MRS-N)

We present spatially-resolved spectroscopic observations of 10 isolated Galactic HII regions using data from the LAMOST Medium-Resolution Spectroscopic Survey of Nebulae (LAMOST MRS-N). The high spatial resolution of the data allows us to investigate the 1D radial profiles of emission line fluxes (Ha, [S II] and [N II]), flux ratios ([N II]/Ha, [S II]/Ha and [S II]/[N II]), and radial velocities of these three emission lines. Among these regions, two are ionization-bounded, while the remaining eight are matter-bounded. The matter-bounded HII regions exhibit shallower slopes in their radial flux profiles compared to the ionization-bounded ones. In most cases, the [N II]/Ha and [S II]/Ha ratios increase with distance from the center of the HII regions, consistent with model predictions that low-ionization emissions dominate the outer zones of these regions. The two ionization-bounded HII regions have kinematic ages (t) of 0.2 and 0.3 Myr, while the matter-bounded regions span ages from 1 to 12 Myr. For the matter-bounded HII regions, the optical emission flux decreases continuously beyond the photodissociation region (PDR), extending to approximately 1-4 times the radius of the PDR (r_PDR). The escape fraction f_esc of ionizing photons, derived from 1D Ha radial flux profiles, is ~ 0% for ionization-bounded HII regions, while it ranges from 50% to 90% for the matter-bounded HII regions. The correlation between f_esc and t suggests that evolved HII regions (with t > 1 Myr) contribute more significantly to ionizing the surrounding diffuse ionized gas compared to younger, newly formed HII regions.

astro-ph.GA↗

arXiv2025

Vortex: Overcoming Memory Capacity Limitations in GPU-Accelerated Large-Scale Data Analytics

Despite the high computational throughput of GPUs, limited memory capacity and bandwidth-limited CPU-GPU communication via PCIe links remain significant bottlenecks for accelerating large-scale data analytics workloads. This paper introduces Vortex, a GPU-accelerated framework designed for data analytics workloads that exceed GPU memory capacity. A key aspect of our framework is an optimized IO primitive that leverages all available PCIe links in multi-GPU systems for the IO demand of a single target GPU. It routes data through other GPUs to such target GPU that handles IO-intensive analytics tasks. This approach is advantageous when other GPUs are occupied with compute-bound workloads, such as popular AI applications that typically underutilize IO resources. We also introduce a novel programming model that separates GPU kernel development from IO scheduling, reducing programmer burden and enabling GPU code reuse. Additionally, we present the design of certain important query operators and discuss a late materialization technique based on GPU's zero-copy memory access. Without caching any data in GPU memory, Vortex improves the performance of the state-of-the-art GPU baseline, Proteus, by 5.7$\times$ on average and enhances price performance by 2.5$\times$ compared to a CPU-based DuckDB baseline.

cs.DB↗

arXiv2024

Scalable Hierarchical Reinforcement Learning for Hyper Scale Multi-Robot Task Planning

To improve the efficiency of warehousing system and meet huge customer orders, we aim to solve the challenges of dimension disaster and dynamic properties in hyper scale multi-robot task planning (MRTP) for robotic mobile fulfillment system (RMFS). Existing research indicates that hierarchical reinforcement learning (HRL) is an effective method to reduce these challenges. Based on that, we construct an efficient multi-stage HRL-based multi-robot task planner for hyper scale MRTP in RMFS, and the planning process is represented with a special temporal graph topology. To ensure optimality, the planner is designed with a centralized architecture, but it also brings the challenges of scaling up and generalization that require policies to maintain performance for various unlearned scales and maps. To tackle these difficulties, we first construct a hierarchical temporal attention network (HTAN) to ensure basic ability of handling inputs with unfixed lengths, and then design multi-stage curricula for hierarchical policy learning to further improve the scaling up and generalization ability while avoiding catastrophic forgetting. Additionally, we notice that policies with hierarchical structure suffer from unfair credit assignment that is similar to that in multi-agent reinforcement learning, inspired of which, we propose a hierarchical reinforcement learning algorithm with counterfactual rollout baseline to improve learning performance. Experimental results demonstrate that our planner outperform other state-of-the-art methods on various MRTP instances in both simulated and real-world RMFS. Also, our planner can successfully scale up to hyper scale MRTP instances in RMFS with up to 200 robots and 1000 retrieval racks on unlearned maps while keeping superior performance over other methods.

cs.RO↗

arXiv2024

Flash Communication: Reducing Tensor Parallelization Bottleneck for Fast Large Language Model Inference

The ever-increasing sizes of large language models necessitate distributed solutions for fast inference that exploit multi-dimensional parallelism, where computational loads are split across various accelerators such as GPU clusters. However, this approach often introduces significant communication overhead, especially on devices with limited bandwidth. In this paper, we introduce Flash Communication, a novel low-bit compression technique designed to alleviate the tensor-parallelism communication bottleneck during inference. Our method substantially boosts intra-node communication speed by more than 3x and reduces the time-to-first-token by 2x, with nearly no sacrifice in model accuracy. Extensive experiments on various up-to-date LLMs demonstrate the effectiveness of our approach.

cs.AI↗

arXiv2024

Diffuse Ionized Gas in the Anti-center of the Milky Way

Using data from the LAMOST Medium-Resolution Spectroscopic Survey of Nebulae, we create a sample of 17,821 diffuse ionized gas (DIG) spectra in the anti-center region of the Milky Way, by excluding fibers in the directions of H II regions and supernova remnants. We then analyze the radial and vertical distributions of three line ratios ([N II]/H$α$, [S II]/H$α$, and [S II]/[N II]), as well as the oxygen abundance. [N II]/H$α$ and [S II]/H$α$ do not exhibit a consistent, monotonic decrease with increasing Galactocentric distance (R$_{gal}$). Instead, they show enhancement within the interarm region, positioned between the Local Arm and the Perseus Arm. [S II]/[N II] has a radial gradient of 0.1415 $\pm$ 0.0646 kpc$^{-1}$ for the inner disk (8.34 $ < R_{gal} < $ 9.65 kpc), and remains nearly flat for the outer disk ($R_{gal} > $ 9.65 kpc). In the vertical direction, [N II]/H$α$, [S II]/H$α$, and [S II]/[N II] increase with increasing Galactic disk height ($|z|$) in both southern and northern disks. Based on the N2S2H$α$ method, which combines [S II]/[N II] and [N II]/H$α$, we estimate the oxygen abundance. The oxygen abundance exhibits a consistent radial gradient with R$_{gal}$, featuring a slope of -0.0559 $\pm$ 0.0209 dex kpc$^{-1}$ for the inner disk and a similar slope of -0.0429 $\pm$ 0.0599 dex kpc$^{-1}$ for the outer disk. A single linear fitting to the entire disk yields a slope of -0.0317 $\pm$ 0.0124 dex kpc$^{-1}$. In the vertical direction, the oxygen abundance decreases with increasing $|z|$ in both southern and northern disks.

astro-ph.GA↗

arXiv2024

RFSR: Improving ISR Diffusion Models via Reward Feedback Learning

Generative diffusion models (DM) have been extensively utilized in image super-resolution (ISR). Most of the existing methods adopt the denoising loss from DDPMs for model optimization. We posit that introducing reward feedback learning to finetune the existing models can further improve the quality of the generated images. In this paper, we propose a timestep-aware training strategy with reward feedback learning. Specifically, in the initial denoising stages of ISR diffusion, we apply low-frequency constraints to super-resolution (SR) images to maintain structural stability. In the later denoising stages, we use reward feedback learning to improve the perceptual and aesthetic quality of the SR images. In addition, we incorporate Gram-KL regularization to alleviate stylization caused by reward hacking. Our method can be integrated into any diffusion-based ISR model in a plug-and-play manner. Experiments show that ISR diffusion models, when fine-tuned with our method, significantly improve the perceptual and aesthetic quality of SR images, achieving excellent subjective results. Code: https://github.com/sxpro/RFSR

cs.CV↗

arXiv2024

TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Ability

Rapid development of large language models (LLMs) has significantly advanced multimodal large language models (LMMs), particularly in vision-language tasks. However, existing video-language models often overlook precise temporal localization and struggle with videos of varying lengths. We introduce TimeMarker, a versatile Video-LLM designed for high-quality dialogue based on video content, emphasizing temporal localization. TimeMarker integrates Temporal Separator Tokens to enhance temporal awareness, accurately marking specific moments within videos. It employs the AnyLength mechanism for dynamic frame sampling and adaptive token merging, enabling effective handling of both short and long videos. Additionally, TimeMarker utilizes diverse datasets, including further transformed temporal-related video QA datasets, to bolster its temporal understanding capabilities. Image and interleaved data are also employed to further enhance the model's semantic perception ability. Evaluations demonstrate that TimeMarker achieves state-of-the-art performance across multiple benchmarks, excelling in both short and long video categories. Our project page is at \url{https://github.com/TimeMarker-LLM/TimeMarker/}.

cs.CV↗

arXiv2024

LESS: Label-Efficient and Single-Stage Referring 3D Segmentation

Referring 3D Segmentation is a visual-language task that segments all points of the specified object from a 3D point cloud described by a sentence of query. Previous works perform a two-stage paradigm, first conducting language-agnostic instance segmentation then matching with given text query. However, the semantic concepts from text query and visual cues are separately interacted during the training, and both instance and semantic labels for each object are required, which is time consuming and human-labor intensive. To mitigate these issues, we propose a novel Referring 3D Segmentation pipeline, Label-Efficient and Single-Stage, dubbed LESS, which is only under the supervision of efficient binary mask. Specifically, we design a Point-Word Cross-Modal Alignment module for aligning the fine-grained features of points and textual embedding. Query Mask Predictor module and Query-Sentence Alignment module are introduced for coarse-grained alignment between masks and query. Furthermore, we propose an area regularization loss, which coarsely reduces irrelevant background predictions on a large scale. Besides, a point-to-point contrastive loss is proposed concentrating on distinguishing points with subtly similar features. Through extensive experiments, we achieve state-of-the-art performance on ScanRefer dataset by surpassing the previous methods about 3.7% mIoU using only binary labels. Code is available at https://github.com/mellody11/LESS.

cs.CV↗

arXiv2024

Tables as Texts or Images: Evaluating the Table Reasoning Ability of LLMs and MLLMs

In this paper, we investigate the effectiveness of various LLMs in interpreting tabular data through different prompting strategies and data formats. Our analyses extend across six benchmarks for table-related tasks such as question-answering and fact-checking. We introduce for the first time the assessment of LLMs' performance on image-based table representations. Specifically, we compare five text-based and three image-based table representations, demonstrating the role of representation and prompting on LLM performance. Our study provides insights into the effective use of LLMs on table-related tasks.

cs.LG↗

arXiv2024

Enhancing Robustness of Vision-Language Models through Orthogonality Learning and Self-Regularization

Efficient fine-tuning of vision-language models (VLMs) like CLIP for specific downstream tasks is gaining significant attention. Previous works primarily focus on prompt learning to adapt the CLIP into a variety of downstream tasks, however, suffering from task overfitting when fine-tuned on a small data set. In this paper, we introduce an orthogonal fine-tuning method for efficiently fine-tuning pretrained weights and enabling enhanced robustness and generalization, while a self-regularization strategy is further exploited to maintain the stability in terms of zero-shot generalization of VLMs, dubbed OrthSR. Specifically, trainable orthogonal matrices are injected seamlessly into the transformer architecture and enforced with orthogonality constraint during the training, benefiting from the norm-preserving property and thus leading to stable and faster convergence, while keeping the pre-trained weights frozen. To alleviate deviation from fine-tuning, a self-regularization strategy is further employed to retain the generalization of the model during the training within a bypass manner. In addition, to enrich the sample diversity for downstream tasks under the small dataset scenario, we first explore attentive CutOut data augmentation to boost the efficient fine-tuning, leading to better model fitting capacity for specific downstream task. Then we support the theoretical analysis on how our approach improves the specific downstream performance and maintains the generalizability. For the first time, we revisit the CLIP and CoOp with our method to effectively improve the model on few-shot image classficiation scenario on par with the elaborated prompt learning methods.

cs.CV↗

arXiv2024

VidCompress: Memory-Enhanced Temporal Compression for Video Understanding in Large Language Models

Video-based multimodal large language models (Video-LLMs) possess significant potential for video understanding tasks. However, most Video-LLMs treat videos as a sequential set of individual frames, which results in insufficient temporal-spatial interaction that hinders fine-grained comprehension and difficulty in processing longer videos due to limited visual token capacity. To address these challenges, we propose VidCompress, a novel Video-LLM featuring memory-enhanced temporal compression. VidCompress employs a dual-compressor approach: a memory-enhanced compressor captures both short-term and long-term temporal relationships in videos and compresses the visual tokens using a multiscale transformer with a memory-cache mechanism, while a text-perceived compressor generates condensed visual tokens by utilizing Q-Former and integrating temporal contexts into query embeddings with cross attention. Experiments on several VideoQA datasets and comprehensive benchmarks demonstrate that VidCompress efficiently models complex temporal-spatial relations and significantly outperforms existing Video-LLMs.

cs.CV↗

arXiv2024

RoboUniView: Visual-Language Model with Unified View Representation for Robotic Manipulation

Utilizing Vision-Language Models (VLMs) for robotic manipulation represents a novel paradigm, aiming to enhance the model's ability to generalize to new objects and instructions. However, due to variations in camera specifications and mounting positions, existing methods exhibit significant performance disparities across different robotic platforms. To address this challenge, we propose RoboUniView in this paper, an innovative approach that decouples visual feature extraction from action learning. We first learn a unified view representation from multi-perspective views by pre-training on readily accessible data, and then derive actions from this unified view representation to control robotic manipulation. This unified view representation more accurately mirrors the physical world and is not constrained by the robotic platform's camera parameters. Thanks to this methodology, we achieve state-of-the-art performance on the demanding CALVIN benchmark, enhancing the success rate in the $D \to D$ setting from 93.0% to 96.2%, and in the $ABC \to D$ setting from 92.2% to 94.2%. Moreover, our model exhibits outstanding adaptability and flexibility: it maintains high performance under unseen camera parameters, can utilize multiple datasets with varying camera parameters, and is capable of joint cross-task learning across datasets. Code is provided for re-implementation. https://github.com/liufanfanlff/RoboUniview

cs.RO↗

arXiv2024

MRStyle: A Unified Framework for Color Style Transfer with Multi-Modality Reference

In this paper, we introduce MRStyle, a comprehensive framework that enables color style transfer using multi-modality reference, including image and text. To achieve a unified style feature space for both modalities, we first develop a neural network called IRStyle, which generates stylized 3D lookup tables for image reference. This is accomplished by integrating an interaction dual-mapping network with a combined supervised learning pipeline, resulting in three key benefits: elimination of visual artifacts, efficient handling of high-resolution images with low memory usage, and maintenance of style consistency even in situations with significant color style variations. For text reference, we align the text feature of stable diffusion priors with the style feature of our IRStyle to perform text-guided color style transfer (TRStyle). Our TRStyle method is highly efficient in both training and inference, producing notable open-set text-guided transfer results. Extensive experiments in both image and text settings demonstrate that our proposed method outperforms the state-of-the-art in both qualitative and quantitative evaluations.

cs.CV↗

arXiv2024

3D Weakly Supervised Semantic Segmentation with 2D Vision-Language Guidance

In this paper, we propose 3DSS-VLG, a weakly supervised approach for 3D Semantic Segmentation with 2D Vision-Language Guidance, an alternative approach that a 3D model predicts dense-embedding for each point which is co-embedded with both the aligned image and text spaces from the 2D vision-language model. Specifically, our method exploits the superior generalization ability of the 2D vision-language models and proposes the Embeddings Soft-Guidance Stage to utilize it to implicitly align 3D embeddings and text embeddings. Moreover, we introduce the Embeddings Specialization Stage to purify the feature representation with the help of a given scene-level label, specifying a better feature supervised by the corresponding text embedding. Thus, the 3D model is able to gain informative supervisions both from the image embedding and text embedding, leading to competitive segmentation performances. To the best of our knowledge, this is the first work to investigate 3D weakly supervised semantic segmentation by using the textual semantic information of text category labels. Moreover, with extensive quantitative and qualitative experiments, we present that our 3DSS-VLG is able not only to achieve the state-of-the-art performance on both S3DIS and ScanNet datasets, but also to maintain strong generalization capability.

cs.CV↗

arXiv2024

Making Large Language Models Better Planners with Reasoning-Decision Alignment

Data-driven approaches for autonomous driving (AD) have been widely adopted in the past decade but are confronted with dataset bias and uninterpretability. Inspired by the knowledge-driven nature of human driving, recent approaches explore the potential of large language models (LLMs) to improve understanding and decision-making in traffic scenarios. They find that the pretrain-finetune paradigm of LLMs on downstream data with the Chain-of-Thought (CoT) reasoning process can enhance explainability and scene understanding. However, such a popular strategy proves to suffer from the notorious problems of misalignment between the crafted CoTs against the consequent decision-making, which remains untouched by previous LLM-based AD methods. To address this problem, we motivate an end-to-end decision-making model based on multimodality-augmented LLM, which simultaneously executes CoT reasoning and carries out planning results. Furthermore, we propose a reasoning-decision alignment constraint between the paired CoTs and planning results, imposing the correspondence between reasoning and decision-making. Moreover, we redesign the CoTs to enable the model to comprehend complex scenarios and enhance decision-making performance. We dub our proposed large language planners with reasoning-decision alignment as RDA-Driver. Experimental evaluations on the nuScenes and DriveLM-nuScenes benchmarks demonstrate the effectiveness of our RDA-Driver in enhancing the performance of end-to-end AD systems. Specifically, our RDA-Driver achieves state-of-the-art planning performance on the nuScenes dataset with 0.80 L2 error and 0.32 collision rate, and also achieves leading results on challenging DriveLM-nuScenes benchmarks with 0.82 L2 error and 0.38 collision rate.

cs.CV↗