arXiv subjects

Tao Chu

Publications and source records attributed to Tao Chu.

At least 19 recordsLinked to original sources

arXiv2026

SenseNova-U1.5: Towards Native Unified Visual Intelligence

We launch SenseNova-U1.5, an 8B-MoT native unified multimodal model that understands, reasons about, and generates visual content within an encoder-free and VAE-free architecture. We strengthen its visual interface through spatially coherent patch reconstruction and scale its training with carefully curated generation and editing data, improved task formulation, structural prompt enhancement, and native resolutions of up to 4K. For post-training, we optimize specialized experts for visual aesthetics, bilingual text rendering, infographic generation, and image editing, and consolidate their capabilities through multi-expert on-policy distillation. Across extensive evaluations, SenseNova-U1.5 largely advances image fidelity, text rendering, complex composition, multi-reference editing, and interleaved generation, while improving instruction following and preserving subject identity, geometry, and unmodified regions. Despite limited exposure to structured formats in its generation data, SenseNova-U1.5 generalizes effectively to long, complex, and structured visual instructions, further proving that multimodal understanding can transfer to visual planning and creation. Together, these findings position native unified modelling as a promising path towards systems that perceive, reason and create within a fully end-to-end framework. We will open-source training code, including supervised fine-tuning, reinforcement learning, and on-policy distillation.

cs.CV↗

arXiv2026

MinerU2.5-Pro: Pushing the Limits of Data-Centric Document Parsing at Scale

Current document parsing methods advance primarily through model architecture innovation, while systematic engineering of training data remains underexplored. Yet state-of-the-art models spanning diverse architectures and parameter scales exhibit highly consistent failure patterns on the same set of hard samples, suggesting that the performance bottleneck stems from shared deficiencies in training data rather than from architectural differences. Building on this finding, we present MinerU2.5-Pro, which advances the state of the art purely through data engineering and training strategy design while retaining the 1.2B-parameter architecture of MinerU2.5 unchanged. At its core is a Data Engine co-designed around coverage, informativeness, and annotation accuracy: Diversity-and-Difficulty-Aware Sampling expands training data from under 10M to 65.5M samples while mitigating distribution shift; Cross-Model Consistency Verification leverages output consensus among heterogeneous models to assess sample difficulty and generate reliable annotations; the Judge-and-Refine pipeline improves annotation quality for hard samples through render-then-verify iterative correction. A three-stage progressive training strategy--large-scale pre-training, hard sample fine-tuning, and GRPO alignment--sequentially exploits these data at different quality tiers. On the evaluation front, we rectify element-matching biases in OmniDocBench v1.5 and introduce a Hard subset, establishing the more discriminative OmniDocBench v1.6 protocol. Without any architectural modification, MinerU2.5-Pro achieves 95.69 on OmniDocBench v1.6, improving over the same-architecture baseline by 2.71 points and surpassing all existing methods, including those based on models with over 200x more parameters.

cs.CV↗

arXiv2025

Observation of magnetically coupled electro-optic effect in LiNbO3/LiTaO3 at room temperature

The magnetoelectric coupling effect serves as a crucial bridge between electrical and magnetic order parameters in condensed matter physics, forming the physical basis for the development of next-generation low-power information storage and sensing technologies. However, material systems exhibiting this effect at room temperature are extremely rare, and the coupling strength is typically very weak, which has long hindered the practical application of such phenomena despite their rich tunability. Here, we break this by reporting a pronounced magnetoelectric phenomenon,the magnetically coupled EO effect in the classic ferroelectric optical materials LiNbO3 and LiTaO3. We trace its origin to an unexpected source,the problematic direct current drift,a major reliability issue in photonic integrated circuits. We unambiguously demonstrate that this drift stems not from mobile ions but from defect-bound unpaired electrons, whose slow polarization relaxation is quenched upon the magnetic-field-induced formation of a room-temperature skyrmion states, as directly visualized by Lorentz transmission electron microscopy. This collective spin ordering not only solves the decades-old drift problem but also transforms the defect states into a magnetically responsive platform, exhibiting an efficiency up to 34 pm/V in LiNbO3 and 15 pm/V in LiTaO3-an orders-of-magnitude enhancement over conventional room-temperature magnetoelectric responses and even surpassing the materials' intrinsic Pockels coefficients. We also measured the magnetic response of this additional electro-optical effect and found that under the condition of applying a magnetic field of 0.1 T, an electro-optical coefficient adjustment of about 8 pm/V can be achieved (0.008 pm/V/Oe), which is equivalent to 30% of the electro-optical coefficient of LiNbO3 itself.

cond-mat.mtrl-sci↗

arXiv2025

High-bandwidth photodetector enabled by frequency-domain equalization

High-bandwidth germanium (Ge) photodetectors are crucial for silicon photonic integrated circuits. However, their bandwidth is restricted by carrier transit time and parasitic parameters. In this work, we propose an equalization photodetector (EqPD) utilizing the frequency response of a high-bandwidth photodetector PDA to subtract the frequency response of a low-bandwidth photodetector PDB. With the response of PDB attenuating more severely than PDA at high frequency, the differential frequency response (the response of EqPD) can get higher values at high frequency than at low frequency, flattening the overall frequency response and expanding the bandwidth. Experimental results show that the EqPD has a bandwidth exceeding 110 GHz with a responsivity of 94 mA/W. A 100 Gbaud non-return-to-zero (NRZ) operation without digital signal processing is also demonstrated. To the best of our knowledge, this represents the highest bandwidth in a vertical Ge photodetector, providing a promising solution for high-speed photodetection in next-generation optical communication.

physics.optics↗

arXiv2025

MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing

We introduce MinerU2.5, a 1.2B-parameter document parsing vision-language model that achieves state-of-the-art recognition accuracy while maintaining exceptional computational efficiency. Our approach employs a coarse-to-fine, two-stage parsing strategy that decouples global layout analysis from local content recognition. In the first stage, the model performs efficient layout analysis on downsampled images to identify structural elements, circumventing the computational overhead of processing high-resolution inputs. In the second stage, guided by the global layout, it performs targeted content recognition on native-resolution crops extracted from the original image, preserving fine-grained details in dense text, complex formulas, and tables. To support this strategy, we developed a comprehensive data engine that generates diverse, large-scale training corpora for both pretraining and fine-tuning. Ultimately, MinerU2.5 demonstrates strong document parsing ability, achieving state-of-the-art performance on multiple benchmarks, surpassing both general-purpose and domain-specific models across various recognition tasks, while maintaining significantly lower computational overhead.

cs.CV↗

arXiv2025

Intern-S1: A Scientific Multimodal Foundation Model

In recent years, a plethora of open-source foundation models have emerged, achieving remarkable progress in some widely attended fields, with performance being quite close to that of closed-source models. However, in high-value but more challenging scientific professional fields, either the fields still rely on expert models, or the progress of general foundation models lags significantly compared to those in popular areas, far from sufficient for transforming scientific research and leaving substantial gap between open-source models and closed-source models in these scientific domains. To mitigate this gap and explore a step further toward Artificial General Intelligence (AGI), we introduce Intern-S1, a specialized generalist equipped with general understanding and reasoning capabilities with expertise to analyze multiple science modal data. Intern-S1 is a multimodal Mixture-of-Experts (MoE) model with 28 billion activated parameters and 241 billion total parameters, continually pre-trained on 5T tokens, including over 2.5T tokens from scientific domains. In the post-training stage, Intern-S1 undergoes offline and then online reinforcement learning (RL) in InternBootCamp, where we propose Mixture-of-Rewards (MoR) to synergize the RL training on more than 1000 tasks simultaneously. Through integrated innovations in algorithms, data, and training systems, Intern-S1 achieved top-tier performance in online RL training. On comprehensive evaluation benchmarks, Intern-S1 demonstrates competitive performance on general reasoning tasks among open-source models and significantly outperforms open-source models in scientific domains, surpassing closed-source state-of-the-art models in professional tasks, such as molecular synthesis planning, reaction condition prediction, predicting thermodynamic stabilities for crystals. Our models are available at https://huggingface.co/internlm/Intern-S1.

cs.LG↗

arXiv2025

Lossless, Non-Volatile Post-Fabrication Trimming of PICs via On-Chip High-Temperature Annealing of Undercut Waveguides

Limited by equipment precision, manufacturing deviations in waveguide width, etch depth, and layer thickness inevitably occur in photonic integrated circuits (PICs). These variations cause initial phase errors, compromising the reliability of phase-sensitive devices such as Mach-Zehnder Interferometers (MZI) and microring resonators. To overcome this, we report a nonvolatile, near-lossless post-trimming method utilizing sufficient high-temperature thermal treatment for undercut waveguides, reported here for the first time to the best of our knowledge. This CMOS-compatible approach requires no additional processes or equipment, enables simple electrical heating for trimming, and retains long-term stability after high-temperature removal, ensuring high energy efficiency. Transmission electron microscopy indicates that high-temperature thermal treatment induces irreversible lattice expansion in silicon waveguides, leading to a reduction in the real refractive index and enabling compensation for process errors. Experimental results using MZIs confirm a permanent refractive index reduction of 0.0173 and high-resolution tuning up to 5.25 bits, effective across a broadband spectrum and stable for over 218 days after final trimming. Furthermore, 15 MZIs on a single wafer are precisely calibrated to BAR, CROSS, or orthogonal states, demonstrating the method universality. This practical and scalable technique enables reliable post-fabrication trimming for next-generation low-cost, energy-efficient PIC applications such as optical switches and optical computing.

physics.optics↗

arXiv2025

Suppressing DC Drift in Thin-Film Lithium Niobate Modulators via Multiferroic Skyrmion Excitation

Thin-film lithium niobate (TFLN) modulators, despite their superior electro-optic performance, face critical DC drift challenges under low-frequency or prolonged operation. In this work, we demonstrate a novel suppression strategy by exciting multiferroic skyrmions in TFLN, achieving drift-free square-wave modulation for voer 1 hour-the first solution eliminating feedback systems. This breakthrough originates from dual carrier suppression mechanisms:(1) charge density reduction via skyrmion-induced polarization nano-regions (PNRs) excitation, and (2) mean free path restriction through polarization gradients at PNRs domain walls. By directly targeting the root cause of DC drift-mobile charge redistribution-our method uniquely preserves the essential SiO2 upper cladding, resolving the longstanding trade-off between drift mitigation and waveguide protection. Crucially, our work also provides the first experimental observation of interconversion between short-term drift (seconds-scale transient overshoot) and long-term drift (hours-scale baseline shift), offering critical insights into their unified origin.

physics.app-ph↗

arXiv2024

Ultralow-Crosstalk Silicon Electro-Optic Switch with Cascaded Phase Shifters for Loss Equivalence

In silicon electro-optic (EO) Mach-Zehnder interferometer (MZI) switches, crosstalk is typically limited by beam imbalance between the MZI arms, primarily caused by the free carrier absorption loss during routing, thus hindering switch scalability. To address this issue, we propose a low-crosstalk push-pull EO MZI switch by cascading a lightly doped, long phase shifter (PSLL) and a heavily doped, short phase shifter (PSHS) to construct phase-shift arms. In both BAR and CROSS states, PSLL in one arm and PSHS in the other arm are simultaneously forward-biased, with PSLL provides a pi/2 greater phase shift for switching, while PSHS balances loss of PSLL, effectively minimizing crosstalk. Simulations indicate that the proposed switch achieves a crosstalk below -51 dB at a 1310 nm wavelength. The fabricated 2 x 2 silicon EO MZI switch exhibited crosstalk between -33 and -44.2 dB at 1316 nm, and maintained crosstalk below -30 dB across an impressive 61 nm optical bandwidth, with response times under 119 ns. Featuring single-pair electrode control, consistent two-state performance, and a compact size, this approach could enable high-radix switch fabrics in data centers and artificial intelligence compute clusters.

physics.optics↗

arXiv2024

Explanation for Trajectory Planning using Multi-modal Large Language Model for Autonomous Driving

End-to-end style autonomous driving models have been developed recently. These models lack interpretability of decision-making process from perception to control of the ego vehicle, resulting in anxiety for passengers. To alleviate it, it is effective to build a model which outputs captions describing future behaviors of the ego vehicle and their reason. However, the existing approaches generate reasoning text that inadequately reflects the future plans of the ego vehicle, because they train models to output captions using momentary control signals as inputs. In this study, we propose a reasoning model that takes future planning trajectories of the ego vehicle as inputs to solve this limitation with the dataset newly collected.

cs.CV↗

arXiv2024

Silicon modulator exceeding 110 GHz using tunable time-frequency equalization

Silicon modulators have garnered considerable attention owing to their potential applications in high-density integration and high-speed modulation. However, they are increasingly challenged by the limited 3 dB bandwidth as the demand for modulation speed in optical communications continues to rise, impeding their ability to compete with modulators made of thin-film lithium niobate. This bandwidth limitation arises because of the parasitic resistance and capacitance in the PN junction of the silicon modulators. This study demonstrates the first silicon modulator exceeding 110 GHz without any resonant structure using a tunable time-frequency equalization technique. This substantial breakthrough enables on-off keying modulation at a rate of 140 Gbaud without digital signal processing. These accomplishments represent the highest bandwidth and maximum baud rate achieved without digital signal processing in an all-silicon modulator, reaching the testing limitations of the experimental system. This opens the possibility of attaining modulation rates of up to 200 or even 300 Gbaud by adopting design strategies such as slow light and technologies such as digital signal processing. This advancement extends the speed capabilities of silicon modulators to the level of thin-film lithium niobate modulators, thereby promoting their application in the broader array of fields, such as linear-drive pluggable transceivers.

physics.optics↗

arXiv2024

Room-Temperature Multiferroic Skyrmions in LiNbO3 with enhancement in electric-optical property

LiNbO3 (LN) is renowned for its exceptional ferroelectric properties, particularly its notable linear electro-optical (EO) effect, which is highly advantageous for various applications such as high-speed communication, optical computation, and quantum information processing. Compared to its ferroelectric properties, the magnetism of LN is not attractive enough due to its weak ferromagnetic nature. Theoretical studies suggest that LN may exhibit a novel magnetoelectric coupling via ferroelectrically-induced ferromagnetism. However, this mechanism has not yet been experimentally validated in any materials, presenting significant challenges for research. In this study, we provide the first experimental evidence supporting the mechanism of ferroelectrically-induced ferromagnetism in LN, including observations of the Dzyaloshinskii-Moriya interaction (DMI) and magnetoelectric coupling. Additionally, we have identified various multiferroic skyrmions, within which ferroelectric polarization signals are detectable. These signals can be influenced by the magnetic vortex structures, indicating a magnetoelectric coupling nature. Currently, they are the only multiferroic skyrmions that can keep stable at room temperature. Moreover, these magnetic textures significantly affect the ferroelectric properties, as demonstrated by an enhancement of the linear electro-optic effect of LN by over 200%. Given the novel magnetoelectric coupling mechanism, the potential of multiferroic skyrmions in spintronics and advanced data storage, and the extensive use of LN EO modulators, our research has significant implications for condensed matter physics, multiferroic materials, and optoelectronics.

cond-mat.mtrl-sci↗

arXiv2024

Stable room-temperature multiferroic skyrmions in lithium niobate with enhanced Pockels effect

Lithium Niobate (LN) is a ferroelectric material with exceptional electrical characteristics, including high piezoelectricity, high Pockels effect, etc. These properties make it a promising platform for numerous fields such as high-speed communication, optical computation, and quantum information processing. Besides these, the introduction of magnetic structures to LN holds significant potential to achieve magnetoelectric coupling, which can be applied in magnetic memory and data-processing devices with high efficiency. Here, for the first time, we observe a special topological magnetic structure called magnetic skyrmion in LN (SK-LN) by the combination of magnetic field annealing and rapid annealing processes . Compared to the magnetic skyrmions reported in magnetic systems, SK-LN exhibit exceptionally high stability. Additionally, the center of the magnetic vortex exhibits spontaneous ferroelectric polarization, indicating its multiferroic characteristic. With the excitation of these multiferroic skyrmions, the modulation efficiency of the electro-optical (EO) modulator fabricated on thin film lithium niobate on insulator (LNOI) wafer was found to be enhanced from 1.98 V*cm to 0.63 V*cm. It is considered that the multiferroic skyrmions significantly enhance the Pockels coefficient of LN to 101 pm/V, nearly three times the result (32pm/V) reported previously.

cond-mat.mtrl-sci↗

arXiv2024

V3Det Challenge 2024 on Vast Vocabulary and Open Vocabulary Object Detection: Methods and Results

Detecting objects in real-world scenes is a complex task due to various challenges, including the vast range of object categories, and potential encounters with previously unknown or unseen objects. The challenges necessitate the development of public benchmarks and challenges to advance the field of object detection. Inspired by the success of previous COCO and LVIS Challenges, we organize the V3Det Challenge 2024 in conjunction with the 4th Open World Vision Workshop: Visual Perception via Learning in an Open World (VPLOW) at CVPR 2024, Seattle, US. This challenge aims to push the boundaries of object detection research and encourage innovation in this field. The V3Det Challenge 2024 consists of two tracks: 1) Vast Vocabulary Object Detection: This track focuses on detecting objects from a large set of 13204 categories, testing the detection algorithm's ability to recognize and locate diverse objects. 2) Open Vocabulary Object Detection: This track goes a step further, requiring algorithms to detect objects from an open set of categories, including unknown objects. In the following sections, we will provide a comprehensive summary and analysis of the solutions submitted by participants. By analyzing the methods and solutions presented, we aim to inspire future research directions in vast vocabulary and open-vocabulary object detection, driving progress in this field. Challenge homepage: https://v3det.openxlab.org.cn/challenge

cs.CV↗

arXiv2024

MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMs

Generating natural and meaningful responses to communicate with multi-modal human inputs is a fundamental capability of Large Vision-Language Models(LVLMs). While current open-source LVLMs demonstrate promising performance in simplified scenarios such as single-turn single-image input, they fall short in real-world conversation scenarios such as following instructions in a long context history with multi-turn and multi-images. Existing LVLM benchmarks primarily focus on single-choice questions or short-form responses, which do not adequately assess the capabilities of LVLMs in real-world human-AI interaction applications. Therefore, we introduce MMDU, a comprehensive benchmark, and MMDU-45k, a large-scale instruction tuning dataset, designed to evaluate and improve LVLMs' abilities in multi-turn and multi-image conversations. We employ the clustering algorithm to ffnd the relevant images and textual descriptions from the open-source Wikipedia and construct the question-answer pairs by human annotators with the assistance of the GPT-4o model. MMDU has a maximum of 18k image+text tokens, 20 images, and 27 turns, which is at least 5x longer than previous benchmarks and poses challenges to current LVLMs. Our in-depth analysis of 15 representative LVLMs using MMDU reveals that open-source LVLMs lag behind closed-source counterparts due to limited conversational instruction tuning data. We demonstrate that ffne-tuning open-source LVLMs on MMDU-45k signiffcantly address this gap, generating longer and more accurate conversations, and improving scores on MMDU and existing benchmarks (MMStar: +1.1%, MathVista: +1.5%, ChartQA:+1.2%). Our contributions pave the way for bridging the gap between current LVLM models and real-world application demands. This project is available at https://github.com/Liuziyu77/MMDU.

cs.CV↗

arXiv2024

Unified Scene Representation and Reconstruction for 3D Large Language Models

Enabling Large Language Models (LLMs) to interact with 3D environments is challenging. Existing approaches extract point clouds either from ground truth (GT) geometry or 3D scenes reconstructed by auxiliary models. Text-image aligned 2D features from CLIP are then lifted to point clouds, which serve as inputs for LLMs. However, this solution lacks the establishment of 3D point-to-point connections, leading to a deficiency of spatial structure information. Concurrently, the absence of integration and unification between the geometric and semantic representations of the scene culminates in a diminished level of 3D scene understanding. In this paper, we demonstrate the importance of having a unified scene representation and reconstruction framework, which is essential for LLMs in 3D scenes. Specifically, we introduce Uni3DR^2 extracts 3D geometric and semantic aware representation features via the frozen pre-trained 2D foundation models (e.g., CLIP and SAM) and a multi-scale aggregate 3D decoder. Our learned 3D representations not only contribute to the reconstruction process but also provide valuable knowledge for LLMs. Experimental results validate that our Uni3DR^2 yields convincing gains over the baseline on the 3D reconstruction dataset ScanNet (increasing F-Score by +1.8\%). When applied to LLMs, our Uni3DR^2-LLM exhibits superior performance over the baseline on the 3D vision-language understanding dataset ScanQA (increasing BLEU-1 by +4.0\% and +4.2\% on the val set and test set, respectively). Furthermore, it outperforms the state-of-the-art method that uses additional GT point clouds on both ScanQA and 3DMV-VQA.

cs.CV↗

arXiv2023

Breaking the Baud Rate Ceiling of Electro-Optic Modulators Using Optical Equalization Technique

This study presents an effective optical equalization technique for generating ultrahigh baud rate signals. The equalization technique was demonstrated using a dual-drive Mach-Zehnder modulator (DDMZM) with two-phase shifters having different bandwidths, which can be achieved by adjusting the structural design of the modulator or by incorporating varying bias voltages. When the two phase shifters are driven by appropriately sized driving signals, their partially offsetting response significantly reduces the rise and fall times of the output signal. The experiments were conducted using silicon DDMZMs without digital signal processing (DSP). In an experiment utilizing a low-speed silicon modulator, the on-off keying (OOK) modulating speed was improved from 30 to 90 Gbaud. In an experiment utilizing a high-speed silicon modulator, the OOK modulating speed was improved from 100 to 128 Gbaud, which approached the limit of our testing system. This remains the highest baud rate achieved by an all-silicon modulator without DSP. This technique breaks the baud rate ceiling of the modulator and has the potential to enable silicon modulators to operate at 200 Gbaud and beyond. The versatility of this method extends to a wide range of optoelectronic platforms, encompassing thin-film lithium niobate and indium phosphide modulators.

physics.optics↗

arXiv2023

A general designing approach for polarization-independent photonic on-chip systems

Due to that the polarization states in optical fibers change randomly during transmission, polarization-independent (PID) devices are demanded to receive lights with arbitrary polarization states. Compared with their orthogonal polarization states, the optical profiles of various modes of the same polarization are similar, and their directions of the main electric field are same. Therefore, it's much easier to design PID devices using multi-modes of one polarization state instead of orthogonal polarizations. This paper firstly presents a scalable method to achieve PID devices and systems by transforming the light orthogonal polarization states into one polarization with different modes. Taking thermo-optical switches as an example, the PDL of the fabricated 2 x 2 switch cell was about 0.8 dB at 1300-1360 nm, and its extinction ratio (ER) was larger than 19 dB for both polarizations. A 4 x 4 switch was also demonstrated and its function of dual polarization switching was implemented, whose ER was larger than 11 dB for both polarizations. This method has potential in constructing PID on-chip systems for optical communications, optical interconnections, etc.

physics.optics↗