arXiv subjects

Chaoran Huang

Publications and source records attributed to Chaoran Huang.

At least 19 recordsLinked to original sources

arXiv2026

Towards Bitstream-corrupted Harsh Visual Understanding: Through Bitstream Language Modeling as Robust Semantic Priors

Bitstream-corrupted Harsh Visual Understanding (BcHVU) aims to understand harshly degraded videos originally decoded from a severely corrupted bitstream in real-world multimedia communication. The ill-posed nature of BcHVU poses a major challenge for existing vision models, as even subtle bitstream corruption can lead to irreversible pixel distortion and significant semantic loss. To address these challenges in BcHVU, we propose Bitstream Language Modeling as Robust Semantic Priors (BLMSP), a framework for learning and injecting bitstream-native semantic cues. Our proposed BLMSP framework learns to extract bitstream-native semantic cues by bitstream language modeling, and leverages them as priors by injecting into off-the-shelf vision models of BcHVU tasks. Specifically, we present a Video Bitstream Byte Model (VBBM) that integrates byte-level modeling and cross-codec semantic distillation, enabling it to interpret robust semantics from byte sequences in multiple corrupted bitstream formats. The learned bitstream semantics are leveraged as robust priors and fused into BcHVU model backbones for improving the quality of video restoration, captioning, and human pose estimation. To train BLMSP, we construct a large-scale multi-source Corrupted-bitstream Harsh-video Paired (CHP) dataset containing 607k corrupted bitstream segments and 287k paired harsh video clips. Extensive experimental results show that the learned bitstream priors improve video restoration, captioning, and human pose estimation by 2.51 dB in PSNR, 0.20 in CIDEr, and 0.18 in PCK@0.2 on average, respectively. These results demonstrate that corrupted bitstream can serve as robust semantic priors in solving pixel distortion and semantic loss in BcHVU.

cs.CV↗

arXiv2026

Bitstream Action Recognition is Byte Modeling

Conventional action recognition typically relies on successful pixel decoding of the bitstream. However, bitstream corruption during storage or transmission may cause severe visual artifacts or even decoding failure, posing a significant challenge to reliable action recognition. Bitstream Action Recognition (BAR) aims to overcome the dependency on decoding and the vulnerability to corruption. In this paper, we propose a novel BAR framework, Bitstream Recognition via Anchoring Corrupted Embeddings (BRACE). BRACE is a dual-branch byte-modeling architecture that treats a corrupted bitstream and its intact counterpart as two byte realizations of the same action. This guides the generation of rich and stable representations for robustness to corruption through Intact-Anchored Representation Alignment (IARA). The intact representation serves as a stable anchor, and the corrupted one is aligned to it at the embedding and decision levels under Unreliable-Anchor Suppression (UAS), entirely in representation space and without repairing the bitstream. To address the scarcity of corrupted bitstreams in practice, we introduce the Real-world Bitstream Corruption Simulator (RBCS), a four-parameter simulator that reproduces bit-flip and byte-loss errors arising in transmission and storage. Building on RBCS, we construct the first large-scale BAR dataset (BAR-D), which comprises the BAR-Stanford40 and BAR-PPMI subsets and spans diverse corruption types and severity levels. Finally, we build a large benchmark on BAR-D involving 14 action recognition methods from the pixel, compressed, and bitstream domains. Extensive experiments demonstrate that BRACE has superior robustness to bitstream corruption than all comparison methods. Ablation studies further validate the effectiveness of the proposed RBCS augmentation and IARA.

cs.CV↗

arXiv2026

Towards Terabit/$\lambda$/s Multidimensional Silicon Photonic Engine

Increasing artificial intelligence (AI) workloads drive co-packaged optics (CPO), which integrates optical engines with electronic components. Optical interconnects can extend transmission distances and reduce latency, allowing distributed clusters in AI factories to operate as a unified computational unit. However, escalating data throughput necessitates greater parallelization of light within ultracompact form factors while maintaining stringent energy efficiency and latency constraints. Here, we present a multidimensional silicon photonic engine that achieves a communication capacity exceeding 1.8 terabit/s/lambda/s. By monolithically integrating transceivers, spatial and polarization (de)multiplexers, and optical signal processors on a single chip, we eliminate bulky discrete (de)multiplexers and power-hungry digital signal processing (DSP). In experiments, the photonic engine can be self-configured to identify two, four, or six concurrent spatial and polarization channels per fiber while mitigating dynamic channel crosstalk. Compared with the state-of-art DSP, our approach achieves >5,000-fold reductions in both power consumption and processing latency at a MIMO processing order of six. Furthermore, we demonstrate full-duplex, modulation-format-transparent inter-chip communication over 300-meter fiber. These results represent a paradigm shift for optical engines in future high-performance computing and AI-driven data centers.

physics.optics↗

arXiv2026

Real-Time Dynamic Crosstalk Tracking and Compensation via Photonic Blind Source Separation

Space-division multiplexing in few-mode fibers can substantially increase optical-link capacity, but its practical deployment is hindered by dynamically varying inter-modal crosstalk induced by environmental perturbations. Although silicon photonic reconfigurable mode processors have been widely demonstrated, continuous adaptation during high-speed intensity-modulation direct-detection (IM/DD) transmission remains challenging. Here, we experimentally demonstrate real-time tracking and compensation of dynamically varying inter-modal crosstalk using a hybrid photonic--electronic framework that combines an integrated silicon photonic processor with an FPGA-based control backend. The photonic processor performs signal separation in the optical domain, while the FPGA extracts signal statistics and implements a blind source separation (BSS)-based feedback algorithm without requiring dedicated training sequences. A two-stage optimization strategy combines rapid crosstalk suppression with stable continuous tracking under dynamic channel variations. We demonstrate photonic blind source separation for signals up to 100 GBaud and FPGA-enabled continuous adaptation for two 64 GBaud data channels in a few-mode-fiber transmission system. The system maintains bit-error rates below $10^{-4}$ under dynamic crosstalk, with a per-update control latency of 4 ms. These results establish a practical route toward low-latency, hardware-efficient adaptive photonic front ends for dynamic high-speed IM/DD space-division-multiplexed links.

physics.optics↗

arXiv2026

An orthogonal-to-non-orthogonal multiplexing format converter

Time-frequency orthogonality has been a foundational principle in the historical development of optical communications, whether in dense wavelength division multiplexing (WDM) within long-reach high-capacity coherent optical transmission or in time-frequency division multiple access within short-reach dense passive optical networks. Towards next-generation agile optical networks, jointly programmable orthogonal and non-orthogonal regulation offers flexible spectral allocation, ultra-dense packet distribution, and increased capacity. For bridging the fundamental differences of physical implementation, we propose and demonstrate a versatile orthogonal to non-orthogonal multiplexing format converter, with application to high-speed coherent optical transmission network enabled by a Talbot-based processor. The programmable Talbot-processed pumps coherently transfer and superpose optical signals of distinct wavelength channels onto a single channel through cross-phase modulation. We first demonstrate flexible conversion of two 80-Gbps WDM QPSK channels separated by 200-250 GHz into a non-orthogonal power-division multiplexing channel, while maintaining the high-quality encoded information in the digital domain. We then validate a digital-subcarrier-multiplexing dense access scenario in which eight 20-Gbps sub-channels are combined, converted, transmitted, and successfully decoded over a field-deployed fiber. The multiplexing format converter promises potential for applications in next-generation optical systems and networks with complex topologies and dense populations.

physics.optics↗

arXiv2025

Programmable 200 GOPS Hopfield-inspired photonic Ising machine

Ising machines offer a compelling approach to addressing NP-hard problems, but physical realizations that are simultaneously scalable, reconfigurable, fast, and stable remain elusive. Quantum annealers, like D-Wave's cryogenic hardware, target combinatorial optimization tasks, but quadratic scaling of qubit requirements with problem size limits their scalability on dense graphs. Here, we introduce a programmable, stable, room-temperature optoelectronic oscillator (OEO)-based Ising machine with linear scaling in spin representation. Inspired by Hopfield networks, our architecture solves fully-connected problems with up to 256 spins (65,536 couplings), and $>$41,000 spins (205,000+ couplings) if sparse. Our system leverages cascaded thin-film lithium niobate modulators, a semiconductor optical amplifier, and a digital signal processing (DSP) engine in a recurrent time-encoded loop, demonstrating potential $>$200 giga-operations per second for spin coupling and nonlinearity. This platform achieves the largest spin configuration in an OEO-based photonic Ising machine, enabled by high intrinsic speed. We experimentally demonstrate best-in-class solution quality for Max-Cut problems of arbitrary graph topologies (2,000 and 20,000 spins) among photonic Ising machines and obtain ground-state solutions for number partitioning and lattice protein folding - benchmarks previously unaddressed by photonic systems. Our system leverages inherent noise from high baud rates to escape local minima and accelerate convergence. Finally, we show that embedding DSP - traditionally used in optical communications - within optical computation enhances convergence and solution quality, opening new frontiers in scalable, ultrafast computing for optimization, neuromorphic processing, and analog AI.

physics.optics↗

arXiv2025

Online training and pruning of multi-wavelength photonic neural networks

CMOS-compatible photonic integrated circuits (PICs) are emerging as a promising platform in artificial intelligence (AI) computing. Owing to the compact footprint of microring resonators (MRRs) and the enhanced interconnect efficiency enabled by wavelength division multiplexing (WDM), MRR-based photonic neural networks (PNNs) are particularly promising for large-scale integration. However, the scalability and energy efficiency of such systems are fundamentally limited by the MRR resonance wavelength variations induced by fabrication process variations (FPVs) and environmental fluctuations. Existing solutions use post-fabrication approaches or thermo-optic tuning, incurring high control power and additional process complexity. In this work, we introduce an online training and pruning method that addresses this challenge, adapting to FPV-induced and thermally induced shifts in MRR resonance wavelength. By incorporating a power-aware pruning term into the conventional loss function, our approach simultaneously optimizes the PNN accuracy and the total power consumption for MRR tuning. In proof-of-concept on-chip experiments on the Iris dataset, our system PNNs can adaptively train to maintain a 96% classification accuracy, while achieving a 44.7% reduction in tuning power via pruning. Additionally, our approach reduces the power consumption by orders-of-magnitude on larger datasets. By addressing chip-to-chip variation and minimizing power requirements, our approach significantly improves the scalability and energy efficiency of MRR-based integrated analog photonic processors, paving the way for large-scale PICs to enable versatile applications including neural networks, photonic switching, LiDAR, and radio-frequency beamforming.

physics.optics↗

arXiv2025

Phase amplification microscopy with femtometer-level accuracy

We demonstrate a major breakthrough in laser interferometry and microscopy achieving femtometer-level measurement accuracy and beyond, termed Phase Amplification microscopy ({\Phi}-Amp). By exploiting the native silicide substrate as a phase cavity, our phase-gain theory predicts that weak sub-atomic phase signals can be magnified over 1000-fold, thus bypassing the shot-noise limit. We experimentally achieved a 158.2-fold phase gain for graphene in ambient air, corresponding ~ 730 femtometer accuracy. To fully unleash the potential of {\Phi}-Amp for atomic fabrication and quantum measurement, we quantified interlayer spacing differences between AB-stacked and 30-degree-twisted bilayer graphene to be ~ 0.77 Angstroms and further detected atomic impurities and defects on large atomic structures. As the first wide-field metrology tool, we envision {\Phi}-Amp may accelerate the scaling up of atomic quantum devices.

physics.optics↗

arXiv2025

Large-scale artificial intelligence with 41 million nanophotonic neurons on a metasurface

Conventional integrated circuits (ICs) struggle to meet the escalating demands of artificial intelligence (AI). This has sparked a renewed interest in an unconventional computing paradigm: neuromorphic (brain-inspired) computing. However, current neuromorphic systems face significant challenges in delivering a large number of parameters (i.e., weights) required for large-scale AI models. As a result, most neuromorphic hardware is limited to basic benchmark demonstrations, hindering its application to real-world AI challenges. Here, we present a large-scale optical neural network (ONN) for machine learning acceleration, featuring over 41 million photonic neurons. This system not only surpasses digital electronics in speed and energy efficiency but more importantly, closes the performance gap with large-scale AI models. Our ONN leverages an innovative optical metasurface device featuring numerous spatial modes. This device integrates over 41 million meta-atoms on a 10 mm$^2$ metasurface chip, enabling the processing of tens of millions of weights in a single operation. For the first time, we demonstrate that an ONN, utilizing a single-layer metasurface, can match the performance of deep and large-scale deep learning models, such as ResNet and Vision Transformer, across various benchmark tasks. Additionally, we show that our system can deliver high-performance solutions to real-world AI challenges through its unprecedented scale, such as accelerating the analysis of multi-gigapixel whole slide images (WSIs) for cancer detection by processing the million-pixel sub-image in a single shot. Our system reduces computing time and energy consumption by over 1,000 times compared to state-of-the-art graphic processing units (GPUs). This work presents a large-scale, low-power, and high-performance neuromorphic computing system, paving the way for future disruptive AI technologies.

physics.optics↗

arXiv2025

Beyond Terabit/s Integrated Neuromorphic Photonic Processor for DSP-Free Optical Interconnects

The rapid expansion of generative AI drives unprecedented demands for high-performance computing. Training large-scale AI models now requires vast interconnected GPU clusters across multiple data centers. Multi-scale AI training and inference demand uniform, ultra-low latency, and energy-efficient links to enable massive GPUs to function as a single cohesive unit. However, traditional electrical and optical interconnects, relying on conventional digital signal processors (DSPs) for signal distortion compensation, increasingly fail to meet these stringent requirements. To overcome these limitations, we present an integrated neuromorphic optical signal processor (OSP) that leverages deep reservoir computing and achieves DSP-free, all-optical, real-time processing. Experimentally, our OSP achieves a 100 Gbaud PAM4 per lane, 1.6 Tbit/s data center interconnect over a 5 km optical fiber in the C-band (equivalent to over 80 km in the O-band), far exceeding the reach of state-of-the-art DSP solutions, which are fundamentally constrained by chromatic dispersion in IMDD systems. Simultaneously, it reduces processing latency by four orders of magnitude and energy consumption by three orders of magnitude. Unlike DSPs, which introduce increased latency at high data rates, our OSP maintains consistent, ultra-low latency regardless of data rate scaling, making it ideal for future optical interconnects. Moreover, the OSP retains full optical field information for better impairment compensation and adapts to various modulation formats, data rates, and wavelengths. Fabricated using a mature silicon photonic process, the OSP can be monolithically integrated with silicon photonic transceivers, enhancing the compactness and reliability of all-optical interconnects. This research provides a highly scalable, energy-efficient, and high-speed solution, paving the way for next-generation AI infrastructure.

physics.optics↗

arXiv2025

Roadmap on Neuromorphic Photonics

This roadmap consolidates recent advances while exploring emerging applications, reflecting the remarkable diversity of hardware platforms, neuromorphic concepts, and implementation philosophies reported in the field. It emphasizes the critical role of cross-disciplinary collaboration in this rapidly evolving field.

cs.ET↗

arXiv2024

Fully integrated hybrid multimode-multiwavelength photonic processor with picosecond latency

High-speed signal processing is essential for maximizing data throughput in emerging communication applications, like multiple-input multiple-output (MIMO) systems and radio-frequency (RF) interference cancellation. However, as these technologies scale, they increase hardware complexity, computing power demands, and create significant digital signal processing (DSP) challenges. While transistor miniaturization has improved digital electronic processors, they still face physical bottlenecks, limiting computational throughput and increasing DSP latency. Photonic processors present a promising alternative, offering large bandwidth, low loss, parallel processing, and low latency. Yet, scalability in photonic processors remains limited by system integration, device size, and on-chip multiplexing challenges. Here, we introduce a scalable on-chip hybrid multiplexed photonic processor, combining mode-division multiplexing (MDM) and wavelength-division multiplexing (WDM). This marks the first implementation of a monolithically integrated MDM-WDM-compatible processor, featuring mode multiplexers, multimode microring resonators, and multimode balanced photodetectors. Furthermore, we demonstrate real-time unscrambling of 5 Gb/s non-return-to-zero optical MIMO signals and RF phase-shift keying signal unjamming. Our system's 30 ps processing latency makes it ideal for real-time MIMO and RF applications. Our analysis reveals that hybrid MDM-WDM multiplexing improves the number of operations per second by 4.1 times over spatially multiplexed WDM configurations, positioning it as a strong candidate for next-generation large-scale photonic processors.

physics.optics↗

arXiv2024

Perfecting Imperfect Physical Neural Networks with Transferable Robustness using Sharpness-Aware Training

AI models are essential in science and engineering, but recent advances are pushing the limits of traditional digital hardware. To address these limitations, physical neural networks (PNNs), which use physical substrates for computation, have gained increasing attention. However, developing effective training methods for PNNs remains a significant challenge. Current approaches, regardless of offline and online training, suffer from significant accuracy loss. Offline training is hindered by imprecise modeling, while online training yields device-specific models that can't be transferred to other devices due to manufacturing variances. Both methods face challenges from perturbations after deployment, such as thermal drift or alignment errors, which make trained models invalid and require retraining. Here, we address the challenges with both offline and online training through a novel technique called Sharpness-Aware Training (SAT), where we innovatively leverage the geometry of the loss landscape to tackle the problems in training physical systems. SAT enables accurate training using efficient backpropagation algorithms, even with imprecise models. PNNs trained by SAT offline even outperform those trained online, despite modeling and fabrication errors. SAT also overcomes online training limitations by enabling reliable transfer of models between devices. Finally, SAT is highly resilient to perturbations after deployment, allowing PNNs to continuously operate accurately under perturbations without retraining. We demonstrate SAT across three types of PNNs, showing it is universally applicable, regardless of whether the models are explicitly known. This work offers a transformative, efficient approach to training PNNs, addressing critical challenges in analog computing and enabling real-world deployment.

physics.optics↗

arXiv2024

High-precision programming of large-scale ring resonator circuits with minimal pre-calibration

Microring resonators (MRRs) are essential components in large-scale photonic integrated circuits (PICs), but programming these circuits with high precision and efficiency remains an unsolved challenge. Conventional methods rely on complex calibration processes that are both time-consuming and often inaccurate, limiting the scalability of PICs. This work introduces an innovative control method called chip-in-the-loop optimization (ChiL) that addresses this challenge by offering high scalability, precision, fast convergence, and robustness. ChiL reduces the calibration complexity for an $N$ devices system from $O(k^N)$ to a single-shot measurement, while maintaining a record-high precision over 9-bit in the presence of system imperfections, including fabrication variances, thermal crosstalk, and temperature drift. Using ChiL, we experimentally demonstrate a photonic solver for computing matrix eigenvalues and eigenvectors with errors on the order of $10^{-4}$. Additionally, we achieve a photonic neural network (PNN) with accuracy and a confusion matrix identical to those of digital computers. ChiL offers a practical approach for programming large-scale PICs and bridges the gap between analog photonic and digital electronic computing and signal processing in both scale and precision.

physics.optics↗

arXiv2024

A 103-TOPS/mm$^2$ Integrated Photonic Computing Engine Enabling Next-Generation Reservoir Computing

Reservoir computing (RC) is a leading machine learning algorithm for information processing due to its rich expressiveness. A new RC paradigm has recently emerged, showcasing superior performance and delivering more interpretable results with shorter training data sets and training times, representing the next generation of RC computing. This work presents the first realization of a high-speed next-generation RC system on an integrated photonic chip. Our experimental results demonstrate state-of-the-art forecasting and classification performances under various machine learning tasks and achieve the fastest speeds of 60 Gbaud and a computing density of 103 tera operations/second/mm$^2$ (TOPS/mm$^2$). The passive system, composed of a simple star coupler with on-chip delay lines, offers several advantages over traditional RC systems, including no speed limitations, compact footprint, extremely high fabrication error tolerance, fewer metaparameters, and greater interpretability. This work lays the foundation for ultrafast on-chip photonic RC, representing significant progress toward developing next-generation high-speed photonic computing and signal processing.

cs.ET↗

arXiv2024

Control-free and efficient integrated photonic neural networks via hardware-aware training and pruning

Integrated photonic neural networks (PNNs) are at the forefront of AI computing, leveraging on light's unique properties, such as large bandwidth, low latency, and potentially low power consumption. Nevertheless, the integrated optical components within PNNs are inherently sensitive to external disturbances and thermal interference, which can detrimentally affect computing accuracy and reliability. Current solutions often use complicated control methods, resulting in high hardware complexity impractical for large-scale PNNs. In response, we propose a novel hardware-aware training and pruning approach. The core idea is to train the parameters of a physical neural network towards its noise-robust and energy-efficient region. This innovation enables control-free and energy-efficient photonic computing. Our method is validated across diverse integrated PNN architectures. Through experimental validation, our approach significantly enhances the computing precision of MRR-based PNN, achieving a notable 4-bit improvement without the need for complex device control mechanisms or energy-intensive temperature stabilization circuits. Specifically, it improves the accuracy of experimental handwritten digit classification from 67.0% to 95.0%, nearing theoretical limits and achieved without a thermoelectric controller. Additionally, this approach reduces the energy by tenfold. We further extend the validation to various architectures, such as PCM-based PNN, demonstrating the broad applicability of our approach across different platforms. This advancement represents a significant step towards the practical, energy-efficient, and noise-resilient implementation of large-scale integrated PNNs.

physics.optics↗

arXiv2024

Photonic RF Channelization Based on Microcombs

In recent decades, microwave photonic channelization techniques have developed significantly. Characterized by low loss, high versatility, large instantaneous bandwidth, and immunity to electromagnetic interference, microwave photonic channelization addresses the requirements of modern radar and electronic warfare for receivers. Microresonator-based optical frequency combs are promising devices for photonic channelized receivers, enabling full advantage of multicarriers, large bandwidths, and accelerating the integration process of microwave photonic channelized receivers. In this paper, we review the research progress and trends in microwave photonic channelization, focusing on schemes that utilize integrated microcombs. We discuss the potential of microcomb-based RF channelization, as well as their challenges and limitations, and provide perspectives for their future development in the context of on-chip silicon-based photonics.

physics.optics↗

arXiv2024

Integrated lithium niobate microwave photonic processing engine

Integrated microwave photonics is an intriguing field that leverages integrated photonic technologies for the generation, transmission, and manipulation of microwave signals in chip-scale optical systems. In particular, ultrafast processing and computation of analog electronic signals in the optical domain with high fidelity and low latency could enable a variety of applications such as MWP filters, microwave signal processing, and image recognition. An ideal photonic platform for achieving these integrated MWP processing tasks shall simultaneously offer an efficient, linear and high-speed electro-optic modulation block to faithfully perform microwave-optic conversion at low power, and a low-loss functional photonic network that can be configured for a variety of signal processing tasks, as well as large-scale, low-cost manufacturability to monolithically integrate the two building blocks on the same chip. In this work, we demonstrate such an integrated MWP processing engine based on a thin-film lithium niobate platform capable of performing multi-purpose processing and computation tasks of analog signals up to 92 giga samples per second at CMOS-compatible voltages. We demonstrate high-speed analog computation, i.e., first- and second-order temporal integration and differentiation with computing accuracies up to 98.1 %, and deploy these functions to showcase three proof-of-concept applications, namely, ordinary differential equation solving, ultra-wideband signal generation and high-speed edge detection of images. We further leverage the image edge detector to enable a photonic-assisted image segmentation model that could effectively outline the boundaries of melanoma lesion in medical diagnostic images, achieving orders of magnitude faster processing speed and lower power consumption than conventional electronic processors.

physics.optics↗