arXiv subjects

Jianyang Shi

Publications and source records attributed to Jianyang Shi.

8 recordsLinked to original sources

arXiv2026

Fully multiplexed photonic tensor computing

Tensor operations dominate modern computational workloads, yet their further acceleration demands hardware platforms with greater parallelism. Although photonic computing provides a compelling route for parallel processing, fully exploiting all native multiplexing dimensions of optical fields is impeded by the challenges in routing and programming light in all dimensions simultaneously. Here we introduce FieldCore, a fully multiplexed photonic tensor core that jointly harnesses wavelength, radio-frequency, guided-mode, time and space dimensions, thereby enabling parallelism to scale multiplicatively within a single optical field. Enabled by inverse-designed silicon photonics, FieldCore preserves a uniform programmed computation across all multiplexed channels in parallel. Experimentally, we validate and benchmark its performance from ultra-high-baudrate arithmetic operations to high-fidelity image convolution and parallel handwritten-digit recognition. We further use FieldCore to unlock applications that naturally require high-dimensional data processing, such as high-dimensional hyperspectral classification and massively parallel mechanical fault diagnosis. Our FieldCore supports an estimated aggregate compute throughput of 69.12 tera operations per second (TOPS) and accommodates up to 1,800 parallel input streams within a single core, establishing a scalable paradigm for fully multiplexed photonic tensor computing and AI inference.

physics.optics↗

arXiv2025

BC-GAN: A Generative Adversarial Network for Synthesizing a Batch of Collocated Clothing

Collocated clothing synthesis using generative networks has become an emerging topic in the field of fashion intelligence, as it has significant potential economic value to increase revenue in the fashion industry. In previous studies, several works have attempted to synthesize visually-collocated clothing based on a given clothing item using generative adversarial networks (GANs) with promising results. These works, however, can only accomplish the synthesis of one collocated clothing item each time. Nevertheless, users may require different clothing items to meet their multiple choices due to their personal tastes and different dressing scenarios. To address this limitation, we introduce a novel batch clothing generation framework, named BC-GAN, which is able to synthesize multiple visually-collocated clothing images simultaneously. In particular, to further improve the fashion compatibility of synthetic results, BC-GAN proposes a new fashion compatibility discriminator in a contrastive learning perspective by fully exploiting the collocation relationship among all clothing items. Our model was examined in a large-scale dataset with compatible outfits constructed by ourselves. Extensive experiment results confirmed the effectiveness of our proposed BC-GAN in comparison to state-of-the-art methods in terms of diversity, visual authenticity, and fashion compatibility.

cs.CV↗

arXiv2024

Seamless Optical Cloud Computing across Edge-Metro Network for Generative AI

The rapid advancement of generative artificial intelligence (AI) in recent years has profoundly reshaped modern lifestyles, necessitating a revolutionary architecture to support the growing demands for computational power. Cloud computing has become the driving force behind this transformation. However, it consumes significant power and faces computation security risks due to the reliance on extensive data centers and servers in the cloud. Reducing power consumption while enhancing computational scale remains persistent challenges in cloud computing. Here, we propose and experimentally demonstrate an optical cloud computing system that can be seamlessly deployed across edge-metro network. By modulating inputs and models into light, a wide range of edge nodes can directly access the optical computing center via the edge-metro network. The experimental validations show an energy efficiency of 118.6 mW/TOPs (tera operations per second), reducing energy consumption by two orders of magnitude compared to traditional electronic-based cloud computing solutions. Furthermore, it is experimentally validated that this architecture can perform various complex generative AI models through parallel computing to achieve image generation tasks.

cs.DC↗

arXiv2024

GiVE: Guiding Visual Encoder to Perceive Overlooked Information

Multimodal Large Language Models have advanced AI in applications like text-to-video generation and visual question answering. These models rely on visual encoders to convert non-text data into vectors, but current encoders either lack semantic alignment or overlook non-salient objects. We propose the Guiding Visual Encoder to Perceive Overlooked Information (GiVE) approach. GiVE enhances visual representation with an Attention-Guided Adapter (AG-Adapter) module and an Object-focused Visual Semantic Learning module. These incorporate three novel loss terms: Object-focused Image-Text Contrast (OITC) loss, Object-focused Image-Image Contrast (OIIC) loss, and Object-focused Image Discrimination (OID) loss, improving object consideration, retrieval accuracy, and comprehensiveness. Our contributions include dynamic visual focus adjustment, novel loss functions to enhance object retrieval, and the Multi-Object Instruction (MOInst) dataset. Experiments show our approach achieves state-of-the-art performance.

cs.CV↗

arXiv2024

Edge-guided inverse design of digital metamaterial-based mode multiplexers for high-capacity multi-dimensional interconnect

The escalating demands of compute-intensive applications urgently necessitate the adoption of optical interconnect technologies to overcome bottlenecks in scaling computing systems. This requires fully exploiting the inherent parallelism of light across scalable dimensions for data loading. Here we experimentally demonstrate a synergy of wavelength- and mode- multiplexing combined with high-order modulation formats to achieve multi-tens-of-terabits-per-second optical interconnects using foundry-compatible silicon photonic circuits. Implementing an edge-guided analog-and-digital optimization method that integrates high efficiency with fabrication robustness, we achieve the inverse design of mode multiplexers based on digital metamaterial waveguides. Furthermore, we employ a packaged five-mode multiplexing chip, achieving a single-wavelength interconnect capacity of 1.62 Tbit s-1 and a record-setting multi-dimensional interconnect capacity of 38.2 Tbit s-1 across 5 modes and 88 wavelength channels, with high-order formats up to 8-ary pulse-amplitude-modulation (PAM). This study highlights the transformative potential of optical interconnect technologies to surmount the constraints of electronic links, thus setting the stage for next-generation datacenter and optical compute interconnects.

physics.optics↗

arXiv2024

WiFi-GEN: High-Resolution Indoor Imaging from WiFi Signals Using Generative AI

Indoor imaging is a critical task for robotics and internet-ofthings. WiFi as an omnipresent signal is a promising candidate for carrying out passive imaging and synchronizing the up-to-date information to all connected devices. This is the first research work to consider WiFi indoor imaging as a multi-modal image generation task that converts the measured WiFi power into a high-resolution indoor image. Our proposedWiFi-GEN network achieves a shape reconstruction accuracy that is 275% of that achieved by physical model-based inversion methods. Additionally, the Frechet Inception Distance score has been significantly reduced by 82%. To examine the effectiveness of models for this task, the first large-scale dataset is released containing 80,000 pairs of WiFi signal and imaging target. Our model absorbs challenges for the model-based methods including the nonlinearity, ill-posedness and non-certainty into massive parameters of our generative AI network. The network is also designed to best fit measured WiFi signals and the desired imaging output. Code: https://github.com/CNFightingSjy/WiFiGEN

cs.CV↗

arXiv2022

GUDN: A novel guide network with label reinforcement strategy for extreme multi-label text classification

In natural language processing, extreme multi-label text classification is an emerging but essential task. The problem of extreme multi-label text classification (XMTC) is to recall some of the most relevant labels for a text from an extremely large label set. Large-scale pre-trained models have brought a new trend to this problem. Though the large-scale pre-trained models have made significant achievements on this problem, the valuable fine-tuned methods have yet to be studied. Though label semantics have been introduced in XMTC, the vast semantic gap between texts and labels has yet to gain enough attention. This paper builds a new guide network (GUDN) to help fine-tune the pre-trained model to instruct classification later. Furthermore, GUDN uses raw label semantics combined with a helpful label reinforcement strategy to effectively explore the latent space between texts and labels, narrowing the semantic gap, which can further improve predicted accuracy. Experimental results demonstrate that GUDN outperforms state-of-the-art methods on Eurlex-4k and has competitive results on other popular datasets. In an additional experiment, we investigated the input lengths' influence on the Transformer-based model's accuracy. Our source code is released at https://t.hk.uy/aFSH.

cs.CL↗

arXiv2021

Beyond 300Gbps Silicon Microring Modulator with AI Acceleration

Silicon microring modulator (Si-MRM) has become one of the most promising compact modulators to meet the increasing capacity requirements of the next generation optical interconnection. The limited electro-optical (E-O) bandwidth, low modulation efficiency, and inherent modulation nonlinearity are the major factors that limit the Si-MRM modulation speed. To address these issues, we comprehensively optimize the Si-MRM from the device to the modulation and the signal processing. Large modulation bandwidth over 67GHz is achieved in our newly fabricated Si-MRM. Additionally, the laser wavelength and bias voltage of Si-MRM are optimized to significantly improve the modulation performance. Finally, we comprehensively study the theoretical model of modulation nonlinearity in Si-MRM, especially transient nonlinearity. A bidirectional gate recurrent unit (Bi-GRU) neural network with minor modification is applied to compensate for the nonlinear impairments. With all these efforts, we experimentally demonstrate a 302 Gbps Si-MRM-based O-band optical interconnection and achieve 300 Gbps transmission over a 1-km standard single-mode fiber using the discrete multitone modulation format with bit and power loading (BPL-DMT).

physics.app-ph↗