arXiv subjects

Yiwen Hu

Publications and source records attributed to Yiwen Hu.

36 records · Page 2Linked to original sources

arXiv2025

Research on stock price forecast of general electric based on mixed CNN-LSTM model

Accurate stock price prediction is crucial for investors and financial institutions, yet the complexity of the stock market makes it highly challenging. This study aims to construct an effective model to enhance the prediction ability of General Electric's stock price trend. The CNN - LSTM model is adopted, combining the feature extraction ability of CNN with the long - term dependency handling ability of LSTM, and the Adam optimizer is used to adjust the parameters. In the data preparation stage, historical trading data of General Electric's stock is collected. After cleaning, handling missing values, and feature engineering, features with strong correlations to the closing price are selected and dimensionality reduction is performed. During model training, the data is divided into training, validation, and testing sets in a ratio of 7:2:1. The Stochastic Gradient Descent algorithm is used with a dynamic learning rate adjustment and L2 regularization, and the Mean Squared Error is used as the loss function, evaluated by variance, R - squared score, and maximum error. Experimental results show that the model loss decreases steadily, and the predicted values align well with the actual values, providing a powerful tool for investment decisions. However, the model's performance in real - time and extreme market conditions remains to be tested, and future improvements could consider incorporating more data sources.

cs.CE↗

arXiv2024

YuLan-Mini: An Open Data-efficient Language Model

Effective pre-training of large language models (LLMs) has been challenging due to the immense resource demands and the complexity of the technical processes involved. This paper presents a detailed technical report on YuLan-Mini, a highly capable base model with 2.42B parameters that achieves top-tier performance among models of similar parameter scale. Our pre-training approach focuses on enhancing training efficacy through three key technical contributions: an elaborate data pipeline combines data cleaning with data schedule strategies, a robust optimization method to mitigate training instability, and an effective annealing approach that incorporates targeted data selection and long context training. Remarkably, YuLan-Mini, trained on 1.08T tokens, achieves performance comparable to industry-leading models that require significantly more data. To facilitate reproduction, we release the full details of the data composition for each training phase. Project details can be accessed at the following link: https://github.com/RUC-GSAI/YuLan-Mini.

cs.CL↗

arXiv2024

Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systems

Recently, slow-thinking reasoning systems, such as o1, have demonstrated remarkable capabilities in solving complex reasoning tasks. These systems typically engage in an extended thinking process before responding to a query, allowing them to generate more thorough, accurate, and well-reasoned solutions. These systems are primarily developed and maintained by industry, with their core techniques not publicly disclosed. In response, an increasing number of studies from the research community aim to explore the technical foundations underlying these powerful reasoning systems. Building on these prior efforts, this paper presents a reproduction report on implementing o1-like reasoning systems. We introduce an ``imitate, explore, and self-improve'' framework, denoted as \textbf{STILL-2}, as our primary technical approach to train the reasoning model. In the initial phase, we use distilled long-form thought data to fine-tune the reasoning model, enabling it to invoke a slow-thinking mode. The model is then encouraged to explore challenging problems by generating multiple rollouts, which can result in increasingly more high-quality trajectories that lead to correct answers. Furthermore, the model undergoes self-improvement by iteratively refining its training dataset. To verify the effectiveness of this approach, we conduct extensive experiments on three challenging benchmarks. The experimental results demonstrate that our approach achieves competitive performance compared to industry-level reasoning systems on these benchmarks.

cs.AI↗

arXiv2024

Fast generation of arbitrary optical focus array

We report a novel method to generate arbitrary optical focus arrays (OFAs). Our approach rapidly produces computer-generated holograms (CGHs) to precisely control the positions and the intensities of the foci. This is achieved by replacing the fast Fourier transform (FFT) operation in the conventional iterative Fourier-transform algorithm (IFTA) with a linear algebra one, identifying/removing zero elements from the matrices, and employing a generalized weighting strategy. On the premise of accelerating the calculation speed by >70 times, we demonstrate OFA with 99% intensity precision in the experiment. Our method proves effective and is applicable for the systems in which real-time OFA generation is essential.

physics.optics↗

arXiv2024

MixPolyp: Integrating Mask, Box and Scribble Supervision for Enhanced Polyp Segmentation

Limited by the expensive labeling, polyp segmentation models are plagued by data shortages. To tackle this, we propose the mixed supervised polyp segmentation paradigm (MixPolyp). Unlike traditional models relying on a single type of annotation, MixPolyp combines diverse annotation types (mask, box, and scribble) within a single model, thereby expanding the range of available data and reducing labeling costs. To achieve this, MixPolyp introduces three novel supervision losses to handle various annotations: Subspace Projection loss (L_SP), Binary Minimum Entropy loss (L_BME), and Linear Regularization loss (L_LR). For box annotations, L_SP eliminates shape inconsistencies between the prediction and the supervision. For scribble annotations, L_BME provides supervision for unlabeled pixels through minimum entropy constraint, thereby alleviating supervision sparsity. Furthermore, L_LR provides dense supervision by enforcing consistency among the predictions, thus reducing the non-uniqueness. These losses are independent of the model structure, making them generally applicable. They are used only during training, adding no computational cost during inference. Extensive experiments on five datasets demonstrate MixPolyp's effectiveness.

cs.CV↗

arXiv2024

In situ fully vectorial tomography and pupil function retrieval of tightly focused fields

Tightly focused optical fields are essential in nano-optics, but their applications have been limited by the challenges of accurate yet efficient characterization. In this article, we develop an in situ method for reconstructing the fully vectorial information of tightly focused fields in three-dimensional (3D) space, while simultaneously retrieving the pupil functions. Our approach encodes these fields using phase-modulated focusing and polarization-split detection, followed by decoding through an algorithm based on least-sampling matrix-based Fourier transform and analytically derived gradient. We further employ a focus scanning strategy. When combined with our decoding algorithm, this strategy mitigates the imperfections in the detection path. This approach requires only 10 frames of 2D measurements to realize approximate 90% accuracy in tomography and pupil function retrieval within 10s. Thus, it serves as a robust and convenient tool for the precise characterization and optimization of light at the nanoscale. We apply this technique to fully vectorial field manipulation, adaptive-optics-assisted nanoscopy, and addressing mixed-state problems.

physics.optics↗

arXiv2024

Towards a Benchmark for Colorectal Cancer Segmentation in Endorectal Ultrasound Videos: Dataset and Model Development

Endorectal ultrasound (ERUS) is an important imaging modality that provides high reliability for diagnosing the depth and boundary of invasion in colorectal cancer. However, the lack of a large-scale ERUS dataset with high-quality annotations hinders the development of automatic ultrasound diagnostics. In this paper, we collected and annotated the first benchmark dataset that covers diverse ERUS scenarios, i.e. colorectal cancer segmentation, detection, and infiltration depth staging. Our ERUS-10K dataset comprises 77 videos and 10,000 high-resolution annotated frames. Based on this dataset, we further introduce a benchmark model for colorectal cancer segmentation, named the Adaptive Sparse-context TRansformer (ASTR). ASTR is designed based on three considerations: scanning mode discrepancy, temporal information, and low computational complexity. For generalizing to different scanning modes, the adaptive scanning-mode augmentation is proposed to convert between raw sector images and linear scan ones. For mining temporal information, the sparse-context transformer is incorporated to integrate inter-frame local and global features. For reducing computational complexity, the sparse-context block is introduced to extract contextual features from auxiliary frames. Finally, on the benchmark dataset, the proposed ASTR model achieves a 77.6% Dice score in rectal cancer segmentation, largely outperforming previous state-of-the-art methods.

eess.IV↗

arXiv2024

LLMBox: A Comprehensive Library for Large Language Models

To facilitate the research on large language models (LLMs), this paper presents a comprehensive and unified library, LLMBox, to ease the development, use, and evaluation of LLMs. This library is featured with three main merits: (1) a unified data interface that supports the flexible implementation of various training strategies, (2) a comprehensive evaluation that covers extensive tasks, datasets, and models, and (3) more practical consideration, especially on user-friendliness and efficiency. With our library, users can easily reproduce existing methods, train new models, and conduct comprehensive performance comparisons. To rigorously test LLMBox, we conduct extensive experiments in a diverse coverage of evaluation settings, and experimental results demonstrate the effectiveness and efficiency of our library in supporting various implementations related to LLMs. The detailed introduction and usage guidance can be found at https://github.com/RUCAIBox/LLMBox.

cs.CL↗

arXiv2024

ECC-PolypDet: Enhanced CenterNet with Contrastive Learning for Automatic Polyp Detection

Accurate polyp detection is critical for early colorectal cancer diagnosis. Although remarkable progress has been achieved in recent years, the complex colon environment and concealed polyps with unclear boundaries still pose severe challenges in this area. Existing methods either involve computationally expensive context aggregation or lack prior modeling of polyps, resulting in poor performance in challenging cases. In this paper, we propose the Enhanced CenterNet with Contrastive Learning (ECC-PolypDet), a two-stage training \& end-to-end inference framework that leverages images and bounding box annotations to train a general model and fine-tune it based on the inference score to obtain a final robust model. Specifically, we conduct Box-assisted Contrastive Learning (BCL) during training to minimize the intra-class difference and maximize the inter-class difference between foreground polyps and backgrounds, enabling our model to capture concealed polyps. Moreover, to enhance the recognition of small polyps, we design the Semantic Flow-guided Feature Pyramid Network (SFFPN) to aggregate multi-scale features and the Heatmap Propagation (HP) module to boost the model's attention on polyp targets. In the fine-tuning stage, we introduce the IoU-guided Sample Re-weighting (ISR) mechanism to prioritize hard samples by adaptively adjusting the loss weight for each sample during fine-tuning. Extensive experiments on six large-scale colonoscopy datasets demonstrate the superiority of our model compared with previous state-of-the-art detectors.

cs.CV↗

arXiv2023

Rigorous and efficient diffraction modeling between arbitrary planes by angular spectrum rearrangement

In computational optics, numerical modeling of diffraction between arbitrary planes offers unparalleled flexibility. However, existing methods suffer from the trade-off between computational accuracy and efficiency. To resolve this dilemma, we present a novel approach that rigorously and efficiently models wave propagation between two arbitrary planes. This is achieved by rearranging the angular spectrum of the source field, coupled with linear algebraic computations. Notably, our method achieves comparable computational efficiency to the control method for both scalar and vectorial diffraction modeling, while eliminating nearly all numerical errors. Furthermore, we selectively merge the angular spectrum to further enhance the efficiency at the expense of precision in a controlled manner. Thereafter, the time consumption is reduced to at most 3% of that before merging.

physics.optics↗

arXiv2023

WeakPolyp: You Only Look Bounding Box for Polyp Segmentation

Limited by expensive pixel-level labels, polyp segmentation models are plagued by data shortage and suffer from impaired generalization. In contrast, polyp bounding box annotations are much cheaper and more accessible. Thus, to reduce labeling cost, we propose to learn a weakly supervised polyp segmentation model (i.e., WeakPolyp) completely based on bounding box annotations. However, coarse bounding boxes contain too much noise. To avoid interference, we introduce the mask-to-box (M2B) transformation. By supervising the outer box mask of the prediction instead of the prediction itself, M2B greatly mitigates the mismatch between the coarse label and the precise prediction. But, M2B only provides sparse supervision, leading to non-unique predictions. Therefore, we further propose a scale consistency (SC) loss for dense supervision. By explicitly aligning predictions across the same image at different scales, the SC loss largely reduces the variation of predictions. Note that our WeakPolyp is a plug-and-play model, which can be easily ported to other appealing backbones. Besides, the proposed modules are only used during training, bringing no computation cost to inference. Extensive experiments demonstrate the effectiveness of our proposed WeakPolyp, which surprisingly achieves a comparable performance with a fully supervised model, requiring no mask annotations at all.

cs.CV↗

arXiv2022

TextBox 2.0: A Text Generation Library with Pre-trained Language Models

To facilitate research on text generation, this paper presents a comprehensive and unified library, TextBox 2.0, focusing on the use of pre-trained language models (PLMs). To be comprehensive, our library covers $13$ common text generation tasks and their corresponding $83$ datasets and further incorporates $45$ PLMs covering general, translation, Chinese, dialogue, controllable, distilled, prompting, and lightweight PLMs. We also implement $4$ efficient training strategies and provide $4$ generation objectives for pre-training new PLMs from scratch. To be unified, we design the interfaces to support the entire research pipeline (from data loading to training and evaluation), ensuring that each step can be fulfilled in a unified way. Despite the rich functionality, it is easy to use our library, either through the friendly Python API or command line. To validate the effectiveness of our library, we conduct extensive experiments and exemplify four types of research scenarios. The project is released at the link: https://github.com/RUCAIBox/TextBox.

cs.CL↗

arXiv2022

BoxPolyp:Boost Generalized Polyp Segmentation Using Extra Coarse Bounding Box Annotations

Accurate polyp segmentation is of great importance for colorectal cancer diagnosis and treatment. However, due to the high cost of producing accurate mask annotations, existing polyp segmentation methods suffer from severe data shortage and impaired model generalization. Reversely, coarse polyp bounding box annotations are more accessible. Thus, in this paper, we propose a boosted BoxPolyp model to make full use of both accurate mask and extra coarse box annotations. In practice, box annotations are applied to alleviate the over-fitting issue of previous polyp segmentation models, which generate fine-grained polyp area through the iterative boosted segmentation model. To achieve this goal, a fusion filter sampling (FFS) module is firstly proposed to generate pixel-wise pseudo labels from box annotations with less noise, leading to significant performance improvements. Besides, considering the appearance consistency of the same polyp, an image consistency (IC) loss is designed. Such IC loss explicitly narrows the distance between features extracted by two different networks, which improves the robustness of the model. Note that our BoxPolyp is a plug-and-play model, which can be merged into any appealing backbone. Quantitative and qualitative experimental results on five challenging benchmarks confirm that our proposed model outperforms previous state-of-the-art methods by a large margin.

cs.CV↗

arXiv2022

Toward Clinically Assisted Colorectal Polyp Recognition via Structured Cross-modal Representation Consistency

The colorectal polyps classification is a critical clinical examination. To improve the classification accuracy, most computer-aided diagnosis algorithms recognize colorectal polyps by adopting Narrow-Band Imaging (NBI). However, the NBI usually suffers from missing utilization in real clinic scenarios since the acquisition of this specific image requires manual switching of the light mode when polyps have been detected by using White-Light (WL) images. To avoid the above situation, we propose a novel method to directly achieve accurate white-light colonoscopy image classification by conducting structured cross-modal representation consistency. In practice, a pair of multi-modal images, i.e. NBI and WL, are fed into a shared Transformer to extract hierarchical feature representations. Then a novel designed Spatial Attention Module (SAM) is adopted to calculate the similarities between the class token and patch tokens %from multi-levels for a specific modality image. By aligning the class tokens and spatial attention maps of paired NBI and WL images at different levels, the Transformer achieves the ability to keep both global and local representation consistency for the above two modalities. Extensive experimental results illustrate the proposed method outperforms the recent studies with a margin, realizing multi-modal prediction with a single Transformer while greatly improving the classification accuracy when only with WL images.

cs.CV↗

arXiv2021

Shallow Attention Network for Polyp Segmentation

Accurate polyp segmentation is of great importance for colorectal cancer diagnosis. However, even with a powerful deep neural network, there still exists three big challenges that impede the development of polyp segmentation. (i) Samples collected under different conditions show inconsistent colors, causing the feature distribution gap and overfitting issue; (ii) Due to repeated feature downsampling, small polyps are easily degraded; (iii) Foreground and background pixels are imbalanced, leading to a biased training. To address the above issues, we propose the Shallow Attention Network (SANet) for polyp segmentation. Specifically, to eliminate the effects of color, we design the color exchange operation to decouple the image contents and colors, and force the model to focus more on the target shape and structure. Furthermore, to enhance the segmentation quality of small polyps, we propose the shallow attention module to filter out the background noise of shallow features. Thanks to the high resolution of shallow features, small polyps can be preserved correctly. In addition, to ease the severe pixel imbalance for small polyps, we propose a probability correction strategy (PCS) during the inference phase. Note that even though PCS is not involved in the training phase, it can still work well on a biased model and consistently improve the segmentation performance. Quantitative and qualitative experimental results on five challenging benchmarks confirm that our proposed SANet outperforms previous state-of-the-art methods by a large margin and achieves a speed about 72FPS.

cs.CV↗

arXiv2019

Towards a multi-input astrophotonic AWG spectrograph

Astrophotonics is the new frontier technology to develop diffraction-limited, miniaturized, and cost-effective instruments for the next generation of large telescopes. For various astronomical studies such as probing the early universe, observing in near infrared (NIR) is crucial. To address this, we are developing moderate resolution (R = 1500) on-chip astrophotonic spectrographs in the NIR bands (J Band: 1.1-1.4 $μm$; H band: 1.45-1.7 $μm$) using the concept of arrayed waveguide gratings (AWGs). We fabricate the AWGs using a silica-on-silicon substrate. The waveguides on these AWGs are 2 $μm$ wide and 0.1 $μm$ high Si3N4 core buried inside a 15 $μm$ thick SiO2 cladding. To make the maximal use of astrophotonic integration such as coupling the AWGs with multiple single-mode fibers coming from photonic lanterns or fiber Bragg gratings (FBGs), we require a multi-input AWG design. In a multi-input AWG, the output spectrum due to each individual input channel overlaps to produce a combined spectrum from all inputs. This on-chip combination of light effectively improves the signal-to-noise ratio as compared to spreading the photons to several AWGs with single inputs. In this paper, we present the design and simulation results of an AWG in the H band with 3 input waveguides (channels). The resolving power of individual input channels is 1500, while the overall resolving power with three inputs together is 500, 600, 750 in three different configurations simulated here. The free spectral range of the device is 9.5 nm around a central wavelength of 1600 nm. For the standard multi-input AWG, the relative shift between the output spectra due to adjacent input channels is about 1.6 nm, which roughly equals one spectral channel spacing. In this paper, we discuss ways to increase the resolving power and the number of inputs without compromising the free spectral range or throughput.

astro-ph.IM↗

arXiv2017

Arrayed Waveguide Grating Spectrometers for Astronomical Applications: New Results

One promising application of photonics to astronomical instrumentation is the miniaturization of near-infrared (NIR) spectrometers for large ground- and space-based astronomical telescopes. Here we present new results from our effort to fabricate arrayed waveguide grating (AWG) spectrometers for astronomical applications entirely in-house. Our latest devices have a peak overall throughput of ~23%, a spectral resolving power ($λ/δλ$) of ~1300, and cover the entire H band (1450-1650 nm) for Transverse Electric (TE) polarization. These AWGs use a silica-on-silicon platform with a very thin layer of Si3N4 as the core of the waveguides. They have a free spectral range of ~10 nm at a wavelength of ~1600 nm and a contrast ratio or crosstalk of about 2% (-17 dB). Various practical aspects of implementing AWGs as astronomical spectrographs are discussed, including the coupling of the light between the fibers and AWGs, high-temperature annealing to improve the throughput of the devices at ~1500 nm, cleaving at the output focal plane of the AWG to provide continuous wavelength coverage, and a novel algorithm to make the devices polarization insensitive over a broad band. These milestones will guide the development of the next generation of AWGs with wider free spectral range and higher resolving power and throughput.

astro-ph.IM↗

arXiv2016

Development of high resolution arrayed waveguide grating spectrometers for astronomical applications: first results

Astrophotonics is the next-generation approach that provides the means to miniaturize near-infrared (NIR) spectrometers for upcoming large telescopes and make them more robust and inexpensive. The target requirements for our spectrograph are: a resolving power of about 3000, wide spectral range (J and H bands), free spectral range of about 30 nm, high on-chip throughput of about 80% (-1dB) and low crosstalk (high contrast ratio) between adjacent on-chip wavelength channels of less than 1% (-20dB). A promising photonic technology to achieve these requirements is Arrayed Waveguide Gratings (AWGs). We have developed our first generation of AWG devices using a silica-on-silicon substrate with a very thin layer of silicon-nitride in the core of our waveguides. The waveguide bending losses are minimized by optimizing the geometry of the waveguides. Our first generation of AWG devices are designed for H band and have a resolving power of around 1500 and free spectral range of about 10 nm around a central wavelength of 1600 nm. The devices have a footprint of only 12 mm x 6 mm. They are broadband (1450-1650 nm), have a peak on-chip throughput of about 80% (-1 dB) and contrast ratio of about 1.5% (-18 dB). These results confirm the robustness of our design, fabrication and simulation methods. Currently, the devices are designed for Transverse Electric (TE) polarization and all the results are for TE mode. We are developing separate J- and H-band AWGs with higher resolving power, higher throughput and lower crosstalk over a wider free spectral range to make them better suited for astronomical applications.

astro-ph.IM↗