Searcharxiv⌕ Search

arXiv subjects

Siqi Zhu

Publications and source records attributed to Siqi Zhu.

20 records · Page 2Linked to original sources

LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs

Current long context large language models (LLMs) can process inputs up to 100,000 tokens, yet struggle to generate outputs exceeding even a modest length of 2,000 words. Through controlled experiments, we find that the model's effective generation length is inherently bounded by the sample it has seen during supervised fine-tuning (SFT). In other words, their output limitation is due to the scarcity of long-output examples in existing SFT datasets. To address this, we introduce AgentWrite, an agent-based pipeline that decomposes ultra-long generation tasks into subtasks, enabling off-the-shelf LLMs to generate coherent outputs exceeding 20,000 words. Leveraging AgentWrite, we construct LongWriter-6k, a dataset containing 6,000 SFT data with output lengths ranging from 2k to 32k words. By incorporating this dataset into model training, we successfully scale the output length of existing models to over 10,000 words while maintaining output quality. We also develop LongBench-Write, a comprehensive benchmark for evaluating ultra-long generation capabilities. Our 9B parameter model, further improved through DPO, achieves state-of-the-art performance on this benchmark, surpassing even much larger proprietary models. In general, our work demonstrates that existing long context LLM already possesses the potential for a larger output window--all you need is data with extended output during model alignment to unlock this capability. Our code & models are at: https://github.com/THUDM/LongWriter.

cs.CL↗

Optical Superoscillatory Poisson-Arago Spots

Optical diffraction limit has been a long-term scientific issue since Ernst Abbe first introduced the concept in 1873. It is a constraint on the smallest light spot that can be achieved. Substantial effort has been invested in the past decade to beat this limit by exploiting evanescent waves. But this method encounters serious near-field limitations. A more promising route to breaking the constraint is to explore optical superoscillation in the far field with engineered metamaterials. However, these particular structures involve with very complicated optimization-based design that requires precisely tailoring the interference of propagating waves with low spatial frequency. To overcome these limitations, here we explore a new approach based on the two-hundred-year-old discovery: Possion-Arago spots. We show for the first time that by using a single disc, constructive interference of propagating waves with high-spatial-frequency wavevectors can be realized, generating a diffraction-unlimited localized Possion-Arago spot with achievable size down to $λ/$20. Actually, such an element permits creation of an ultra-long nearly nondiffracting superoscillatory needle with appreciable field of view. This easy-to-fabrication element provides a promising route to overcome the diffraction limit, thus might open new avenues to exploit various applications in different fields.

physics.optics↗