Jiali Sun | Searcharxiv

arXiv2026

OpenHarmony Bench: Evaluating LLMs and Coding Agents on OpenHarmony App Development

We present OPENHARMONY BENCH, an app-level coding benchmark for evaluating LLM-based coding agents on OpenHarmony ArkTS applications. Unlike function-level benchmarks, it evaluates complete app-level changes: each task requires an agent to modify a buildable ArkTS project so that a requested behavior works end to end, involving UI state, data persistence, build configuration, and platform APIs. The benchmark installs and drives the delivered application on a device to check whether the behavior is observable. It covers three input sources: natural-language feature requests (new-feature), structured scenario specifications (spec-driven), and bug descriptions (bug-fix). The benchmark contains 153 top-level tasks and 242 Feature points (F-points), where an F-point is one executable behavior check. The snapshot includes 32 new-feature tasks, 50 spec-driven tasks with 139 F-points, and 71 bug-fix tasks. The main leaderboard is scored over top-level tasks rather than independently weighted F-points. We describe the benchmark construction, statistics, and build-and-test evaluation pipeline, and evaluate DevEco Code with eight LLMs across three independent full-suite runs per configuration. Three findings emerge. First, newer generations complete more tasks than their predecessors within evaluated model-family pairs. Second, buildability is close to saturated while behavioral correctness is not: mean Final Build Success Rate is 94.77% to 100.00%, whereas mean Task Completion is 48.36% to 58.39%. Third, spec-driven tasks have the lowest Task Completion under all-checks task scoring, with no configuration exceeding 35%. The code, data, tasks, reference solutions, tests, evaluation scripts, and leaderboard are released through the official OPENHARMONY BENCH website at https://bench.matrix.openharmony.cn/.

cs.SE↗

arXiv2025

Demultiplexing through a multimode fiber using chip-scale diffractive neural networks

In today's information age, advanced fiber optic transmission technology is of paramount importance. Multimode fibers (MMFs) using space-division multiplexing (SDM) are promising for improved transmission capacity, connection flexibility, and security of data. However, the complex transmission characteristics of MMFs significantly hinder precise mode demultiplexing. Conventional approaches, including holographic measurements, phase retrieval algorithms, photonic lanterns, and multiplane light conversion, are limited by system complexity, size, and flexibility. In this paper, we demonstrate for the first time a purely optical, chip-scale AI solution for high-mode isolation, speed-of-light demultiplexing of MMF modes using a three-dimensional diffractive neural network (DNN). The DNN is trained with synthetic modal data and fabricated using two-photon nanolithography. It features a compact size of $120{\mu}m \times 120{\mu}m \times 80{\mu}m$ and a diffractive structure size of $1{\mu}m^{2}$ for the neurons at the hidden layers of the network. Experimentally, the DNN demultiplexer achieves a relative demultiplexing accuracy of over 80%. The AI approach of DNN allows for flexible design and overcomes the size and performance limitations of digital-optical demultiplexers. This work paves the way for compact, low-latency optical processors for high-performance demultiplexers and enables scalable, chip-integrated solutions for next-generation fiber optic networks.

physics.optics↗