ChipVerilog: A Large-Scale OpenCores-Derived Benchmark for LLM-Based Verilog RTL Generation

Abstract

Large language models have shown strong potential for Verilog RTL generation. However, many existing benchmarks are built from short, self-contained module-level tasks. These tasks are useful for controlled evaluation, but they do not fully capture the code scale, hierarchy, and module interactions found in practical IP and processor-core RTL. We present ChipVerilog, a description-to-Verilog generation benchmark built from OpenCores IP/core designs. The benchmark contains 64 generation targets from five design families: OR1200, double-precision FPU, MIPS-16, I2C, and CORDIC. It includes both single-module targets and cross-module targets that instantiate or interact with other RTL modules. Several targets exceed 1,000 lines of Verilog, making ChipVerilog substantially larger and structurally more complex than typical module-level suites. Each benchmark instance is constructed from a pair of specification documents and reference RTL. We extract the target functionality, write a detailed natural-language description, and manually review the description for correctness and clarity. Generated RTL is checked by compilation and validated through equivalence checking for local modules, or by simulation for integrated IP/core targets. Results show that large-scale RTL remains challenging, especially for hierarchical and cross-module designs.