AcupunctureBench: A Multimodal Benchmark for Evaluating Generative Models in the Acupuncture Domain
As large language models (LLMs) are increasingly applied in the field of traditional Chinese medicine (TCM), standardized benchmark evaluations for TCM have gradually emerged. Acupuncture is a crucial component of TCM. However, existing evaluation benchmarks are lacking in this domain. To address this gap, this paper proposes AcupunctureBench, the first independently constructed large-scale multimodal benchmark for the acupuncture domain, covering text, image, and video modalities. The AcupunctureBench spans multidimen sional knowledge in acupuncture, including acupuncture theory, meridians and acupoints, clinical reasoning, TCM syndromes, and practical acupoint operation. To achieve large-scale construction while ensuring medical professionalism, this work designs a human-machine collaborative data construction pipeline and proposes efficient deduplication strategies to reduce redundant samples, while extensively adopting data synthesis to mitigate data leak age risks. AcupunctureBench contains a total of 58,526 question samples and 1,216 video samples, covering multiple task types including objective questions, subjective question answering, case analysis, image understanding, image reasoning, and video understanding. Based on this benchmark, this paper evaluates 14 large language models, 14 large multimodal models, and 4 video foundation models, and further releases the first dedicated evaluation website for the acupuncture domain.