PRISM-Net: Dynamic Routing Network with VLM-Guided Disentangled Attribute Distillation for Explainable Fine-Grained Product Retrieval
Fine-grained visual product retrieval in large-scale e-commerce multimedia systems demands discriminative representations capable of simultaneously resolving macro-level category semantics and micro-level visual attributes such as structural contour (line) and chromatic appearance (color). However, conventional backbones including convolutional neural networks (CNNs) and vision Transformers (ViTs) inherently suffer from feature entanglement and shortcut learning. To learn the micro-level visual attributes without costly manual attribute annotations, we propose phase-routed interpretable semantic mixture-of-experts network (PRISM-Net), an explainable multi-expert architecture that disentangles holistic visual features into disentangled line and color expert pathways governed by a conditional meta-router. In phase 1, we introduce an external VLM-Guided sigmoid semantic distillation mechanism with gradient-detached routing gates, utilizing zero-shot text prompts from SigLIP to supervise specialized structural and chromatic experts. In phase 2, the disentangled expert embeddings are adaptively fused with macro base features via a learned meta-router gate to calibrate category boundaries. Extensive 5-fold cross-validation across 275,454 e-commerce images (18 categories) and full-scale retrieval benchmarks (55,091 queries against a 220,363-image gallery) across SwinV2, ConvNeXt-V2, and DINOv3 demonstrate substantial gains. On SwinV2, PRISM-Net boosts full-fold product mAP@10 from 80.98% to 83.90% (+2.92%), line mAP@10 by +5.74%, and color mAP@10 by +9.12%. Furthermore, unsupervised distribution analysis of the meta-router gate reveals that both ViT and CNN backbones emergently allocate up to 6.8x higher structural line weights to rigid appliances and hardware while dedicating >96% chromatic weight to clothing.