arXiv subjects

Soumya Swagata Biswas

Publications and source records attributed to Soumya Swagata Biswas.

2 recordsLinked to original sources

arXiv2026

Seeing Through the Forgery Is Not Enough: Regressive Sycophancy under Fabricated Evidence in Medical Vision-Language Models

A medical vision-language model may analyze a given CT scan and correctly detect both kidneys in view. Continuing this conversation, if a user provides a prescription with an opposite direction, medical VLMs often sycophantically choose to realign their diagnosis. In this work, we study regressive sycophancy under fabricated evidence, where a model abandons a correct answer when pushback comes with a fake document. We test four open VLMs on 400 VQA-RAD yes/no questions in 19,200 multi-turn conversations, with and without a fabricated prescription. Without a disclosure that documents are synthetic, the models abandon 80.1% of 6,488 correct answers. Notably, under one script each, the same prescription raises the flip rate of Qwen2.5-VL-7B from 45.9% to 89.6% and lowers that of Qwen3-VL-8B from 77.7% to 50.9%. A model rejected the prescription while still preserving the correct answer in 915 of 3,232 conversations with the prescription (28.3%), and 40% of these flipped in later turns. System-prompt sentences that disclose the documents as synthetic lower one model's flip rate from 77.8% to 47.3%. Thus, we conclude that awareness about a forgery does not prevent a model from adopting its answer.

cs.CL↗

arXiv2026

Embedding Asymmetry as Attack Signal: Detecting and Defending Active Label Inference in Vertical Federated Learning

Vertical federated learning lets organizations train a shared model over features they hold separately, while one party keeps the labels private. The active label inference attack of Fu et al. [12] breaks this: a feature-holding party runs a modified optimizer, MaliciousSGD, that forces class structure into its embeddings and then recovers the labels with semi-supervised learning. Defenses that perturb the shared gradient fail, because the attack acts on the party's internal optimizer, not on the message it sends. We show the attack leaves a signal the server can read from the embeddings it already receives. The adversary's Fisher class-separability pulls away from the honest party's, and the gap is visible within a few epochs. A defense triggered by this signal acts on the adversary alone. Scaling its gradient down handles few-class tasks, and a norm non-expansive projection handles many-class tasks, with the Discriminative Coverage Ratio explaining why the projection rank must grow with the class count. The defense breaks the alignment between embedding clusters and labels rather than destroying the embedding. Across four seeds on CIFAR-10, CIFAR-100, and CINIC-10, label inference falls below what honest training already leaks, at under two points of task-accuracy cost.

cs.CR↗