arXiv subjects

Salma Elmalaki

Publications and source records attributed to Salma Elmalaki.

At least 19 recordsLinked to original sources

arXiv2026

PEARL: Structural Privacy-Utility Control in Human-Centric CPS via Personalized Early-Exit Deep Reinforcement Learning

In human-centric Cyber-Physical Systems (CPS), personalized Deep Reinforcement Learning (DRL) agents must share fine-grained control actions with cloud services, exposing sensitive private states to inference attacks by honest-but-curious adversaries. Static privacy models fail to address the dynamic nature of human interactions. This paper introduces PEARL (Personalized Early-exit Adaptive Reinforcement Learning), a novel framework that addresses this challenge through structural privacy control rather than data perturbation. PEARL deploys a dual-path Early-Exit Deep Q-Network (EE-DQN) at the edge, using Mutual Information (MI) between private states and observable actions to train per-branch binary labels: Utility Confidence Labels (UCL), verifying action quality, and Privacy Confidence Labels (PCL), verifying MI leakage remains below a user-defined threshold. At inference, PEARL selects the shallowest exit branch satisfying both UCL and PCL, structurally limiting shared action descriptive power without noise injection. An MI-based feedback loop tracks behavioral drift and triggers retraining when privacy-utility profiles shift, ensuring long-term robustness. Validated on a personalized smart-home HVAC system and a VR smart classroom, PEARL reduces adversarial state-inference accuracy by 25.67% on average with a controlled 10-16% utility cost, establishing a practical, dynamically enforceable privacy-utility tradeoff.

cs.LG↗

arXiv2026

PACIFIC: Can LLMs Discern the Psychometric Traits Influencing Your Preferences? Personality-Driven Preference Alignment in LLMs

User preferences are increasingly used to personalize Large Language Model (LLM) responses, yet reliably leveraging preference signals remains under-explored. In practice, preferences can be noisy, incomplete, or even misleading, which can degrade answer quality when applied naively. Motivated by the observation that stable personality traits shape everyday preferences, we introduce PACIFIC (Preference Alignment for Choices Inference via Five-factor Identity Characterization), a personality-driven preference alignment framework that uses Big-Five (OCEAN) traits as a principled "latent" signal for organizing and reasoning over user preference history. To systematically evaluate this framework, we construct a psychometrics-based dataset containing 1,200 preference-query pairs spanning diverse domains (e.g., travel, movies, and education), with comprehensive coverage of high and low Big-Five trait directions. Extensive experiments show that trait-aligned preferences substantially improve personalized QA: given clean, trait-aligned context, LLMs reach near-ceiling accuracy (up to 99%), confirming that reasoning capability is not the bottleneck. The challenge is that real histories are mixed-trait and unlabeled. We show the true bottleneck is retrieval: a persona-aware contrastive retriever (PiRAG) raises label-free accuracy from 30% to 43% over standard semantic retrieval, without any trait annotations at inference.

cs.CL↗

arXiv2026

Bern2Edge: A Neurosymbolic Compiler for Edge Deployment via Bernstein Polynomial Networks

Deploying high-accuracy neural networks on resource-constrained edge devices remains challenging, as existing approaches treat training, compression, and hardware synthesis as separate stages, leaving a gap between software-trained models and efficient end-to-end deployment with limited support for interpretability. We propose Bern2Edge, an end-to-end framework that uses knowledge distillation to convert a pretrained teacher feed-forward network into hardware-efficient representations via Bernstein polynomial activations. This representation enables two deployment paths: (i) a high-fidelity LUT-based realization that preserves model fidelity under compression, and (ii) a symbolic rule-based representation derived from Bernstein activation geometry, enabling interpretable inference with explicit input-space constraints. The resulting BNNs achieve up to 2.12 percentage-point (pp) accuracy improvement over ReLU under identical compression constraints. At the system level, Bern2Edge achieves up to 99.8% latency reduction and 95.2% BRAM reduction relative to a W8A8 quantized teacher on an AMD Xilinx KV260 FPGA, while maintaining accuracy within 0.5 pp, and further deploys on a low-power Spartan-7 XC7S15 FPGA. The rule-based path reduces DSP usage by up to 89.0% at a cost of 1.5 pp in total accuracy.

cs.LG↗

arXiv2026

Follow My Eyes: Backdoor Attacks on Goal-Directed Scanpath Prediction

Scanpath prediction models forecast the sequence of fixations a person makes while searching a scene, and increasingly serve as the upstream perception layer for foveated rendering, intent inference, and gaze-driven assistive interfaces. Since eye-tracking data is expensive to collect, these models are routinely fine-tuned from public datasets or third-party pretrained weights, exposing them to training-time poisoning. We present the first backdoor attacks on multimodal scanpath prediction. This task differs from classification: its output is a continuous, variable-length sequence of fixations, opening new avenues of attack. A fixed-trajectory backdoor is easy to implant, but it clusters poisoned samples away from clean data, making it detectable. We instead design two backdoor attacks that condition the malicious supervision on each scene to keep triggered outputs diverse and plausible: a spatial misdirection attack that redirects the predicted search toward an attacker-chosen object instead of the queried one, and a duration inflation attack that lengthens the predicted search by inserting extra fixations while preserving correct localization. Our attacks succeed across visual, textual, and multimodal triggers, with duration inflation reaching up to 93.5% attack success from as few as 540 poisoned samples (2.5% of training data), and spatial misdirection redirecting the search in up to 61% of triggered inputs. We evaluate our attacks against five existing backdoor defenses, spanning fine-tuning, fine-pruning, neural attention distillation, contrastive learning, and trigger inversion, and show none removes the backdoor without degrading model output below the usable threshold. Our attacks generalize across models and datasets, showing scanpath prediction models are vulnerable to backdoor attacks through data poisoning, and that designing an effective defense remains an open problem.

cs.CR↗

arXiv2026

FinP: Fairness-in-Privacy in Federated Learning by Addressing Disparities in Privacy Risk

Federated Learning (FL) inherently mitigates mass data centralization risks; however, its privacy protections are not equally distributed - leaving vulnerable individuals disproportionately exposed to sophisticated privacy attacks. Crucially, statistical heterogeneity in human-centric FL environments often results in an inequitable distribution of privacy risks, particularly affecting those whose sensitive attributes or behaviors make them outliers. To address this critical gap, we introduce FinP, a novel framework designed to formalize and enforce fairness-in-privacy by mitigating disproportionate client vulnerability to Source Inference Attacks (SIA). FinP operationalizes a two-pronged defense strategy that tackles both the symptoms and root causes of privacy disparity, ensuring that no group of clients bears an excessive privacy burden. It combines a server-side adaptive aggregation mechanism, which dynamically weights client contributions based on their estimated privacy risk, with a client-side regularization technique to curb localized overfitting that drives unique data memorization. Extensive empirical evaluations on FEMNIST, Human Activity Recognition (HAR), and CIFAR-10 datasets demonstrate that FinP effectively aligns privacy fairness with primary task utility. Notably, FinP successfully mitigates SIA risks and reduces disparities in privacy exposure, establishing that strong fairness-in-privacy guarantees need not compromise model utility. Ultimately, FinP establishes equitable privacy protections by reducing vulnerability disparities by up to 57.14%, while preserving global model utility within a marginal +/- 1.75% of standard federated baselines.

cs.LG↗

arXiv2026

PersonaDrive: Human-Style Retrieval-Augmented VLA Agents for Closed-Loop Driving Simulation

Closed-loop driving simulators typically populate their environments with non-ego traffic agents that behave largely the same way, produced either by rule-based traffic managers or by learned models trained toward a single behavioral mode. Recent work introduces style variation through post-hoc labels on observational data or LLM-inferred reward weights, but these signals act as proxies for what a style should reward rather than demonstrations of humans explicitly asked to drive in that style. We introduce PersonaDrive, a pipeline that conditions a vision-language-action (VLA) driving agent on retrieved demonstrations from a style-instructed human driving dataset, in which participants drive CARLA leaderboard routes under aggressive, neutral, and conservative instructions on a driver-in-the-loop rig. The pipeline has three stages: (i) offline triplet mining over per-style human driving data using a combined image-text similarity score; (ii) training a lightweight retrieval head that fuses frozen visual features with a small control encoder over per-style databases; and (iii) fine-tuning a single VLA backbone to treat retrieved context points as in-context behavioral demonstrations during waypoint prediction. At inference, the same backbone is conditioned on any style by swapping which per-style database the retrieval head queries, so selecting a style requires no per-style retraining while enabling human-style, style-diverse non-ego agents for closed-loop simulation. On Bench2Drive, PersonaDrive (no style) improves the driving score by 4.6% over SimLingo and 2.5% over HiP-AD, and under style conditioning attains the highest driving score in every style within a roughly 2% band (its weakest style surpassing the strongest baseline, DMW, by 5.4%), while average speed and acceleration rise by 18% and 25% from the conservative to the aggressive instruction.

cs.AI↗

arXiv2026

Exponential Approximation Rates and Parameter Efficiency of Learnable Bernstein Activations

The choice of activation function fundamentally shapes the representational capacity and parameter efficiency of deep neural networks, yet most widely used activations lack rigorous theoretical guarantees on these properties. We provide a theoretical analysis of DeepBern-Nets (DBNs) -- networks employing learnable Bernstein polynomial activations -- showing that their approximation error decays with the network depth $L$ and the polynomial order $n$ with a rate of $\mathcal{O}(n^{-L})$, exponentially faster than the polynomial rate of ReLU architectures while remaining fully differentiable. We validate these predictions through $1{,}344$ experiments on large scientific datasets (HIGGS and SUSY), comparing DBNs against ReLU, Leaky ReLU, SELU, and GeLU. DBNs achieve over $70\%$ parameter reduction across the majority of architectures -- reaching $99.9\%$ at scale -- converge to ReLU's final loss in as few as $26\%$ of the training epochs, and attain up to $45\%$ lower final loss. These advantages hold over all tested activations, confirming that DBN's gains stem from the learnable polynomial structure rather than mere smoothness.

cs.LG↗

arXiv2026

TeamLLM: Exploring the Capabilities of LLMs for Multimodal Group Interaction Prediction

Predicting group behavior, how individuals coordinate, communicate, and interact during collaborative tasks, is essential for designing systems that can support team performance through real-time prediction and realistic simulation of collaborative scenarios. Large Language Models (LLMs) have shown promise for processing sensor data for human-activity recognition (HAR), yet their capabilities for team dynamics or group-level multimodal sensing remain unexplored. This paper investigates whether LLMs can predict group coordination patterns from multimodal sensor data in collaborative Mixed Reality (MR) environments. We encode hierarchical context -- individual behavioral profiles, group structural properties, and temporal activity context -- as natural language and evaluate three LLM adaptation paradigms (zero-shot, few-shot, and supervised fine-tuning) against statistical baselines. Our evaluation on 16 groups (64 participants, $\sim$25 hours of sensor data) reveals that LLMs achieve 3.2$\times$ improvement over LSTM baselines for linguistically-grounded behaviors, with fine-tuning reaching 96\% accuracy for conversation prediction while maintaining sub-35ms latency. Beyond performance gains, we characterize the boundaries of text-based LLMs for multimodal sensing conversation prediction succeeds because turn-taking maps to linguistic patterns, while shared or joint attention may require spatial and visual reasoning that text only LLMs cannot capture. We further identify simulation mode brittleness (83\% degradation from cascading context errors) and minimal few-shot sensitivity to example selection strategy. These findings establish guidelines when LLMs are appropriate for CPS/IoT sensing for team dynamics and inform the design of future multimodal foundation models.

cs.HC↗

arXiv2026

APPA: Adaptive Preference Pluralistic Alignment for Fair Federated RLHF of LLMs

Aligning large language models (LLMs) with diverse human preferences requires pluralistic alignment, where a single model must respect the values of multiple distinct groups simultaneously. In federated reinforcement learning from human feedback (FedRLHF), these groups align a shared policy without centralizing preference data, which makes fair reward aggregation essential. Existing aggregation methods exhibit clear trade offs: average based aggregation systematically under aligns worst performing groups, while min aggregation prioritizes worst group performance at the cost of overall alignment. We propose APPA, an Adaptive Preference Pluralistic Alignment framework that dynamically reweights group level rewards based on historical alignment rewards. Our approach prioritizes under aligned groups without degrading well aligned ones, while requiring no access to raw preference data. Integrated into a proximal policy optimization (PPO) based FedRLHF pipeline and evaluated on GLOBALQA and OQA across three model families (Gemma 2 2B, Llama 3.2 3B, Qwen3 0.6B), APPA achieves strong fairness alignment trade offs, improving worst group alignment by up to 28% over average aggregation while maintaining higher overall alignment than min aggregation across most configurations.

cs.LG↗

arXiv2026

MURMR: A Multimodal Sensing Framework for Automated Group Behavior Analysis in Mixed Reality

When teams coordinate in immersive environments, collaboration breakdowns can go undetected without automated analysis, directly affecting task performance. Yet existing methods rely on external observation and manual annotation, offering no annotation-free method for analyzing temporal collaboration dynamics from headset-native data. We introduce \sysname, a passive sensing pipeline that captures and analyzes multimodal interaction data from commodity MR headsets without external instrumentation. Two complementary modules address different levels of analysis: a structural module that generates automated multimodal sociograms and network metrics at both session and intra-session granularities, and a temporal module that applies unsupervised deep clustering to identify moment-to-moment dyadic behavioral phases without predefined taxonomies. An exploratory deployment with 48 participants in a co-located object-sorting task reveals that intra-session structural analysis captures significant within-session variability lost in session-level aggregation, with gaze, audio, and position contributing non-redundantly. The temporal module identifies five behavioral phases with 83\% correspondence to video observations. Cross-tabulation shows that behavioral transitions consistently occur within structurally stable states, demonstrating that the two modules capture complementary dynamics. These results establish that passive headset sensing provides meaningful signal for automated, multi-level collaboration analysis in immersive environments.

cs.HC↗

arXiv2026

What Sensors See, What People Feel: An Exploratory Study of Subjective Collaboration Perception in Mixed Reality

Mixed Reality (MR) enables rich, embodied collaboration; however, it is uncertain whether sensor- and system-logged behavioral signals capture how users experience that collaboration. This disconnect stems from a fundamental gap. Behavioral signals are observable and continuous, while collaboration is interpreted subjectively and shaped by internal states like presence, cognitive availability, and social awareness. Our core insight is that sensor signals serve as observable manifestations of subjective experiences in MR collaboration, and they can be captured through sensor data such as shared gaze, speech, spatial movement, and other system-logged performance metrics. We propose the Sensor-to-Subjective (S2S) Mapping Framework, a conceptual model that links observable interaction patterns to users' subjective perceptions of collaboration and internal cognitive states through sensor-based indicators and task performance metrics. To evaluate this model, we conducted an exploratory study with 48 participants across 12 MR groups engaged in a collaborative image-sorting task. Our findings show a correlation between sensed behavior and perceived collaboration, particularly through shared attention and proximity.

cs.HC↗

arXiv2025

A Systematic Evaluation of Preference Aggregation in Federated RLHF for Pluralistic Alignment of LLMs

This paper addresses the challenge of aligning large language models (LLMs) with diverse human preferences within federated learning (FL) environments, where standard methods often fail to adequately represent diverse viewpoints. We introduce a comprehensive evaluation framework that systematically assesses the trade-off between alignment quality and fairness when using different aggregation strategies for human preferences. In our federated setting, each group locally evaluates rollouts and produces reward signals, and the server aggregates these group-level rewards without accessing any raw data. Specifically, we evaluate standard reward aggregation techniques (min, max, and average) and introduce a novel adaptive scheme that dynamically adjusts preference weights based on a group's historical alignment performance. Our experiments on question-answering (Q/A) tasks using a PPO-based RLHF pipeline demonstrate that our adaptive approach consistently achieves superior fairness while maintaining competitive alignment scores. This work offers a robust methodology for evaluating LLM behavior across diverse populations and provides a practical solution for developing truly pluralistic and fairly aligned models.

cs.CL↗

arXiv2025

GaitGuard: Protecting Video-Based Gait Privacy in Mixed Reality

Mixed Reality (MR) systems capture continuous video streams that expose bystanders' and collaborators' gait patterns -- a biometric revealing sensitive attributes including age, gender, and health conditions. We show that video-based gait profiling achieves 78\% accuracy (15.6$\times$ random chance) on unprotected MR feeds, motivating \textbf{GaitGuard}, a real-time defense operating on a companion mobile device. GaitGuard introduces \textbf{GaitExtract}, an automated gait feature extraction pipeline adapted from clinical analysis for egocentric MR perspectives. Through systematic evaluation of 233 mitigation configurations, we characterize privacy-utility-performance trade-offs. A key insight is that gait features derive primarily from transient events (heel strikes, toe-offs). We exploit this temporal sparsity through adaptive mitigation that selectively processes only gait-critical frames, achieving a 68\% reduction in profiling accuracy while preserving visual quality (SSIM: 0.97) at 29~FPS. \textbf{GaitGuard} scales to 10 simultaneous users with under 10ms latency. A qualitative study of 20-participants confirms that the users preferred a solution such as \textbf{GaitGuard} which provides privacy guarantees.

cs.HC↗

arXiv2025

M-CALLM: Multi-level Context Aware LLM Framework for Group Interaction Prediction

This paper explores how large language models can leverage multi-level contextual information to predict group coordination patterns in collaborative mixed reality environments. We demonstrate that encoding individual behavioral profiles, group structural properties, and temporal dynamics as natural language enables LLMs to break through the performance ceiling of statistical models. We build M-CALLM, a framework that transforms multimodal sensor streams into hierarchical context for LLM-based prediction, and evaluate three paradigms (zero-shot prompting, few-shot learning, and supervised fine-tuning) against statistical baselines across intervention mode (real-time prediction) and simulation mode (autoregressive forecasting) Head-to-head comparison on 16 groups (64 participants, ~25 hours) demonstrates that context-aware LLMs achieve 96% accuracy for conversation prediction, a 3.2x improvement over LSTM baselines, while maintaining sub-35ms latency. However, simulation mode reveals brittleness with 83% degradation due to cascading errors. Deep-dive into modality-specific performance shows conversation depends on temporal patterns, proximity benefits from group structure (+6%), while shared attention fails completely (0% recall), exposing architectural limitations. We hope this work spawns new ideas for building intelligent collaborative sensing systems that balance semantic reasoning capabilities with fundamental constraints.

cs.HC↗

arXiv2025

PluralLLM: Pluralistic Alignment in LLMs via Federated Learning

Ensuring Large Language Models (LLMs) align with diverse human preferences while preserving privacy and fairness remains a challenge. Existing methods, such as Reinforcement Learning from Human Feedback (RLHF), rely on centralized data collection, making them computationally expensive and privacy-invasive. We introduce PluralLLM a federated learning-based approach that enables multiple user groups to collaboratively train a transformer-based preference predictor without sharing sensitive data, which can also serve as a reward model for aligning LLMs. Our method leverages Federated Averaging (FedAvg) to aggregate preference updates efficiently, achieving 46% faster convergence, a 4% improvement in alignment scores, and nearly the same group fairness measure as in centralized training. Evaluated on a Q/A preference alignment task, PluralLLM demonstrates that federated preference learning offers a scalable and privacy-preserving alternative for aligning LLMs with diverse human values.

cs.LG↗

arXiv2025

MoCoMR: A Collaborative MR Simulator with Individual Behavior Modeling

Studying collaborative behavior in Mixed Reality (MR) often requires extensive, challenging data collection. This paper introduces MoCoMR, a novel simulator designed to address this by generating synthetic yet realistic collaborative MR data. MoCoMR captures individual behavioral modalities such as speaking, gaze, and locomotion during a collaborative image-sorting task with 48 participants to identify distinct behavioral patterns. MoCoMR simulates individual actions and interactions within a virtual space, enabling researchers to investigate the impact of individual behaviors on group dynamics and task performance. This simulator facilitates the development of more effective and human-centered MR applications by providing insights into user behavior and interaction patterns. The simulator's API allows for flexible configuration and data analysis, enabling researchers to explore various scenarios and generate valuable insights for optimizing collaborative MR experiences.

cs.HC↗

arXiv2025

AutoLike: Auditing Social Media Recommendations through User Interactions

Modern social media platforms, such as TikTok, Facebook, and YouTube, rely on recommendation systems to personalize content for users based on user interactions with endless streams of content, such as "For You" pages. However, these complex algorithms can inadvertently deliver problematic content related to self-harm, mental health, and eating disorders. We introduce AutoLike, a framework to audit recommendation systems in social media platforms for topics of interest and their sentiments. To automate the process, we formulate the problem as a reinforcement learning problem. AutoLike drives the recommendation system to serve a particular type of content through interactions (e.g., liking). We apply the AutoLike framework to the TikTok platform as a case study. We evaluate how well AutoLike identifies TikTok content automatically across nine topics of interest; and conduct eight experiments to demonstrate how well it drives TikTok's recommendation system towards particular topics and sentiments. AutoLike has the potential to assist regulators in auditing recommendation systems for problematic content. (Warning: This paper contains qualitative examples that may be viewed as offensive or harmful.)

cs.LG↗

arXiv2025

GroupBeaMR: Analyzing Collaborative Group Behavior in Mixed Reality Through Passive Sensing and Sociometry

Understanding group behavior is crucial for enhancing collaboration and productivity in mixed reality (MR). This paper introduces a framework for group behavior analysis in MR, or GroupBeaMR for short for analyzing group behavior in MR. GroupBeaMR leverages MR headsets' sensors to analyze group behavior through conversation, shared attention, and proximity, identifying cohesive, fragmented, and competitive interaction patterns. Using social network analysis, GroupBeaMR provides quantitative assessments of group dynamics, offering insights into collaboration structures. A user study with 48 participants across 12 groups validates the framework's ability to distinguish interaction patterns in MR environments. Our analyses show that group behavior is independent of task performance, emphasizing the significance of social interaction patterns. Our group-type assignments indicate that sensor-based assessments in MR can provide meaningful insights into collaborative experiences, supporting the design of systems that adapt and optimize group behaviors.

cs.HC↗