arXiv subjects

Mengyu Chen

Publications and source records attributed to Mengyu Chen.

16 recordsLinked to original sources

arXiv2026

Inverse Theory of Mind Modeling for Content Recommendation: From Web Browsing to Dynamic Intelligent Interfaces

Modern recommender systems treat observed actions as reliable proxies for user preferences, yet interactions often reflect exploration or comparison rather than stable preference expression. As interfaces evolve from static layouts toward generative UIs and immersive extended reality (XR), the need for deeper, modality-agnostic user understanding grows: these adaptive environments must decide not only what to present but where, when, how prominently, and most importantly why a user acts. We propose an Inverse Theory of Mind (IToM) pipeline that reasons backward from observed interactions to infer the beliefs, preferences, and decision-making traits that explain behavior. The pipeline reconstructs each user's decision context, including what was chosen and what alternatives were available, applies LLM-driven counterfactual reasoning to produce evidence-grounded natural-language belief statements, and synthesizes these beliefs through multi-hypothesis abductive inference into a structured user persona. We evaluate on the OPeRA dataset against ground-truth personality assessments, attitudinal surveys, and interview-based personas across four tasks: next action prediction, shopping attitude alignment, Big Five personality inference, and held-out category prediction. Results show that inferred personas match or exceed ground-truth personas and that multi-hypothesis reasoning is essential for accurate personality prediction. We further demonstrate cross-modal transferability with a persona-driven spatial banking application on VisionOS.

cs.AI↗

arXiv2026

Transition Techniques for Externally-Guided Multi-Scale Viewpoint Changes

Extended reality (XR) is increasingly used to help users understand complex virtual environments through multiple viewpoints across different immersion levels, positions, and scales. While numerous techniques address viewpoint transitions for self-guided exploration, many scenarios require externally-guided transitions where a system or presenter controls the user's viewpoint, leaving the user with limited spatial knowledge and control over the transition process, which can increase susceptibility to disorientation and discomfort. We present three transition techniques for externally-guided multi-scale XR viewpoint changes and evaluate them against a fade-to-black baseline in a within-subjects study (N=20). Participants transitioned between world-in-miniature, street-level, and indoor destination views. We combined spatial recall measures, standardized questionnaires, and semi-structured interviews to assess orientation, workload, comfort, and continuity. Results showed that in our setup, techniques externalizing reference frames and the user's pose improved multi-scale spatial recall relative to a fade, while same-scale recall was insensitive to transition technique. We conclude with design implications for multi-scale XR viewpoint transitions.

cs.HC↗

arXiv2026

Designing Digital Humans with Ambient Intelligence

Digital humans are lifelike virtual agents capable of natural conversation and are increasingly deployed in domains like retail and finance. However, most current digital humans operate in isolation from their surroundings and lack contextual awareness beyond the dialogue itself. We address this limitation by integrating ambient intelligence (AmI) - i.e., environmental sensors, IoT data, and contextual modeling - with digital human systems. This integration enables situational awareness of the user's environment, anticipatory and proactive assistance, seamless cross-device interactions, and personalized long-term user support. We present a conceptual framework defining key roles that AmI can play in shaping digital human behavior, a design space highlighting dimensions such as proactivity levels and privacy strategies, and application-driven patterns with case studies in financial and retail services. We also discuss an architecture for ambient-enabled digital humans and provide guidelines for responsible design regarding privacy and data governance. Together, our work positions ambient intelligent digital humans as a new class of interactive agents powered by AI that respond not only to users' queries but also to the context and situations in which the interaction occurs.

cs.HC↗

arXiv2026

Adaptive information-maximization encoding for ghost imaging--A general Bayesian framework under experimental physical constraints

Ghost imaging (GI) has demonstrated diverse imaging capabilities enabled by its encoding-decoding-based computational imaging mechanism. Accordingly, information-theoretic studies have emerged as a promising avenue for probing the fundamental performance bounds of of GI and related computational imaging paradigms. However, the design of information-theoretically optimal encoding strategies remains largely unexplored, primarily due to the intractability of the prior probability density function (PDF) of an unknown scene. Here, by leveraging the ability of recursively estimating the PDF of the object to be imaged via Bayesian filtering, we propose to establish an adaptive information-maximization encoding (AIME) design framework. Based on the adaptively estimated posterior PDF from previously acquired measurements, the expected information gain of subsequent detections is evaluated and maximized to design the corresponding encoding patterns in a closed-loop manner. Within this framework, the theoretical form of the information-optimal encoding under representative physical constraints is analytically derived. Corresponding experimental results show that, GI systems employing information-optimal encoding achieve markedly improved imaging performance compared with conventional fixed point-to-point imaging without relying on additional heuristic regularization schemes, particularly in low signal-to-noise ratio regimes. Moreover, the proposed strategy consistently enables significantly enhanced information acquisition capability compared with existing encoding strategies, leading to substantially improved imaging quality. These results establish a principled information-theoretic foundation for optimal encoding design in computational imaging paradigms,provided that the forward model can be accurately characterized.

physics.optics↗

arXiv2026

Secure and Private Spatial Sharing for Mixed Reality Remote Collaboration in Enterprise Settings

Mixed Reality (MR) technologies are increasingly adopted by enterprises to enhance remote collaboration, enabling users to share real-time views of their physical environments through head-mounted displays (HMDs). While MR spatial sharing offers significant benefits, it introduces complex security and privacy risks, particularly in balancing employee collaboration needs with enterprise data protection requirements across office and personal spaces. This paper investigates these challenges through formative interviews with employees and expert consultations with professionals in cybersecurity, IoT, technology risk, and corporate legal domains. We present a conceptual framework for secure MR spatial sharing in enterprise contexts and identify critical concerns and requirements for system design. Based on our findings, we offer actionable recommendations to guide the development of secure and privacy-preserving MR spatial sharing solutions for future enterprise deployments.

cs.HC↗

arXiv2025

Adaptive 3D UI Placement in Mixed Reality Using Deep Reinforcement Learning

Mixed Reality (MR) could assist users' tasks by continuously integrating virtual content with their view of the physical environment. However, where and how to place these content to best support the users has been a challenging problem due to the dynamic nature of MR experiences. In contrast to prior work that investigates optimization-based methods, we are exploring how reinforcement learning (RL) could assist with continuous 3D content placement that is aware of users' poses and their surrounding environments. Through an initial exploration and preliminary evaluation, our results demonstrate the potential of RL to position content that maximizes the reward for users on the go. We further identify future directions for research that could harness the power of RL for personalized and optimized UI and content placement in MR.

cs.HC↗

arXiv2025

Examining the Effects of Immersive and Non-Immersive Presenter Modalities on Engagement and Social Interaction in Co-located Augmented Presentations

Head-worn augmented reality (AR) allows audiences to be immersed and engaged in stories told by live presenters. While presenters may also be in AR to have the same level of immersion and awareness as their audience, this symmetric presentation style may diminish important social cues such as eye contact. In this work, we examine the effects this (a)symmetry has on engagement, group awareness, and social interaction in co-located one-on-one augmented presentations. We developed a presentation system incorporating 2D/3D content that audiences can view and interact with in AR, with presenters controlling and delivering the presentation in either a symmetric style in AR, or an asymmetric style with a handheld tablet. We conducted a within- and between-subjects evaluation with 12 participant pairs to examine the differences between these symmetric and asymmetric presentation modalities. From our findings, we extracted four themes and derived strategies and guidelines for designers interested in augmented presentations.

cs.HC↗

arXiv2024

Enabling Data-Driven and Empathetic Interactions: A Context-Aware 3D Virtual Agent in Mixed Reality for Enhanced Financial Customer Experience

In this paper, we introduce a novel system designed to enhance customer service in the financial and retail sectors through a context-aware 3D virtual agent, utilizing Mixed Reality (MR) and Vision Language Models (VLMs). Our approach focuses on enabling data-driven and empathetic interactions that ensure customer satisfaction by introducing situational awareness of the physical location, personalized interactions based on customer profiles, and rigorous privacy and security standards. We discuss our design considerations critical for deployment in real-world customer service environments, addressing challenges in user data management and sensitive information handling. We also outline the system architecture and key features unique to banking and retail environments. Our work demonstrates the potential of integrating MR and VLMs in service industries, offering practical insights in customer service delivery while maintaining high standards of security and personalization.

cs.HC↗

arXiv2024

ConnectVR: A Trigger-Action Interface for Creating Agent-based Interactive VR Stories

The demand for interactive narratives is growing with increasing popularity of VR and video gaming. This presents an opportunity to create interactive storytelling experiences that allow players to engage with a narrative from a first person perspective, both, immersively in VR and in 3D on a computer. However, for artists and storytellers without programming experience, authoring such experiences is a particularly complex task as it involves coding a series of story events (character animation, movements, time control, dialogues, etc.) to be connected and triggered by a variety of player behaviors. In this work, we present ConnectVR, a trigger-action interface to enable non-technical creators design agent-based narrative experiences. Our no-code authoring method specifically focuses on the design of narratives driven by a series of cause-effect relationships triggered by the player's actions. We asked 15 participants to use ConnectVR in a preliminary workshop study as well as two artists to extensively use our system to create VR narrative projects in a three-week in-depth study. Our findings shed light on the creative opportunities facilitated by ConnectVR's trigger-action approach, particularly its capability to establish chained behavioral effects between virtual characters and objects. The results of both studies underscore the positive feedback from participants regarding our system's capacity to not only support creativity but also to simplify the creation of interactive narrative experiences. Results indicate compatibility with non-technical narrative creator's workflows, showcasing its potential to enhance the overall creative process in the realm of VR narrative design.

cs.HC↗

arXiv2024

EntangleVR++: Evaluating the Potential of using Entanglement in an Interactive VR Scene Creation System

Interactive digital stories provide a sense of flexibility and freedom to players by allowing them to make choices at key junctions. These choices advance the narrative and determine, to some degree, how the story evolves for that player. As shown in prior work, the ability to control or participate in the construction of the narrative can give the player a high level of agency that results in a stronger sense of immersion in the narrative experience. To support the design of this type of interactive storytelling, our system, EntangleVR++, borrows the idea of entanglement from quantum computing. Our use of entanglement allows creators and storytellers control over which sequences of story events take place in correlation with each other, initiated by the choices a player makes. In this work, we evaluated how well our idea of entanglement enables creators to easily and quickly design interactive VR narratives. We asked 16 participants to use our system and based on user interviews, analyses of screen recordings, and questionnaire feedback, we extracted four themes. From these themes and the study overall, we derived four authoring strategies for tool designers interested in the design of future visual interface for interactively creating virtual scenes that include relational objects and multiple outcomes driven by player interactions.

cs.HC↗

arXiv2023

Simple and Efficient Partial Graph Adversarial Attack: A New Perspective

As the study of graph neural networks becomes more intensive and comprehensive, their robustness and security have received great research interest. The existing global attack methods treat all nodes in the graph as their attack targets. Although existing methods have achieved excellent results, there is still considerable space for improvement. The key problem is that the current approaches rigidly follow the definition of global attacks. They ignore an important issue, i.e., different nodes have different robustness and are not equally resilient to attacks. From a global attacker's view, we should arrange the attack budget wisely, rather than wasting them on highly robust nodes. To this end, we propose a totally new method named partial graph attack (PGA), which selects the vulnerable nodes as attack targets. First, to select the vulnerable items, we propose a hierarchical target selection policy, which allows attackers to only focus on easy-to-attack nodes. Then, we propose a cost-effective anchor-picking policy to pick the most promising anchors for adding or removing edges, and a more aggressive iterative greedy-based attack method to perform more efficient attacks. Extensive experimental results demonstrate that PGA can achieve significant improvements in both attack effect and attack efficiency compared to other existing graph global attack methods.

cs.LG↗

arXiv2023

Enhancing Virtual Assistant Intelligence: Precise Area Targeting for Instance-level User Intents beyond Metadata

Virtual assistants have been widely used by mobile phone users in recent years. Although their capabilities of processing user intents have been developed rapidly, virtual assistants in most platforms are only capable of handling pre-defined high-level tasks supported by extra manual efforts of developers. However, instance-level user intents containing more detailed objectives with complex practical situations, are yet rarely studied so far. In this paper, we explore virtual assistants capable of processing instance-level user intents based on pixels of application screens, without the requirements of extra extensions on the application side. We propose a novel cross-modal deep learning pipeline, which understands the input vocal or textual instance-level user intents, predicts the targeting operational area, and detects the absolute button area on screens without any metadata of applications. We conducted a user study with 10 participants to collect a testing dataset with instance-level user intents. The testing dataset is then utilized to evaluate the performance of our model, which demonstrates that our model is promising with the achievement of 64.43% accuracy on our testing dataset.

cs.HC↗

arXiv2022

CardsVR: A Two-Person VR Experience with Passive Haptic Feedback from a Deck of Playing Cards

Presence in virtual reality (VR) is meaningful for remotely connecting with others and facilitating social interactions despite great distance while providing a sense of "being there." This work presents CardsVR, a two-person VR experience that allows remote participants to play a game of cards together. An entire deck of tracked cards are used to recreate the sense of playing cards in-person. Prior work in VR commonly provides passive haptic feedback either through a single object or through static objects in the environment. CardsVR is novel in providing passive haptic feedback through multiple cards that are individually tracked and represented in the virtual environment. Participants interact with the physical cards by picking them up, holding them, playing them, or moving them on the physical table. Our participant study (N=23) shows that passive haptic feedback provides significant improvement in three standard measures of presence: Possibility to Act, Realism, and Haptics.

cs.HC↗

arXiv2022

Efficient and ultra-stable perovskite light-emitting diodes

Perovskite light-emitting diodes (PeLEDs) have emerged as a strong contender for next-generation display and information technologies. However, similar to perovskite solar cells, the poor operational stability remains the main obstacle toward commercial applications. Here we demonstrate ultra-stable and efficient PeLEDs with extraordinary operational lifetimes (T50) of 1.0x10^4 h, 2.8x10^4 h, 5.4x10^5 h, and 1.9x10^6 h at initial radiance (or current densities) of 3.7 W/sr/m2 (~5 mA/cm2), 2.1 W/sr/m2 (~3.2 mA/cm2), 0.42 W/sr/m2 (~1.1 mA/cm2), and 0.21 W/sr/m2 (~0.7 mA/cm2) respectively, and external quantum efficiencies of up to 22.8%. Key to this breakthrough is the introduction of a dipolar molecular stabilizer, which serves two critical roles simultaneously. First, it prevents the detrimental transformation and decomposition of the alpha-phase FAPbI3 perovskite, by inhibiting the formation of lead and iodide intermediates. Secondly, hysteresis-free device operation and microscopic luminescence imaging experiments reveal substantially suppressed ion migration in the emissive perovskite. The record-long PeLED lifespans are encouraging, as they now satisfy the stability requirement for commercial organic LEDs (OLEDs). These results remove the critical concern that halide perovskite devices may be intrinsically unstable, paving the path toward industrial applications.

cond-mat.mtrl-sci↗

arXiv2021

SceneAR: Scene-based Micro Narratives for Sharing and Remixing in Augmented Reality

Short-form digital storytelling has become a popular medium for millions of people to express themselves. Traditionally, this medium uses primarily 2D media such as text (e.g., memes), images (e.g., Instagram), gifs (e.g., Giphy), and videos (e.g., TikTok, Snapchat). To expand the modalities from 2D to 3D media, we present SceneAR, a smartphone application for creating sequential scene-based micro narratives in augmented reality (AR). What sets SceneAR apart from prior work is the ability to share the scene-based stories as AR content -- no longer limited to sharing images or videos, these narratives can now be experienced in people's own physical environments. Additionally, SceneAR affords users the ability to remix AR, empowering them to build-upon others' creations collectively. We asked 18 people to use SceneAR in a 3-day study. Based on user interviews, analysis of screen recordings, and the stories they created, we extracted three themes. From those themes and the study overall, we derived six strategies for designers interested in supporting short-form AR narratives.

cs.HC↗

arXiv2021

Convolutional Neural Network with Pruning Method for Handwritten Digit Recognition

CNN model is a popular method for imagery analysis, so it could be utilized to recognize handwritten digits based on MNIST datasets. For higher recognition accuracy, various CNN models with different fully connected layer sizes are exploited to figure out the relationship between the CNN fully connected layer size and the recognition accuracy. Inspired by previous pruning work, we performed pruning methods of distinctiveness on CNN models and compared the pruning performance with NN models. For better pruning performances on CNN, the effect of angle threshold on the pruning performance was explored. The evaluation results show that: for the fully connected layer size, there is a threshold, so that when the layer size increases, the recognition accuracy grows if the layer size smaller than the threshold, and falls if the layer size larger than the threshold; the performance of pruning performed on CNN is worse than on NN; as pruning angle threshold increases, the fully connected layer size and the recognition accuracy decreases. This paper also shows that for CNN models trained by the MNIST dataset, they are capable of handwritten digit recognition and achieve the highest recognition accuracy with fully connected layer size 400. In addition, for same dataset MNIST, CNN models work better than big, deep, simple NN models in a published paper.

cs.CV↗