arXiv subjects

Shixiao Wang

Publications and source records attributed to Shixiao Wang.

9 recordsLinked to original sources

arXiv2026

Copilot-Assisted Second-Thought Framework for Brain-to-Robot Hand Motion Decoding

Motor kinematics prediction (MKP) from electroencephalography (EEG) is an important research area for developing movement-related brain-computer interfaces (BCIs). While traditional methods often rely on convolutional neural networks (CNNs) or recurrent neural networks (RNNs), Transformer-based models have shown strong ability in modeling long sequential EEG data. In this study, we propose a CNN-attention hybrid model for decoding hand kinematics from EEG during grasp-and-lift tasks, achieving strong performance in within-subject experiments. We further extend this approach to EEG-EMG multimodal decoding, which yields substantially improved results. Within-subject tests achieve PCC values of 0.9854, 0.9946, and 0.9065 for the X, Y, and Z axes, respectively, computed on the midpoint trajectory between the thumb and index finger, while cross-subject tests result in 0.9643, 0.9795, and 0.5852. The decoded trajectories from both modalities are then used to control a Franka Panda robotic arm in a MuJoCo simulation. To enhance trajectory fidelity, we introduce a copilot framework that filters low-confidence decoded points using a motion-state-aware critic within a finite-state machine. This post-processing step improves the overall within-subject PCC of EEG-only decoding to 0.93 while excluding fewer than 20% of the data points.

cs.RO↗

arXiv2026

Generative Modeling of Human-Computer Interfaces with Diffusion Processes and Conditional Control

This study investigates human-computer interface generation based on diffusion models to overcome the limitations of traditional template-based design and fixed rule-driven methods. It first analyzes the key challenges of interface generation, including the diversity of interface elements, the complexity of layout logic, and the personalization of user needs. A generative framework centered on the diffusion-reverse diffusion process is then proposed, with conditional control introduced in the reverse diffusion stage to integrate user intent, contextual states, and task constraints, enabling unified modeling of visual presentation and interaction logic. In addition, regularization constraints and optimization objectives are combined to ensure the rationality and stability of the generated interfaces. Experiments are conducted on a public interface dataset with systematic evaluations, including comparative experiments, hyperparameter sensitivity tests, environmental sensitivity tests, and data sensitivity tests. Results show that the proposed method outperforms representative models in mean squared error, structural similarity, peak signal-to-noise ratio, and mean absolute error, while maintaining strong robustness under different parameter settings and environmental conditions. Overall, the diffusion model framework effectively improves the diversity, rationality, and intelligence of interface generation, providing a feasible solution for automated interface generation in complex interaction scenarios.

cs.HC↗

arXiv2025

Transformer-Based Modeling of User Interaction Sequences for Dwell Time Prediction in Human-Computer Interfaces

This study investigates the task of dwell time prediction and proposes a Transformer framework based on interaction behavior modeling. The method first represents user interaction sequences on the interface by integrating dwell duration, click frequency, scrolling behavior, and contextual features, which are mapped into a unified latent space through embedding and positional encoding. On this basis, a multi-head self-attention mechanism is employed to capture long-range dependencies, while a feed-forward network performs deep nonlinear transformations to model the dynamic patterns of dwell time. Multiple comparative experiments are conducted with BILSTM, DRFormer, FedFormer, and iTransformer as baselines under the same conditions. The results show that the proposed method achieves the best performance in terms of MSE, RMSE, MAPE, and RMAE, and more accurately captures the complex patterns in interaction behavior. In addition, sensitivity experiments are carried out on hyperparameters and environments to examine the impact of the number of attention heads, sequence window length, and device environment on prediction performance, which further demonstrates the robustness and adaptability of the method. Overall, this study provides a new solution for dwell time prediction from both theoretical and methodological perspectives and verifies its effectiveness in multiple aspects.

cs.HC↗

arXiv2025

Graph Neural Networks for User Satisfaction Classification in Human-Computer Interaction

This study focuses on the problem of user satisfaction classification and proposes a framework based on graph neural networks to address the limitations of traditional methods in handling complex interaction relationships and multidimensional features. User behaviors, interface elements, and their potential connections are abstracted into a graph structure, and joint modeling of nodes and edges is used to capture semantics and dependencies in the interaction process. Graph convolution and attention mechanisms are introduced to fuse local features and global context, and global pooling with a classification layer is applied to achieve automated satisfaction classification. The method extracts deep patterns from structured data and improves adaptability and robustness in multi-source heterogeneous and dynamic environments. To verify effectiveness, a public user satisfaction survey dataset from Kaggle is used, and results are compared with multiple baseline models across several performance metrics. Experiments show that the method outperforms existing approaches in accuracy, F1-Score, AUC, and Precision, demonstrating the advantage of graph-based modeling in satisfaction prediction tasks. The study not only enriches the theoretical framework of user modeling but also highlights its practical value in optimizing human-computer interaction experience.

cs.HC↗

arXiv2025

Capsule Network-Based Semantic Intent Modeling for Human-Computer Interaction

This paper proposes a user semantic intent modeling algorithm based on Capsule Networks to address the problem of insufficient accuracy in intent recognition for human-computer interaction. The method represents semantic features in input text through a vectorized capsule structure. It uses a dynamic routing mechanism to transfer information across multiple capsule layers. This helps capture hierarchical relationships and part-whole structures between semantic entities more effectively. The model uses a convolutional feature extraction module as the low-level encoder. After generating initial semantic capsules, it forms high-level abstract intent representations through an iterative routing process. To further enhance performance, a margin-based mechanism is introduced into the loss function. This improves the model's ability to distinguish between intent classes. Experiments are conducted using a public natural language understanding dataset. Multiple mainstream models are used for comparison. Results show that the proposed model outperforms traditional methods and other deep learning structures in terms of accuracy, F1-score, and intent detection rate. The study also analyzes the effect of the number of dynamic routing iterations on model performance. A convergence curve of the loss function during training is provided. These results verify the stability and effectiveness of the proposed method in semantic modeling. Overall, this study presents a new structured modeling approach to improve intent recognition under complex semantic conditions.

cs.CL↗

arXiv2025

A Deep Learning Approach to Interface Color Quality Assessment in HCI

In this paper, a quantitative evaluation model for the color quality of human-computer interaction interfaces is proposed by combining deep convolutional neural networks (CNN). By extracting multidimensional features of interface images, including hue, brightness, purity, etc., CNN is used for efficient feature modeling and quantitative analysis, and the relationship between interface design and user perception is studied. The experiment is based on multiple international mainstream website interface datasets, covering e-commerce platforms, social media, education platforms, etc., and verifies the evaluation effect of the model on indicators such as contrast, clarity, color coordination, and visual appeal. The results show that the CNN evaluation is highly consistent with the user rating, with a correlation coefficient of up to 0.96, and it also shows high accuracy in mean square error and absolute error. Compared with traditional experience-based evaluation methods, the proposed model can efficiently and scientifically capture the visual characteristics of the interface and avoid the influence of subjective factors. Future research can explore the introduction of multimodal data (such as text and interactive behavior) into the model to further enhance the evaluation ability of dynamic interfaces and expand it to fields such as smart homes, medical systems, and virtual reality. This paper provides new methods and new ideas for the scientific evaluation and optimization of interface design.

cs.HC↗

arXiv2024

Dynamic User Interface Generation for Enhanced Human-Computer Interaction Using Variational Autoencoders

This study presents a novel approach for intelligent user interaction interface generation and optimization, grounded in the variational autoencoder (VAE) model. With the rapid advancement of intelligent technologies, traditional interface design methods struggle to meet the evolving demands for diversity and personalization, often lacking flexibility in real-time adjustments to enhance the user experience. Human-Computer Interaction (HCI) plays a critical role in addressing these challenges by focusing on creating interfaces that are functional, intuitive, and responsive to user needs. This research leverages the RICO dataset to train the VAE model, enabling the simulation and creation of user interfaces that align with user aesthetics and interaction habits. By integrating real-time user behavior data, the system dynamically refines and optimizes the interface, improving usability and underscoring the importance of HCI in achieving a seamless user experience. Experimental findings indicate that the VAE-based approach significantly enhances the quality and precision of interface generation compared to other methods, including autoencoders (AE), generative adversarial networks (GAN), conditional GANs (cGAN), deep belief networks (DBN), and VAE-GAN. This work contributes valuable insights into HCI, providing robust technical solutions for automated interface generation and enhanced user experience optimization.

cs.HC↗

arXiv2024

Emotion-Aware Interaction Design in Intelligent User Interface Using Multi-Modal Deep Learning

In an era where user interaction with technology is ubiquitous, the importance of user interface (UI) design cannot be overstated. A well-designed UI not only enhances usability but also fosters more natural, intuitive, and emotionally engaging experiences, making technology more accessible and impactful in everyday life. This research addresses this growing need by introducing an advanced emotion recognition system to significantly improve the emotional responsiveness of UI. By integrating facial expressions, speech, and textual data through a multi-branch Transformer model, the system interprets complex emotional cues in real-time, enabling UIs to interact more empathetically and effectively with users. Using the public MELD dataset for validation, our model demonstrates substantial improvements in emotion recognition accuracy and F1 scores, outperforming traditional methods. These findings underscore the critical role that sophisticated emotion recognition plays in the evolution of UIs, making technology more attuned to user needs and emotions. This study highlights how enhanced emotional intelligence in UIs is not only about technical innovation but also about fostering deeper, more meaningful connections between users and the digital world, ultimately shaping how people interact with technology in their daily lives.

cs.HC↗

arXiv2024

Efficient and Aesthetic UI Design with a Deep Learning-Based Interface Generation Tree Algorithm

This paper presents a novel method for user interface (UI) generation based on the Transformer architecture, addressing the increasing demand for efficient and aesthetically pleasing UI designs in software development. Traditional UI design relies heavily on designers' expertise, which can be time-consuming and costly. Leveraging the capabilities of Transformers, particularly their ability to capture complex design patterns and long-range dependencies, we propose a Transformer-based interface generation tree algorithm. This method constructs a hierarchical representation of UI components as nodes in a tree structure, utilizing pre-trained Transformer models for encoding and decoding. We define a markup language to describe UI components and their properties and use a rich dataset of real-world web and mobile application interfaces for training. The experimental results demonstrate that our approach not only significantly enhances design quality and efficiency but also outperforms traditional models in user satisfaction and aesthetic appeal. We also provide a comparative analysis with existing models, illustrating the advantages of our method in terms of accuracy, user ratings, and design similarity. Overall, our study underscores the potential of the Transformer based approach to revolutionize the UI design process, making it accessible for non-professionals while maintaining high standards of quality.

cs.HC↗