arXiv subjects

Nicholas Hawkins

Publications and source records attributed to Nicholas Hawkins.

4 recordsLinked to original sources

arXiv2026

A Dialogue-Based Human-Robot Interaction Protocol for Wheelchair and Robotic Arm Integrated Control

People with lower and upper body disabilities can benefit from wheelchairs and robotic arms to improve mobility and independence. Prior assistive interfaces, such as touchscreens and voice-driven predefined commands, often remain unintuitive and struggle to capture complex user intent. We propose a natural, dialogue based human robot interaction protocol that simulates an intelligent agent capable of communicating with users to understand intent and execute assistive actions. In a pilot study, five participants completed five assistive tasks (cleaning, drinking, feeding, drawer opening, and door opening) through dialogue-based interaction with a wheelchair and robotic arm. As a baseline, participants were required to open a door using the manual control (a wheelchair joystick and a game controller for the arm) and complete a questionnaire to gather their feedback. By analyzing the post-study questionnaires, we found that most participants enjoyed the dialogue-based interaction and assistive robot autonomy.

cs.RO↗

arXiv2026

A Multimodal Data Collection Framework for Dialogue-Driven Assistive Robotics to Clarify Ambiguities: A Wizard-of-Oz Pilot Study

Integrated control of wheelchairs and wheelchair-mounted robotic arms (WMRAs) has strong potential to increase independence for users with severe motor limitations, yet existing interfaces often lack the flexibility needed for intuitive assistive interaction. Although data-driven AI methods show promise, progress is limited by the lack of multimodal datasets that capture natural Human-Robot Interaction (HRI), particularly conversational ambiguity in dialogue-driven control. To address this gap, we propose a multimodal data collection framework that employs a dialogue-based interaction protocol and a two-room Wizard-of-Oz (WoZ) setup to simulate robot autonomy while eliciting natural user behavior. The framework records five synchronized modalities: RGB-D video, conversational audio, inertial measurement unit (IMU) signals, end-effector Cartesian pose, and whole-body joint states across five assistive tasks. Using this framework, we collected a pilot dataset of 53 trials from five participants and validated its quality through motion smoothness analysis and user feedback. The results show that the framework effectively captures diverse ambiguity types and supports natural dialogue-driven interaction, demonstrating its suitability for scaling to a larger dataset for learning, benchmarking, and evaluation of ambiguity-aware assistive control.

cs.RO↗

arXiv2023

Nanoscale rheology: Dynamic Mechanical Analysis over a broad and continuous frequency range using Photothermal Actuation Atomic Force Microscopy

Polymeric materials are widely used in industries ranging from automotive to biomedical. Their mechanical properties play a crucial role in their application and function and arise from the nanoscale structures and interactions of their constitutive polymer molecules. Polymeric materials behave viscoelastically, i.e. their mechanical responses depend on the time scale of the measurements; quantifying these time-dependent rheological properties at the nanoscale is relevant to develop, for example, accurate models and simulations of those materials, which are needed for advanced industrial applications. In this paper, an atomic force microscopy (AFM) method based on the photothermal actuation of an AFM cantilever is developed to quantify the nanoscale loss tangent, storage modulus, and loss modulus of polymeric materials. The method is then validated on a styrene-butadiene rubber (SBR), demonstrating the method's ability to quantify nanoscale viscoelasticity over a continuous frequency range up to five orders of magnitude (0.2 Hz to 20,200 Hz). Furthermore, this method is combined with AFM viscoelastic mapping obtained with amplitude-modulation frequency-modulation (AM-FM) AFM, enabling the extension of viscoelastic quantification over an even broader frequency range, and demonstrating that the novel technique synergizes with preexisting AFM techniques for quantitative measurement of viscoelastic properties. The method presented here introduces a way to characterize the viscoelasticity of polymeric materials, and soft matter in general at the nanoscale, for any application.

cond-mat.mes-hall↗

arXiv2020

Whole Slide Images based Cancer Survival Prediction using Attention Guided Deep Multiple Instance Learning Networks

Traditional image-based survival prediction models rely on discriminative patch labeling which make those methods not scalable to extend to large datasets. Recent studies have shown Multiple Instance Learning (MIL) framework is useful for histopathological images when no annotations are available in classification task. Different to the current image-based survival models that limit to key patches or clusters derived from Whole Slide Images (WSIs), we propose Deep Attention Multiple Instance Survival Learning (DeepAttnMISL) by introducing both siamese MI-FCN and attention-based MIL pooling to efficiently learn imaging features from the WSI and then aggregate WSI-level information to patient-level. Attention-based aggregation is more flexible and adaptive than aggregation techniques in recent survival models. We evaluated our methods on two large cancer whole slide images datasets and our results suggest that the proposed approach is more effective and suitable for large datasets and has better interpretability in locating important patterns and features that contribute to accurate cancer survival predictions. The proposed framework can also be used to assess individual patient's risk and thus assisting in delivering personalized medicine. Codes are available at https://github.com/uta-smile/DeepAttnMISL_MEDIA.

eess.IV↗