arXiv subjects

Fillia Makedon

Publications and source records attributed to Fillia Makedon.

14 recordsLinked to original sources

arXiv2026

A Contrastive Fewshot RGBD Traversability Segmentation Framework for Indoor Robotic Navigation

Indoor traversability segmentation aims to identify safe, navigable free space for autonomous agents, which is critical for robotic navigation. Pure vision-based models often fail to detect thin obstacles, such as chair legs, which can pose serious safety risks. We propose a multi-modal segmentation framework that leverages RGB images and sparse 1D laser depth information to capture geometric interactions and improve the detection of challenging obstacles. To reduce the reliance on large labeled datasets, we adopt the few-shot segmentation (FSS) paradigm, enabling the model to generalize from limited annotated examples. Traditional FSS methods focus solely on positive prototypes, often leading to overfitting to the support set and poor generalization. To address this, we introduce a negative contrastive learning (NCL) branch that leverages negative prototypes (obstacles) to refine free-space predictions. Additionally, we design a two-stage attention depth module to align 1D depth vectors with RGB images both horizontally and vertically. Extensive experiments on our custom-collected indoor RGB-D traversability dataset demonstrate that our method outperforms state-of-the-art FSS and RGB-D segmentation baselines, achieving up to 9\% higher mIoU under both 1-shot and 5-shot settings. These results highlight the effectiveness of leveraging negative prototypes and sparse depth for robust and efficient traversability segmentation.

cs.RO↗

arXiv2025

NavVI: A Telerobotic Simulation with Multimodal Feedback for Visually Impaired Navigation in Warehouse Environments

Industrial warehouses are congested with moving forklifts, shelves and personnel, making robot teleoperation particularly risky and demanding for blind and low-vision (BLV) operators. Although accessible teleoperation plays a key role in inclusive workforce participation, systematic research on its use in industrial environments is limited, and few existing studies barely address multimodal guidance designed for BLV users. We present a novel multimodal guidance simulator that enables BLV users to control a mobile robot through a high-fidelity warehouse environment while simultaneously receiving synchronized visual, auditory, and haptic feedback. The system combines a navigation mesh with regular re-planning so routes remain accurate avoiding collisions as forklifts and human avatars move around the warehouse. Users with low vision are guided with a visible path line towards destination; navigational voice cues with clockwise directions announce upcoming turns, and finally proximity-based haptic feedback notifies the users of static and moving obstacles in the path. This real-time, closed-loop system offers a repeatable testbed and algorithmic reference for accessible teleoperation research. The simulator's design principles can be easily adapted to real robots due to the alignment of its navigation, speech, and haptic modules with commercial hardware, supporting rapid feasibility studies and deployment of inclusive telerobotic tools in actual warehouses.

cs.HC↗

arXiv2024

Enhancing Robustness of Indoor Robotic Navigation with Free-Space Segmentation Models Against Adversarial Attacks

Endeavors in indoor robotic navigation rely on the accuracy of segmentation models to identify free space in RGB images. However, deep learning models are vulnerable to adversarial attacks, posing a significant challenge to their real-world deployment. In this study, we identify vulnerabilities within the hidden layers of neural networks and introduce a practical approach to reinforce traditional adversarial training. Our method incorporates a novel distance loss function, minimizing the gap between hidden layers in clean and adversarial images. Experiments demonstrate satisfactory performance in improving the model's robustness against adversarial perturbations.

cs.CV↗

arXiv2023

Depth-guided Free-space Segmentation for a Mobile Robot

Accurate indoor free-space segmentation is a challenging task due to the complexity and the dynamic nature that indoor environments exhibit. We propose an indoors free-space segmentation method that associates large depth values with navigable regions. Our method leverages an unsupervised masking technique that, using positive instances, generates segmentation labels based on textural homogeneity and depth uniformity. Moreover, we generate superpixels corresponding to areas of higher depth and align them with features extracted from a Dense Prediction Transformer (DPT). Using the estimated free-space masks and the DPT feature representation, a SegFormer model is fine-tuned on our custom-collected indoor dataset. Our experiments demonstrate sufficient performance in intricate scenarios characterized by cluttered obstacles and challenging identification of free space.

cs.RO↗

arXiv2022

Assessing Fatigue with Multimodal Wearable Sensors and Machine Learning

Fatigue is a loss in cognitive or physical performance due to physiological factors such as insufficient sleep, long work hours, stress, and physical exertion. It adversely affects the human body and can slow reaction times, reduce attention, and limit short-term memory. Hence, there is a need to monitor a person's state to avoid extreme fatigue conditions that can result in physiological complications. However, tools to understand and assess fatigue are minimal. This paper primarily focuses on building an experimental setup that induces cognitive fatigue (CF) and physical fatigue (PF) through multiple cognitive and physical tasks while simultaneously recording physiological data. First, we built a prototype sensor suit embedded with numerous physiological sensors for easy use during data collection. Second, participants' self-reported visual analog scores (VAS) are reported after each task to confirm fatigue induction. Finally, an evaluation system is built that utilizes machine learning (ML) models to detect states of CF and PF from sensor data, thus providing an objective measure. Our methods beat state-of-the-art approaches, where Random Forest performs the best in detecting PF with an accuracy of 80.5% while correctly predicting the true PF condition 88% of the time. On the other hand, the long short-term memory (LSTM) recurrent neural network produces the best results in detecting CF in the subjects (with 84.1% accuracy, 0.9 recall).

cs.HC↗

arXiv2021

Understanding Cognitive Fatigue from fMRI Scans with Self-supervised Learning

Functional magnetic resonance imaging (fMRI) is a neuroimaging technique that records neural activations in the brain by capturing the blood oxygen level in different regions based on the task performed by a subject. Given fMRI data, the problem of predicting the state of cognitive fatigue in a person has not been investigated to its full extent. This paper proposes tackling this issue as a multi-class classification problem by dividing the state of cognitive fatigue into six different levels, ranging from no-fatigue to extreme fatigue conditions. We built a spatio-temporal model that uses convolutional neural networks (CNN) for spatial feature extraction and a long short-term memory (LSTM) network for temporal modeling of 4D fMRI scans. We also applied a self-supervised method called MoCo (Momentum Contrast) to pre-train our model on a public dataset BOLD5000 and fine-tuned it on our labeled dataset to predict cognitive fatigue. Our novel dataset contains fMRI scans from Traumatic Brain Injury (TBI) patients and healthy controls (HCs) while performing a series of N-back cognitive tasks. This method establishes a state-of-the-art technique to analyze cognitive fatigue from fMRI data and beats previous approaches to solve this problem.

cs.CV↗

arXiv2021

Sequential Late Fusion Technique for Multi-modal Sentiment Analysis

Multi-modal sentiment analysis plays an important role for providing better interactive experiences to users. Each modality in multi-modal data can provide different viewpoints or reveal unique aspects of a user's emotional state. In this work, we use text, audio and visual modalities from MOSI dataset and we propose a novel fusion technique using a multi-head attention LSTM network. Finally, we perform a classification task and evaluate its performance.

cs.LG↗

arXiv2021

Towards a Multi-purpose Robotic Nursing Assistant

Robotic nursing aid is one of the heavily researched areas in robotics nowadays. Several robotic assistants exist that only focus on a specific function related to nurses assistance or functions related to patient aid. There is a need for a unified system that not only performs tasks that would assist nurses and reduce their burden but also perform tasks that help a patient. In recent times, due to the COVID-19 pandemic, there is also an increase in the need for robotic assistants that have teleoperation capabilities to provide better protection against the virus spread. To address these requirements, we propose a novel Multi-purpose Intelligent Nurse Aid (MINA) robotic system that is capable of providing walking assistance to the patients and perform teleoperation tasks with an easy-to-use and intuitive Graphical User Interface (GUI). This paper also presents preliminary results from the walking assistant task that improves upon the current state-of-the-art methods and shows the developed GUI for teleoperation.

cs.RO↗

arXiv2021

A Survey on Contrastive Self-supervised Learning

Self-supervised learning has gained popularity because of its ability to avoid the cost of annotating large-scale datasets. It is capable of adopting self-defined pseudo labels as supervision and use the learned representations for several downstream tasks. Specifically, contrastive learning has recently become a dominant component in self-supervised learning methods for computer vision, natural language processing (NLP), and other domains. It aims at embedding augmented versions of the same sample close to each other while trying to push away embeddings from different samples. This paper provides an extensive review of self-supervised methods that follow the contrastive approach. The work explains commonly used pretext tasks in a contrastive learning setup, followed by different architectures that have been proposed so far. Next, we have a performance comparison of different methods for multiple downstream tasks such as image classification, object detection, and action recognition. Finally, we conclude with the limitations of the current methods and the need for further techniques and future directions to make substantial progress.

cs.CV↗

arXiv2020

Self-Supervised Human Activity Recognition by Augmenting Generative Adversarial Networks

This article proposes a novel approach for augmenting generative adversarial network (GAN) with a self-supervised task in order to improve its ability for encoding video representations that are useful in downstream tasks such as human activity recognition. In the proposed method, input video frames are randomly transformed by different spatial transformations, such as rotation, translation and shearing or temporal transformations such as shuffling temporal order of frames. Then discriminator is encouraged to predict the applied transformation by introducing an auxiliary loss. Subsequently, results prove superiority of the proposed method over baseline methods for providing a useful representation of videos used in human activity recognition performed on datasets such as KTH, UCF101 and Ball-Drop. Ball-Drop dataset is a specifically designed dataset for measuring executive functions in children through physically and cognitively demanding tasks. Using features from proposed method instead of baseline methods caused the top-1 classification accuracy to increase by more then 4%. Moreover, ablation study was performed to investigate the contribution of different transformations on downstream task.

cs.CV↗

arXiv2020

Automated system to measure Tandem Gait to assess executive functions in children

As mobile technologies have become ubiquitous in recent years, computer-based cognitive tests have become more popular and efficient. In this work, we focus on assessing motor function in children by analyzing their gait movements. Although there has been a lot of research on designing automated assessment systems for gait analysis, most of these efforts use obtrusive wearable sensors for measuring body movements. We have devised a computer vision-based assessment system that only requires a camera which makes it easier to employ in school or home environments. A dataset has been created with 27 children performing the test. Furthermore in order to improve the accuracy of the system, a deep learning based model was pre-trained on NTU-RGB+D 120 dataset and then it was fine-tuned on our gait dataset. The results highlight the efficacy of proposed work for automating the assessment of children's performances by achieving 76.61% classification accuracy.

cs.CV↗

arXiv2018

Using Humanoid Robot to Instruct and Evaluate Performance of a Physical Task

In this paper, we present a tool to assess users ability to change tasks. To do this, we use a variation of the Box and Blocks Test. In this version, a humanoid robot instructs a user to perform a task involving the movement of certain colored blocks. The robot changes randomly change the color of blocks that the user is supposed to move. Canny Edge Detection and Hough Transformation are used to assess user perform the robot's built-in camera. This will allow the robot to inform the user and keep a log of their progress. We present this method for monitoring user progress by describing how the moved blocks are detected. We also present the results of a pilot study where users used this system to perform the task. Preliminary results show that users do not perform differently when the task is changed in this scenario.

cs.HC↗

arXiv2018

Towards Deep Learning based Hand Keypoints Detection for Rapid Sequential Movements from RGB Images

Hand keypoints detection and pose estimation has numerous applications in computer vision, but it is still an unsolved problem in many aspects. An application of hand keypoints detection is in performing cognitive assessments of a subject by observing the performance of that subject in physical tasks involving rapid finger motion. As a part of this work, we introduce a novel hand key-points benchmark dataset that consists of hand gestures recorded specifically for cognitive behavior monitoring. We explore the state of the art methods in hand keypoint detection and we provide quantitative evaluations for the performance of these methods on our dataset. In future, these results and our dataset can serve as a useful benchmark for hand keypoint recognition for rapid finger movements.

cs.CV↗

arXiv2017

Improving the Accuracy of the CogniLearn System for Cognitive Behavior Assessment

HTKS is a game-like cognitive assessment method, designed for children between four and eight years of age. During the HTKS assessment, a child responds to a sequence of requests, such as "touch your head" or "touch your toes". The cognitive challenge stems from the fact that the children are instructed to interpret these requests not literally, but by touching a different body part than the one stated. In prior work, we have developed the CogniLearn system, that captures data from subjects performing the HTKS game, and analyzes the motion of the subjects. In this paper we propose some specific improvements that make the motion analysis module more accurate. As a result of these improvements, the accuracy in recognizing cases where subjects touch their toes has gone from 76.46% in our previous work to 97.19% in this paper.

cs.CV↗