arXiv subjects

Haofeng Chen

Publications and source records attributed to Haofeng Chen.

13 recordsLinked to original sources

arXiv2026

Toward Geometry-Scalable Whole-Body Touch for Humanoids: A 3D-Printed Conformal EIT Skin

Whole-body tactile sensing is a prerequisite for humanoids that operate in contact-rich human environments, but conventional taxel arrays scale poorly with surface area, wiring complexity, and robot-specific curvature. We present a conformal electrical impedance tomography tactile skin fabricated through a geometry-adaptable additive-manufacturing workflow. A flexible conductive TPU layer forms a continuous sensing domain, while contact-induced coupling with conductive patches produces boundary voltage changes that are reconstructed using a one-step Gauss-Newton EIT solver. We first characterize the electromechanical design space of the layered structure and show that low-resistance contact-enhancement patches and a porous conductive TPU sensing layer improve sensitivity while preserving printability. We then validate contact localization on a planar prototype, a curved U-shaped prototype, and a qualitative iCub-face-shaped geometry. The curved sensor achieves a mean localization error of 6 mm over 18 contact positions without supervised post-processing. These results suggest that additively manufactured tomographic skins can reduce the morphology-specific redesign burden for humanoid tactile coverage and provide a practical route toward large-area contact sensing for human-centered deployment.

cs.RO↗

arXiv2025

Physics-Driven Learning Framework for Tomographic Tactile Sensing

Electrical impedance tomography (EIT) provides an attractive solution for large-area tactile sensing due to its minimal wiring and shape flexibility, but its nonlinear inverse problem often leads to severe artifacts and inaccurate contact reconstruction. This work presents PhyDNN, a physics-driven deep reconstruction framework that embeds the EIT forward model directly into the learning objective. By jointly minimizing the discrepancy between predicted and ground-truth conductivity maps and enforcing consistency with the forward PDE, PhyDNN reduces the black-box nature of deep networks and improves both physical plausibility and generalization. To enable efficient backpropagation, we design a differentiable forward-operator network that accurately approximates the nonlinear EIT response, allowing fast physics-guided training. Extensive simulations and real tactile experiments on a 16-electrode soft sensor show that PhyDNN consistently outperforms NOSER, TV, and standard DNNs in reconstructing contact shape, location, and pressure distribution. PhyDNN yields fewer artifacts, sharper boundaries, and higher metric scores, demonstrating its effectiveness for high-quality tomographic tactile sensing.

cs.LG↗

arXiv2025

SO-Bench: A Structural Output Evaluation of Multimodal LLMs

Multimodal large language models (MLLMs) are increasingly deployed in real-world, agentic settings where outputs must not only be correct, but also conform to predefined data schemas. Despite recent progress in structured generation in textual domain, there is still no benchmark that systematically evaluates schema-grounded information extraction and reasoning over visual inputs. In this work, we conduct a comprehensive study of visual structural output capabilities for MLLMs with our carefully designed SO-Bench benchmark. Covering four visual domains, including UI screens, natural images, documents, and charts, SO-Bench is built from over 6.5K diverse JSON schemas and 1.8K curated image-schema pairs with human-verified quality. Benchmarking experiments on open-sourced and frontier proprietary models reveal persistent gaps in predicting accurate, schema compliant outputs, highlighting the need for better multimodal structured reasoning. Beyond benchmarking, we further conduct training experiments to largely improve the model's structured output capability. We make the benchmark and evaluation publicly available at https://github.com/apple/ml-sobench

cs.CV↗

arXiv2025

NTIRE 2025 challenge on Text to Image Generation Model Quality Assessment

This paper reports on the NTIRE 2025 challenge on Text to Image (T2I) generation model quality assessment, which will be held in conjunction with the New Trends in Image Restoration and Enhancement Workshop (NTIRE) at CVPR 2025. The aim of this challenge is to address the fine-grained quality assessment of text-to-image generation models. This challenge evaluates text-to-image models from two aspects: image-text alignment and image structural distortion detection, and is divided into the alignment track and the structural track. The alignment track uses the EvalMuse-40K, which contains around 40K AI-Generated Images (AIGIs) generated by 20 popular generative models. The alignment track has a total of 371 registered participants. A total of 1,883 submissions are received in the development phase, and 507 submissions are received in the test phase. Finally, 12 participating teams submitted their models and fact sheets. The structure track uses the EvalMuse-Structure, which contains 10,000 AI-Generated Images (AIGIs) with corresponding structural distortion mask. A total of 211 participants have registered in the structure track. A total of 1155 submissions are received in the development phase, and 487 submissions are received in the test phase. Finally, 8 participating teams submitted their models and fact sheets. Almost all methods have achieved better results than baseline methods, and the winning methods in both tracks have demonstrated superior prediction performance on T2I model quality assessment.

cs.CV↗

arXiv2025

Pneumatic-Tomographic Tactile Skin for Multicontact Localization and Force Estimation

Tactile skins based on electrical impedance tomography (EIT) enable large-area contact localization with few electrodes, but suffer from nonuniform sensitivity that limits force estimation accuracy. This work introduces a dual-channel tactile skin that integrates an EIT layer with a pneumatic pressure layer and a calibration framework that leverages their complementary strengths. The EIT layer provides robust multicontact localization, while the pneumatic pressure layer supplies a stable scalar measurement that serves as contact force estimation. A location-aware correction method is introduced, learning smooth spatial gain and offset fields from a single-session calibration, enabling spatially consistent multicontact force estimation. With location-aware correction, the proposed system achieves a single-contact force estimation root-mean-square error (RMSE) of 0.59 N across 10-25-mm indenters, representing a 35%-60% reduction over EIT-only approaches (1.45-1.48 N). In multicontact experiments, the per-contact RMSE is reduced by 39.6% compared to the uncorrected pneumatic baseline. The proposed system achieves accurate force estimation across diverse contact configurations, generalizes to varying indenter sizes, and preserves EIT's inherent advantages in multicontact localization. By letting the pneumatic pressure layer handle the force estimation and using the EIT layer to determine where each contact occurs, the method avoids the need for large datasets, complicated calibration setups, and heavy machine-learning pipelines often required by previous EIT-only approaches. This dual-channel design provides a practical, scalable, and easy-to-calibrate solution for building large-area robotic skins.

cs.RO↗

arXiv2025

Multi-Touch and Bending Sensing Using Electrical Impedance Tomography for Robotics

Electrical Impedance Tomography (EIT) offers a promising solution for distributed tactile sensing with minimal wiring and full-surface coverage in robotic applications. However, EIT-based tactile sensors face significant challenges during surface bending. Deformation alters the baseline impedance distribution and couples with touch-induced conductivity variations, complicating signal interpretation. To address this challenge, we present a novel sensing framework that integrates a deep neural network for interaction state classification with a dynamic adaptive reference strategy to decouple touch and deformation signals, while a data-driven regression model translates EIT voltage changes into continuous bending angles. The framework is validated using a magnetic hydrogel composite sensor that conforms to bendable surfaces. Experimental evaluations demonstrate that the proposed framework achieves precise and robust bending angle estimation, high accuracy in distinguishing touch, bending, and idle states, and significantly improves touch localization quality under bending deformation compared to conventional fixed-reference methods. Real-time experiments confirm the system's capability to reliably detect multi-touch interactions and track bending angles across varying deformation conditions. This work paves the way for flexible EIT-based robotic skins capable of rich multimodal sensing in robotics and human-robot interaction.

cs.RO↗

arXiv2024

A Physics-Embedded Dual-Learning Imaging Framework for Electrical Impedance Tomography

Electrical Impedance Tomography (EIT) is a promising noninvasive imaging technique that reconstructs the spatial conductivity distribution from boundary voltage measurements. However, it poses a highly nonlinear and ill-posed inverse problem. Traditional regularization-based methods are sensitive to noise and often produce significant artifacts. Physics-Embedded learning frameworks, particularly Physics-Informed Neural Networks (PINNs), have shown success in solving such inverse problems under ideal conditions with abundant internal data. Yet in practical EIT applications, only sparse and noisy boundary measurements are available. Moreover, changing boundary excitations require the simultaneous training of multiple forward networks and one inverse network, which significantly increases computational complexity and hampers convergence. To overcome these limitations, we propose a Physics-Embedded Dual-Learning Imaging Framework for EIT. The dual-learning strategy is composed of a supervised CNN-based forward network, which learns to predict a discrete internal potential distribution under fixed Neumann-to-Dirichlet boundary conditions, and an unsupervised PINN-based inverse network, which reconstructs the conductivity by enforcing the governing PDE through discrete numerical differentiation of the predicted potentials. This decoupled architecture removes the need for smooth conductivity assumptions, reduces the number of forward networks required from $K$ to 1, and improves reconstruction robustness and efficiency under realistic measurement constraints.(https://github.com/XuanxuanYang/CNN-PINNframework.git)

eess.SP↗

arXiv2024

A Two-Stage Imaging Framework Combining CNN and Physics-Informed Neural Networks for Full-Inverse Tomography: A Case Study in Electrical Impedance Tomography (EIT)

Electrical Impedance Tomography (EIT) is a highly ill-posed inverse problem, with the challenge of reconstructing internal conductivities using only boundary voltage measurements. Although Physics-Informed Neural Networks (PINNs) have shown potential in solving inverse problems, existing approaches are limited in their applicability to EIT, as they often rely on impractical prior knowledge and assumptions that cannot be satisfied in real-world scenarios. To address these limitations, we propose a two-stage hybrid learning framework that combines Convolutional Neural Networks (CNNs) and PINNs. This framework integrates data-driven and model-driven paradigms, blending supervised and unsupervised learning to reconstruct conductivity distributions while ensuring adherence to the underlying physical laws, thereby overcoming the constraints of existing methods.

cs.LG↗

arXiv2023

QDTrack: Quasi-Dense Similarity Learning for Appearance-Only Multiple Object Tracking

Similarity learning has been recognized as a crucial step for object tracking. However, existing multiple object tracking methods only use sparse ground truth matching as the training objective, while ignoring the majority of the informative regions in images. In this paper, we present Quasi-Dense Similarity Learning, which densely samples hundreds of object regions on a pair of images for contrastive learning. We combine this similarity learning with multiple existing object detectors to build Quasi-Dense Tracking (QDTrack), which does not require displacement regression or motion priors. We find that the resulting distinctive feature space admits a simple nearest neighbor search at inference time for object association. In addition, we show that our similarity learning scheme is not limited to video data, but can learn effective instance similarity even from static input, enabling a competitive tracking performance without training on videos or using tracking supervision. We conduct extensive experiments on a wide variety of popular MOT benchmarks. We find that, despite its simplicity, QDTrack rivals the performance of state-of-the-art tracking methods on all benchmarks and sets a new state-of-the-art on the large-scale BDD100K MOT benchmark, while introducing negligible computational overhead to the detector.

cs.CV↗

arXiv2022

Analyzing Material Recognition Performance of Thermal Tactile Sensing using a Large Materials Database and a Real Robot

In this paper we focus on analyzing the thermal modality of tactile sensing for material recognition using a large materials database. Many factors affect thermal recognition performance, including sensor noise, the initial temperatures of the sensor and the object, the thermal effusivities of the materials, and the duration of contact. To analyze the influence of these factors on thermal recognition, we used a semi-infinite solid based thermal model to simulate heat-transfer data from all the materials in the CES Edupack Level-1 database. We used support-vector machines (SVMs) to predict F1 scores for binary material recognition for 2346 material pairs. We also collected data using a real robot equipped with a thermal sensor and analyzed its material recognition performance on 66 real-world material pairs. Additionally, we analyzed the performance when the models were trained on the simulated data and tested on the real-robot data. Our models predicted the material recognition performance with a 0.980 F1 score for the simulated data, a 0.994 F1 score for real-world data with constant initial sensor temperatures, a 0.966 F1 score for real-world data with varied initial sensor temperatures, and a 0.815 F1 score for sim-to-real transfer. Finally, we present some guidelines on sensor design and parameter choice for thermal recognition based on the insights gained from these results that would hopefully enable robotics researchers to use this less-explored tactile sensing modality more effectively during physical human-robot and robot-object interactions. We release our simulated and real-robot datasets for further use by the robotics community.

cs.RO↗

arXiv2021

Quasi-Dense Similarity Learning for Multiple Object Tracking

Similarity learning has been recognized as a crucial step for object tracking. However, existing multiple object tracking methods only use sparse ground truth matching as the training objective, while ignoring the majority of the informative regions on the images. In this paper, we present Quasi-Dense Similarity Learning, which densely samples hundreds of region proposals on a pair of images for contrastive learning. We can directly combine this similarity learning with existing detection methods to build Quasi-Dense Tracking (QDTrack) without turning to displacement regression or motion priors. We also find that the resulting distinctive feature space admits a simple nearest neighbor search at the inference time. Despite its simplicity, QDTrack outperforms all existing methods on MOT, BDD100K, Waymo, and TAO tracking benchmarks. It achieves 68.7 MOTA at 20.3 FPS on MOT17 without using external training data. Compared to methods with similar detectors, it boosts almost 10 points of MOTA and significantly decreases the number of ID switches on BDD100K and Waymo datasets. Our code and trained models are available at http://vis.xyz/pub/qdtrack.

cs.CV↗

arXiv2021

Home Action Genome: Cooperative Compositional Action Understanding

Existing research on action recognition treats activities as monolithic events occurring in videos. Recently, the benefits of formulating actions as a combination of atomic-actions have shown promise in improving action understanding with the emergence of datasets containing such annotations, allowing us to learn representations capturing this information. However, there remains a lack of studies that extend action composition and leverage multiple viewpoints and multiple modalities of data for representation learning. To promote research in this direction, we introduce Home Action Genome (HOMAGE): a multi-view action dataset with multiple modalities and view-points supplemented with hierarchical activity and atomic action labels together with dense scene composition labels. Leveraging rich multi-modal and multi-view settings, we propose Cooperative Compositional Action Understanding (CCAU), a cooperative learning framework for hierarchical action recognition that is aware of compositional action elements. CCAU shows consistent performance improvements across all modalities. Furthermore, we demonstrate the utility of co-learning compositions in few-shot action recognition by achieving 28.6% mAP with just a single sample.

cs.CV↗

arXiv2020

BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning

Datasets drive vision progress, yet existing driving datasets are impoverished in terms of visual content and supported tasks to study multitask learning for autonomous driving. Researchers are usually constrained to study a small set of problems on one dataset, while real-world computer vision applications require performing tasks of various complexities. We construct BDD100K, the largest driving video dataset with 100K videos and 10 tasks to evaluate the exciting progress of image recognition algorithms on autonomous driving. The dataset possesses geographic, environmental, and weather diversity, which is useful for training models that are less likely to be surprised by new conditions. Based on this diverse dataset, we build a benchmark for heterogeneous multitask learning and study how to solve the tasks together. Our experiments show that special training strategies are needed for existing models to perform such heterogeneous tasks. BDD100K opens the door for future studies in this important venue.

cs.CV↗