arXiv subjects

Jin Pan

Publications and source records attributed to Jin Pan.

24 records · Page 2Linked to original sources

arXiv2024

VRCopilot: Authoring 3D Layouts with Generative AI Models in VR

Immersive authoring provides an intuitive medium for users to create 3D scenes via direct manipulation in Virtual Reality (VR). Recent advances in generative AI have enabled the automatic creation of realistic 3D layouts. However, it is unclear how capabilities of generative AI can be used in immersive authoring to support fluid interactions, user agency, and creativity. We introduce VRCopilot, a mixed-initiative system that integrates pre-trained generative AI models into immersive authoring to facilitate human-AI co-creation in VR. VRCopilot presents multimodal interactions to support rapid prototyping and iterations with AI, and intermediate representations such as wireframes to augment user controllability over the created content. Through a series of user studies, we evaluated the potential and challenges in manual, scaffolded, and automatic creation in immersive authoring. We found that scaffolded creation using wireframes enhanced the user agency compared to automatic creation. We also found that manual creation via multimodal specification offers the highest sense of creativity and agency.

cs.HC↗

arXiv2024

Computation of Characteristic Mode for Regional Structure of Interconnected Metal Bodies

Existing methods for calculating substructure characteristic modes require treating interconnected metal structures as a single entity to ensure current continuity between different metal bodies. However, when these structures are treated as separate entities, existing methods exhibit inaccuracies, affecting the assessment of structural performance. To address this challenge, we propose an enhanced electromagnetic model that enables accurate characteristic mode analysis for regional structures within interconnected metal bodies. Numerical results validate the accuracy of the proposed method, and an antenna design example demonstrates its practical utility.

physics.class-ph↗

arXiv2024

BLOS-BEV: Navigation Map Enhanced Lane Segmentation Network, Beyond Line of Sight

Bird's-eye-view (BEV) representation is crucial for the perception function in autonomous driving tasks. It is difficult to balance the accuracy, efficiency and range of BEV representation. The existing works are restricted to a limited perception range within 50 meters. Extending the BEV representation range can greatly benefit downstream tasks such as topology reasoning, scene understanding, and planning by offering more comprehensive information and reaction time. The Standard-Definition (SD) navigation maps can provide a lightweight representation of road structure topology, characterized by ease of acquisition and low maintenance costs. An intuitive idea is to combine the close-range visual information from onboard cameras with the beyond line-of-sight (BLOS) environmental priors from SD maps to realize expanded perceptual capabilities. In this paper, we propose BLOS-BEV, a novel BEV segmentation model that incorporates SD maps for accurate beyond line-of-sight perception, up to 200m. Our approach is applicable to common BEV architectures and can achieve excellent results by incorporating information derived from SD maps. We explore various feature fusion schemes to effectively integrate the visual BEV representations and semantic features from the SD map, aiming to leverage the complementary information from both sources optimally. Extensive experiments demonstrate that our approach achieves state-of-the-art performance in BEV segmentation on nuScenes and Argoverse benchmark. Through multi-modal inputs, BEV segmentation is significantly enhanced at close ranges below 50m, while also demonstrating superior performance in long-range scenarios, surpassing other methods by over 20% mIoU at distances ranging from 50-200m.

cs.CV↗

arXiv2021

A Large-Scale Dataset for Benchmarking Elevator Button Segmentation and Character Recognition

Human activities are hugely restricted by COVID-19, recently. Robots that can conduct inter-floor navigation attract much public attention, since they can substitute human workers to conduct the service work. However, current robots either depend on human assistance or elevator retrofitting, and fully autonomous inter-floor navigation is still not available. As the very first step of inter-floor navigation, elevator button segmentation and recognition hold an important position. Therefore, we release the first large-scale publicly available elevator panel dataset in this work, containing 3,718 panel images with 35,100 button labels, to facilitate more powerful algorithms on autonomous elevator operation. Together with the dataset, a number of deep learning based implementations for button segmentation and recognition are also released to benchmark future methods in the community. The dataset will be available at \url{https://github.com/zhudelong/elevator_button_recognition

cs.CV↗

arXiv2020

Multibeam Optimization for Joint Communication and Radio Sensing Using Analog Antenna Arrays

Multibeam technology enables the use of two or more subbeams for joint communication and radio sensing, to meet different requirements of beamwidth and pointing directions. Generating and optimizing multibeam subject to the requirements is critical and challenging, particularly for systems using analog arrays. This paper develops optimal solutions to a range of multibeam design problems, where both communication and sensing are considered. We first study the optimal combination of two pre-generated subbeams, and their beamforming vectors, using a combining phase coefficient. Closed-form optimal solutions are derived to the constrained optimization problems, where the received signal powers for communication and the beamforming waveforms are alternatively used as the objective and constraint functions. We also develop global optimization methods which directly find optimal solutions for a single beamforming vector. By converting the original intractable complex NP-hard global optimization problems to real quadratically constrained quadratic programs, near-optimal solutions are obtained using semidefinite relaxation techniques. Extensive simulations validate the effectiveness of the proposed constrained multibeam generation and optimization methods.

eess.SP↗

arXiv2018

Learning to Interrupt: A Hierarchical Deep Reinforcement Learning Framework for Efficient Exploration

To achieve scenario intelligence, humans must transfer knowledge to robots by developing goal-oriented algorithms, which are sometimes insensitive to dynamically changing environments. While deep reinforcement learning achieves significant success recently, it is still extremely difficult to be deployed in real robots directly. In this paper, we propose a hybrid structure named Option-Interruption in which human knowledge is embedded into a hierarchical reinforcement learning framework. Our architecture has two key components: options, represented by existing human-designed methods, can significantly speed up the training process and interruption mechanism, based on learnable termination functions, enables our system to quickly respond to the external environment. To implement this architecture, we derive a set of update rules based on policy gradient methods and present a complete training process. In the experiment part, our method is evaluated in Four-room navigation and exploration task, which shows the efficiency and flexibility of our framework.

cs.AI↗