Searcharxiv⌕ Search

arXiv subjects

Yulong Li

Publications and source records attributed to Yulong Li.

At least 73 records · Page 4Linked to original sources
arXiv2024

Bimanual Dexterity for Complex Tasks

To train generalist robot policies, machine learning methods often require a substantial amount of expert human teleoperation data. An ideal robot for humans collecting data is one that closely mimics them: bimanual arms and dexterous hands. However, creating such a bimanual teleoperation system with over 50 DoF is a significant challenge. To address this, we introduce Bidex, an extremely dexterous, low-cost, low-latency and portable bimanual dexterous teleoperation system which relies on motion capture gloves and teacher arms. We compare Bidex to a Vision Pro teleoperation system and a SteamVR system and find Bidex to produce better quality data for more complex tasks at a faster rate. Additionally, we show Bidex operating a mobile bimanual robot for in the wild tasks. The robot hands (5k USD) and teleoperation system (7k USD) is readily reproducible and can be used on many robot arms including two xArms (16k USD). Website at https://bidex-teleop.github.io/

cs.RO↗
arXiv2024

RoomDiffusion: A Specialized Diffusion Model in the Interior Design Industry

Recent advancements in text-to-image diffusion models have significantly transformed visual content generation, yet their application in specialized fields such as interior design remains underexplored. In this paper, we present RoomDiffusion, a pioneering diffusion model meticulously tailored for the interior design industry. To begin with, we build from scratch a whole data pipeline to update and evaluate data for iterative model optimization. Subsequently, techniques such as multiaspect training, multi-stage fine-tune and model fusion are applied to enhance both the visual appeal and precision of the generated results. Lastly, leveraging the latent consistency Distillation method, we distill and expedite the model for optimal efficiency. Unlike existing models optimized for general scenarios, RoomDiffusion addresses specific challenges in interior design, such as lack of fashion, high furniture duplication rate, and inaccurate style. Through our holistic human evaluation protocol with more than 20 professional human evaluators, RoomDiffusion demonstrates industry-leading performance in terms of aesthetics, accuracy, and efficiency, surpassing all existing open source models such as stable diffusion and SDXL.

cs.CV↗
arXiv2024

Shared and Private Information Learning in Multimodal Sentiment Analysis with Deep Modal Alignment and Self-supervised Multi-Task Learning

Designing an effective representation learning method for multimodal sentiment analysis tasks is a crucial research direction. The challenge lies in learning both shared and private information in a complete modal representation, which is difficult with uniform multimodal labels and a raw feature fusion approach. In this work, we propose a deep modal shared information learning module based on the covariance matrix to capture the shared information between modalities. Additionally, we use a label generation module based on a self-supervised learning strategy to capture the private information of the modalities. Our module is plug-and-play in multimodal tasks, and by changing the parameterization, it can adjust the information exchange relationship between the modes and learn the private or shared information between the specified modes. We also employ a multi-task learning strategy to help the model focus its attention on the modal differentiation training data. We provide a detailed formulation derivation and feasibility proof for the design of the deep modal shared information learning module. We conduct extensive experiments on three common multimodal sentiment analysis baseline datasets, and the experimental results validate the reliability of our model. Furthermore, we explore more combinatorial techniques for the use of the module. Our approach outperforms current state-of-the-art methods on most of the metrics of the three public datasets.

cs.CL↗
arXiv2023

CornerFormer: Boosting Corner Representation for Fine-Grained Structured Reconstruction

Structured reconstruction is a non-trivial dense prediction problem, which extracts structural information (\eg, building corners and edges) from a raster image, then reconstructs it to a 2D planar graph accordingly. Compared with common segmentation or detection problems, it significantly relays on the capability that leveraging holistic geometric information for structural reasoning. Current transformer-based approaches tackle this challenging problem in a two-stage manner, which detect corners in the first model and classify the proposed edges (corner-pairs) in the second model. However, they separate two-stage into different models and only share the backbone encoder. Unlike the existing modeling strategies, we present an enhanced corner representation method: 1) It fuses knowledge between the corner detection and edge prediction by sharing feature in different granularity; 2) Corner candidates are proposed in four heatmap channels w.r.t its direction. Both qualitative and quantitative evaluations demonstrate that our proposed method can better reconstruct fine-grained structures, such as adjacent corners and tiny edges. Consequently, it outperforms the state-of-the-art model by +1.9\%@F-1 on Corner and +3.0\%@F-1 on Edge.

cs.CV↗
arXiv2023

Teach me with a Whisper: Enhancing Large Language Models for Analyzing Spoken Transcripts using Speech Embeddings

Speech data has rich acoustic and paralinguistic information with important cues for understanding a speaker's tone, emotion, and intent, yet traditional large language models such as BERT do not incorporate this information. There has been an increased interest in multi-modal language models leveraging audio and/or visual information and text. However, current multi-modal language models require both text and audio/visual data streams during inference/test time. In this work, we propose a methodology for training language models leveraging spoken language audio data but without requiring the audio stream during prediction time. This leads to an improved language model for analyzing spoken transcripts while avoiding an audio processing overhead at test time. We achieve this via an audio-language knowledge distillation framework, where we transfer acoustic and paralinguistic information from a pre-trained speech embedding (OpenAI Whisper) teacher model to help train a student language model on an audio-text dataset. In our experiments, the student model achieves consistent improvement over traditional language models on tasks analyzing spoken transcripts.

cs.CL↗
arXiv2023

Rearrangement Planning for General Part Assembly

Most successes in autonomous robotic assembly have been restricted to single target or category. We propose to investigate general part assembly, the task of creating novel target assemblies with unseen part shapes. As a fundamental step to a general part assembly system, we tackle the task of determining the precise poses of the parts in the target assembly, which we we term ``rearrangement planning''. We present General Part Assembly Transformer (GPAT), a transformer-based model architecture that accurately predicts part poses by inferring how each part shape corresponds to the target shape. Our experiments on both 3D CAD models and real-world scans demonstrate GPAT's generalization abilities to novel and diverse target and part shapes.

cs.RO↗
arXiv2023

Semi-Supervised 2D Human Pose Estimation Driven by Position Inconsistency Pseudo Label Correction Module

In this paper, we delve into semi-supervised 2D human pose estimation. The previous method ignored two problems: (i) When conducting interactive training between large model and lightweight model, the pseudo label of lightweight model will be used to guide large models. (ii) The negative impact of noise pseudo labels on training. Moreover, the labels used for 2D human pose estimation are relatively complex: keypoint category and keypoint position. To solve the problems mentioned above, we propose a semi-supervised 2D human pose estimation framework driven by a position inconsistency pseudo label correction module (SSPCM). We introduce an additional auxiliary teacher and use the pseudo labels generated by the two teacher model in different periods to calculate the inconsistency score and remove outliers. Then, the two teacher models are updated through interactive training, and the student model is updated using the pseudo labels generated by two teachers. To further improve the performance of the student model, we use the semi-supervised Cut-Occlude based on pseudo keypoint perception to generate more hard and effective samples. In addition, we also proposed a new indoor overhead fisheye human keypoint dataset WEPDTOF-Pose. Extensive experiments demonstrate that our method outperforms the previous best semi-supervised 2D human pose estimation method. We will release the code and dataset at https://github.com/hlz0606/SSPCM.

cs.CV↗
arXiv2023

PrimeQA: The Prime Repository for State-of-the-Art Multilingual Question Answering Research and Development

The field of Question Answering (QA) has made remarkable progress in recent years, thanks to the advent of large pre-trained language models, newer realistic benchmark datasets with leaderboards, and novel algorithms for key components such as retrievers and readers. In this paper, we introduce PRIMEQA: a one-stop and open-source QA repository with an aim to democratize QA re-search and facilitate easy replication of state-of-the-art (SOTA) QA methods. PRIMEQA supports core QA functionalities like retrieval and reading comprehension as well as auxiliary capabilities such as question generation.It has been designed as an end-to-end toolkit for various use cases: building front-end applications, replicating SOTA methods on pub-lic benchmarks, and expanding pre-existing methods. PRIMEQA is available at : https://github.com/primeqa.

cs.CL↗
arXiv2022

Spectral analysis of a family of nonsymmetric fractional elliptic operators

In this work, we investigate the spectral problem $Au = λu$ where $A$ is a fractional elliptic operator involving left- and right-sided Riemann-Liouville derivatives. These operators are nonlocal and nonsymmetric, however, share certain classic elliptic properties. The eigenvalues correspond to the roots of a class of certain special functions. Compared with classic Sturm-Liouville problems, the most challenging part is to set up the framework for analyzing these nonlocal operators, which requires developing new tools. We prove the existence of the real eigenvalues, find the range for all possible complex eigenvalues, explore the graphs of eigenfunctions, and show numerical findings on the distribution of eigenvalues on the complex plane.

math.AP↗
arXiv2022

Scene Editing as Teleoperation: A Case Study in 6DoF Kit Assembly

Studies in robot teleoperation have been centered around action specifications -- from continuous joint control to discrete end-effector pose control. However, these robot-centric interfaces often require skilled operators with extensive robotics expertise. To make teleoperation accessible to non-expert users, we propose the framework "Scene Editing as Teleoperation" (SEaT), where the key idea is to transform the traditional "robot-centric" interface into a "scene-centric" interface -- instead of controlling the robot, users focus on specifying the task's goal by manipulating digital twins of the real-world objects. As a result, a user can perform teleoperation without any expert knowledge of the robot hardware. To achieve this goal, we utilize a category-agnostic scene-completion algorithm that translates the real-world workspace (with unknown objects) into a manipulable virtual scene representation and an action-snapping algorithm that refines the user input before generating the robot's action plan. To train the algorithms, we procedurally generated a large-scale, diverse kit-assembly dataset that contains object-kit pairs that mimic real-world object-kitting tasks. Our experiments in simulation and on a real-world system demonstrate that our framework improves both the efficiency and success rate for 6DoF kit-assembly tasks. A user study demonstrates that SEaT framework participants achieve a higher task success rate and report a lower subjective workload compared to an alternative robot-centric interface. Video can be found at https://www.youtube.com/watch?v=-NdR3mkPbQQ .

cs.RO↗
arXiv2022

Waviness affects friction and abrasive wear

To prolong lifetime and reduce energy consumption, a thorough understanding of abrasive wear is essential. The potentially crucial influence of surface topography intricacies on tribological behavior have been obscured, since roughness and waviness are often considered simple scalar quantities. Here, the complete waviness profile of the sliding track was used to shed light on the influence of surface topography on abrasive wear. Bearing steel pins and disks were tribologically tested with Al2O3-based slurries as interfacial medium. It was found that even small surface deviations (albeit minimized and controlled for) can significantly increase the friction coefficient - up to 91 %. Not only are frictional fluctuations strongly correlated with the disks' initial waviness profile, these small fluctuations correlate with unevenly-distributed high wear.

cond-mat.mtrl-sci↗
arXiv2022

Learning Cross-Lingual IR from an English Retriever

We present DR.DECR (Dense Retrieval with Distillation-Enhanced Cross-Lingual Representation), a new cross-lingual information retrieval (CLIR) system trained using multi-stage knowledge distillation (KD). The teacher of DR.DECR relies on a highly effective but computationally expensive two-stage inference process consisting of query translation and monolingual IR, while the student, DR.DECR, executes a single CLIR step. We teach DR.DECR powerful multilingual representations as well as CLIR by optimizing two corresponding KD objectives. Learning useful representations of non-English text from an English-only retriever is accomplished through a cross-lingual token alignment algorithm that relies on the representation capabilities of the underlying multilingual encoders. In both in-domain and zero-shot out-of-domain evaluation, DR.DECR demonstrates far superior accuracy over direct fine-tuning with labeled CLIR data. It is also the best single-model retriever on the XOR-TyDi benchmark at the time of this writing.

cs.CL↗
arXiv2022

A Simple Proof of PreciseQMA = PSPACE

We give an alternative proof of PreciseQMA = PSPACE, first proved by Fefferman and Lin (Innov. Theor. Comp. Sci. 2018), where PreciseQMA is the class Quantum Merlin-Arthur with inverse exponential completeness-soundness gap. We adapt the proof of Quantum Cook-Levin Theorem to prove the inclusion PSPACE in PreciseQMA.

quant-ph↗
arXiv2020

Raising the regularity of generalized Abel equations in fractional Sobolev spaces with homogeneous boundary conditions

The generalized (or coupled) Abel equations on the bounded interval have been well investigated in H$\ddot{\text{o}}$lderian spaces that admit integrable singularities at the endpoints and relatively inadequate in other functional spaces. In recent years, such operators have appeared in BVPs of fractional-order differential equations such as fractional diffusion equations that are usually studied in the frame of fractional Sobolev spaces for weak solution and numerical approximation; and their analysis plays the key role during the process of converting weak solutions to the true solutions. This article develops the mapping properties of generalized Abel operators $α{_aD_x^{-s}}+β{_xD_b^{-s}}$ in fractional Sobolev spaces, where $0<α,β$, $α+β=1$, $ 0<s<1$ and $ {_aD_x^{-s}}$, $ {_xD_b^{-s}}$ are fractional Riemann-Liouville integrals. It is mainly concerned with the regularity property of $(α{_aD_x^{-s}}+β{_xD_b^{-s}})u=f$ by taking into account homogeneous boundary conditions. Namely, we investigate the regularity behavior of $u(x)$ while letting $f(x)$ become smoother and imposing homogeneous boundary restrictions $u(a)=u(b)=0$.

math.CA↗
arXiv2020

On the Skewed Fractional Diffusion Advection Reaction Equation on the Interval

This article provides techniques of raising the regularity of fractional order equations and resolves fundamental questions on the one-dimensional homogeneous boundary-value problem of skewed (double-sided) fractional diffusion advection reaction equation (FDARE) with variable coefficients on the bounded interval. The existence of the true (classical) solution together with norm estimation is established and the precise regularity bound is found; also, the structure of the solution is unraveled, capturing the essence of regularity, singularity, and other features of the solution. The key analysis lies in exploring the properties of Gauss hypergeometric functions, solving coupled Abel integral equations and dominant singular integral equations, and connecting the functions from fractional Sobolev spaces to the ones from H$\ddot{\text{o}}$lderian spaces that admit integrable singularities at the endpoints.

math.CA↗
arXiv2019

Investigation of the force closure for Eulerian-Eulerian simulations: a validation study of nine gas-liquid flow cases

Gas-liquid flows can be simulated by the Eulerian-Eulerian (E-E) method. Whether to include a specific momentum interfacial exchange force model remains as a question with no answer. In this work, our aim is to seek a general numerical settings for the E-E method, which can provide competent results for industrial bubbly flows with different geometries under different operations. They were selected from different industries including chemical, nuclear, bio-processing and metallurgical engineering. Simulations were launched by the OpenFOAM solver reactingTwoPhaseEulerFoam, in which the E-E method was implemented with sophisticated numerical techniques to ensure stabilities. Predictions were compared against experimental data. It was found that the drag force and turbulent dispersion force play the most important role on the predictions and should be included for all simulations. The first one accounts for the two-way coupling while the second one accounts for the turbulence effect and ensures the E-E equations to be well-posed. The lift force and wall lubrication force should be included to address the phase fraction accumulation in the vicinity of the wall, especially for pipe flows with large aspect ratio. In other cases the lateral forces can be safely neglected. All the test case are open-sourced and are available as supplementary data for anyone to download as baseline test cases.

physics.flu-dyn↗
arXiv2019

Fragmentation of shells: An analogy with the crack formation in tree bark

How does a shell explode into a series of fragments upon impact? The well accepted explanation is Mott's theory, which considers the fragmentation of shells as a random process controlled by defects. However, Mott's theory is inadequate due to its assumption of energy conversion, and it is incapable of explaining the lack of change in saturation fragment length with the increase in expansion velocity. In this paper, we present a theory to explain the physical mechanism for fragmentation of shells and propose a highly efficient model for predicting the number of necks after fragmentation. We recognize that the fragmentation problem in shells is analogous to the cracking behavior of tree bark, and closed form solutions is obtained to describe the relationship between the expansion velocity and the number of necks with consideration of the strain rate dependent strength of the shell material. The theoretical results show excellent correlation with the experimental results.

cond-mat.soft↗