arXiv subjects

Fei Liu

Publications and source records attributed to Fei Liu.

At least 217 records · Page 12Linked to original sources

arXiv2022

Multi-task Learning for Monocular Depth and Defocus Estimations with Real Images

Monocular depth estimation and defocus estimation are two fundamental tasks in computer vision. Most existing methods treat depth estimation and defocus estimation as two separate tasks, ignoring the strong connection between them. In this work, we propose a multi-task learning network consisting of an encoder with two decoders to estimate the depth and defocus map from a single focused image. Through the multi-task network, the depth estimation facilitates the defocus estimation to get better results in the weak texture region and the defocus estimation facilitates the depth estimation by the strong physical connection between the two maps. We set up a dataset (named ALL-in-3D dataset) which is the first all-real image dataset consisting of 100K sets of all-in-focus images, focused images with focus depth, depth maps, and defocus maps. It enables the network to learn features and solid physical connections between the depth and real defocus images. Experiments demonstrate that the network learns more solid features from the real focused images than the synthetic focused images. Benefiting from this multi-task structure where different tasks facilitate each other, our depth and defocus estimations achieve significantly better performance than other state-of-art algorithms. The code and dataset will be publicly available at https://github.com/cubhe/MDDNet.

cs.CV↗

arXiv2022

Deriving a kinetic uncertainty relation for piecewise deterministic processes: from classical to quantum

From the perspective of Markovian piecewise deterministic processes (PDPs), we investigate the derivation of a kinetic uncertainty relation (KUR), which was originally proposed in Markovian open quantum systems. First, stationary distributions of classical PDPs are explicitly constructed. Then, a tilting method is used to derive a rate functional of large deviations. Finally, based on an improved approximation scheme, we recover the KUR. These classical results are directly extended to the open quantum systems. We use a driven two-level quantum system to exemplify the quantum results.

cond-mat.stat-mech↗

arXiv2022

Lidar Point Cloud Guided Monocular 3D Object Detection

Monocular 3D object detection is a challenging task in the self-driving and computer vision community. As a common practice, most previous works use manually annotated 3D box labels, where the annotating process is expensive. In this paper, we find that the precisely and carefully annotated labels may be unnecessary in monocular 3D detection, which is an interesting and counterintuitive finding. Using rough labels that are randomly disturbed, the detector can achieve very close accuracy compared to the one using the ground-truth labels. We delve into this underlying mechanism and then empirically find that: concerning the label accuracy, the 3D location part in the label is preferred compared to other parts of labels. Motivated by the conclusions above and considering the precise LiDAR 3D measurement, we propose a simple and effective framework, dubbed LiDAR point cloud guided monocular 3D object detection (LPCG). This framework is capable of either reducing the annotation costs or considerably boosting the detection accuracy without introducing extra annotation costs. Specifically, It generates pseudo labels from unlabeled LiDAR point clouds. Thanks to accurate LiDAR 3D measurements in 3D space, such pseudo labels can replace manually annotated labels in the training of monocular 3D detectors, since their 3D location information is precise. LPCG can be applied into any monocular 3D detector to fully use massive unlabeled data in a self-driving system. As a result, in KITTI benchmark, we take the first place on both monocular 3D and BEV (bird's-eye-view) detection with a significant margin. In Waymo benchmark, our method using 10% labeled data achieves comparable accuracy to the baseline detector using 100% labeled data. The codes are released at https://github.com/SPengLiang/LPCG.

cs.CV↗

arXiv2022

Learning as Conversation: Dialogue Systems Reinforced for Information Acquisition

We propose novel AI-empowered chat bots for learning as conversation where a user does not read a passage but gains information and knowledge through conversation with a teacher bot. Our information-acquisition-oriented dialogue system employs a novel adaptation of reinforced self-play so that the system can be transferred to various domains without in-domain dialogue data, and can carry out conversations both informative and attentive to users. Our extensive subjective and objective evaluations on three large public data corpora demonstrate the effectiveness of our system to deliver knowledge-intensive and attentive conversations and help end users substantially gain knowledge without reading passages. Our code and datasets are publicly available for follow-up research.

cs.CL↗

arXiv2022

Recurrent Affine Transformation for Text-to-image Synthesis

Text-to-image synthesis aims to generate natural images conditioned on text descriptions. The main difficulty of this task lies in effectively fusing text information into the image synthesis process. Existing methods usually adaptively fuse suitable text information into the synthesis process with multiple isolated fusion blocks (e.g., Conditional Batch Normalization and Instance Normalization). However, isolated fusion blocks not only conflict with each other but also increase the difficulty of training (see first page of the supplementary). To address these issues, we propose a Recurrent Affine Transformation (RAT) for Generative Adversarial Networks that connects all the fusion blocks with a recurrent neural network to model their long-term dependency. Besides, to improve semantic consistency between texts and synthesized images, we incorporate a spatial attention model in the discriminator. Being aware of matching image regions, text descriptions supervise the generator to synthesize more relevant image contents. Extensive experiments on the CUB, Oxford-102 and COCO datasets demonstrate the superiority of the proposed model in comparison to state-of-the-art models \footnote{https://github.com/senmaoy/Recurrent-Affine-Transformation-for-Text-to-image-Synthesis.git}

cs.CV↗

arXiv2022

Electron transport in the single-layer semiconductor

Two-dimensional (2D) materials are a new class of materials with interesting physical properties and applications ranging from nanoelectronics to sensing and photonics. In addition to graphene, the most studied 2D material, monolayers of other layered materials such as semiconducting dichalcogenides MoS2 or WSe2 are gaining in importance as promising channel materials for field-effect transistors (FETs) and phototransistors. However, it is unclear that how the specific process of electron transport is affected by temperature. So, nowadays the electron dynamics of single-layer semiconductor cannot be understood fundamentally. Here, we develop an analytical theory distinguishing from traditional energy band theory, backed up by Monte-Carlo simulations, that predicts the process of electron transport and the effect of temperature on the electron transport in the single-layer semiconductor. In this paper, A new model is built to deal with electron transporting in the sing-layer semiconductor. The resistance is decided by the barrier rather than the electron scattering in the single-layer semiconductor, which is macroscopic quantum effect. Electron transport in FETs with different dielectric configurations are investigated at different temperatures and a new control factor that is decided by top-gate voltage or bottom-gate voltage is introduced to describe the effect of gate voltage on the electron transport in 2D semiconductor. The results of simulation show the drain current is mainly determined by some elements, such as temperature, top-gate voltage, bottom-gate voltage and source-drain voltage.

cond-mat.mtrl-sci↗

arXiv2022

Charge transport in monolayers of metal nanoparticles

Two-dimensional (2D) nanoparticle films are a new class of materials with interesting physical properties and applications ranging from nanoelectronics to sensing and photonics. The importance of conducting nanoparticle films makes the fundamental understanding of their charge transport extremely important for materials and process design. Various hopping and transport mechanisms have been proposed and the nanoparticle monolayer is consistent with the electrical equivalent RC circuit, but their theoretical methods are limited to the model of the single electron tunneling between capacitively coupled nanoparticles with a characteristic time constant RC and the conductivity of thin film is the experimental conductivity, which cannot be deduced from these theoretical models. It is also unclear that how the specific process of electron transpot is affected by temperature. So, nowadays the electron dynamics of thin film cannot be understood fundamentally. Here, we develop an analytical theory based on the model of Sommerfeld, backed up by Monte-Carlo simulations, that predicts the process of charge transport and the effect of temperature on the electron transport in the thin film. In this paper two different nanoparticle models were built to cope with different types of morphology: triangular array and rectangular array. The transport properties of these different kinds of arrays including 2D ordered nanoparticle arrays with/without local structural disorder and 2D gradient nanoparticle arrays were investigated at different temperatures. For 2D well-ordered nanoparticle array without local structural disorder, the I-V curves are non-linear and highly symmetric.

cond-mat.mtrl-sci↗

arXiv2022

Towards Abstractive Grounded Summarization of Podcast Transcripts

Podcasts have recently shown a rapid rise in popularity. Summarization of podcast transcripts is of practical benefit to both content providers and consumers. It helps consumers to quickly decide whether they will listen to the podcasts and reduces the cognitive load of content providers to write summaries. Nevertheless, podcast summarization faces significant challenges including factual inconsistencies with respect to the inputs. The problem is exacerbated by speech disfluencies and recognition errors in transcripts of spoken language. In this paper, we explore a novel abstractive summarization method to alleviate these challenges. Specifically, our approach learns to produce an abstractive summary while grounding summary segments in specific portions of the transcript to allow for full inspection of summary details. We conduct a series of analyses of the proposed approach on a large podcast dataset and show that the approach can achieve promising results. Grounded summaries bring clear benefits in locating the summary and transcript segments that contain inconsistent information, and hence significantly improve summarization quality in both automatic and human evaluation metrics.

cs.CL↗

arXiv2022

Differentiable Robotic Manipulation of Deformable Rope-like Objects Using Compliant Position-based Dynamics

Robot manipulation of rope-like objects is an interesting problem that has some critical applications, such as autonomous robotic suturing. Solving for and controlling rope is difficult due to the complexity of rope physics and the challenge of building fast and accurate models of deformable materials. While more data-driven approaches have become more popular for finding controllers that learn to do a single task, there is still a strong motivation for a model-based method that could be used to solve a large variety of optimization problems. Towards this end, we introduced compliant, position-based dynamics (XPBD) to model rope-like objects. Using geometric constraints, the model can represent the coupling of shear/stretch and bend/twist effects. Of crucial importance is that our formulation is differentiable, which can solve parameter estimation problems and improve the matching of rope physics to real-life scenarios (i.e., the real-to-sim problem). For the generality of rope-like objects, two different solvers are proposed to handle the inextensible and extensible effects of varied material stiffness for the rope. We demonstrate our framework's robustness and accuracy on real-to-sim experimental setups using the Baxter robot and the da Vinci research kit (DVRK). Our work leads to a new path for robotic manipulation of the deformable rope-like object taking advantage of the ready-to-use gradients.

cs.RO↗

arXiv2022

Parameter Identification and Motion Control for Articulated Rigid Body Robots Using Differentiable Position-based Dynamics

Simulation modeling of robots, objects, and environments is the backbone for all model-based control and learning. It is leveraged broadly across dynamic programming and model-predictive control, as well as data generation for imitation, transfer, and reinforcement learning. In addition to fidelity, key features of models in these control and learning contexts are speed, stability, and native differentiability. However, many popular simulation platforms for robotics today lack at least one of the features above. More recently, position-based dynamics (PBD) has become a very popular simulation tool for modeling complex scenes of rigid and non-rigid object interactions, due to its speed and stability, and is starting to gain significant interest in robotics for its potential use in model-based control and learning. Thus, in this paper, we present a mathematical formulation for coupling position-based dynamics (PBD) simulation and optimal robot design, model-based motion control and system identification. Our framework breaks down PBD definitions and derivations for various types of joint-based articulated rigid bodies. We present a back-propagation method with automatic differentiation, which can integrate both positional and angular geometric constraints. Our framework can critically provide the native gradient information and perform gradient-based optimization tasks. We also propose articulated joint model representations and simulation workflow for our differentiable framework. We demonstrate the capability of the framework in efficient optimal robot design, accurate trajectory torque estimation and supporting spring stiffness estimation, where we achieve minor errors. We also implement impedance control in real robots to demonstrate the potential of our differentiable framework in human-in-the-loop applications.

cs.RO↗

arXiv2021

Direct evidence for intermediate multiferroic phase in LiCuFe2(VO4)3

Magnetic susceptibility, specific heat, dielectric, and electric polarization of LiCuFe2(VO4)3 have been investigated. Two sequential antiferromagnetic transitions at TN1 ~ 9.95 K and TN2 ~ 8.17 K are observed under zero magnetic field. While a dielectric peak at TN1 is clearly identified, the measured pyroelectric current also exhibits a sharp peak at TN1, implying the magnetically relevant ferroelectricity. Interestingly, another pyroelectric peak around TN2 with opposite signal is observed, resulting in the disappearance of electric polarization below TN2. Besides, the electric polarization is significantly suppressed in response to external magnetic field, evidencing remarkable magnetoelectric effect. These results suggest the essential relevance of the magnetic structure with the ferroelectricity in LiCuFe2(VO4)3, deserving for further investigation of the underlying mechanism.

cond-mat.mtrl-sci↗

arXiv2021

Dirac-Source Diode with Sub-unity Ideality Factor

An increase in power consumption necessitates a low-power circuit technology to extend Moore's law. Low-power transistors, such as tunnel field-effect transistors (TFETs), negative-capacitance field-effect transistors (NC-FETs), and Dirac-source field-effect transistors (DS-FETs), have been realised to break the thermionic limit of the subthreshold swing (SS). However, a low-power diode rectifier, which breaks the thermionic limit of an ideality factor (n) of 1 at room temperature, has not been proposed yet. In this study, we have realised a DS Schottky diode, which exhibits a steep-slope characteristic curve, by utilising the linear density of states (DOSs) of graphene. For the developed DS Schottky diode, n<1 for more than two decades of drain current with a minimum value of 0.8, and the rectifying ratio is large (100000). The realisation of a DS Schottky diode paves the way for the development of low-power electronic circuits.

cond-mat.mes-hall↗

arXiv2021

Binary Response Models for Heterogeneous Panel Data with Interactive Fixed Effects

In this paper, we investigate binary response models for heterogeneous panel data with interactive fixed effects by allowing both the cross-sectional dimension and the temporal dimension to diverge. From a practical point of view, the proposed framework can be applied to predict the probability of corporate failure, conduct credit rating analysis, etc. Theoretically and methodologically, we establish a link between a maximum likelihood estimation and a least squares approach, provide a simple information criterion to detect the number of factors, and achieve the asymptotic distributions accordingly. In addition, we conduct intensive simulations to examine the theoretical findings. In the empirical study, we focus on the sign prediction of stock returns, and then use the results of sign forecast to conduct portfolio analysis.

econ.EM↗

arXiv2021

Incommensurate-commensurate magnetic phase transition in the double tungstate Li2Co(WO4)2

Magnetic susceptibility, specific heat, and neutron powder diffraction measurements have been performed on polycrystalline Li2Co(WO4)2 samples. Under zero magnetic field, two successive magnetic transitions at TN1 ~ 9.4 K and TN2 ~ 7.4 K are observed. The magnetic ordering temperatures gradually decrease as the magnetic field increases. Neutron diffraction reveals that Li2Co(WO4)2 enters an incommensurate magnetic state with a temperature dependent k between TN1 and TN2. The magnetic propagation vector locks-in to a commensurate value k = (1/2, 1/4, 1/4) below TN2. The antiferromagnetic structure is refined at 1.7 K with Co2+ magnetic moment 2.8(1) uB, consistent with our first-principles calculations.

cond-mat.mtrl-sci↗

arXiv2021

Modeling Endorsement for Multi-Document Abstractive Summarization

A crucial difference between single- and multi-document summarization is how salient content manifests itself in the document(s). While such content may appear at the beginning of a single document, essential information is frequently reiterated in a set of documents related to a particular topic, resulting in an endorsement effect that increases information salience. In this paper, we model the cross-document endorsement effect and its utilization in multiple document summarization. Our method generates a synopsis from each document, which serves as an endorser to identify salient content from other documents. Strongly endorsed text segments are used to enrich a neural encoder-decoder model to consolidate them into an abstractive summary. The method has a great potential to learn from fewer examples to identify salient content, which alleviates the need for costly retraining when the set of documents is dynamically adjusted. Through extensive experiments on benchmark multi-document summarization datasets, we demonstrate the effectiveness of our proposed method over strong published baselines. Finally, we shed light on future research directions and discuss broader challenges of this task using a case study.

cs.CL↗

arXiv2021

StreamHover: Livestream Transcript Summarization and Annotation

With the explosive growth of livestream broadcasting, there is an urgent need for new summarization technology that enables us to create a preview of streamed content and tap into this wealth of knowledge. However, the problem is nontrivial due to the informal nature of spoken language. Further, there has been a shortage of annotated datasets that are necessary for transcript summarization. In this paper, we present StreamHover, a framework for annotating and summarizing livestream transcripts. With a total of over 500 hours of videos annotated with both extractive and abstractive summaries, our benchmark dataset is significantly larger than currently existing annotated corpora. We explore a neural extractive summarization model that leverages vector-quantized variational autoencoder to learn latent vector representations of spoken utterances and identify salient utterances from the transcripts to form summaries. We show that our model generalizes better and improves performance over strong baselines. The results of this study provide an avenue for future research to improve summarization solutions for efficient browsing of livestreams.

cs.CL↗

arXiv2021

Some Dynamical Properties on Manifolds with no Conjugate Points

In this article, we study the dynamics of geodesic flows on Riemannian (not necessarily compact) manifolds with no conjugate points. We prove the Anosov Closing Lemma, the local product structure, and the transitivity of the geodesic flows on $Ω_1$ under the conditions of bounded asymptote and uniform visibility. As an application, we further discuss about some generic properties of the set of invariant probability measures

math.DS↗

arXiv2021

OPT: Omni-Perception Pre-Trainer for Cross-Modal Understanding and Generation

In this paper, we propose an Omni-perception Pre-Trainer (OPT) for cross-modal understanding and generation, by jointly modeling visual, text and audio resources. OPT is constructed in an encoder-decoder framework, including three single-modal encoders to generate token-based embeddings for each modality, a cross-modal encoder to encode the correlations among the three modalities, and two cross-modal decoders to generate text and image respectively. For the OPT's pre-training, we design a multi-task pretext learning scheme to model multi-modal resources from three different data granularities, \ie, token-, modality-, and sample-level modeling, through which OPT learns to align and translate among different modalities. The pre-training task is carried out on a large amount of image-text-audio triplets from Open Images. Experimental results show that OPT can learn strong image-text-audio multi-modal representations and achieve promising results on a variety of cross-modal understanding and generation tasks.

cs.CV↗