arXiv subjects

Mingxiao Li

Publications and source records attributed to Mingxiao Li.

At least 55 records · Page 3Linked to original sources

arXiv2024

DMON: A Simple yet Effective Approach for Argument Structure Learning

Argument structure learning~(ASL) entails predicting relations between arguments. Because it can structure a document to facilitate its understanding, it has been widely applied in many fields~(medical, commercial, and scientific domains). Despite its broad utilization, ASL remains a challenging task because it involves examining the complex relationships between the sentences in a potentially unstructured discourse. To resolve this problem, we have developed a simple yet effective approach called Dual-tower Multi-scale cOnvolution neural Network~(DMON) for the ASL task. Specifically, we organize arguments into a relationship matrix that together with the argument embeddings forms a relationship tensor and design a mechanism to capture relations with contextual arguments. Experimental results on three different-domain argument mining datasets demonstrate that our framework outperforms state-of-the-art models. The code is available at https://github.com/VRCMF/DMON.git .

cs.CL↗

arXiv2024

Elucidating the Exposure Bias in Diffusion Models

Diffusion models have demonstrated impressive generative capabilities, but their \textit{exposure bias} problem, described as the input mismatch between training and sampling, lacks in-depth exploration. In this paper, we systematically investigate the exposure bias problem in diffusion models by first analytically modelling the sampling distribution, based on which we then attribute the prediction error at each sampling step as the root cause of the exposure bias issue. Furthermore, we discuss potential solutions to this issue and propose an intuitive metric for it. Along with the elucidation of exposure bias, we propose a simple, yet effective, training-free method called Epsilon Scaling to alleviate the exposure bias. We show that Epsilon Scaling explicitly moves the sampling trajectory closer to the vector field learned in the training phase by scaling down the network output, mitigating the input mismatch between training and sampling. Experiments on various diffusion frameworks (ADM, DDIM, EDM, LDM, DiT, PFGM++) verify the effectiveness of our method. Remarkably, our ADM-ES, as a state-of-the-art stochastic sampler, obtains 2.17 FID on CIFAR-10 under 100-step unconditional generation. The code is available at \url{https://github.com/forever208/ADM-ES} and \url{https://github.com/forever208/EDM-ES}.

cs.LG↗

arXiv2023

Contrast, Attend and Diffuse to Decode High-Resolution Images from Brain Activities

Decoding visual stimuli from neural responses recorded by functional Magnetic Resonance Imaging (fMRI) presents an intriguing intersection between cognitive neuroscience and machine learning, promising advancements in understanding human visual perception and building non-invasive brain-machine interfaces. However, the task is challenging due to the noisy nature of fMRI signals and the intricate pattern of brain visual representations. To mitigate these challenges, we introduce a two-phase fMRI representation learning framework. The first phase pre-trains an fMRI feature learner with a proposed Double-contrastive Mask Auto-encoder to learn denoised representations. The second phase tunes the feature learner to attend to neural activation patterns most informative for visual reconstruction with guidance from an image auto-encoder. The optimized fMRI feature learner then conditions a latent diffusion model to reconstruct image stimuli from brain activities. Experimental results demonstrate our model's superiority in generating high-resolution and semantically accurate images, substantially exceeding previous state-of-the-art methods by 39.34% in the 50-way-top-1 semantic classification accuracy. Our research invites further exploration of the decoding task's potential and contributes to the development of non-invasive brain-machine interfaces.

cs.CV↗

arXiv2023

Electrically empowered microcomb laser

Optical frequency comb underpins a wide range of applications from communication, metrology, to sensing. Its development on a chip-scale platform -- so called soliton microcomb -- provides a promising path towards system miniaturization and functionality integration via photonic integrated circuit (PIC) technology. Although extensively explored in recent years, challenges remain in key aspects of microcomb such as complex soliton initialization, high threshold, low power efficiency, and limited comb reconfigurability. Here we present an on-chip laser that directly outputs microcomb and resolves all these challenges, with a distinctive mechanism created from synergetic interaction among resonant electro-optic effect, optical Kerr effect, and optical gain inside the laser cavity. Realized with integration between a III-V gain chip and a thin-film lithium niobate (TFLN) PIC, the laser is able to directly emit mode-locked microcomb on demand with robust turnkey operation inherently built in, with individual comb linewidth down to 600 Hz, whole-comb frequency tuning rate exceeding $\rm 2.4\times10^{17}$ Hz/s, and 100% utilization of optical power fully contributing to comb generation. The demonstrated approach unifies architecture and operation simplicity, high-speed reconfigurability, and multifunctional capability enabled by TFLN PIC, opening up a great avenue towards on-demand generation of mode-locked microcomb that is expected to have profound impact on broad applications.

physics.optics↗

arXiv2023

Generating Explanations in Medical Question-Answering by Expectation Maximization Inference over Evidence

Medical Question Answering~(medical QA) systems play an essential role in assisting healthcare workers in finding answers to their questions. However, it is not sufficient to merely provide answers by medical QA systems because users might want explanations, that is, more analytic statements in natural language that describe the elements and context that support the answer. To do so, we propose a novel approach for generating natural language explanations for answers predicted by medical QA systems. As high-quality medical explanations require additional medical knowledge, so that our system extract knowledge from medical textbooks to enhance the quality of explanations during the explanation generation process. Concretely, we designed an expectation-maximization approach that makes inferences about the evidence found in these texts, offering an efficient way to focus attention on lengthy evidence passages. Experimental results, conducted on two datasets MQAE-diag and MQAE, demonstrate the effectiveness of our framework for reasoning with textual evidence. Our approach outperforms state-of-the-art models, achieving a significant improvement of \textbf{6.86} and \textbf{9.43} percentage points on the Rouge-1 score; \textbf{8.23} and \textbf{7.82} percentage points on the Bleu-4 score on the respective datasets.

cs.CL↗

arXiv2023

Decoding Realistic Images from Brain Activity with Contrastive Self-supervision and Latent Diffusion

Reconstructing visual stimuli from human brain activities provides a promising opportunity to advance our understanding of the brain's visual system and its connection with computer vision models. Although deep generative models have been employed for this task, the challenge of generating high-quality images with accurate semantics persists due to the intricate underlying representations of brain signals and the limited availability of parallel data. In this paper, we propose a two-phase framework named Contrast and Diffuse (CnD) to decode realistic images from functional magnetic resonance imaging (fMRI) recordings. In the first phase, we acquire representations of fMRI data through self-supervised contrastive learning. In the second phase, the encoded fMRI representations condition the diffusion model to reconstruct visual stimulus through our proposed concept-aware conditioning method. Experimental results show that CnD reconstructs highly plausible images on challenging benchmarks. We also provide a quantitative interpretation of the connection between the latent diffusion model (LDM) components and the human brain's visual system. In summary, we present an effective approach for reconstructing visual stimuli based on human brain activity and offer a novel framework to understand the relationship between the diffusion model and the human brain visual system.

cs.CV↗

arXiv2023

Crossword: A Semantic Approach to Data Compression via Masking

The traditional methods for data compression are typically based on the symbol-level statistics, with the information source modeled as a long sequence of i.i.d. random variables or a stochastic process, thus establishing the fundamental limit as entropy for lossless compression and as mutual information for lossy compression. However, the source (including text, music, and speech) in the real world is often statistically ill-defined because of its close connection to human perception, and thus the model-driven approach can be quite suboptimal. This study places careful emphasis on English text and exploits its semantic aspect to enhance the compression efficiency further. The main idea stems from the puzzle crossword, observing that the hidden words can still be precisely reconstructed so long as some key letters are provided. The proposed masking-based strategy resembles the above game. In a nutshell, the encoder evaluates the semantic importance of each word according to the semantic loss and then masks the minor ones, while the decoder aims to recover the masked words from the semantic context by means of the Transformer. Our experiments show that the proposed semantic approach can achieve much higher compression efficiency than the traditional methods such as Huffman code and UTF-8 code, while preserving the meaning in the target text to a great extent.

cs.CL↗

arXiv2022

Layout-aware Dreamer for Embodied Referring Expression Grounding

In this work, we study the problem of Embodied Referring Expression Grounding, where an agent needs to navigate in a previously unseen environment and localize a remote object described by a concise high-level natural language instruction. When facing such a situation, a human tends to imagine what the destination may look like and to explore the environment based on prior knowledge of the environmental layout, such as the fact that a bathroom is more likely to be found near a bedroom than a kitchen. We have designed an autonomous agent called Layout-aware Dreamer (LAD), including two novel modules, that is, the Layout Learner and the Goal Dreamer to mimic this cognitive decision process. The Layout Learner learns to infer the room category distribution of neighboring unexplored areas along the path for coarse layout estimation, which effectively introduces layout common sense of room-to-room transitions to our agent. To learn an effective exploration of the environment, the Goal Dreamer imagines the destination beforehand. Our agent achieves new state-of-the-art performance on the public leaderboard of the REVERIE dataset in challenging unseen test environments with improvement in navigation success (SR) by 4.02% and remote grounding success (RGS) by 3.43% compared to the previous state-of-the-art. The code is released at https://github.com/zehao-wang/LAD

cs.CV↗

arXiv2022

Find a Way Forward: a Language-Guided Semantic Map Navigator

In this paper, we introduce the map-language navigation task where an agent executes natural language instructions and moves to the target position based only on a given 3D semantic map. To tackle the task, we design the instruction-aware Path Proposal and Discrimination model (iPPD). Our approach leverages map information to provide instruction-aware path proposals, i.e., it selects all potential instruction-aligned candidate paths to reduce the solution space. Next, to represent the map observations along a path for a better modality alignment, a novel Path Feature Encoding scheme tailored for semantic maps is proposed. An attention-based Language Driven Discriminator is designed to evaluate path candidates and determine the best path as the final result. Our method can naturally avoid error accumulation compared with single-step greedy decision methods. Comparing to a single-step imitation learning approach, iPPD has performance gains above 17% on navigation success and 0.18 on path matching measurement nDTW in challenging unseen environments.

cs.AI↗

arXiv2022

High-speed tunable microwave-rate soliton microcomb

Microwave signal generation with fast frequency tuning underlies many applications including sensing, imaging, ranging, time keeping, wireless communication, and high-speed electronics. Soliton microcombs are a promising new approach for photonic-based microwave signal synthesis. To date, however, tuning rate has been limited in microcombs (and in frequency combs generally). Here, we demonstrate the first microwave-rate soliton microcomb whose repetition rate can be tuned at a high speed. By integrating an electro-optic tuning/modulation element into a lithium niobate comb microresonator, a modulation bandwidth up to 75 MHz and a continuous frequency modulation rate up to 5.0 * 10^14 Hz/s are achieved, several orders-of-magnitude faster than existing microcomb technology. These features are especially useful for disciplining an optical VCO to a long-term reference such as an optical clock or for tight phase lock in dual-microcomb clocks or synthesizers. Moreover, the microwave signal rate can be set to any X - W band rate. Besides its positive impact on microwave photonics, fast repetition rate control is generally important in all applications of frequency combs.

physics.optics↗

arXiv2022

Self-injection-locked second-harmonic integrated source

High coherence visible and near-visible laser sources are centrally important to the operation of advanced position/navigation/timing systems as well as classical/quantum sensing systems. However, the complexity and size of these bench-top lasers is an impediment to their transitioning beyond the laboratory. Here, a system-on-a-chip that emits high-coherence visible and near-visible lightwaves is demonstrated. The devices rely upon a new approach wherein wavelength conversion and coherence increase by self-injection-locking are combined within in a single nonlinear resonator. This simplified approach is demonstrated in a hybridly-integrated device and provides a short-term linewidth around 10-30 kHz. On-chip, converted optical power over 2 mW is also obtained. Moreover, measurements show that heterogeneous integration can result in conversion efficiency higher than 25% with output power over 11 mW. Because the approach uses mature III-V pump lasers in combination with thin-film lithium niobate, it can be scaled for low-cost manufacturing of high-coherence visible emitters. Also, the coherence generation process can be transferred to other frequency conversion processes including optical parametric oscillation, sum/difference frequency generation, and third-harmonic generation.

physics.optics↗

arXiv2022

Integrated Pockels Laser

The development of integrated semiconductor lasers has miniaturized traditional bulky laser systems, enabling a wide range of photonic applications. A progression from pure III-V based lasers to III-V/external cavity structures has harnessed low-loss waveguides in different material systems, leading to significant improvements in laser coherence and stability. Despite these successes, however, key functions remain absent. In this work, we address a critical missing function by integrating the Pockels effect into a semiconductor laser. Using a hybrid integrated III-V/Lithium Niobate structure, we demonstrate several essential capabilities that have not existed in previous integrated lasers. These include a record-high frequency modulation speed of 2 exahertz/s (2.0$\times$10$^{18}$ Hz/s) and fast switching at 50 MHz, both of which are made possible by integration of the electro-optic effect. Moreover, the device co-lases at infrared and visible frequencies via the second-harmonic frequency conversion process, the first such integrated multi-color laser. Combined with its narrow linewidth and wide tunability, this new type of integrated laser holds promise for many applications including LiDAR, microwave photonics, atomic physics, and AR/VR.

physics.optics↗

arXiv2022

Dynamic Key-value Memory Enhanced Multi-step Graph Reasoning for Knowledge-based Visual Question Answering

Knowledge-based visual question answering (VQA) is a vision-language task that requires an agent to correctly answer image-related questions using knowledge that is not presented in the given image. It is not only a more challenging task than regular VQA but also a vital step towards building a general VQA system. Most existing knowledge-based VQA systems process knowledge and image information similarly and ignore the fact that the knowledge base (KB) contains complete information about a triplet, while the extracted image information might be incomplete as the relations between two objects are missing or wrongly detected. In this paper, we propose a novel model named dynamic knowledge memory enhanced multi-step graph reasoning (DMMGR), which performs explicit and implicit reasoning over a key-value knowledge memory module and a spatial-aware image graph, respectively. Specifically, the memory module learns a dynamic knowledge representation and generates a knowledge-aware question representation at each reasoning step. Then, this representation is used to guide a graph attention operator over the spatial-aware image graph. Our model achieves new state-of-the-art accuracy on the KRVQR and FVQA datasets. We also conduct ablation experiments to prove the effectiveness of each component of the proposed model.

cs.CV↗

arXiv2022

Modeling Coreference Relations in Visual Dialog

Visual dialog is a vision-language task where an agent needs to answer a series of questions grounded in an image based on the understanding of the dialog history and the image. The occurrences of coreference relations in the dialog makes it a more challenging task than visual question-answering. Most previous works have focused on learning better multi-modal representations or on exploring different ways of fusing visual and language features, while the coreferences in the dialog are mainly ignored. In this paper, based on linguistic knowledge and discourse features of human dialog we propose two soft constraints that can improve the model's ability of resolving coreferences in dialog in an unsupervised way. Experimental results on the VisDial v1.0 dataset shows that our model, which integrates two novel and linguistically inspired soft constraints in a deep transformer neural architecture, obtains new state-of-the-art performance in terms of recall at 1 and other evaluation metrics compared to current existing models and this without pretraining on other vision-language datasets. Our qualitative results also demonstrate the effectiveness of the method that we propose.

cs.CV↗

arXiv2021

Towards Understanding Iterative Magnitude Pruning: Why Lottery Tickets Win

The lottery ticket hypothesis states that sparse subnetworks exist in randomly initialized dense networks that can be trained to the same accuracy as the dense network they reside in. However, the subsequent work has failed to replicate this on large-scale models and required rewinding to an early stable state instead of initialization. We show that by using a training method that is stable with respect to linear mode connectivity, large networks can also be entirely rewound to initialization. Our subsequent experiments on common vision tasks give strong credence to the hypothesis in Evci et al. (2020b) that lottery tickets simply retrain to the same regions (although not necessarily to the same basin). These results imply that existing lottery tickets could not have been found without the preceding dense training by iterative magnitude pruning, raising doubts about the use of the lottery ticket hypothesis.

cs.LG↗

arXiv2021

Ultra-broadband Entangled Photons on a Nanophotonic Chip

Nanophotonic entangled-photon sources are a critical building block of chip-scale quantum photonic architecture and have seen significant development over the past two decades. These sources generate photon pairs that typically span over a narrow frequency bandwidth. Generating entanglement over a wide spectral region has proven to be useful in a wide variety of applications including quantum metrology, spectroscopy and sensing, and optical communication. However, generation of broadband photon pairs with temporal coherence approaching an optical cycle on a chip is yet to be seen. Here we demonstrate generation of ultra-broadband entangled photons using spontaneous parametric down-conversion in a periodically-poled lithium niobate nanophotonic waveguide. We employ dispersion engineering to achieve a bandwidth of 100 THz (1.2 - 2 $μ$m), at a high efficiency of 13 GHz/mW. The photons show strong temporal correlations and purity with the coincidence-to-accidental ratio exceeding $10^5$ and $>$ 98\% two-photon interference visibility. These properties together with the piezo-electric and electro-optic control and reconfigurability, make thin-film lithium niobate an excellent platform for a controllable entanglement source for quantum communication and computing, and open a path towards femtosecond metrology and spectroscopy with non-classical light on a nanophotonic chip.

physics.optics↗

arXiv2020

Multiscale Dynamic Human Mobility Flow Dataset in the U.S. during the COVID-19 Epidemic

Understanding dynamic human mobility changes and spatial interaction patterns at different geographic scales is crucial for assessing the impacts of non-pharmaceutical interventions (such as stay-at-home orders) during the COVID-19 pandemic. In this data descriptor, we introduce a regularly-updated multiscale dynamic human mobility flow dataset across the United States, with data starting from March 1st, 2020. By analyzing millions of anonymous mobile phone users' visits to various places provided by SafeGraph, the daily and weekly dynamic origin-to-destination (O-D) population flows are computed, aggregated, and inferred at three geographic scales: census tract, county, and state. There is high correlation between our mobility flow dataset and openly available data sources, which shows the reliability of the produced data. Such a high spatiotemporal resolution human mobility flow dataset at different geographic scales over time may help monitor epidemic spreading dynamics, inform public health policy, and deepen our understanding of human behavior changes under the unprecedented public health crisis. This up-to-date O-D flow open data can support many other social sensing and transportation applications.

cs.SI↗

arXiv2020

Lithium niobate photonic-crystal electro-optic modulator

Modern advanced photonic integrated circuits require dense integration of high-speed electro-optic functional elements on a compact chip that consumes only moderate power. Energy efficiency, operation speed, and device dimension are thus crucial metrics underlying almost all current developments of photonic signal processing units. Recently, thin-film lithium niobate (LN) emerges as a promising platform for photonic integrated circuits. Here we make an important step towards miniaturizing functional components on this platform, reporting probably the smallest high-speed LN electro-optic modulators, based upon photonic crystal nanobeam resonators. The devices exhibit a significant tuning efficiency up to 1.98 GHz/V, a broad modulation bandwidth of 17.5 GHz, while with a tiny electro-optic modal volume of only 0.58 $μ{\rm m}^3$. The modulators enable efficient electro-optic driving of high-Q photonic cavity modes in both adiabatic and non-adiabatic regimes, and allow us to achieve electro-optic switching at 11 Gb/s with a bit-switching energy as low as 22 fJ. The demonstration of energy efficient and high-speed electro-optic modulation at the wavelength scale paves a crucial foundation for realizing large-scale LN photonic integrated circuits that are of immense importance for broad applications in data communication, microwave photonics, and quantum photonics.

physics.app-ph↗