arXiv subjects

Jingwei Xu

Publications and source records attributed to Jingwei Xu.

At least 37 records · Page 2Linked to original sources

arXiv2024

Neuro-symbolic Learning Yielding Logical Constraints

Neuro-symbolic systems combine the abilities of neural perception and logical reasoning. However, end-to-end learning of neuro-symbolic systems is still an unsolved challenge. This paper proposes a natural framework that fuses neural network training, symbol grounding, and logical constraint synthesis into a coherent and efficient end-to-end learning process. The capability of this framework comes from the improved interactions between the neural and the symbolic parts of the system in both the training and inference stages. Technically, to bridge the gap between the continuous neural network and the discrete logical constraint, we introduce a difference-of-convex programming technique to relax the logical constraints while maintaining their precision. We also employ cardinality constraints as the language for logical constraint learning and incorporate a trust region method to avoid the degeneracy of logical constraint in learning. Both theoretical analyses and empirical evaluations substantiate the effectiveness of the proposed framework.

cs.AI↗

arXiv2024

LASER: Script Execution by Autonomous Agents for On-demand Traffic Simulation

Autonomous Driving Systems (ADS) require diverse and safety-critical traffic scenarios for effective training and testing, but the existing data generation methods struggle to provide flexibility and scalability. We propose LASER, a novel frame-work that leverage large language models (LLMs) to conduct traffic simulations based on natural language inputs. The framework operates in two stages: it first generates scripts from user-provided descriptions and then executes them using autonomous agents in real time. Validated in the CARLA simulator, LASER successfully generates complex, on-demand driving scenarios, significantly improving ADS training and testing data generation.

cs.RO↗

arXiv2024

Executing Arithmetic: Fine-Tuning Large Language Models as Turing Machines

Large Language Models (LLMs) have demonstrated remarkable capabilities across a wide range of natural language processing and reasoning tasks. However, their performance in the foundational domain of arithmetic remains unsatisfactory. When dealing with arithmetic tasks, LLMs often memorize specific examples rather than learning the underlying computational logic, limiting their ability to generalize to new problems. In this paper, we propose a Composable Arithmetic Execution Framework (CAEF) that enables LLMs to learn to execute step-by-step computations by emulating Turing Machines, thereby gaining a genuine understanding of computational logic. Moreover, the proposed framework is highly scalable, allowing composing learned operators to significantly reduce the difficulty of learning complex operators. In our evaluation, CAEF achieves nearly 100% accuracy across seven common mathematical operations on the LLaMA 3.1-8B model, effectively supporting computations involving operands with up to 100 digits, a level where GPT-4o falls short noticeably in some settings.

cs.AI↗

arXiv2024

MeteoRA: Multiple-tasks Embedded LoRA for Large Language Models

The pretrain+fine-tune paradigm is foundational for deploying large language models (LLMs) across various downstream applications. Within this framework, Low-Rank Adaptation (LoRA) stands out for its parameter-efficient fine-tuning (PEFT), producing numerous reusable task-specific LoRA adapters. However, this approach requires explicit task intention selection, posing challenges for autonomous task sensing and switching during inference with multiple existing LoRA adapters embedded in a single LLM. In this work, we introduce MeteoRA (Multiple-tasks embedded LoRA), a scalable and efficient framework that reuses multiple task-specific LoRA adapters into the base LLM via a full-mode Mixture-of-Experts (MoE) architecture. This framework also includes novel MoE forward acceleration strategies to address the efficiency challenges of traditional MoE implementations. Our evaluation, using the LlaMA2-13B and LlaMA3-8B base models equipped with 28 existing LoRA adapters through MeteoRA, demonstrates equivalent performance with the traditional PEFT method. Moreover, the LLM equipped with MeteoRA achieves superior performance in handling composite tasks, effectively solving ten sequential problems in a single inference pass, thereby demonstrating the framework's enhanced capability for timely adapter switching.

cs.CL↗

arXiv2024

A lower bound on the number of edges in DP-critical graphs. II. Four colors

A graph $G$ is $k$-critical (list $k$-critical, DP $k$-critical) if $χ(G)= k$ ($χ_\ell(G)= k$, $χ_\mathrm{DP}(G)= k$) and for every proper subgraph $G'$ of $G$, $χ(G') \left(3 + \frac{1}{5} \right) \frac{n}{2}. $$ This is the first bound on $f_\mathrm{DP}(n,4)$ that is asymptotically better than the well-known bound $f(n,4)\geq \left(3 + \frac{1}{13} \right) \frac{n}{2}$ by Gallai from 1963. The result also yields a better bound on $f_{\ell}(n,4)$ than the one known before.

math.CO↗

arXiv2024

DebSDF: Delving into the Details and Bias of Neural Indoor Scene Reconstruction

In recent years, the neural implicit surface has emerged as a powerful representation for multi-view surface reconstruction due to its simplicity and state-of-the-art performance. However, reconstructing smooth and detailed surfaces in indoor scenes from multi-view images presents unique challenges. Indoor scenes typically contain large texture-less regions, making the photometric loss unreliable for optimizing the implicit surface. Previous work utilizes monocular geometry priors to improve the reconstruction in indoor scenes. However, monocular priors often contain substantial errors in thin structure regions due to domain gaps and the inherent inconsistencies when derived independently from different views. This paper presents \textbf{DebSDF} to address these challenges, focusing on the utilization of uncertainty in monocular priors and the bias in SDF-based volume rendering. We propose an uncertainty modeling technique that associates larger uncertainties with larger errors in the monocular priors. High-uncertainty priors are then excluded from optimization to prevent bias. This uncertainty measure also informs an importance-guided ray sampling and adaptive smoothness regularization, enhancing the learning of fine structures. We further introduce a bias-aware signed distance function to density transformation that takes into account the curvature and the angle between the view direction and the SDF normals to reconstruct fine details better. Our approach has been validated through extensive experiments on several challenging datasets, demonstrating improved qualitative and quantitative results in reconstructing thin structures in indoor scenes, thereby outperforming previous work.

cs.CV↗

arXiv2024

Softened Symbol Grounding for Neuro-symbolic Systems

Neuro-symbolic learning generally consists of two separated worlds, i.e., neural network training and symbolic constraint solving, whose success hinges on symbol grounding, a fundamental problem in AI. This paper presents a novel, softened symbol grounding process, bridging the gap between the two worlds, and resulting in an effective and efficient neuro-symbolic learning framework. Technically, the framework features (1) modeling of symbol solution states as a Boltzmann distribution, which avoids expensive state searching and facilitates mutually beneficial interactions between network training and symbolic reasoning;(2) a new MCMC technique leveraging projection and SMT solvers, which efficiently samples from disconnected symbol solution spaces; (3) an annealing mechanism that can escape from %being trapped into sub-optimal symbol groundings. Experiments with three representative neuro symbolic learning tasks demonstrate that, owining to its superior symbol grounding capability, our framework successfully solves problems well beyond the frontier of the existing proposals.

cs.AI↗

arXiv2024

Learning with Logical Constraints but without Shortcut Satisfaction

Recent studies in neuro-symbolic learning have explored the integration of logical knowledge into deep learning via encoding logical constraints as an additional loss function. However, existing approaches tend to vacuously satisfy logical constraints through shortcuts, failing to fully exploit the knowledge. In this paper, we present a new framework for learning with logical constraints. Specifically, we address the shortcut satisfaction issue by introducing dual variables for logical connectives, encoding how the constraint is satisfied. We further propose a variational framework where the encoded logical constraint is expressed as a distributional loss that is compatible with the model's original training loss. The theoretical analysis shows that the proposed approach bears salient properties, and the experimental evaluations demonstrate its superior performance in both model generalizability and constraint satisfaction.

cs.AI↗

arXiv2024

ADMap: Anti-disturbance framework for reconstructing online vectorized HD map

In the field of autonomous driving, online high-definition (HD) map reconstruction is crucial for planning tasks. Recent research has developed several high-performance HD map reconstruction models to meet this necessity. However, the point sequences within the instance vectors may be jittery or jagged due to prediction bias, which can impact subsequent tasks. Therefore, this paper proposes the Anti-disturbance Map reconstruction framework (ADMap). To mitigate point-order jitter, the framework consists of three modules: Multi-Scale Perception Neck, Instance Interactive Attention (IIA), and Vector Direction Difference Loss (VDDL). By exploring the point-order relationships between and within instances in a cascading manner, the model can monitor the point-order prediction process more effectively. ADMap achieves state-of-the-art performance on the nuScenes and Argoverse2 datasets. Extensive results demonstrate its ability to produce stable and reliable map elements in complex and changing driving scenarios. Code and more demos are available at https://github.com/hht1996ok/ADMap.

cs.CV↗

arXiv2024

Securing Reliability: A Brief Overview on Enhancing In-Context Learning for Foundation Models

As foundation models (FMs) continue to shape the landscape of AI, the in-context learning (ICL) paradigm thrives but also encounters issues such as toxicity, hallucination, disparity, adversarial vulnerability, and inconsistency. Ensuring the reliability and responsibility of FMs is crucial for the sustainable development of the AI ecosystem. In this concise overview, we investigate recent advancements in enhancing the reliability and trustworthiness of FMs within ICL frameworks, focusing on four key methodologies, each with its corresponding subgoals. We sincerely hope this paper can provide valuable insights for researchers and practitioners endeavoring to build safe and dependable FMs and foster a stable and consistent ICL environment, thereby unlocking their vast potential.

cs.LG↗

arXiv2024

Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Survey

Transformer-based Large Language Models (LLMs) have been applied in diverse areas such as knowledge bases, human interfaces, and dynamic agents, and marking a stride towards achieving Artificial General Intelligence (AGI). However, current LLMs are predominantly pretrained on short text snippets, which compromises their effectiveness in processing the long-context prompts that are frequently encountered in practical scenarios. This article offers a comprehensive survey of the recent advancement in Transformer-based LLM architectures aimed at enhancing the long-context capabilities of LLMs throughout the entire model lifecycle, from pre-training through to inference. We first delineate and analyze the problems of handling long-context input and output with the current Transformer-based models. We then provide a taxonomy and the landscape of upgrades on Transformer architecture to solve these problems. Afterwards, we provide an investigation on wildly used evaluation necessities tailored for long-context LLMs, including datasets, metrics, and baseline models, as well as optimization toolkits such as libraries, frameworks, and compilers to boost the efficacy of LLMs across different stages in runtime. Finally, we discuss the challenges and potential avenues for future research. A curated repository of relevant literature, continuously updated, is available at https://github.com/Strivin0311/long-llms-learning.

cs.CL↗

arXiv2023

Sparse critical graphs for defective $(1,3)$-coloring

A graph $G$ is $(1,3)$-colorable if its vertices can be partitioned into subsets $V_1$ and $V_2$ so that every vertex in $G[V_1]$ has degree at most $1$ and every vertex in $G[V_2]$ has degree at most $3$. We prove that every graph with maximum average degree at most 28/9 is $(1, 3)$-colorable.

math.CO↗

arXiv2023

EA-LSS: Edge-aware Lift-splat-shot Framework for 3D BEV Object Detection

In recent years, great progress has been made in the Lift-Splat-Shot-based (LSS-based) 3D object detection method. However, inaccurate depth estimation remains an important constraint to the accuracy of camera-only and multi-model 3D object detection models, especially in regions where the depth changes significantly (i.e., the "depth jump" problem). In this paper, we proposed a novel Edge-aware Lift-splat-shot (EA-LSS) framework. Specifically, edge-aware depth fusion (EADF) module is proposed to alleviate the "depth jump" problem and fine-grained depth (FGD) module to further enforce refined supervision on depth. Our EA-LSS framework is compatible for any LSS-based 3D object detection models, and effectively boosts their performances with negligible increment of inference time. Experiments on nuScenes benchmarks demonstrate that EA-LSS is effective in either camera-only or multi-model models. It is worth mentioning that EA-LSS achieved the state-of-the-art performance on nuScenes test benchmarks with mAP and NDS of 76.5% and 77.6%, respectively.

cs.CV↗

arXiv2023

Injective edge colorings of degenerate graphs and the oriented chromatic number

Given a graph $G$, an injective edge-coloring of $G$ is a function $ψ:E(G) \rightarrow \mathbb N$ such that if $ψ(e) = ψ(e')$, then no third edge joins an endpoint of $e$ and an endpoint of $e'$. The injective chromatic index of a graph $G$, written $χ_{inj}'(G)$, is the minimum number of colors needed for an injective edge coloring of $G$. In this paper, we investigate the injective chromatic index of certain classes of degenerate graphs. First, we show that if $G$ is a $d$-degenerate graph of maximum degree $Δ$, then $χ_{inj}'(G) = O(d^3 \log Δ)$. Next, we show that if $G$ is a graph of Euler genus $g$, then $χ_{inj}'(G) \leq (3+o(1))g$, which is tight when $G$ is a clique. Finally, we show that the oriented chromatic number of a graph is at most exponential in its injective chromatic index. Using this fact, we prove that the oriented chromatic number of a graph embedded on a surface of Euler genus $g$ has oriented chromatic number at most $O(g^{6400})$, improving the previously known upper bound of $2^{O(g^{\frac{1}{2} + ε})}$ and resolving a conjecture of Aravind and Subramanian.

math.CO↗

arXiv2023

Sparse critical graphs for defective DP-colorings

An interesting generalization of list coloring is so called DP-coloring (named after Dvořák and Postle). We study $(i,j)$-defective DP-colorings of simple graphs. Define $g_{DP}(i,j,n)$ to be the minimum number of edges in an $n$-vertex DP-$(i,j)$-critical graph. We prove sharp bounds on $g_{DP}(i,j,n)$ for $i=1,2$ and $j\geq 2i$ for infinitely many $n$.

math.CO↗

arXiv2022

System response analysis in wavenumber domain for linear space-invariant time-varying problems

Being a powerful tool for linear time-invariant (LTI) systems, system response analysis can also be applied to the so-called linear space-invariant (LSI) but time-varying systems, which is a dual of the conventional LTI problems. In this paper, we propose a system response analysis method for LSI problems by conducting Fourier transform of the field distribution on the space instead of time coordinate. Specifically, input and output signals can be expressed in the wavenumber (spatial frequency) domain. In this way, the system function in wavenumber domain can also be obtained for LSI systems. Given an arbitrary input and temporal profile of the medium, the output can be easily predicted using the system function. Moreover, for a complex temporal system, the proposed method allows for decomposing it into multiple simpler subsystems that appear in sequence in time. The system function of the whole system can be efficiently calculated by multiplying those of the individual subsystems.

physics.optics↗

arXiv2022

Fundamental asymmetries between spatial and temporal boundaries in electromagnetics

Time-varying materials bring an extra degree of design freedom compared to their conventional time-invariant counterparts. However, few discussions have focused on the underlying physical difference between spatial and temporal boundaries. In this letter, we thoroughly investigate those differences from the perspective of conservation laws. By doing so, the building blocks of optics and electromagnetics such as the reflection law, Snell's law, and Fresnel's equations can be analogously derived in a temporal context, but with completely different interpretations. Furthermore, we study the unique features of temporal boundaries, such as their nonconformance to energy conservation and causality.

physics.optics↗

arXiv2021

Out-of-Domain Human Mesh Reconstruction via Dynamic Bilevel Online Adaptation

We consider a new problem of adapting a human mesh reconstruction model to out-of-domain streaming videos, where performance of existing SMPL-based models are significantly affected by the distribution shift represented by different camera parameters, bone lengths, backgrounds, and occlusions. We tackle this problem through online adaptation, gradually correcting the model bias during testing. There are two main challenges: First, the lack of 3D annotations increases the training difficulty and results in 3D ambiguities. Second, non-stationary data distribution makes it difficult to strike a balance between fitting regular frames and hard samples with severe occlusions or dramatic changes. To this end, we propose the Dynamic Bilevel Online Adaptation algorithm (DynaBOA). It first introduces the temporal constraints to compensate for the unavailable 3D annotations, and leverages a bilevel optimization procedure to address the conflicts between multi-objectives. DynaBOA provides additional 3D guidance by co-training with similar source examples retrieved efficiently despite the distribution shift. Furthermore, it can adaptively adjust the number of optimization steps on individual frames to fully fit hard samples and avoid overfitting regular frames. DynaBOA achieves state-of-the-art results on three out-of-domain human mesh reconstruction benchmarks.

cs.CV↗