arXiv subjects

Jiawei Wang

Publications and source records attributed to Jiawei Wang.

At least 127 records · Page 7Linked to original sources

arXiv2024

Large Language Models as Urban Residents: An LLM Agent Framework for Personal Mobility Generation

This paper introduces a novel approach using Large Language Models (LLMs) integrated into an agent framework for flexible and effective personal mobility generation. LLMs overcome the limitations of previous models by effectively processing semantic data and offering versatility in modeling various tasks. Our approach addresses three research questions: aligning LLMs with real-world urban mobility data, developing reliable activity generation strategies, and exploring LLM applications in urban mobility. The key technical contribution is a novel LLM agent framework that accounts for individual activity patterns and motivations, including a self-consistency approach to align LLMs with real-world activity data and a retrieval-augmented strategy for interpretable activity generation. We evaluate our LLM agent framework and compare it with state-of-the-art personal mobility generation approaches, demonstrating the effectiveness of our approach and its potential applications in urban mobility. Overall, this study marks the pioneering work of designing an LLM agent framework for activity generation based on real-world human activity data, offering a promising tool for urban mobility analysis.

cs.AI↗

arXiv2024

Decentralized Robust Data-driven Predictive Control for Smoothing Mixed Traffic Flow

In a mixed traffic with connected automated vehicles (CAVs) and human-driven vehicles (HDVs) coexisting, data-driven predictive control of CAVs promises system-wide traffic performance improvements. Yet, most existing approaches focus on a centralized setup, which is not computationally scalable while failing to protect data privacy. The robustness against unknown disturbances has not been well addressed either, causing safety concerns. In this paper, we propose a decentralized robust DeeP-LCC (Data-EnablEd Predictive Leading Cruise Control) approach for CAVs to smooth mixed traffic flow. In particular, each CAV computes its control input based on locally available data from its involved subsystem. Meanwhile, the interaction between neighboring subsystems is modeled as a bounded disturbance, for which appropriate estimation methods are proposed. Then, we formulate a robust optimization problem and present its tractable computational solutions. Compared with the centralized formulation, our method greatly reduces computation burden with better safety performance, while naturally preserving data privacy. Extensive traffic simulations validate its wave-dampening ability, safety performance, and computational benefits.

math.OC↗

arXiv2024

Sub-100 Hz Intrinsic Linewidth 852 nm Silicon Nitride External Cavity Laser

We demonstrate an external cavity laser with intrinsic linewidth below 100 Hz around an operating wavelength of 852 nm, selected for its relevance to laser cooling and manipulation of cesium atoms. This system achieves a maximum CW output power of 24 mW, wavelength tunability over 15 nm, and a side-mode suppression ratio exceeding 50 dB. This performance level is facilitated by careful design of a low-loss integrated silicon nitride photonic circuit serving as the external cavity combined with commercially available semiconductor gain chips. This approach demonstrates the feasibility of compact integrated lasers with sub-kHz linewidth centering on the needs of emerging sensor concepts based on ultracold atoms and can be further extended to shorter wavelengths via selection of suitable semiconductor gain media.

physics.optics↗

arXiv2024

Robust Data-EnablEd Predictive Leading Cruise Control via Reachability Analysis

Data-driven predictive control promises model-free wave-dampening strategies for Connected and Autonomous Vehicles (CAVs) in mixed traffic flow. However, its performance relies on data quality, which suffers from unknown noise and disturbances. This paper introduces a Robust Data-EnablEd Predictive Leading Cruise Control (RDeeP-LCC) method based on reachability analysis, aiming to achieve safe and optimal CAV control under bounded process noise and external disturbances. Precisely, the matrix zonotope set technique and Willems' Fundamental Lemma are employed to derive the over-approximated system dynamics directly from data, and a data-driven feedback control technique is utilized to obtain an additional feedback input for stability. We decouple the mixed platoon into an error system and a nominal system, where the error system provides data-driven reachability sets for the enhanced safety constraints in the nominal system. Finally, a data-driven predictive control framework is formulated in a tube-based control manner for robustness guarantees. Nonlinear simulations with noise-corrupted data demonstrate that the proposed method outperforms baseline methods in mitigating traffic waves.

eess.SY↗

arXiv2024

Tarsier: Recipes for Training and Evaluating Large Video Description Models

Generating fine-grained video descriptions is a fundamental challenge in video understanding. In this work, we introduce Tarsier, a family of large-scale video-language models designed to generate high-quality video descriptions. Tarsier employs CLIP-ViT to encode frames separately and then uses an LLM to model temporal relationships. Despite its simple architecture, we demonstrate that with a meticulously designed two-stage training procedure, the Tarsier models exhibit substantially stronger video description capabilities than any existing open-source model, showing a $+51.4\%$ advantage in human side-by-side evaluation over the strongest model. Additionally, they are comparable to state-of-the-art proprietary models, with a $+12.3\%$ advantage against GPT-4V and a $-6.7\%$ disadvantage against Gemini 1.5 Pro. When upgraded to Tarsier2 by building upon SigLIP and Qwen2-7B, it further improves significantly with a $+4.8\%$ advantage against GPT-4o. Besides video description, Tarsier proves to be a versatile generalist model, achieving new state-of-the-art results across nine public benchmarks, including multi-choice VQA, open-ended VQA, and zero-shot video captioning. Our second contribution is the introduction of a new benchmark -- DREAM-1K (https://tarsier-vlm.github.io/) for evaluating video description models, consisting of a new challenging dataset featuring videos from diverse sources and varying complexity, along with an automatic method specifically designed to assess the quality of fine-grained video descriptions. We make our models and evaluation benchmark publicly available at https://github.com/bytedance/tarsier.

cs.CV↗

arXiv2024

Static Analysis Driven Enhancements for Comprehension in Machine Learning Notebooks

Jupyter notebooks enable developers to interleave code snippets with rich-text and in-line visualizations. Data scientists use Jupyter notebook as the de-facto standard for creating and sharing machine-learning based solutions, primarily written in Python. Recent studies have demonstrated, however, that a large portion of Jupyter notebooks available on public platforms are undocumented and lacks a narrative structure. This reduces the readability of these notebooks. To address this shortcoming, this paper presents HeaderGen, a novel tool-based approach that automatically annotates code cells with categorical markdown headers based on a taxonomy of ML operations, and classifies and displays function calls according to this taxonomy. For this functionality to be realized, HeaderGen enhances an existing call graph analysis in PyCG. To improve precision, HeaderGen extends PyCG's analysis with support for handling external library code and flow-sensitivity. The former is realized by facilitating the resolution of function return-types. The evaluation on 15 real-world Jupyter notebooks from Kaggle shows that HeaderGen's underlying call graph analysis yields high accuracy (95.6% precision and 95.3% recall). This is because HeaderGen can resolve return-types of external libraries where existing type inference tools such as pytype (by Google), pyright (by Microsoft), and Jedi fall short. The header generation has a precision of 85.7% and a recall rate of 92.8%. In a user study, HeaderGen helps participants finish comprehension and navigation tasks faster. To further evaluate the type inference capability of tools, we introduce TypeEvalPy, a framework for evaluating type inference tools with a micro-benchmark containing 154 code snippets and 845 type annotations. Our comparative analysis on four tools revealed that HeaderGen outperforms other tools in exact matches with the ground truth.

cs.SE↗

arXiv2024

Nonlinear Eigen-approach ADMM for Sparse Optimization on Stiefel Manifold

With the growing interest and applications in machine learning and data science, finding an efficient method to sparse analysis the high-dimensional data and optimizing a dimension reduction model to extract lower dimensional features has becoming more and more important. Orthogonal constraints (Stiefel manifold) is a commonly met constraint in these applications, and the sparsity is usually enforced through the element-wise L1 norm. Many applications can be found on optimization over Stiefel manifold within the area of physics and machine learning. In this paper, we propose a novel idea by tackling the Stiefel manifold through an nonlinear eigen-approach by first using ADMM to split the problem into smooth optimization over manifold and convex non-smooth optimization, and then transforming the former into the form of nonlinear eigenvalue problem with eigenvector dependency (NEPv) which is solved by self-consistent field (SCF) iteration, and the latter can be found to have an closed-form solution through proximal gradient. Compared with existing methods, our proposed algorithm takes the advantage of specific structure of the objective function, and has efficient convergence results under mild assumptions.

math.OC↗

arXiv2024

DLAFormer: An End-to-End Transformer For Document Layout Analysis

Document layout analysis (DLA) is crucial for understanding the physical layout and logical structure of documents, serving information retrieval, document summarization, knowledge extraction, etc. However, previous studies have typically used separate models to address individual sub-tasks within DLA, including table/figure detection, text region detection, logical role classification, and reading order prediction. In this work, we propose an end-to-end transformer-based approach for document layout analysis, called DLAFormer, which integrates all these sub-tasks into a single model. To achieve this, we treat various DLA sub-tasks (such as text region detection, logical role classification, and reading order prediction) as relation prediction problems and consolidate these relation prediction labels into a unified label space, allowing a unified relation prediction module to handle multiple tasks concurrently. Additionally, we introduce a novel set of type-wise queries to enhance the physical meaning of content queries in DETR. Moreover, we adopt a coarse-to-fine strategy to accurately identify graphical page objects. Experimental results demonstrate that our proposed DLAFormer outperforms previous approaches that employ multi-branch or multi-stage architectures for multiple tasks on two document layout analysis benchmarks, DocLayNet and Comp-HRDoc.

cs.CV↗

arXiv2024

AMCEN: An Attention Masking-based Contrastive Event Network for Two-stage Temporal Knowledge Graph Reasoning

Temporal knowledge graphs (TKGs) can effectively model the ever-evolving nature of real-world knowledge, and their completeness and enhancement can be achieved by reasoning new events from existing ones. However, reasoning accuracy is adversely impacted due to an imbalance between new and recurring events in the datasets. To achieve more accurate TKG reasoning, we propose an attention masking-based contrastive event network (AMCEN) with local-global temporal patterns for the two-stage prediction of future events. In the network, historical and non-historical attention mask vectors are designed to control the attention bias towards historical and non-historical entities, acting as the key to alleviating the imbalance. A local-global message-passing module is proposed to comprehensively consider and capture multi-hop structural dependencies and local-global temporal evolution for the in-depth exploration of latent impact factors of different event types. A contrastive event classifier is used to classify events more accurately by incorporating local-global temporal patterns into contrastive learning. Therefore, AMCEN refines the prediction scope with the results of the contrastive event classification, followed by utilizing attention masking-based decoders to finalize the specific outcomes. The results of our experiments on four benchmark datasets highlight the superiority of AMCEN. Especially, the considerable improvements in Hits@1 prove that AMCEN can make more precise predictions about future occurrences.

cs.LG↗

arXiv2024

Scalable single-microring hybrid III-V/Si lasers for emerging narrow-linewidth applications

Silicon photonics, compatible with large-scale silicon manufacturing, is a disruptive photonic platform that has indicated significant implications in industry and research areas (e.g., quantum, neuromorphic computing, LiDAR). Cutting-edge applications such as high-capacity coherent optical communication and heterodyne LiDAR have escalated the demand for integrated narrow-linewidth laser sources. To that effect, this work seeks to address this requirement through the development of a high-performance hybrid III-V/silicon laser. The developed integrated laser, utilizes a single microring resonator (MRR), demonstrating single-mode operation with a side mode suppression ratio (SMSR) exceeding 40 dB, with laser output power as high as 16.4 mW. Moving away from current hybrid/heterogeneous laser architectures that necessitate multiple complex control, the developed laser architecture requires only two control parameters. Importantly, this serves to streamline industrial adoption by reducing the complexity involved in characterizing these lasers, at-scale. Through the succinct structure and control framework, a narrow laser linewidth of 2.79 kHz and low relative intensity noise (RIN) of -135 dB/Hz are achieved. Furthermore, optical data transmission at 12.5 Gb/s is demonstrated where a signal-to-noise ratio (SNR) of 10 dB is measured.

physics.optics↗

arXiv2024

Detect-Order-Construct: A Tree Construction based Approach for Hierarchical Document Structure Analysis

Document structure analysis (aka document layout analysis) is crucial for understanding the physical layout and logical structure of documents, with applications in information retrieval, document summarization, knowledge extraction, etc. In this paper, we concentrate on Hierarchical Document Structure Analysis (HDSA) to explore hierarchical relationships within structured documents created using authoring software employing hierarchical schemas, such as LaTeX, Microsoft Word, and HTML. To comprehensively analyze hierarchical document structures, we propose a tree construction based approach that addresses multiple subtasks concurrently, including page object detection (Detect), reading order prediction of identified objects (Order), and the construction of intended hierarchical structure (Construct). We present an effective end-to-end solution based on this framework to demonstrate its performance. To assess our approach, we develop a comprehensive benchmark called Comp-HRDoc, which evaluates the above subtasks simultaneously. Our end-to-end system achieves state-of-the-art performance on two large-scale document layout analysis datasets (PubLayNet and DocLayNet), a high-quality hierarchical document structure reconstruction dataset (HRDoc), and our Comp-HRDoc benchmark. The Comp-HRDoc benchmark will be released to facilitate further research in this field.

cs.CV↗

arXiv2024

Dynamic Phase Enabled Topological Mode Steering in Composite Su-Schrieffer-Heeger Waveguide Arrays

Topological boundary states localize at interfaces whenever the interface implies a change of the associated topological invariant encoded in the geometric phase. The generically present dynamic phase, however, which is energy and time dependent, has been known to be non-universal, and hence not to intertwine with any topological geometric phase. Using the example of topological zero modes in composite Su-Schrieffer-Heeger (c-SSH) waveguide arrays with a central defect, we report on the selective excitation and transition of topological boundary mode based on dynamic phase-steered interferences. Our work thus provides a new knob for the control and manipulation of topological states in composite photonic devices, indicating promising applications where topological modes and their bandwidth can be jointly controlled by the dynamic phase, geometric phase, and wavelength in on-chip topological devices.

physics.optics↗

arXiv2024

ContextSeg: Sketch Semantic Segmentation by Querying the Context with Attention

Sketch semantic segmentation is a well-explored and pivotal problem in computer vision involving the assignment of pre-defined part labels to individual strokes. This paper presents ContextSeg - a simple yet highly effective approach to tackling this problem with two stages. In the first stage, to better encode the shape and positional information of strokes, we propose to predict an extra dense distance field in an autoencoder network to reinforce structural information learning. In the second stage, we treat an entire stroke as a single entity and label a group of strokes within the same semantic part using an auto-regressive Transformer with the default attention mechanism. By group-based labeling, our method can fully leverage the context information when making decisions for the remaining groups of strokes. Our method achieves the best segmentation accuracy compared with state-of-the-art approaches on two representative datasets and has been extensively evaluated demonstrating its superior performance. Additionally, we offer insights into solving part imbalance in training data and the preliminary experiment on cross-category training, which can inspire future research in this field.

cs.CV↗

arXiv2024

EM-TTS: Efficiently Trained Low-Resource Mongolian Lightweight Text-to-Speech

Recently, deep learning-based Text-to-Speech (TTS) systems have achieved high-quality speech synthesis results. Recurrent neural networks have become a standard modeling technique for sequential data in TTS systems and are widely used. However, training a TTS model which includes RNN components requires powerful GPU performance and takes a long time. In contrast, CNN-based sequence synthesis techniques can significantly reduce the parameters and training time of a TTS model while guaranteeing a certain performance due to their high parallelism, which alleviate these economic costs of training. In this paper, we propose a lightweight TTS system based on deep convolutional neural networks, which is a two-stage training end-to-end TTS model and does not employ any recurrent units. Our model consists of two stages: Text2Spectrum and SSRN. The former is used to encode phonemes into a coarse mel spectrogram and the latter is used to synthesize the complete spectrum from the coarse mel spectrogram. Meanwhile, we improve the robustness of our model by a series of data augmentations, such as noise suppression, time warping, frequency masking and time masking, for solving the low resource mongolian problem. Experiments show that our model can reduce the training time and parameters while ensuring the quality and naturalness of the synthesized speech compared to using mainstream TTS models. Our method uses NCMMSC2022-MTTSC Challenge dataset for validation, which significantly reduces training time while maintaining a certain accuracy.

cs.SD↗

arXiv2024

Trapped ion qubit and clock operations with a visible wavelength photonic coil resonator stabilized integrated Brillouin laser

Integrating precise, stable, ultra-low noise visible light lasers into atomic systems is critical for advancing quantum information sciences and improving scalability and portability. Trapped ions are a leading approach for high-fidelity quantum computing, high-accuracy optical clocks, and precision quantum sensors. However, current ion-based systems rely on bulky, lab-scale precision lasers and optical stabilization cavities for optical clock and qubit operations, constraining the size, weight, scalability, and portability of atomic systems. Chip-scale integration of ultra-low noise lasers and reference cavities operating directly at optical clock transitions and capable of qubit and clock operations will represent a major transformation in atom and trapped ion-based quantum technologies. However, this goal has remained elusive. Here we report the first demonstration of chip-scale optical clock and qubit operations on a trapped ion using a photonic integrated direct-drive visible wavelength Brillouin laser stabilized to an integrated 3-meter coil-resonator reference cavity and the optical clock transition of a $^{88}$Sr$^+$ ion trapped on a surface electrode chip. We also demonstrate for the first time, to the best of our knowledge, trapped-ion spectroscopy and qubit operations such as Rabi oscillations and high fidelity (99%) qubit state preparation and measurement (SPAM) using direct drive integrated photonic technologies without bulk optic stabilization cavities or second harmonic generation. Our chip-scale stabilized Brillouin laser exhibits a 6 kHz linewidth with the 0.4 Hz quadrupole transition of $^{88}$Sr$^+$ and a self-consistent coherence time of 60 $μ$s via Ramsey interferometry on the trapped ion qubit. Furthermore, we demonstrate the stability of the locked Brillouin laser to 5$\times10^{-13}/ \sqrtτ$ at 1 second using dual optical clocks.

quant-ph↗

arXiv2024

Single-species population models with age structure and psychological effect in a polluted environment

This paper considers a single-population model with age structure and psychological effects in a polluted environment. We divide the single population into two stages of larval and adult structure. The model uses Logistic input, and the larvae are converted into adult bodies by constant ratio. We only consider adulthood. The role of psychological effects makes the contact between adult and environmental toxins a functional form, while the contact between larvae and environmental toxins is linear. For the deterministic model embodied as a nonlinear time-varying system, we discuss the asymptotic stability of the system by Lyapunov one-time approximation theory, and give a sufficient condition for stability to be established. Considering that the contact rate between biological and environmental toxins in nature is not always constant, we make the contact rate interfere with white noise, and then modify the contact rate into a stochastic process, thus establishing a corresponding random single-population model. According to Itô formula and Lyapunov in the function method, we first prove the existence of globally unique positive solutions for stochastic models under arbitrary initial conditions, and then give sufficient conditions for weak average long-term survival and random long-term survival for single populations in the expected sense.

q-bio.PE↗

arXiv2024

Kerr optical frequency division with integrated photonics for stable microwave and mmWave generation

Optical frequency division (OFD) has revolutionized microwave and mmWave generation and set spectral purity records owing to its unique capability to transfer high fractional stability from optical to electronic frequencies. Recently, rapid developments in integrated optical reference cavities and microresonator-based optical frequency combs (microcombs) have created a path to transform OFD technology to chip scale. Here, we demonstrate an ultra-low phase noise mmWave oscillator by leveraging integrated photonic components and Kerr optical frequency division. The oscillator derives its stability from an integrated CMOS-compatible SiN coil cavity, and the optical frequency division is achieved spontaneously through Kerr interaction between the injected reference lasers and soliton microcombs in the integrated SiN microresonator. Besides achieving record-low phase noise for integrated mmWave oscillators, our demonstration greatly simplifies the implementation of integrated OFD oscillators and could be useful in applications of Radar, spectroscopy, and astronomy.

physics.optics↗

arXiv2024

Anneal-free ultra-low loss silicon nitride integrated photonics

Heterogeneous and monolithic integration of the versatile low loss silicon nitride platform with low temperature materials such as silicon electronics and photonics, III-V compound semiconductors, lithium niobate, organics, and glasses, has been inhibited by the need for high temperature annealing as well as the need for different process flows for thin and thick waveguides. New techniques are needed to maintain the state-of-the-art losses, nonlinear properties, and CMOS compatible processes while enabling this next generation of 3D silicon nitride integration. We report a significant advance in silicon nitride integrated photonics, demonstrating the lowest losses to date for an anneal-free process at a maximum temperature of 250 C, with the same deuterated silane based fabrication flow, for nitride and oxide, for an order of magnitude range in nitride thickness without requiring stress mitigation or polishing. We report record low losses for anneal-free nitride core and oxide cladding, enabling 1.77 dB/m loss and 14.9 million Q for 80 nm nitride core waveguides, more than half an order magnitude lower loss than previously reported 270 C processes, and 8.66 dB/m loss and 4.03 million Q for 800 nm thick nitride. We demonstrate laser stabilization with over 4 orders of magnitude frequency noise reduction using a thin nitride reference cavity. And using a thick nitride micro-resonator, we demonstrate parametric gain and Optical Parametric Oscillation (OPO) with the lowest reported OPO threshold per unit resonator length for low temperature fabricated nitride, and supercontinuum generation over two octaves. These results represent a significant step towards a uniform ultra-low loss silicon nitride homogeneous and heterogeneous platform for both thin and thick waveguides capable of linear and nonlinear photonic circuits and integration with low temperature materials and processes.

physics.optics↗