arXiv subjects

Anna Wang

Publications and source records attributed to Anna Wang.

At least 19 recordsLinked to original sources

arXiv2026

MotionAtlas: Detailed Region Captioning for Motion-Centric Videos

We propose MotionAtlas, a system for detailed captioning of motion-centric videos, comprising (1) a dedicated human-annotated benchmark, (2) a scalable, high-quality pipeline to construct training samples, and (3) a family of powerful Video-MLLMs. Unlike conventional global motion captioning datasets, we focus on region-aware motion captioning: given a video and a spatiotemporal mask, the model generates precise descriptions of motion within the target region, thereby alleviating visual clutter and motion entanglement and enabling reliable, quantifiable evaluation. Concretely, we first build MotionAtlas-Bench, a comprehensive benchmark comprising 2,073 multiple-choice questions, meticulously annotated for a curated set of high-quality, motion-centric videos, to evaluate fine-grained motion understanding of the objects in question. Second, we design a rigorous and scalable data pipeline that leverages self-bootstrap refinement to suppress fine-grained hallucinations, yielding 159k high-quality motion captioning data. Third, we design a tailored training data composition strategy, which achieves consistent and substantial performance gains across diverse baseline Video-MLLMs, including Molmo2 and Qwen3-VL. For instance, MotionAtlas-4B surpasses Qwen3-VL-4B by an average of 5.2 percentage points across general motion benchmarks. The benchmark, dataset, and code have been released.

cs.CV↗

arXiv2026

Towards One-to-Many Temporal Grounding

Temporal Grounding (TG) aims to localize video segments corresponding to a textual query. Prior research predominantly focuses on single-segment retrieval. Real-world scenarios, however, often require localizing multiple disjoint segments for a single query -- a setting we term One-to-Many Temporal Grounding (OMTG). Previous state-of-the-art MLLMs, optimized for one-to-one settings, struggle in this context, often yielding near-zero scores due to a lack of event cardinality perception. To bridge this gap, we present a systematic solution with three key contributions. First, we establish the first comprehensive OMTG benchmark, introducing Count Accuracy (C-Acc) and Effective Temporal F1 (EtF1) as evaluation metrics. Second, we curate a high-quality OMTG dataset comprising 56k samples through a sophisticated construction pipeline. Third, we develop novel temporal and caption reward functions specifically designed for OMTG. In particular, the caption reward leverages Chain-of-Thought reasoning over dense video captions to explicitly guide policy optimization toward both preciseness and completeness. Extensive experiments show our model achieves a new state-of-the-art EtF1 of 43.65\% on OMTG Bench, outperforming Gemini 2.5 Pro and Seed-1.8 by 15.85\% and 15.61\%, respectively. Project Page: https://insomniaaac.github.io/OMTG/

cs.CV↗

arXiv2026

MergeTok: Unified Continuous and Discrete Visual Tokenization via Token Merging

Most visual tokenizers for image generation are bifurcated into two families with complementary limitations: continuous VAEs offer high-fidelity reconstruction but suffer from dense, entangled latents that are poorly suited for semantic control, whereas discrete VQ-based models enable autoregressive generation yet struggle with gradient sparsity, unstable training, and codebook collapse. In this work, we introduce MergeTok, a unified tokenizer that jointly optimizes continuous (VAE) and discrete (VQ) tokenizers within a encoder-decoder architecture, leveraging token merging techniques as a semantic bridge. By clustering similar tokens during encoding, MergeTok establishes a structural prior that provides dual supervision signals: (i) it imposes merged-token semantic alignment in the VAE branch, regularizing its latent space toward disentangled, semantic-aware representations; (ii) it derives group-wise constraints, promoting intra-group diversity and inter-group exclusivity that stabilize VQ training. MergeTok shows competitive reconstruction and generation performance on ImageNet-256, with substantially lower rFID than strong VAE and VQ models under matched token budgets, while producing semantically-organized token representations compatible with both autoregressive and diffusion generators. This shows that a single architecture can endow visual tokenizers with robust semantic organization and generator-friendly discreteness.

cs.CV↗

arXiv2026

A Modular 3D-Printed Design to Investigate Prebiotic Chemical Systems in Hot Spring Pools

The emergence of membranous compartments (protocells) with encapsulated genetic material was a crucial step life's origin and evolution. The hot spring hypothesis for the origin of life suggests that protocells could have formed in hot spring pools and encapsulated organic matter. Previous investigations have focused on mimicking wet-dry (WD) cycles within a single pool, which precludes simulation of many hydrothermal field conditions, such as differential mineralogy, variable temperature and pH and water flow between multiple hot spring pools. Here, we present a modular 3D-printed hydrothermal field simulator that mimics the complex nature of hot spring fields by controlling the variability of a series of linked pools, including WD cycles, temperature, pH, mineralogy, and mixing of different fluids. Results from using the prototype hot spring field design demonstrate the ability to spontaneously form lipid vesicles that encapsulate organic matter within membranous compartments comprised of decanoic acid:decanol (4:1) or the phospholipids POPC:POPG (1:1). We observed distinct morphological differences in the vesicles, ranging from thick-walled multilamellar, thin-walled oligolamellar and unilamellar as well as giant unilamellar vesicles formed under multiple WD cycles in the simulator pools. Cargo encapsulation was favoured in the cell-like giant unilamellar and small oligolamellar vesicles. Overall, hot-spring simulator offers a customisable avenue for studying other hot spring processes such as prebiotic chemical reactions, mineral surface catalysis, and the complexity of hydrothermal field dynamics.

cond-mat.soft↗

arXiv2025

MergeDNA: Context-aware Genome Modeling with Dynamic Tokenization through Token Merging

Modeling genomic sequences faces two unsolved challenges: the information density varies widely across different regions, while there is no clearly defined minimum vocabulary unit. Relying on either four primitive bases or independently designed DNA tokenizers, existing approaches with naive masked language modeling pre-training often fail to adapt to the varying complexities of genomic sequences. Leveraging Token Merging techniques, this paper introduces a hierarchical architecture that jointly optimizes a dynamic genomic tokenizer and latent Transformers with context-aware pre-training tasks. As for network structures, the tokenization module automatically chunks adjacent bases into words by stacking multiple layers of the differentiable token merging blocks with local-window constraints, then a Latent Encoder captures the global context of these merged words by full-attention blocks. Symmetrically employing a Latent Decoder and a Local Decoder, MergeDNA learns with two pre-training tasks: Merged Token Reconstruction simultaneously trains the dynamic tokenization module and adaptively filters important tokens, while Adaptive Masked Token Modeling learns to predict these filtered tokens to capture informative contents. Extensive experiments show that MergeDNA achieves superior performance on three popular DNA benchmarks and several multi-omics tasks with fine-tuning or zero-shot evaluation, outperforming typical tokenization methods and large-scale DNA foundation models.

q-bio.GN↗

arXiv2025

An Approach to Technical AGI Safety and Security

Artificial General Intelligence (AGI) promises transformative benefits but also presents significant risks. We develop an approach to address the risk of harms consequential enough to significantly harm humanity. We identify four areas of risk: misuse, misalignment, mistakes, and structural risks. Of these, we focus on technical approaches to misuse and misalignment. For misuse, our strategy aims to prevent threat actors from accessing dangerous capabilities, by proactively identifying dangerous capabilities, and implementing robust security, access restrictions, monitoring, and model safety mitigations. To address misalignment, we outline two lines of defense. First, model-level mitigations such as amplified oversight and robust training can help to build an aligned model. Second, system-level security measures such as monitoring and access control can mitigate harm even if the model is misaligned. Techniques from interpretability, uncertainty estimation, and safer design patterns can enhance the effectiveness of these mitigations. Finally, we briefly outline how these ingredients could be combined to produce safety cases for AGI systems.

cs.AI↗

arXiv2025

Evaluating Compositional Scene Understanding in Multimodal Generative Models

The visual world is fundamentally compositional. Visual scenes are defined by the composition of objects and their relations. Hence, it is essential for computer vision systems to reflect and exploit this compositionality to achieve robust and generalizable scene understanding. While major strides have been made toward the development of general-purpose, multimodal generative models, including both text-to-image models and multimodal vision-language models, it remains unclear whether these systems are capable of accurately generating and interpreting scenes involving the composition of multiple objects and relations. In this work, we present an evaluation of the compositional visual processing capabilities in the current generation of text-to-image (DALL-E 3) and multimodal vision-language models (GPT-4V, GPT-4o, Claude Sonnet 3.5, QWEN2-VL-72B, and InternVL2.5-38B), and compare the performance of these systems to human participants. The results suggest that these systems display some ability to solve compositional and relational tasks, showing notable improvements over the previous generation of multimodal models, but with performance nevertheless well below the level of human participants, particularly for more complex scenes involving many ($>5$) objects and multiple relations. These results highlight the need for further progress toward compositional understanding of visual scenes.

cs.CV↗

arXiv2025

A Framework for Evaluating Emerging Cyberattack Capabilities of AI

As frontier AI models become more capable, evaluating their potential to enable cyberattacks is crucial for ensuring the safe development of Artificial General Intelligence (AGI). Current cyber evaluation efforts are often ad-hoc, lacking systematic analysis of attack phases and guidance on targeted defenses. This work introduces a novel evaluation framework that addresses these limitations by: (1) examining the end-to-end attack chain, (2) identifying gaps in AI threat evaluation, and (3) helping defenders prioritize targeted mitigations and conduct AI-enabled adversary emulation for red teaming. Our approach adapts existing cyberattack chain frameworks for AI systems. We analyzed over 12,000 real-world instances of AI involvement in cyber incidents, catalogued by Google's Threat Intelligence Group, to curate seven representative attack chain archetypes. Through a bottleneck analysis on these archetypes, we pinpointed phases most susceptible to AI-driven disruption. We then identified and utilized externally developed cybersecurity model evaluations focused on these critical phases. We report on AI's potential to amplify offensive capabilities across specific attack stages, and offer recommendations for prioritizing defenses. We believe this represents the most comprehensive AI cyber risk evaluation framework published to date.

cs.CR↗

arXiv2024

Turbo your multi-modal classification with contrastive learning

Contrastive learning has become one of the most impressive approaches for multi-modal representation learning. However, previous multi-modal works mainly focused on cross-modal understanding, ignoring in-modal contrastive learning, which limits the representation of each modality. In this paper, we propose a novel contrastive learning strategy, called $Turbo$, to promote multi-modal understanding by joint in-modal and cross-modal contrastive learning. Specifically, multi-modal data pairs are sent through the forward pass twice with different hidden dropout masks to get two different representations for each modality. With these representations, we obtain multiple in-modal and cross-modal contrastive objectives for training. Finally, we combine the self-supervised Turbo with the supervised multi-modal classification and demonstrate its effectiveness on two audio-text classification tasks, where the state-of-the-art performance is achieved on a speech emotion recognition benchmark dataset.

cs.LG↗

arXiv2024

M$^{3}$V: A multi-modal multi-view approach for Device-Directed Speech Detection

With the goal of more natural and human-like interaction with virtual voice assistants, recent research in the field has focused on full duplex interaction mode without relying on repeated wake-up words. This requires that in scenes with complex sound sources, the voice assistant must classify utterances as device-oriented or non-device-oriented. The dual-encoder structure, which is jointly modeled by text and speech, has become the paradigm of device-directed speech detection. However, in practice, these models often produce incorrect predictions for unaligned input pairs due to the unavoidable errors of automatic speech recognition (ASR).To address this challenge, we propose M$^{3}$V, a multi-modal multi-view approach for device-directed speech detection, which frames we frame the problem as a multi-view learning task that introduces unimodal views and a text-audio alignment view in the network besides the multi-modal. Experimental results show that M$^{3}$V significantly outperforms models trained using only single or multi-modality and surpasses human judgment performance on ASR error data for the first time.

cs.SD↗

arXiv2024

DSCLAP: Domain-Specific Contrastive Language-Audio Pre-Training

Analyzing real-world multimodal signals is an essential and challenging task for intelligent voice assistants (IVAs). Mainstream approaches have achieved remarkable performance on various downstream tasks of IVAs with pre-trained audio models and text models. However, these models are pre-trained independently and usually on tasks different from target domains, resulting in sub-optimal modality representations for downstream tasks. Moreover, in many domains, collecting enough language-audio pairs is extremely hard, and transcribing raw audio also requires high professional skills, making it difficult or even infeasible to joint pre-training. To address these painpoints, we propose DSCLAP, a simple and effective framework that enables language-audio pre-training with only raw audio signal input. Specifically, DSCLAP converts raw audio signals into text via an ASR system and combines a contrastive learning objective and a language-audio matching objective to align the audio and ASR transcriptions. We pre-train DSCLAP on 12,107 hours of in-vehicle domain audio. Empirical results on two downstream tasks show that while conceptually simple, DSCLAP significantly outperforms the baseline models in all metrics, showing great promise for domain-specific IVAs applications.

cs.SD↗

arXiv2023

Lipid exchange promotes fusion of model protocells

Vesicle fusion is an important process underlying cell division, transport, and membrane trafficking. In phospholipid systems, a range of fusogens including divalent cations and depletants have been shown to induce adhesion, hemifusion, and then full content fusion between vesicles. This works shows that these fusogens do not perform the same function for fatty acid vesicles, which are used as model protocells (primitive cells). Even when fatty acid vesicles appear adhered or hemifused to each other, the intervening barriers between vesicles do not rupture. This difference is likely because fatty acids have a single aliphatic tail, and are more dynamic than their phospholipid counterparts. To address this, we postulate that fusion could instead occur under conditions, such as lipid exchange, that disrupt lipid packing. Using both experiments and molecular dynamics simulations, we verify that fusion in fatty acid systems can indeed be induced by lipid exchange. These results begin to probe how membrane biophysics could constrain the evolutionary dynamics of protocells.

cond-mat.soft↗

arXiv2023

Using holographic microscopy to measure the effect of confinement on crowding agents in lipid vesicles

The hydrodynamic effects of macromolecular crowding inside cells is often studied in vitro by using polymers as crowding reagents. Confinement of polymers inside cell-sized droplets has been shown to affect the diffusion of small molecules. Here we develop a method, based on digital holographic microscopy, to measure the diffusion of polystyrene microspheres that are confined within lipid vesicles containing a high concentration of solute. We apply the method to three solutes of varying complexity: sucrose, dextran, and PEG, prepared at ~7 % (w/w). We find that diffusion inside and outside the vesicles is the same when the solute is sucrose or dextran that is prepared below the critical overlap concentration. For polyethylene glycol, which is present at a concentration higher than the critical overlap concentration, the diffusion of microspheres inside vesicles is slower, hinting at the potential effects of confinement on crowding agents.

cond-mat.soft↗

arXiv2022

MAI-SIM: interferometric multicolor structured illumination microscopy for everybody

Sub-diffraction resolution, gentle sample illumination, and the possibility to image in multiple colors make Structured Illumination Microscopy (SIM) an imaging technique which is particularly well suited for live cell observations. Here, we present Machine learning Assisted Interferometric-SIM (MAI-SIM), an easy-to-implement method for high speed SIM imaging in multiple colors. The instrument is based on an interferometer design in which illumination patterns are generated, rotated, and stepped in phase through movement of a single galvanometric mirror element. The design is robust, flexible, and the pattern generation process works for all wavelengths. We complement the unique properties of interferometric SIM with a machine learning toolbox that is simple and efficient to use and is superior to existing methods for the reconstruction of super-resolved images recorded by the instrument. The framework permits real-time SIM reconstructions to be performed in multiple colors, providing the user with instant visualization of the super-resolved images. We demonstrate the capability of MAI-SIM on live biological samples and capture super-resolution images in multiple colors simultaneously over large fields of view. Finally, we embrace a fully open design philosophy to bring the advantages of MAI-SIM to as many users as possible and provide full details on system design and software.

physics.ins-det↗

arXiv2022

Measuring vesicle loading with holographic microscopy and bulk light scattering

We report efforts to quantify the loading of cell-sized lipid vesicles using in-line digital holographic microscopy. This method does not require fluorescent reporters, fluorescent tracers, or radioactive tracers. A single-color LED light source takes the place of conventional illumination to generate holograms rather than bright field images. By modelling the vesicle's scattering in a microscope with a Lorenz-Mie light scattering model, and comparing the results to data holograms, we are able to measure the vesicle's refractive index and thus loading. Performing the same comparison for bulk light scattering measurements enables retrieval of vesicle loading for nanoscale vesicles.

cond-mat.soft↗

arXiv2022

Deep learning-assisted imaging through stationary scattering media

Imaging through scattering media is a challenging problem owing to speckle decorrelations from perturbations in the media itself. For in-line imaging modalities, which are appealing because they are compact, require no moving parts, and are robust, negating the effects of such scattering becomes particularly challenging. Here we explore the effect of stationary scattering media on light scattering in in-line geometries, including digital holographic microscopy. We consider various object-scatterer scenarios where the object is distorted or obscured by additional stationary scatterers, and use an advanced deep learning (DL) generative methodology, generative adversarial networks (GANs), to mitigate the effects of the additional scatterers. Using light scattering simulations and experiments on objects of interest with and without additional scatterers, we find that conditional GANs can be quickly trained with minuscule datasets and can also efficiently learn the one-to-one statistical mapping between the cross-domain input-output image pairs. Training such a network yields a standalone model, that can be used later to inverse or negate the effect of scattering, yielding clear object reconstructions for object retrieval and downstream processing. Moreover, it is well-known that the coherent point spread function (c-PSF) of a stationary scattering optical system is a speckle pattern which is spatially shift variant. We show that with rapid training using only 20 image pairs, it is possible to negate this undesired scattering to accurately localize diffraction-limited impulses with high spatial accuracy, therefore transforming the earlier shift variant system to a linear shift invariant (LSI) system.

physics.optics↗

arXiv2020

Bulk self-assembly of giant, unilamellar vesicles

The desire to create cell-like models for fundamental science and applications has spurred extensive effort towards creating giant unilamellar vesicles (GUVs). However, a route to selectively self-assemble GUVs in bulk has remained elusive. In bulk solution, membrane-forming molecules such as phospholipids, single-tailed surfactants, and block copolymers typically self-assemble into multilamellar, onion-like structures. So although self-assembly processes can form nanoscale unilamellar vesicles, scaffolding by droplets or surfaces is required to create GUVs. Here we show that surprisingly, it is possible to bulk self-assemble cell-sized GUVs with almost complete selectivity over other vesicle topologies. The seemingly paradoxical pair of features that enables this appears to be having very dynamic molecules at the nanoscale, that create unusually rigid membranes. The resultant self-assembly pathway enables encapsulation of molecules and colloids, and can also generate model primitive cells that can grow and divide.

cond-mat.soft↗

arXiv2019

Attosecond Transient Absorption Spooktroscopy: a ghost imaging approach to ultrafast absorption spectroscopy

The recent demonstration of isolated attosecond pulses from an X-ray free-electron laser (XFEL) opens the possibility for probing ultrafast electron dynamics at X-ray wavelengths. An established experimental method for probing ultrafast dynamics is X-ray transient absorption spectroscopy, where the X-ray absorption spectrum is measured by scanning the central photon energy and recording the resultant photoproducts. The spectral bandwidth inherent to attosecond pulses is wide compared to the resonant features typically probed, which generally precludes the application of this technique in the attosecond regime. In this paper we propose and demonstrate a new technique to conduct transient absorption spectroscopy with broad bandwidth attosecond pulses with the aid of ghost imaging, recovering sub-bandwidth resolution in photoproduct-based absorption measurements.

physics.chem-ph↗