arXiv subjects

Nan Chen

Publications and source records attributed to Nan Chen.

At least 73 records · Page 4Linked to original sources

arXiv2025

A Dual-Core Model for ENSO Diversity: Unifying Model Hierarchies for Realistic Simulations

Despite advances in climate modeling, simulating the El Niño-Southern Oscillation (ENSO) remains challenging due to its spatiotemporal diversity and complexity. To address this, we build upon existing model hierarchies to develop a new unified modeling platform, which provides practical, scalable, and accurate tools for advancing ENSO research. Within this framework, we introduce a dual-core ENSO model (DCM) that integrates two widely used ENSO modeling approaches: a linear stochastic model confined to the equator and a nonlinear intermediate model extending off-equator. The stochastic model ensures computational efficiency and statistical accuracy. It captures essential ENSO characteristics and reproduces the observed non-Gaussian statistics. Meanwhile, the nonlinear model assimilates pseudo-observations from the stochastic model while resolving key air-sea interactions, such as feedback balances and spatial patterns of sea surface temperature anomalies (SSTA) during El Niño peaks and improving western-central Pacific SSTA magnitudes and spatial accuracy. The DCM effectively captures the realistic dynamical and statistical features of the ENSO diversity and complexity. Notably, the computational efficiency of the DCM facilitates a rapid generation of extended ENSO datasets, overcoming observational limitations. The outcome facilitates the analysis of long-term variations, advancing our understanding of ENSO and many other climate phenomena.

physics.ao-ph↗

arXiv2025

CustomContrast: A Multilevel Contrastive Perspective For Subject-Driven Text-to-Image Customization

Subject-driven text-to-image (T2I) customization has drawn significant interest in academia and industry. This task enables pre-trained models to generate novel images based on unique subjects. Existing studies adopt a self-reconstructive perspective, focusing on capturing all details of a single image, which will misconstrue the specific image's irrelevant attributes (e.g., view, pose, and background) as the subject intrinsic attributes. This misconstruction leads to both overfitting or underfitting of irrelevant and intrinsic attributes of the subject, i.e., these attributes are over-represented or under-represented simultaneously, causing a trade-off between similarity and controllability. In this study, we argue an ideal subject representation can be achieved by a cross-differential perspective, i.e., decoupling subject intrinsic attributes from irrelevant attributes via contrastive learning, which allows the model to focus more on intrinsic attributes through intra-consistency (features of the same subject are spatially closer) and inter-distinctiveness (features of different subjects have distinguished differences). Specifically, we propose CustomContrast, a novel framework, which includes a Multilevel Contrastive Learning (MCL) paradigm and a Multimodal Feature Injection (MFI) Encoder. The MCL paradigm is used to extract intrinsic features of subjects from high-level semantics to low-level appearance through crossmodal semantic contrastive learning and multiscale appearance contrastive learning. To facilitate contrastive learning, we introduce the MFI encoder to capture cross-modal representations. Extensive experiments show the effectiveness of CustomContrast in subject similarity and text controllability.

cs.CV↗

arXiv2024

Minimum Reduced-Order Models via Causal Inference

Constructing sparse, effective reduced-order models (ROMs) for high-dimensional dynamical data is an active area of research in applied sciences. In this work, we study an efficient approach to identifying such sparse ROMs using an information-theoretic indicator called causation entropy. Given a feature library of possible building block terms for the sought ROMs, the causation entropy ranks the importance of each term to the dynamics conveyed by the training data before a parameter estimation procedure is performed. It thus allows for an efficient construction of a hierarchy of ROMs with varying degrees of sparsity to effectively handle different tasks. This article examines the ability of the causation entropy to identify skillful sparse ROMs when a relatively high-dimensional ROM is required to emulate the dynamics conveyed by the training dataset. We demonstrate that a Gaussian approximation of the causation entropy still performs exceptionally well even in presence of highly non-Gaussian statistics. Such approximations provide an efficient way to access the otherwise hard to compute causation entropies when the selected feature library contains a large number of candidate functions. Besides recovering long-term statistics, we also demonstrate good performance of the obtained ROMs in recovering unobserved dynamics via data assimilation with partial observations, a test that has not been done before for causation-based ROMs of partial differential equations. The paradigmatic Kuramoto-Sivashinsky equation placed in a chaotic regime with highly skewed, multimodal statistics is utilized for these purposes.

math.DS↗

arXiv2024

Simulation and Data Assimilation in an Idealized Coupled Atmosphere-Ocean-Sea Ice Floe Model with Cloud Effects

Sea ice plays a crucial role in the climate system, particularly in the Marginal Ice Zone (MIZ), a transitional area consisting of fragmented ice between the open ocean and consolidated pack ice. As the MIZ expands, understanding its dynamics becomes essential for predicting climate change impacts. However, the role of clouds in these processes has been largely overlooked. This paper addresses that gap by developing an idealized coupled atmosphere-ocean-ice model incorporating cloud and precipitation effects, tackling both forward (simulation) and inverse (data assimilation) problems. Sea ice dynamics are modeled using the discrete element method, which simulates floes driven by atmospheric and oceanic forces. The ocean is represented by a two-layer quasi-geostrophic (QG) model, capturing mesoscale eddies and ice-ocean drag. The atmosphere is modeled using a two-layer saturated precipitating QG system, accounting for variable evaporation over sea surfaces and ice. Cloud cover affects radiation, influencing ice melting. The idealized coupled modeling framework allows us to study the interactions between atmosphere, ocean, and sea ice floes. Specifically, it focuses on how clouds and precipitation affect energy balance, melting, and freezing processes. It also serves as a testbed for data assimilation, which allows the recovery of unobserved floe trajectories and ocean fields in cloud-induced uncertainties. Numerical results show that appropriate reduced-order models help improve data assimilation efficiency with partial observations, allowing the skillful inference of missing floe trajectories and lower atmospheric winds. These results imply the potential of integrating idealized models with data assimilation to improve our understanding of Arctic dynamics and predictions.

physics.geo-ph↗

arXiv2024

Probabilistic Eddy Identification with Uncertainty Quantification

Mesoscale eddies are critical in ocean circulation and the global climate system. Standard eddy identification methods are usually based on deterministic optimal point estimates of the ocean flow field. However, uncertainty exists in estimating the flow field due to noisy, sparse, and indirect observations and turbulent flow models. Because of the intrinsic strong nonlinearity in the eddy identification diagnostics, even a small uncertainty in estimating the flow field can cause a significant error in the identified eddies. This paper presents a general probabilistic eddy identification framework that adapts existing identification methods to incorporate uncertainty into the diagnostic, emphasizing the interaction between the uncertainty in state estimation and the nonlinearity in diagnostics for affecting the identification results. The probabilistic eddy identification framework starts by sampling an ensemble of flow realizations from the probabilistic state estimation, followed by applying traditional nonlinear eddy diagnostics to individual realizations. The corresponding eddy statistics are then aggregated from the diagnostic results based on these realizations. The framework is applied to a scenario mimicking the Beaufort Gyre marginal ice zone, where large uncertainty appears in estimating the ocean field using Lagrangian data assimilation with sparse ice floe trajectories. The skills in counting the number of eddies and computing the probability of each eddy event are significantly improved under the probabilistic framework. Notably, incorporating the nonlinear propagation of uncertainty in diagnostics provides a more accurate mean estimate than standard deterministic methods in estimating eddy lifetime. It also facilitates uncertainty quantification in inferring such a crucial dynamical quantity.

math.DS↗

arXiv2024

FRAP: A Flexible Resource Accessing Protocol for Multiprocessor Real-Time Systems

Fully-partitioned fixed-priority scheduling (FP-FPS) multiprocessor systems are widely found in real-time applications, where spin-based protocols are often deployed to manage the mutually exclusive access of shared resources. Unfortunately, existing approaches either enforce rigid spin priority rules for resource accessing or carry significant pessimism in the schedulability analysis, imposing substantial blocking time regardless of task execution urgency or resource over-provisioning. This paper proposes FRAP, a spin-based flexible resource accessing protocol for FP-FPS systems. A task under FRAP can spin at any priority within a range for accessing a resource, allowing flexible and fine-grained resource control with predictable worst-case behaviour. Under flexible spinning, we demonstrate that the existing analysis techniques can lead to incorrect timing bounds and present a novel MCMF (minimum cost maximum flow)-based blocking analysis, providing predictability guarantee for FRAP. A spin priority assignment is reported that fully exploits flexible spinning to reduce the blocking time of tasks with high urgency, enhancing the performance of FRAP. Experimental results show that FRAP outperforms the existing spin-based protocols in schedulability by 15.20%-32.73% on average, up to 65.85%.

cs.OS↗

arXiv2024

VisEval: A Benchmark for Data Visualization in the Era of Large Language Models

Translating natural language to visualization (NL2VIS) has shown great promise for visual data analysis, but it remains a challenging task that requires multiple low-level implementations, such as natural language processing and visualization design. Recent advancements in pre-trained large language models (LLMs) are opening new avenues for generating visualizations from natural language. However, the lack of a comprehensive and reliable benchmark hinders our understanding of LLMs' capabilities in visualization generation. In this paper, we address this gap by proposing a new NL2VIS benchmark called VisEval. Firstly, we introduce a high-quality and large-scale dataset. This dataset includes 2,524 representative queries covering 146 databases, paired with accurately labeled ground truths. Secondly, we advocate for a comprehensive automated evaluation methodology covering multiple dimensions, including validity, legality, and readability. By systematically scanning for potential issues with a number of heterogeneous checkers, VisEval provides reliable and trustworthy evaluation outcomes. We run VisEval on a series of state-of-the-art LLMs. Our evaluation reveals prevalent challenges and delivers essential insights for future advancements.

cs.HC↗

arXiv2024

A Stochastic Precipitating Quasi-Geostrophic Model

Efficient and effective modeling of complex systems, incorporating cloud physics and precipitation, is essential for accurate climate modeling and forecasting. However, simulating these systems is computationally demanding since microphysics has crucial contributions to the dynamics of moisture and precipitation. In this paper, appropriate stochastic models are developed for the phase-transition dynamics of water, focusing on the precipitating quasi-geostrophic (PQG) model as a prototype. By treating the moisture, phase transitions, and latent heat release as integral components of the system, the PQG model constitutes a set of partial differential equations (PDEs) that involve Heaviside nonlinearities due to phase changes of water. Despite systematically characterizing the precipitation physics, expensive iterative algorithms are needed to find a PDE inversion at each numerical integration time step. As a crucial step toward building an effective stochastic model, a computationally efficient Markov jump process is designed to randomly simulate transitions between saturated and unsaturated states that avoids using the expensive iterative solver. The transition rates, which are deterministic, are derived from the physical fields, guaranteeing physical and statistical consistency with nature. Furthermore, to maintain the consistent spatial pattern of precipitation, the stochastic model incorporates an adaptive parameterization that automatically adjusts the transitions based on spatial information. Numerical tests show the stochastic model retains critical properties of the original PQG system while significantly reducing computational demands. It accurately captures observed precipitation patterns, including the spatial distribution and temporal variability of rainfall, alongside reproducing essential dynamic features such as potential vorticity fields and zonal mean flows.

physics.flu-dyn↗

arXiv2024

gFlora: a topology-aware method to discover functional co-response groups in soil microbial communities

We aim to learn the functional co-response group: a group of taxa whose co-response effect (the representative characteristic of the group showing the total topological abundance of taxa) co-responds (associates well statistically) to a functional variable. Different from the state-of-the-art method, we model the soil microbial community as an ecological co-occurrence network with the taxa as nodes (weighted by their abundance) and their relationships (a combination from both spatial and functional ecological aspects) as edges (weighted by the strength of the relationships). Then, we design a method called gFlora which notably uses graph convolution over this co-occurrence network to get the co-response effect of the group, such that the network topology is also considered in the discovery process. We evaluate gFlora on two real-world soil microbiome datasets (bacteria and nematodes) and compare it with the state-of-the-art method. gFlora outperforms this on all evaluation metrics, and discovers new functional evidence for taxa which were so far under-studied. We show that the graph convolution step is crucial to taxa with relatively low abundance (thus removing the bias towards taxa with higher abundance), and the discovered bacteria of different genera are distributed in the co-occurrence network but still tightly connected among themselves, demonstrating that topologically they fill different but collaborative functional roles in the ecological community.

cs.LG↗

arXiv2024

CGNSDE: Conditional Gaussian Neural Stochastic Differential Equation for Modeling Complex Systems and Data Assimilation

A new knowledge-based and machine learning hybrid modeling approach, called conditional Gaussian neural stochastic differential equation (CGNSDE), is developed to facilitate modeling complex dynamical systems and implementing analytic formulae of the associated data assimilation (DA). In contrast to the standard neural network predictive models, the CGNSDE is designed to effectively tackle both forward prediction tasks and inverse state estimation problems. The CGNSDE starts by exploiting a systematic causal inference via information theory to build a simple knowledge-based nonlinear model that nevertheless captures as much explainable physics as possible. Then, neural networks are supplemented to the knowledge-based model in a specific way, which not only characterizes the remaining features that are challenging to model with simple forms but also advances the use of analytic formulae to efficiently compute the nonlinear DA solution. These analytic formulae are used as an additional computationally affordable loss to train the neural networks that directly improve the DA accuracy. This DA loss function promotes the CGNSDE to capture the interactions between state variables and thus advances its modeling skills. With the DA loss, the CGNSDE is more capable of estimating extreme events and quantifying the associated uncertainty. Furthermore, crucial physical properties in many complex systems, such as the translate-invariant local dependence of state variables, can significantly simplify the neural network structures and facilitate the CGNSDE to be applied to high-dimensional systems. Numerical experiments based on chaotic systems with intermittency and strong non-Gaussian features indicate that the CGNSDE outperforms knowledge-based regression models, and the DA loss further enhances the modeling skills of the CGNSDE.

cs.LG↗

arXiv2024

Benchmarking Data Science Agents

In the era of data-driven decision-making, the complexity of data analysis necessitates advanced expertise and tools of data science, presenting significant challenges even for specialists. Large Language Models (LLMs) have emerged as promising aids as data science agents, assisting humans in data analysis and processing. Yet their practical efficacy remains constrained by the varied demands of real-world applications and complicated analytical process. In this paper, we introduce DSEval -- a novel evaluation paradigm, as well as a series of innovative benchmarks tailored for assessing the performance of these agents throughout the entire data science lifecycle. Incorporating a novel bootstrapped annotation method, we streamline dataset preparation, improve the evaluation coverage, and expand benchmarking comprehensiveness. Our findings uncover prevalent obstacles and provide critical insights to inform future advancements in the field.

cs.AI↗

arXiv2024

Integrating Active Learning in Causal Inference with Interference: A Novel Approach in Online Experiments

In the domain of causal inference research, the prevalent potential outcomes framework, notably the Rubin Causal Model (RCM), often overlooks individual interference and assumes independent treatment effects. This assumption, however, is frequently misaligned with the intricate realities of real-world scenarios, where interference is not merely a possibility but a common occurrence. Our research endeavors to address this discrepancy by focusing on the estimation of direct and spillover treatment effects under two assumptions: (1) network-based interference, where treatments on neighbors within connected networks affect one's outcomes, and (2) non-random treatment assignments influenced by confounders. To improve the efficiency of estimating potentially complex effects functions, we introduce an novel active learning approach: Active Learning in Causal Inference with Interference (ACI). This approach uses Gaussian process to flexibly model the direct and spillover treatment effects as a function of a continuous measure of neighbors' treatment assignment. The ACI framework sequentially identifies the experimental settings that demand further data. It further optimizes the treatment assignments under the network interference structure using genetic algorithms to achieve efficient learning outcome. By applying our method to simulation data and a Tencent game dataset, we demonstrate its feasibility in achieving accurate effects estimations with reduced data requirements. This ACI approach marks a significant advancement in the realm of data efficiency for causal inference, offering a robust and efficient alternative to traditional methodologies, particularly in scenarios characterized by complex interference patterns.

stat.ME↗

arXiv2024

A Causation-Based Computationally Efficient Strategy for Deploying Lagrangian Drifters to Improve Real-Time State Estimation

Deploying Lagrangian drifters that facilitate the state estimation of the underlying flow field within a future time interval is practically important. However, the uncertainty in estimating the flow field prevents using standard deterministic approaches for designing strategies and applying trajectory-wise skill scores to evaluate performance. In this paper an information measurement is developed to quantitatively assess the information gain in the estimated flow field by deploying an additional set of drifters. This information measurement is derived by exploiting causal inference. It is characterized by the inferred probability density function of the flow field, which naturally considers the uncertainty. Although the information measurement is an ideal theoretical metric, using it as the direct cost makes the optimization problem computationally expensive. To this end, an effective surrogate cost function is developed. It is highly efficient to compute while capturing the essential features of the information measurement when solving the optimization problem. Based upon these properties, a practical strategy for deploying drifter observations to improve future state estimation is designed. Due to the forecast uncertainty, the approach exploits the expected value of spatial maps of the surrogate cost associated with different forecast realizations to seek the optimal solution. Numerical experiments justify the effectiveness of the surrogate cost. The proposed strategy significantly outperforms the method by randomly deploying the drifters. It is also shown that, under certain conditions, the drifters determined by the expected surrogate cost remain skillful for the state estimation of a single forecast realization of the flow field as in reality.

math.DS↗

arXiv2024

A Physics-Informed Auto-Learning Framework for Developing Stochastic Conceptual Models for ENSO Diversity

Understanding ENSO dynamics has tremendously improved over the past decades. However, one aspect still poorly understood or represented in conceptual models is the ENSO diversity in spatial pattern, peak intensity, and temporal evolution. In this paper, a physics-informed auto-learning framework is developed to derive ENSO stochastic conceptual models with varying degrees of freedom. The framework is computationally efficient and easy to apply. Once the state vector of the target model is set, causal inference is exploited to build the right-hand side of the equations based on a mathematical function library. Fundamentally different from standard nonlinear regression, the auto-learning framework provides a parsimonious model by retaining only terms that improve the dynamical consistency with observations. It can also identify crucial latent variables and provide physical explanations. Exploiting a realistic six-dimensional reference recharge oscillator-based ENSO model, a hierarchy of three- to six-dimensional models is derived using the auto-learning framework and is systematically validated by a unified set of validation criteria assessing the dynamical and statistical features of the ENSO diversity. It is shown that the minimum model characterizing ENSO diversity is four-dimensional, with three interannual variables describing the western Pacific thermocline depth, the eastern and central Pacific sea surface temperatures (SSTs), and one intraseasonal variable for westerly wind events. Without the intraseasonal variable, the resulting three-dimensional model underestimates extreme events and is too regular. The limited number of weak nonlinearities in the model are essential in reproducing the observed extreme El Niños and nonlinear relationship between the eastern and western Pacific SSTs.

math.DS↗

arXiv2024

LEMDA: A Lagrangian-Eulerian Multiscale Data Assimilation Framework

Lagrangian trajectories are widely used as observations for recovering the underlying flow field via Lagrangian data assimilation (DA). However, the strong nonlinearity in the observational process and the high dimensionality of the problems often cause challenges in applying standard Lagrangian DA. In this paper, a Lagrangian-Eulerian multiscale DA (LEMDA) framework is developed. It starts with exploiting the Boltzmann kinetic description of the particle dynamics to derive a set of continuum equations, which characterize the statistical quantities of particle motions at fixed grids and serve as Eulerian observations. Despite the nonlinearity in the continuum equations and the processes of Lagrangian observations, the time evolutions of the posterior distribution from LEMDA can be written down using closed analytic formulae. This offers an exact and efficient way of carrying out DA, which avoids using ensemble approximations and the associated tunings. The analytically solvable properties also facilitate the derivation of an effective reduced-order Lagrangian DA scheme that further enhances computational efficiency. The Lagrangian DA within the framework has advantages when a moderate number of particles is used, while the Eulerian DA can effectively save computational costs when the number of particle observations becomes large. The Eulerian DA is also valuable when particles collide, such as using sea ice floe trajectories as observations. LEMDA naturally applies to multiscale turbulent flow fields, where the Eulerian DA recovers the large-scale structures, and the Lagrangian DA efficiently resolves the small-scale features in each grid cell via parallel computing. Numerical experiments demonstrate the skilful results of LEMDA and its two components.

physics.flu-dyn↗

arXiv2023

FlowHON: Representing Flow Fields Using Higher-Order Networks

Flow fields are often partitioned into data blocks for massively parallel computation and analysis based on blockwise relationships. However, most of the previous techniques only consider the first-order dependencies among blocks, which is insufficient in describing complex flow patterns. In this work, we present FlowHON, an approach to construct higher-order networks (HONs) from flow fields. FlowHON captures the inherent higher-order dependencies in flow fields as nodes and estimates the transitions among them as edges. We formulate the HON construction as an optimization problem with three linear transformations. The first two layers correspond to the node generation and the third one corresponds to edge estimation. Our formulation allows the node generation and edge estimation to be solved in a unified framework. With FlowHON, the rich set of traditional graph algorithms can be applied without any modification to analyze flow fields, while leveraging the higher-order information to understand the inherent structure and manage flow data for efficiency. We demonstrate the effectiveness of FlowHON using a series of downstream tasks, including estimating the density of particles during tracing, partitioning flow fields for data management, and understanding flow fields using the node-link diagram representation of networks.

cs.LG↗

arXiv2023

Swashplateless-elevon Actuation for a Dual-rotor Tail-sitter VTOL UAV

In this paper, we propose a novel swashplateless-elevon actuation (SEA) for dual-rotor tail-sitter vertical takeoff and landing (VTOL) unmanned aerial vehicles (UAVs). In contrast to the conventional elevon actuation (CEA) which controls both pitch and yaw using elevons, the SEA adopts swashplateless mechanisms to generate an extra moment through motor speed modulation to control pitch and uses elevons solely for controlling yaw, without requiring additional actuators. This decoupled control strategy mitigates the saturation of elevons' deflection needed for large pitch and yaw control actions, thus improving the UAV's control performance on trajectory tracking and disturbance rejection performance in the presence of large external disturbances. Furthermore, the SEA overcomes the actuation degradation issues experienced by the CEA when the UAV is in close proximity to the ground, leading to a smoother and more stable take-off process. We validate and compare the performances of the SEA and the CEA in various real-world flight conditions, including take-off, trajectory tracking, and hover flight and position steps under external disturbance. Experimental results demonstrate that the SEA has better performances than the CEA. Moreover, we verify the SEA's feasibility in the attitude transition process and fixed-wing-mode flight of the VTOL UAV. The results indicate that the SEA can accurately control pitch in the presence of high-speed incoming airflow and maintain a stable attitude during fixed-wing mode flight. Video of all experiments can be found in youtube.com/watch?v=Sx9Rk4Zf7sQ

cs.RO↗

arXiv2023

A Survey of Imbalanced Learning on Graphs: Problems, Techniques, and Future Directions

Graphs represent interconnected structures prevalent in a myriad of real-world scenarios. Effective graph analytics, such as graph learning methods, enables users to gain profound insights from graph data, underpinning various tasks including node classification and link prediction. However, these methods often suffer from data imbalance, a common issue in graph data where certain segments possess abundant data while others are scarce, thereby leading to biased learning outcomes. This necessitates the emerging field of imbalanced learning on graphs, which aims to correct these data distribution skews for more accurate and representative learning outcomes. In this survey, we embark on a comprehensive review of the literature on imbalanced learning on graphs. We begin by providing a definitive understanding of the concept and related terminologies, establishing a strong foundational understanding for readers. Following this, we propose two comprehensive taxonomies: (1) the problem taxonomy, which describes the forms of imbalance we consider, the associated tasks, and potential solutions; (2) the technique taxonomy, which details key strategies for addressing these imbalances, and aids readers in their method selection process. Finally, we suggest prospective future directions for both problems and techniques within the sphere of imbalanced learning on graphs, fostering further innovation in this critical area.

cs.LG↗