arXiv subjects

Andrea L. Bertozzi

Publications and source records attributed to Andrea L. Bertozzi.

At least 37 records · Page 2Linked to original sources

arXiv2024

Narrative Analysis of True Crime Podcasts With Knowledge Graph-Augmented Large Language Models

Narrative data spans all disciplines and provides a coherent model of the world to the reader or viewer. Recent advancement in machine learning and Large Language Models (LLMs) have enable great strides in analyzing natural language. However, Large language models (LLMs) still struggle with complex narrative arcs as well as narratives containing conflicting information. Recent work indicates LLMs augmented with external knowledge bases can improve the accuracy and interpretability of the resulting models. In this work, we analyze the effectiveness of applying knowledge graphs (KGs) in understanding true-crime podcast data from both classical Natural Language Processing (NLP) and LLM approaches. We directly compare KG-augmented LLMs (KGLLMs) with classical methods for KG construction, topic modeling, and sentiment analysis. Additionally, the KGLLM allows us to query the knowledge base in natural language and test its ability to factually answer questions. We examine the robustness of the model to adversarial prompting in order to test the model's ability to deal with conflicting information. Finally, we apply classical methods to understand more subtle aspects of the text such as the use of hearsay and sentiment in narrative construction and propose future directions. Our results indicate that KGLLMs outperform LLMs on a variety of metrics, are more robust to adversarial prompts, and are more capable of summarizing the text into topics.

cs.CL↗

arXiv2024

Detection and tracking of gas plumes in LWIR hyperspectral video sequence data

Automated detection of chemical plumes presents a segmentation challenge. The segmentation problem for gas plumes is difficult due to the diffusive nature of the cloud. The advantage of considering hyperspectral images in the gas plume detection problem over the conventional RGB imagery is the presence of non-visual data, allowing for a richer representation of information. In this paper we present an effective method of visualizing hyperspectral video sequences containing chemical plumes and investigate the effectiveness of segmentation techniques on these post-processed videos. Our approach uses a combination of dimension reduction and histogram equalization to prepare the hyperspectral videos for segmentation. First, Principal Components Analysis (PCA) is used to reduce the dimension of the entire video sequence. This is done by projecting each pixel onto the first few Principal Components resulting in a type of spectral filter. Next, a Midway method for histogram equalization is used. These methods redistribute the intensity values in order to reduce flicker between frames. This properly prepares these high-dimensional video sequences for more traditional segmentation techniques. We compare the ability of various clustering techniques to properly segment the chemical plume. These include K-means, spectral clustering, and the Ginzburg-Landau functional.

cs.CV↗

arXiv2024

A comparative study of dynamic models for gravity-driven particle-laden flows

The dynamics of viscous thin-film particle-laden flows down inclined surfaces are commonly modeled with one of two approaches: a diffusive flux model or a suspension balance model. The diffusive flux model assumes that the particles migrate via a diffusive flux induced by gradients in both the particle concentration and the effective suspension viscosity. The suspension balance model introduces non-Newtonian bulk stress with shear-induced normal stresses, the gradients of which cause particle migration. Both models have appeared in the literature of particle-laden flow with virtually no comparison between the two models. For particle-laden viscous flow on an incline, in a thin-film geometry, one can use lubrication theory to derive a compact dynamic model in the form of a $2\times 2$ system of conservation laws. We can then directly compare the two theories side by side by looking at similarities and differences in the flux functions for the conservation laws, and in exact and numerical simulations of the equations. We compare the flux profiles over a range of parameters, showing fairly good agreement between the models, with the biggest difference involving the behavior at the free surface. We also consider less dense suspensions at lower inclination angles where the dynamics involve two shock waves that can be clearly measured in experiments. In this context the solutions differ by no more than about 10%, suggesting that either model could be used for this configuration.

physics.flu-dyn↗

arXiv2024

Equilibrium theory of bidensity particle-laden suspensions in thin-film flow down a spiral separator

Spiral gravity separators are designed to separate multi-species slurry components based on differences in density and size. Previous studies have investigated steady-state solutions for mixtures of liquids and single particle species in thin-film flows. However, these models are constrained to single-species systems and cannot describe the dynamics of multi-species separation. In contrast, our analysis extends to mixtures containing two particle species of differing densities, revealing that they undergo radial separation, which is an essential mechanism for practical applications in separating particles of varying densities. This work models gravity-driven bidensity slurries in a spiral trough by incorporating particle interactions, using empirically derived formulas for particle fluxes from previous bidensity studies on inclined planes. Specifically, we study a thin-film bidensity slurry flowing down a rectangular channel helically wound around a vertical axis. Through a thin-film approximation, we derive equilibrium profiles for the concentration of each particle species and the fluid depth. Additionally, we analyze the influence of key design parameters, such as spiral radius and channel width, on particle concentration profiles. Our findings provide valuable insights into optimizing spiral separator designs for enhanced applicability and adaptability.

physics.flu-dyn↗

arXiv2024

Optimal policy for control of epidemics with constrained time intervals and region-based interactions

We introduce a policy model coupled with the susceptible-infected-recovered (SIR) epidemic model to study interactions between policy-making and the dynamics of epidemics. We consider both single-region policies, as well as game-theoretic models involving interactions among several regions, and hierarchical interactions among policy-makers modeled as multi-layer games. We assume that the policy functions are piece-wise constant with a minimum time interval for each policy stage, considering policies cannot change frequently in time or they cannot be easily followed. The optimal policy is obtained by minimizing a cost function which consists of an implementation cost, an impact cost, and, in the case of multi-layer games, a non-compliance cost. We show in a case study of COVID-19 in France that when the cost function is reduced to the impact cost and is parameterized as the final epidemic size, the solution approximates that of the optimal control in Bliman et al, J. Optim. Theory Appl., 189, 2021, for sufficiently small minimum policy time interval. For a larger time interval however the optimal policy is a step down function, quite different from the step up structure typically deployed during the COVID-19 pandemic. In addition, we present a counterfactual study of how the pandemic would have evolved if herd immunity was reached during the second wave in the county of Los Angeles, California. Lastly, we study a case of three interacting counties with and without a governing state.

math.OC↗

arXiv2024

A Primal-Dual Framework for Transformers and Neural Networks

Self-attention is key to the remarkable success of transformers in sequence modeling tasks including many applications in natural language processing and computer vision. Like neural network layers, these attention mechanisms are often developed by heuristics and experience. To provide a principled framework for constructing attention layers in transformers, we show that the self-attention corresponds to the support vector expansion derived from a support vector regression problem, whose primal formulation has the form of a neural network layer. Using our framework, we derive popular attention layers used in practice and propose two new attentions: 1) the Batch Normalized Attention (Attention-BN) derived from the batch normalization layer and 2) the Attention with Scaled Head (Attention-SH) derived from using less training data to fit the SVR model. We empirically demonstrate the advantages of the Attention-BN and Attention-SH in reducing head redundancy, increasing the model's accuracy, and improving the model's efficiency in a variety of practical applications including image and time-series classification.

cs.LG↗

arXiv2023

AutoKG: Efficient Automated Knowledge Graph Generation for Language Models

Traditional methods of linking large language models (LLMs) to knowledge bases via the semantic similarity search often fall short of capturing complex relational dynamics. To address these limitations, we introduce AutoKG, a lightweight and efficient approach for automated knowledge graph (KG) construction. For a given knowledge base consisting of text blocks, AutoKG first extracts keywords using a LLM and then evaluates the relationship weight between each pair of keywords using graph Laplace learning. We employ a hybrid search scheme combining vector similarity and graph-based associations to enrich LLM responses. Preliminary experiments demonstrate that AutoKG offers a more comprehensive and interconnected knowledge retrieval mechanism compared to the semantic similarity search, thereby enhancing the capabilities of LLMs in generating more insightful and relevant outputs.

cs.CL↗

arXiv2023

Hate speech and hate crimes: a data-driven study of evolving discourse around marginalized groups

This study explores the dynamic relationship between online discourse, as observed in tweets, and physical hate crimes, focusing on marginalized groups. Leveraging natural language processing techniques, including keyword extraction and topic modeling, we analyze the evolution of online discourse after events affecting these groups. Examining sentiment and polarizing tweets, we establish correlations with hate crimes in Black and LGBTQ+ communities. Using a knowledge graph, we connect tweets, users, topics, and hate crimes, enabling network analyses. Our findings reveal divergent patterns in the evolution of user communities for Black and LGBTQ+ groups, with notable differences in sentiment among influential users. This analysis sheds light on distinctive online discourse patterns and emphasizes the need to monitor hate speech to prevent hate crimes, especially following significant events impacting marginalized communities.

cs.SI↗

arXiv2023

Fluid dynamics alters liquid-liquid phase separation in confined aqueous two-phase systems

Liquid-liquid phase separation is key to understanding aqueous two-phase systems (ATPS) arising throughout cell biology, medical science, and the pharmaceutical industry. Controlling the detailed morphology of phase-separating compound droplets leads to new technologies for efficient single-cell analysis, targeted drug delivery, and effective cell scaffolds for wound healing. We present a computational model of liquid-liquid phase separation relevant to recent laboratory experiments with gelatin-polyethylene glycol mixtures. We include buoyancy and surface-tension-driven finite viscosity fluid dynamics with thermally induced phase separation. We show that the fluid dynamics greatly alters the evolution and equilibria of the phase separation problem. Notably, buoyancy plays a critical role in driving the ATPS to energy-minimizing crescent-shaped morphologies and shear flows can generate a tenfold speedup in particle formation. Neglecting fluid dynamics produces incorrect minimum-energy droplet shapes. The model allows for optimization of current manufacturing procedures for structured microparticles and improves understanding of ATPS evolution in confined and flowing settings important in biology and biotechnology.

cond-mat.soft↗

arXiv2023

A positivity-preserving numerical method for a thin liquid film on a vertical cylindrical fiber

When a thin liquid film flows down on a vertical fiber, one can observe the complex and captivating interfacial dynamics of an unsteady flow. Such dynamics are applicable in various fluid experiments due to their high surface area-to-volume ratio. Recent studies verified that when the flow undergoes regime transitions, the magnitude of the film thickness changes dramatically, making numerical simulations challenging. In this paper, we present a computationally efficient numerical method that can maintain the positivity of the film thickness as well as conserve the volume of the fluid under the coarse mesh setting. A series of comparisons to laboratory experiments and previously proposed numerical methods supports the validity of our numerical method. We also prove that our method is second-order consistent in space and satisfies the entropy estimate.

math.NA↗

arXiv2023

Novel Batch Active Learning Approach and Its Application to Synthetic Aperture Radar Datasets

Active learning improves the performance of machine learning methods by judiciously selecting a limited number of unlabeled data points to query for labels, with the aim of maximally improving the underlying classifier's performance. Recent gains have been made using sequential active learning for synthetic aperture radar (SAR) data arXiv:2204.00005. In each iteration, sequential active learning selects a query set of size one while batch active learning selects a query set of multiple datapoints. While batch active learning methods exhibit greater efficiency, the challenge lies in maintaining model accuracy relative to sequential active learning methods. We developed a novel, two-part approach for batch active learning: Dijkstra's Annulus Core-Set (DAC) for core-set generation and LocalMax for batch sampling. The batch active learning process that combines DAC and LocalMax achieves nearly identical accuracy as sequential active learning but is more efficient, proportional to the batch size. As an application, a pipeline is built based on transfer learning feature embedding, graph learning, DAC, and LocalMax to classify the FUSAR-Ship and OpenSARShip datasets. Our pipeline outperforms the state-of-the-art CNN-based methods.

cs.LG↗

arXiv2023

An electro-hydrodynamics modeling of droplet actuation on solid surface by surfactant-mediated electro-dewetting

We propose an electro-hydrodynamics model to describe the dynamic evolution of a slender drop containing a dilute ionic surfactant on a naturally wettable surface, with a varying external electric field. This unified model reproduces fundamental microfluidic operations controlled by electrical signals, including dewetting, rewetting, and droplet shifting. In this paper, lubrication theory analysis and numerical simulations illustrate how to electrically control the wettability of surface via the charged surfactant. Our numerical results show that electric field promotes dewetting by attracting ionic surfactants onto the transition thin-film region and promotes rewetting by attracting them away from the region.

physics.flu-dyn↗

arXiv2023

Graph-based Active Learning for Surface Water and Sediment Detection in Multispectral Images

We develop a graph active learning pipeline (GAP) to detect surface water and in-river sediment pixels in satellite images. The active learning approach is applied within the training process to optimally select specific pixels to generate a hand-labeled training set. Our method obtains higher accuracy with far fewer training pixels than both standard and deep learning models. According to our experiments, our GAP trained on a set of 3270 pixels reaches a better accuracy than the neural network method trained on 2.1 million pixels.

eess.IV↗

arXiv2023

Active Learning of Non-semantic Speech Tasks with Pretrained Models

Pretraining neural networks with massive unlabeled datasets has become popular as it equips the deep models with a better prior to solve downstream tasks. However, this approach generally assumes that the downstream tasks have access to annotated data of sufficient size. In this work, we propose ALOE, a novel system for improving the data- and label-efficiency of non-semantic speech tasks with active learning. ALOE uses pretrained models in conjunction with active learning to label data incrementally and learn classifiers for downstream tasks, thereby mitigating the need to acquire labeled data beforehand. We demonstrate the effectiveness of ALOE on a wide range of tasks, uncertainty-based acquisition functions, and model architectures. Training a linear classifier on top of a frozen encoder with ALOE is shown to achieve performance similar to several baselines that utilize the entire labeled data.

cs.SD↗

arXiv2022

Proximal Implicit ODE Solvers for Accelerating Learning Neural ODEs

Learning neural ODEs often requires solving very stiff ODE systems, primarily using explicit adaptive step size ODE solvers. These solvers are computationally expensive, requiring the use of tiny step sizes for numerical stability and accuracy guarantees. This paper considers learning neural ODEs using implicit ODE solvers of different orders leveraging proximal operators. The proximal implicit solver consists of inner-outer iterations: the inner iterations approximate each implicit update step using a fast optimization algorithm, and the outer iterations solve the ODE system over time. The proximal implicit ODE solver guarantees superiority over explicit solvers in numerical stability and computational efficiency. We validate the advantages of proximal implicit solvers over existing popular neural ODE solvers on various challenging benchmark tasks, including learning continuous-depth graph neural networks and continuous normalizing flows.

math.NA↗

arXiv2022

Graph-based Active Learning for Semi-supervised Classification of SAR Data

We present a novel method for classification of Synthetic Aperture Radar (SAR) data by combining ideas from graph-based learning and neural network methods within an active learning framework. Graph-based methods in machine learning are based on a similarity graph constructed from the data. When the data consists of raw images composed of scenes, extraneous information can make the classification task more difficult. In recent years, neural network methods have been shown to provide a promising framework for extracting patterns from SAR images. These methods, however, require ample training data to avoid overfitting. At the same time, such training data are often unavailable for applications of interest, such as automatic target recognition (ATR) and SAR data. We use a Convolutional Neural Network Variational Autoencoder (CNNVAE) to embed SAR data into a feature space, and then construct a similarity graph from the embedded data and apply graph-based semi-supervised learning techniques. The CNNVAE feature embedding and graph construction requires no labeled data, which reduces overfitting and improves the generalization performance of graph learning at low label rates. Furthermore, the method easily incorporates a human-in-the-loop for active learning in the data-labeling process. We present promising results and compare them to other standard machine learning methods on the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset for ATR with small amounts of labeled data.

cs.LG↗

arXiv2022

Dynamics of small particle inertial migration in curved square ducts

Microchannels are well-known in microfluidic applications for the control and separation of microdroplets and cells. Often the objects in the flow experience inertial effects, resulting in dynamics that is a departure from the underlying channel flow dynamics. This paper considers small neutrally buoyant spherical particles suspended in flow through a curved duct having a square cross-section. The particle experiences a combination of inertial lift force induced by the disturbance from the primary flow along the duct, and drag from the secondary vortices in the cross-section, which drive migration of the particle within the cross-section. We construct a simplified model that preserves the core topology of the force field yet depends on a single parameter $κ$, quantifying the relative strength of the two forces. We show that $κ$ is a bifurcation parameter for the dynamical system that describes motion of the particle in the cross section of the duct. At large values of $κ$ there exists an attracting limit cycle, in each of the upper and lower halves of the duct. At small $κ$ we find that particles migrate to one of four stable foci. Between these extremes, there is an intermediate-range of $κ$ for which all particles migrate to a single stable focus. Noting that the positions of the limit cycles and foci vary with the value of $κ$, this behavior indicates that, for a suitable particle mixture, duct bend radius might be chosen to segregate particles by size. We evaluate the time and axial distance required to focus particles near the unique stable node, which determines the duct length required for particle segregation.

physics.flu-dyn↗

arXiv2021

Efficient and Reliable Overlay Networks for Decentralized Federated Learning

We propose near-optimal overlay networks based on $d$-regular expander graphs to accelerate decentralized federated learning (DFL) and improve its generalization. In DFL a massive number of clients are connected by an overlay network, and they solve machine learning problems collaboratively without sharing raw data. Our overlay network design integrates spectral graph theory and the theoretical convergence and generalization bounds for DFL. As such, our proposed overlay networks accelerate convergence, improve generalization, and enhance robustness to clients failures in DFL with theoretical guarantees. Also, we present an efficient algorithm to convert a given graph to a practical overlay network and maintaining the network topology after potential client failures. We numerically verify the advantages of DFL with our proposed networks on various benchmark tasks, ranging from image classification to language modeling using hundreds of clients.

cs.NI↗