arXiv subjects

Mustafa Mustafa

Publications and source records attributed to Mustafa Mustafa.

17 recordsLinked to original sources

arXiv2026

VulReaD: Knowledge-Graph-guided Software Vulnerability Reasoning and Detection

Software vulnerability detection (SVD) is a critical challenge in modern systems. Large language models (LLMs) offer natural-language explanations alongside predictions, but most work focuses on binary evaluation, and explanations often lack semantic consistency with Common Weakness Enumeration (CWE) categories. We propose VulReaD, a knowledge-graph-guided approach for vulnerability reasoning and detection that moves beyond binary classification toward CWE-level reasoning. VulReaD leverages a security knowledge graph (KG) as a semantic backbone and uses a strong teacher LLM to generate CWE-consistent contrastive reasoning supervision, enabling student model training without manual annotations. Students are fine-tuned with Odds Ratio Preference Optimization (ORPO) to encourage taxonomy-aligned reasoning while suppressing unsupported explanations. Across three real-world datasets, VulReaD improves binary F1 by 8-10% and multi-class classification by 30% Macro-F1 and 18% Micro-F1 compared to state-of-the-art baselines. Results show that LLMs outperform deep learning baselines in binary detection and that KG-guided reasoning enhances CWE coverage and interpretability.

cs.SE↗

arXiv2021

Fast, high-fidelity Lyman $\alpha$ forests with convolutional neural networks

Full-physics cosmological simulations are powerful tools for studying the formation and evolution of structure in the universe but require extreme computational resources. Here, we train a convolutional neural network to use a cheaper N-body-only simulation to reconstruct the baryon hydrodynamic variables (density, temperature, and velocity) on scales relevant to the Lyman-$\alpha$ (Ly$\alpha$) forest, using data from Nyx simulations. We show that our method enables rapid estimation of these fields at a resolution of $\sim$20kpc, and captures the statistics of the Ly$\alpha$ forest with much greater accuracy than existing approximations. Because our model is fully-convolutional, we can train on smaller simulation boxes and deploy on much larger ones, enabling substantial computational savings. Furthermore, as our method produces an approximation for the hydrodynamic fields instead of Ly$\alpha$ flux directly, it is not limited to a particular choice of ionizing background or mean transmitted flux.

astro-ph.CO↗

arXiv2021

Towards physically consistent data-driven weather forecasting: Integrating data assimilation with equivariance-preserving deep spatial transformers

There is growing interest in data-driven weather prediction (DDWP), for example using convolutional neural networks such as U-NETs that are trained on data from models or reanalysis. Here, we propose 3 components to integrate with commonly used DDWP models in order to improve their physical consistency and forecast accuracy. These components are 1) a deep spatial transformer added to the latent space of the U-NETs to preserve a property called equivariance, which is related to correctly capturing rotations and scalings of features in spatio-temporal data, 2) a data-assimilation (DA) algorithm to ingest noisy observations and improve the initial conditions for next forecasts, and 3) a multi-time-step algorithm, which combines forecasts from DDWP models with different time steps through DA, improving the accuracy of forecasts at short intervals. To show the benefit/feasibility of each component, we use geopotential height at 500~hPa (Z500) from ERA5 reanalysis and examine the short-term forecast accuracy of specific setups of the DDWP framework. Results show that the equivariance-preserving networks (U-STNs) clearly outperform the U-NETs, for example improving the forecast skill by $45\%$. Using a sigma-point ensemble Kalman (SPEnKF) algorithm for DA and U-STN as the forward model, we show that stable, accurate DA cycles are achieved even with high observation noise. The DDWP+DA framework substantially benefits from large ($O(1000)$) ensembles that are inexpensively generated with the data-driven forward model in each DA cycle. The multi-time-step DDWP+DA framework also shows promises, e.g., it reduces the average error by factors of 2-3.

physics.ao-ph↗

arXiv2021

Estimating Galactic Distances From Images Using Self-supervised Representation Learning

We use a contrastive self-supervised learning framework to estimate distances to galaxies from their photometric images. We incorporate data augmentations from computer vision as well as an application-specific augmentation accounting for galactic dust. We find that the resulting visual representations of galaxy images are semantically useful and allow for fast similarity searches, and can be successfully fine-tuned for the task of redshift estimation. We show that (1) pretraining on a large corpus of unlabeled data followed by fine-tuning on some labels can attain the accuracy of a fully-supervised model which requires 2-4x more labeled data, and (2) that by fine-tuning our self-supervised representations using all available data labels in the Main Galaxy Sample of the Sloan Digital Sky Survey (SDSS), we outperform the state-of-the-art supervised learning method.

astro-ph.IM↗

arXiv2020

Self-Supervised Representation Learning for Astronomical Images

Sky surveys are the largest data generators in astronomy, making automated tools for extracting meaningful scientific information an absolute necessity. We show that, without the need for labels, self-supervised learning recovers representations of sky survey images that are semantically useful for a variety of scientific tasks. These representations can be directly used as features, or fine-tuned, to outperform supervised methods trained only on labeled data. We apply a contrastive learning framework on multi-band galaxy photometry from the Sloan Digital Sky Survey (SDSS) to learn image representations. We then use them for galaxy morphology classification, and fine-tune them for photometric redshift estimation, using labels from the Galaxy Zoo 2 dataset and SDSS spectroscopy. In both downstream tasks, using the same learned representations, we outperform the supervised state-of-the-art results, and we show that our approach can achieve the accuracy of supervised models while using 2-4 times fewer labels for training.

astro-ph.IM↗

arXiv2020

Using Machine Learning to Augment Coarse-Grid Computational Fluid Dynamics Simulations

Simulation of turbulent flows at high Reynolds number is a computationally challenging task relevant to a large number of engineering and scientific applications in diverse fields such as climate science, aerodynamics, and combustion. Turbulent flows are typically modeled by the Navier-Stokes equations. Direct Numerical Simulation (DNS) of the Navier-Stokes equations with sufficient numerical resolution to capture all the relevant scales of the turbulent motions can be prohibitively expensive. Simulation at lower-resolution on a coarse-grid introduces significant errors. We introduce a machine learning (ML) technique based on a deep neural network architecture that corrects the numerical errors induced by a coarse-grid simulation of turbulent flows at high-Reynolds numbers, while simultaneously recovering an estimate of the high-resolution fields. Our proposed simulation strategy is a hybrid ML-PDE solver that is capable of obtaining a meaningful high-resolution solution trajectory while solving the system PDE at a lower resolution. The approach has the potential to dramatically reduce the expense of turbulent flow simulations. As a proof-of-concept, we demonstrate our ML-PDE strategy on a two-dimensional turbulent (Rayleigh Number $Ra=10^9$) Rayleigh-B\'enard Convection (RBC) problem.

physics.comp-ph↗

arXiv2020

ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning

We introduce ADAHESSIAN, a second order stochastic optimization algorithm which dynamically incorporates the curvature of the loss function via ADAptive estimates of the HESSIAN. Second order algorithms are among the most powerful optimization algorithms with superior convergence properties as compared to first order methods such as SGD and Adam. The main disadvantage of traditional second order methods is their heavier per iteration computation and poor accuracy as compared to first order methods. To address these, we incorporate several novel approaches in ADAHESSIAN, including: (i) a fast Hutchinson based method to approximate the curvature matrix with low computational overhead; (ii) a root-mean-square exponential moving average to smooth out variations of the Hessian diagonal across different iterations; and (iii) a block diagonal averaging to reduce the variance of Hessian diagonal elements. We show that ADAHESSIAN achieves new state-of-the-art results by a large margin as compared to other adaptive optimization methods, including variants of Adam. In particular, we perform extensive tests on CV, NLP, and recommendation system tasks and find that ADAHESSIAN: (i) achieves 1.80%/1.45% higher accuracy on ResNets20/32 on Cifar10, and 5.55% higher accuracy on ImageNet as compared to Adam; (ii) outperforms AdamW for transformers by 0.13/0.33 BLEU score on IWSLT14/WMT14 and 2.7/1.0 PPL on PTB/Wikitext-103; (iii) outperforms AdamW for SqueezeBert by 0.41 points on GLUE; and (iv) achieves 0.032% better score than Adagrad for DLRM on the Criteo Ad Kaggle dataset. Importantly, we show that the cost per iteration of ADAHESSIAN is comparable to first order methods, and that it exhibits robustness towards its hyperparameters.

cs.LG↗

arXiv2020

MeshfreeFlowNet: A Physics-Constrained Deep Continuous Space-Time Super-Resolution Framework

We propose MeshfreeFlowNet, a novel deep learning-based super-resolution framework to generate continuous (grid-free) spatio-temporal solutions from the low-resolution inputs. While being computationally efficient, MeshfreeFlowNet accurately recovers the fine-scale quantities of interest. MeshfreeFlowNet allows for: (i) the output to be sampled at all spatio-temporal resolutions, (ii) a set of Partial Differential Equation (PDE) constraints to be imposed, and (iii) training on fixed-size inputs on arbitrarily sized spatio-temporal domains owing to its fully convolutional encoder. We empirically study the performance of MeshfreeFlowNet on the task of super-resolution of turbulent flows in the Rayleigh-Benard convection problem. Across a diverse set of evaluation metrics, we show that MeshfreeFlowNet significantly outperforms existing baselines. Furthermore, we provide a large scale implementation of MeshfreeFlowNet and show that it efficiently scales across large clusters, achieving 96.80% scaling efficiency on up to 128 GPUs and a training time of less than 4 minutes.

cs.LG↗

arXiv2019

Towards Physics-informed Deep Learning for Turbulent Flow Prediction

While deep learning has shown tremendous success in a wide range of domains, it remains a grand challenge to incorporate physical principles in a systematic manner to the design, training, and inference of such models. In this paper, we aim to predict turbulent flow by learning its highly nonlinear dynamics from spatiotemporal velocity fields of large-scale fluid flow simulations of relevance to turbulence modeling and climate modeling. We adopt a hybrid approach by marrying two well-established turbulent flow simulation techniques with deep learning. Specifically, we introduce trainable spectral filters in a coupled model of Reynolds-averaged Navier-Stokes (RANS) and Large Eddy Simulation (LES), followed by a specialized U-net for prediction. Our approach, which we call turbulent-Flow Net (TF-Net), is grounded in a principled physics model, yet offers the flexibility of learned representations. We compare our model, TF-Net, with state-of-the-art baselines and observe significant reductions in error for predictions 60 frames ahead. Most importantly, our method predicts physical fields that obey desirable physical characteristics, such as conservation of mass, whilst faithfully emulating the turbulent kinetic energy field and spectrum, which are critical for accurate prediction of turbulent flows.

physics.comp-ph↗

arXiv2017

Scaling GRPC Tensorflow on 512 nodes of Cori Supercomputer

We explore scaling of the standard distributed Tensorflow with GRPC primitives on up to 512 Intel Xeon Phi (KNL) nodes of Cori supercomputer with synchronous stochastic gradient descent (SGD), and identify causes of scaling inefficiency at higher node counts. To our knowledge, this is the first exploration of distributed GRPC Tensorflow scalability on a HPC supercomputer at such large scale with synchronous SGD. We studied scaling of two convolution neural networks - ResNet-50, a state-of-the-art deep network for classification with roughly 25.5 million parameters, and HEP-CNN, a shallow topology with less than 1 million parameters for common scientific usages. For ResNet-50, we achieve >80% scaling efficiency on up to 128 workers, using 32 parameter servers (PS tasks) with a steep decline down to 23% for 512 workers using 64 PS tasks. Our analysis of the efficiency drop points to low network bandwidth utilization due to combined effect of three factors. (a) Heterogeneous distributed parallelization algorithm which uses PS tasks as centralized servers for gradient averaging is suboptimal for utilizing interconnect bandwidth. (b) Load imbalance among PS tasks hinders their efficient scaling. (c) Underlying communication primitive GRPC is currently inefficient on Cori high-speed interconnect. The HEP-CNN demands less interconnect bandwidth, and shows >80% weak scaling efficiency for up to 256 nodes with only 1 PS task. Our findings are applicable to other deep learning networks. Big networks with millions of parameters stumble upon the issues discussed here. Shallower networks like HEP-CNN with relatively lower number of parameters can efficiently enjoy weak scaling even with a single parameter server.

cs.DC↗

arXiv2017

CosmoGAN: creating high-fidelity weak lensing convergence maps using Generative Adversarial Networks

Inferring model parameters from experimental data is a grand challenge in many sciences, including cosmology. This often relies critically on high fidelity numerical simulations, which are prohibitively computationally expensive. The application of deep learning techniques to generative modeling is renewing interest in using high dimensional density estimators as computationally inexpensive emulators of fully-fledged simulations. These generative models have the potential to make a dramatic shift in the field of scientific simulations, but for that shift to happen we need to study the performance of such generators in the precision regime needed for science applications. To this end, in this work we apply Generative Adversarial Networks to the problem of generating weak lensing convergence maps. We show that our generator network produces maps that are described by, with high statistical confidence, the same summary statistics as the fully simulated maps.

astro-ph.IM↗

arXiv2017

STAR Data Reconstruction at NERSC/Cori, an adaptable Docker container approach for HPC

As HPC facilities grow their resources, adaptation of classic HEP/NP workflows becomes a need. Linux containers may very well offer a way to lower the bar to exploiting such resources and at the time, help collaboration to reach vast elastic resources on such facilities and address their massive current and future data processing challenges. In this proceeding, we showcase STAR data reconstruction workflow at Cori HPC system at NERSC. STAR software is packaged in a Docker image and runs at Cori in Shifter containers. We highlight two of the typical end-to-end optimization challenges for such pipelines: 1) data transfer rate which was carried over ESnet after optimizing end points and 2) scalable deployment of conditions database in an HPC environment. Our tests demonstrate equally efficient data processing workflows on Cori/HPC, comparable to standard Linux clusters.

physics.data-an↗

arXiv2015

Overview of recent results from the STAR experiment

The Solenoidal Tracker at RHIC (STAR) experiment utilizes its excellent mid-rapidity tracking and particle identification capabilities to study the emergent properties of Quantum Chromodynamics (QCD). The STAR heavy-ion program at vanishingly small baryon density is aimed to address questions about the quantitative properties of the strongly-interacting Quark Gluon Plasma (QGP) matter created in high energy collisions ($\eta/s$, $\hat{q}$, chirality, transport parameters, heavy quark diffusion coefficients ...). At finite baryon density, the questions concern the phases of nuclear matter (the QCD phase diagram) and the nature of the phase transition, namely: what is the onset collision energy for the formation of QGP? What is the nature of phase transition in heavy-ion collisions? Are there two phase transition regions? If yes, where is the critical point situated? At Quark Matter 2015, the STAR collaboration has presented a wealth of new experimental results which address these questions. In these proceedings I highlight a few of those results.

nucl-ex↗

arXiv2015

A study of charm quark correlations in ultra-relativistic $p$ + $p$ collisions with PYTHIA

Measurements of heavy flavor quark (charm and bottom) correlations in heavy ion collisions are instrumental to understand the flavor dependence of energy loss mechanisms in hot and dense QCD media. Experimental measurements of these correlations in baseline $p$+$p$ collisions are crucial to understand the contributions of perturbative and non-perturbative QCD processes to the correlation functions and further help in interpreting correlation measurements in heavy ion collisions. In this paper, we investigate $D$-$\bar{D}$ meson correlations and $D$ with one particle from $D$ meson decay daughter correlations using PYTHIA Event Generator in $p$ + $p$ collisions at $\sqrt{s}$ = 200, 500 and 5500 GeV. Charm/bottom events are found to contribute mainly to the away side/near side pattern of $D$-electron correlations, respectively. In the energy region of RHIC, $D$-$\bar{D}$ correlations inherit initial $c$-$\bar{c}$ correlations and $B\rightarrow DX$ decay contribution is insignificant. Furthermore, Bottom quark correlations are suggested to be applicable at LHC energy, as the bottom contributions on $D$ related correlations are relatively large.

nucl-th↗

arXiv2014

Measurements of Non-photonic Electron Production and Azimuthal Anisotropy in $\sqrt {s_{NN}} = 39$, 62.4 and 200 GeV \auau\ Collisions from STAR at RHIC

During RHIC 2010 run, STAR has collected a large amount of minimum-bias, central and high $p_{T}$ trigger data in Au+Au collisions at $\sqrt{s_{NN}} = 39$, 62.4 and 200 GeV with detector configuration for minimum photonic conversion background. In this article we report on a new high precision measurement of non-photonic electron mid-rapidity invariant yield, improved nuclear modification factor and $v_{2}$ in Au+Au collisions at $\sqrt{s_{NN}} = 200$ GeV. We also present measurements of mid-rapidity invariant yield at $\sqrt{s_{NN}} = 62.4$ and $v_{2}$ at $\sqrt{s_{NN}} = 39$ and 62.4 GeV.

nucl-ex↗

arXiv2011

Supersymmetry identifies molecular Stark states whose eigenproperties can be obtained analytically

We made use of supersymmetric (SUSY) quantum mechanics to find a condition under which the Stark effect problem for a polar and polarizable closed-shell diatomic molecule subject to collinear electrostatic and nonresonant radiative fields becomes exactly solvable. The condition, $Δω= \frac{ω^2}{4 (m+1)^2}$, connects values of the dimensionless parameters $ω$ and $Δω$ that characterize the strengths of the permanent and induced dipole interactions of the molecule with the respective fields. The exact solutions are obtained for the $|\tilde{J}=m,m;ω,Δω>$ family of "stretched" states. The field-free and strong-field limits of the combined-fields problem were found to exhibit supersymmetry and shape-invariance, which is indeed the reason why they are analytically solvable. By making use of the analytic form of the $|\tilde{J}=m,m;ω,Δω>$ wavefunctions, we obtained simple formulae for the expectation values of the space-fixed electric dipole moment, the alignment cosine, the angular momentum squared, and derived a "sum rule" which combines the above expectation values into a formula for the eigenenergy. The analytic expressions for the characteristics of the strongly oriented and aligned states provide a direct access to the values of the interaction parameters required for creating such states in the laboratory.

physics.chem-ph↗

arXiv2011

Supersymmetric factorization yields exact solutions to the molecular Stark effect problem for "stretched" states

By invoking supersymmetry, we found a condition under which the Stark effect problem for a polar and polarizable molecule subject to nonresonant electric fields becomes exactly solvable. The exact solvability condition for the interaction parameters involved yields exact wavefunction for the "stretched" states, $|J=m,m>$, and for the $|1,1>$ state in the case of a purely induced-dipole interaction. The analytic expressions for the eigenenergy, the space-fixed dipole moment, the alignment cosine, and the expectation value of the angular momentum allow to readily reverse-engineer the problem of finding the values of the interaction parameters required for creating quantum states with preordained characteristics.

physics.chem-ph↗