arXiv subjects

Koushik Sen

Publications and source records attributed to Koushik Sen.

At least 37 records · Page 2Linked to original sources

arXiv2025

A new sample of massive B-type contact binary candidates from the OGLE survey of the Magellanic Clouds

Massive contact binaries (CBs) are key to understanding close-binary evolution and stellar mergers, yet their study has been limited by the scarcity of observed systems, particularly of B-type binaries expected to dominate this class. We bridge this gap by mining a large sample of massive CB candidates from the OGLE-IV database, increasing their known numbers in the Magellanic Clouds by nearly an order of magnitude. Using main-sequence colour-magnitude limits, an observationally informed period-luminosity-colour relation for CBs, and a high morph-parameter cut ($c\geq0.7$), we identified 68 O- and B-type binaries that exhibit smooth, sinusoidal light curves with nearly equal eclipse depths. We then isolated a bona fide sample of 37 CB candidates (28 in the LMC and 9 in the SMC) that match theoretical colour-magnitude and period distributions derived from an extensive grid of MESA binary models. The bona fide sample, dominated by B-type systems with $P\approx0.6-1$ d, agrees with the predicted population and may contain many $q\approx1$ binaries, as expected from models showing mass equalization preceding temperature equalization during nuclear-timescale contact. Synthetic PHOEBE light curves of contact and near-contact phases of MESA models reveal a degeneracy between these configurations, suggesting possible misidentifications among these systems. Spectroscopic follow-up is required to test these predictions and refine the evolutionary framework of massive CBs.

astro-ph.SR↗

arXiv2025

SCGAgent: Recreating the Benefits of Reasoning Models for Secure Code Generation with Agentic Workflows

Large language models (LLMs) have seen widespread success in code generation tasks for different scenarios, both everyday and professional. However current LLMs, despite producing functional code, do not prioritize security and may generate code with exploitable vulnerabilities. In this work, we propose techniques for generating code that is more likely to be secure and introduce SCGAgent, a proactive secure coding agent that implements our techniques. We use security coding guidelines that articulate safe programming practices, combined with LLM-generated unit tests to preserve functional correctness. In our evaluation, we find that SCGAgent is able to preserve nearly 98% of the functionality of the base Sonnet-3.7 LLM while achieving an approximately 25% improvement in security. Moreover, SCGAgent is able to match or best the performance of sophisticated reasoning LLMs using a non-reasoning model and an agentic workflow.

cs.CR↗

arXiv2025

Type-Constrained Code Generation with Language Models

Large language models (LLMs) have achieved notable success in code generation. However, they still frequently produce uncompilable output because their next-token inference procedure does not model formal aspects of code. Although constrained decoding is a promising approach to alleviate this issue, it has only been applied to handle either domain-specific languages or syntactic features of general-purpose programming languages. However, LLMs frequently generate code with typing errors, which are beyond the domain of syntax and generally hard to adequately constrain. To address this challenge, we introduce a type-constrained decoding approach that leverages type systems to guide code generation. For this purpose, we develop novel prefix automata and a search over inhabitable types, forming a sound approach to enforce well-typedness on LLM-generated code. We formalize our approach on a foundational simply-typed language and extend it to TypeScript to demonstrate practicality. Our evaluation on the HumanEval and MBPP datasets shows that our approach reduces compilation errors by more than half and significantly increases functional correctness in code synthesis, translation, and repair tasks across LLMs of various sizes and model families, including state-of-the-art open-weight models with more than 30B parameters. The results demonstrate the generality and effectiveness of our approach in constraining LLM code generation with formal rules of type systems.

cs.LG↗

arXiv2025

R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents

Improving open-source models on real-world SWE tasks (solving GITHUB issues) faces two key challenges: 1) scalable curation of execution environments to train these models, and, 2) optimal scaling of test-time compute. We introduce AgentGym, the largest procedurally-curated executable gym environment for training real-world SWE-agents, consisting of more than 8.7K tasks. AgentGym is powered by two main contributions: 1) SYNGEN: a synthetic data curation recipe that enables scalable curation of executable environments using test-generation and back-translation directly from commits, thereby reducing reliance on human-written issues or unit tests. We show that this enables more scalable training leading to pass@1 performance of 34.4% on SWE-Bench Verified benchmark with our 32B model. 2) Hybrid Test-time Scaling: we provide an in-depth analysis of two test-time scaling axes; execution-based and execution-free verifiers, demonstrating that they exhibit complementary strengths and limitations. Test-based verifiers suffer from low distinguishability, while execution-free verifiers are biased and often rely on stylistic features. Surprisingly, we find that while each approach individually saturates around 42-43%, significantly higher gains can be obtained by leveraging their complementary strengths. Overall, our approach achieves 51% on the SWE-Bench Verified benchmark, reflecting a new state-of-the-art for open-weight SWE-agents and for the first time showing competitive performance with proprietary models such as o1, o1-preview and sonnet-3.5-v2 (with tools). We will open-source our environments, models, and agent trajectories.

cs.SE↗

arXiv2025

Challenges and Paths Towards AI for Software Engineering

AI for software engineering has made remarkable progress recently, becoming a notable success within generative AI. Despite this, there are still many challenges that need to be addressed before automated software engineering reaches its full potential. It should be possible to reach high levels of automation where humans can focus on the critical decisions of what to build and how to balance difficult tradeoffs while most routine development effort is automated away. Reaching this level of automation will require substantial research and engineering efforts across academia and industry. In this paper, we aim to discuss progress towards this in a threefold manner. First, we provide a structured taxonomy of concrete tasks in AI for software engineering, emphasizing the many other tasks in software engineering beyond code generation and completion. Second, we outline several key bottlenecks that limit current approaches. Finally, we provide an opinionated list of promising research directions toward making progress on these bottlenecks, hoping to inspire future research in this rapidly maturing field.

cs.SE↗

arXiv2025

LangProBe: a Language Programs Benchmark

Composing language models (LMs) into multi-step language programs and automatically optimizing their modular prompts is now a mainstream paradigm for building AI systems, but the tradeoffs in this space have only scarcely been studied before. We introduce LangProBe, the first large-scale benchmark for evaluating the architectures and optimization strategies for language programs, with over 2000 combinations of tasks, architectures, optimizers, and choices of LMs. Using LangProBe, we are the first to study the impact of program architectures and optimizers (and their compositions together and with different models) on tradeoffs of quality and cost. We find that optimized language programs offer strong cost--quality Pareto improvement over raw calls to models, but simultaneously demonstrate that human judgment (or empirical decisions) about which compositions to pursue is still necessary for best performance. We will open source the code and evaluation data for LangProBe.

cs.CL↗

arXiv2025

X-ray emission from helium star+black hole binaries as probes of tidally induced spin-up of second-born black holes

Tidally induced spin-up of stripped helium stars in short-period (<\,1\,d) binaries with black holes has been presented as one of the possible mechanisms to reproduce the high-spin tail of the black hole spin distribution derived from gravitational wave (GW) merger observations. At such short periods, a fraction of the strong stellar wind from the stripped helium stars may be accreted by the black holes, and its gravitational potential energy may be released as observable radiation in the X-ray regime. We estimate the X-ray luminosity and its observability from black holes in orbit with stripped helium stars, which evolve into binary black hole or black hole+neutron star binaries that merge within Hubble time. We post-process recent advancements for estimating X-ray luminosities (via wind accretion onto stellar mass black holes) into two rapid population synthesis codes, BSE and StarTrack. We derive lower limits on the X-ray luminosity distribution from populations of stripped helium+black hole binaries at four metallicities (0.01, 0.1, 0.5, 1 $Z_{\odot}$) and two mass transfer stability criteria. We find that a large fraction (0.1-0.5) of stripped-helium stars that get spun up by tides also transfer enough wind matter onto the black hole to produce X-ray luminosities above $10^{35}$\,erg\,s$^{-1}$, up to $\sim10^{39}$\,erg\,s$^{-1}$. Such binaries should be observable as X-ray bright systems at 0.1\,$Z_{\odot}$, 0.5\,$Z_{\odot}$ and $Z_{\odot}$, representative of Sextans A, the Large Magellanic Cloud (LMC) and the Solar neighbourhood, respectively. The formation efficiency of these systems increases with decreasing metallicity. However, accounting for the local star formation rates, our population synthesis predicts $\sim$2 and $\sim$1 such observable binaries in the Milky Way and LMC, respectively, that will produce a binary compact object merger within a Hubble time. (Abridged)

astro-ph.HE↗

arXiv2024

Syzygy: Dual Code-Test C to (safe) Rust Translation using LLMs and Dynamic Analysis

Despite extensive usage in high-performance, low-level systems programming applications, C is susceptible to vulnerabilities due to manual memory management and unsafe pointer operations. Rust, a modern systems programming language, offers a compelling alternative. Its unique ownership model and type system ensure memory safety without sacrificing performance. In this paper, we present Syzygy, an automated approach to translate C to safe Rust. Our technique uses a synergistic combination of LLM-driven code and test translation guided by dynamic-analysis-generated execution information. This paired translation runs incrementally in a loop over the program in dependency order of the code elements while maintaining per-step correctness. Our approach exposes novel insights on combining the strengths of LLMs and dynamic analysis in the context of scaling and combining code generation with testing. We apply our approach to successfully translate Zopfli, a high-performance compression library with ~3000 lines of code and 98 functions. We validate the translation by testing equivalence with the source C program on a set of inputs. To our knowledge, this is the largest automated and test-validated C to safe Rust code translation achieved so far.

cs.SE↗

arXiv2024

Specifications: The missing link to making the development of LLM systems an engineering discipline

Despite the significant strides made by generative AI in just a few short years, its future progress is constrained by the challenge of building modular and robust systems. This capability has been a cornerstone of past technological revolutions, which relied on combining components to create increasingly sophisticated and reliable systems. Cars, airplanes, computers, and software consist of components-such as engines, wheels, CPUs, and libraries-that can be assembled, debugged, and replaced. A key tool for building such reliable and modular systems is specification: the precise description of the expected behavior, inputs, and outputs of each component. However, the generality of LLMs and the inherent ambiguity of natural language make defining specifications for LLM-based components (e.g., agents) both a challenging and urgent problem. In this paper, we discuss the progress the field has made so far-through advances like structured outputs, process supervision, and test-time compute-and outline several future directions for research to enable the development of modular and reliable LLM-based systems through improved specifications.

cs.SE↗

arXiv2024

Multiversion Hindsight Logging for Continuous Training

Production Machine Learning involves continuous training: hosting multiple versions of models over time, often with many model versions running at once. When model performance does not meet expectations, Machine Learning Engineers (MLEs) debug issues by exploring and analyzing numerous prior versions of code and training data to identify root causes and mitigate problems. Traditional debugging and logging tools often fall short in managing this experimental, multi-version context. FlorDB introduces Multiversion Hindsight Logging, which allows engineers to use the most recent version's logging statements to query past versions, even when older versions logged different data. Log statement propagation enables consistent injection of logging statements into past code versions, regardless of changes to the codebase. Once log statements are propagated across code versions, the remaining challenge in Multiversion Hindsight Logging is to efficiently replay the new log statements based on checkpoints from previous runs. Finally, a coherent user experience is required to help MLEs debug across all versions of code and data. To this end, FlorDB presents a unified relational model for efficient handling of historical queries, offering a comprehensive view of the log history to simplify the exploration of past code iterations. We present a performance evaluation on diverse benchmarks confirming its scalability and the ability to deliver real-time query responses, leveraging query-based filtering and checkpoint-based parallelism for efficient replay.

cs.SE↗

arXiv2024

Whispering in the dark: Faint X-ray emission from black holes with OB star companions

Context. Recent astrometric and spectroscopic surveys of OB stars have revealed a few stellar-mass black holes (BHs) with orbital periods as low as 10 days. No X-ray counterpart has been detected, due to the absence of a radiatively efficient accretion disk around the BH. Yet, dissipative processes in the hot, dilute and strongly magnetized plasma around the BH (so-called BH corona) can still lead to non-thermal X-ray emission (e.g. synchrotron). Aims. We determine the X-ray luminosity distribution from BH+OB star binaries up to orbital periods of a few thousand days. Methods. We use detailed binary evolution models computed with MESA for initial primary masses of 10-90 $M_{\odot}$ and orbital periods from 1-3000 d. The X-ray luminosity is computed for a broad range of radiative efficiencies. Results. We show that particle acceleration through magnetic reconnection can heat the BH corona. A substantial fraction of the gravitational potential energy from the accreted plasma is converted into non-thermal X-ray emission. Our population synthesis analysis predicts at least 28 (up to 72) BH+OB star binaries in the Large Magellanic Cloud (LMC) to produce X-ray luminosity above 10$^{31}$ erg$\,$s$^{-1}$, observable through focused Chandra observations. We identify a population of SB1 systems in the LMC and HD96670 in the Milky Way comprising O stars with unseen companions of masses above 2.3 $M_{\odot}$ that aligns well with our predictions. The predicted luminosities of the OB companions to these X-ray-emitting BHs are 10$^{4.5-5.5}$ $L_{\odot}$. Conclusions. These results make the case for long-time exposure in X-rays of the stellar-mass BH candidates identified around OB stars. It will constrain the underlying population of X-ray-faint BHs, the evolution from single to double degenerate binaries, and the progenitors of gravitational wave mergers. (Abridged)

astro-ph.HE↗

arXiv2024

BACKRUNNER: Mitigating Smart Contract Attacks in the Real World

Billions of dollars have been lost due to vulnerabilities in smart contracts. To counteract this, researchers have proposed attack frontrunning protections designed to preempt malicious transactions by inserting "whitehat" transactions ahead of them to protect the assets. In this paper, we demonstrate that existing frontrunning protections have become ineffective in real-world scenarios. Specifically, we collected 158 recent real-world attack transactions and discovered that 141 of them can bypass state-of-the-art frontrunning protections. We systematically analyze these attacks and show how inherent limitations of existing frontrunning techniques hinder them from protecting valuable assets in the real world. We then propose a new approach involving 1) preemptive hijack, and 2) attack backrunning, which circumvent the existing limitations and can help protect assets before and after an attack. Our approach adapts the exploit used in the attack to the same or similar contracts before and after the attack to safeguard the assets. We conceptualize adapting exploits as a program repair problem and apply established techniques to implement our approach into a full-fledged framework, BACKRUNNER. Running on previous attacks in 2023, BACKRUNNER can successfully rescue more than \$410M. In the real world, it has helped rescue over \$11.2M worth of assets in 28 separate incidents within two months.

cs.CR↗

arXiv2024

LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code

Large Language Models (LLMs) applied to code-related applications have emerged as a prominent field, attracting significant interest from both academia and industry. However, as new and improved LLMs are developed, existing evaluation benchmarks (e.g., HumanEval, MBPP) are no longer sufficient for assessing their capabilities. In this work, we propose LiveCodeBench, a comprehensive and contamination-free evaluation of LLMs for code, which continuously collects new problems over time from contests across three competition platforms, namely LeetCode, AtCoder, and CodeForces. Notably, our benchmark also focuses on a broader range of code related capabilities, such as self-repair, code execution, and test output prediction, beyond just code generation. Currently, LiveCodeBench hosts four hundred high-quality coding problems that were published between May 2023 and May 2024. We have evaluated 18 base LLMs and 34 instruction-tuned LLMs on LiveCodeBench. We present empirical findings on contamination, holistic performance comparisons, potential overfitting in existing benchmarks as well as individual model comparisons. We will release all prompts and model completions for further community analysis, along with a general toolkit for adding new scenarios and model

cs.SE↗

arXiv2024

Analytic approximations for massive close post-mass transfer binary systems

Massive binary evolution models are needed to predict massive star populations in star forming galaxies, the supernova diversity, and the number and properties of gravitational wave sources. Such models are often computed using so called rapid binary evolution codes, which approximate the evolution of the binary components based on detailed single star models. However, about one third of the interacting massive binary stars undergo mass transfer during core hydrogen burning (Case A mass transfer), whose outcome is difficult to derive from single star models. Here, we use a large grid of detailed binary evolution models for primaries in the initial mass range 10 to 40 Solar masses of LMC and SMC composition, to derive analytic fits for the key quantities needed in rapid binary evolution codes, i.e., the duration of core hydrogen burning, and the resulting donor star mass. Systems with shorter orbital periods produce up to 50% lighter stripped donors and have a up to 30% larger lifetime than wider systems. We find that both quantities depend strongly on the initial binary orbital period, but that the initial mass ratio and the mass transfer efficiency of the binary have little impact on the outcome. Our results are easily parameterisable and can be used to capture the effects of Case A mass transfer more accurately in rapid binary evolution codes.

astro-ph.SR↗

arXiv2024

The Counterfeit Conundrum: Can Code Language Models Grasp the Nuances of Their Incorrect Generations?

While language models are increasingly more proficient at code generation, they still frequently generate incorrect programs. Many of these programs are obviously wrong, but others are more subtle and pass weaker correctness checks such as being able to compile. In this work, we focus on these counterfeit samples: programs sampled from a language model that 1) have a high enough log-probability to be generated at a moderate temperature and 2) pass weak correctness checks. Overall, we discover that most models have a very shallow understanding of counterfeits through three clear failure modes. First, models mistakenly classify them as correct. Second, models are worse at reasoning about the execution behaviour of counterfeits and often predict their execution results as if they were correct. Third, when asking models to fix counterfeits, the likelihood of a model successfully repairing a counterfeit is often even lower than that of sampling a correct program from scratch. Counterfeits also have very unexpected properties: first, counterfeit programs for problems that are easier for a model to solve are not necessarily easier to detect and only slightly easier to execute and repair. Second, counterfeits from a given model are just as confusing to the model itself as they are to other models. Finally, both strong and weak models are able to generate counterfeit samples that equally challenge all models. In light of our findings, we recommend that care and caution be taken when relying on models to understand their own samples, especially when no external feedback is incorporated.

cs.SE↗

arXiv2024

DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelines

Chaining language model (LM) calls as composable modules is fueling a new way of programming, but ensuring LMs adhere to important constraints requires heuristic "prompt engineering". We introduce LM Assertions, a programming construct for expressing computational constraints that LMs should satisfy. We integrate our constructs into the recent DSPy programming model for LMs, and present new strategies that allow DSPy to compile programs with LM Assertions into more reliable and accurate systems. We also propose strategies to use assertions at inference time for automatic self-refinement with LMs. We report on four diverse case studies for text generation and find that LM Assertions improve not only compliance with imposed rules but also downstream task performance, passing constraints up to 164% more often and generating up to 37% more higher-quality responses. Our reference implementation of LM Assertions is integrated into DSPy at https://github.com/stanfordnlp/dspy

cs.CL↗

arXiv2024

LLM4Fuzz: Guided Fuzzing of Smart Contracts with Large Language Models

As blockchain platforms grow exponentially, millions of lines of smart contract code are being deployed to manage extensive digital assets. However, vulnerabilities in this mission-critical code have led to significant exploitations and asset losses. Thorough automated security analysis of smart contracts is thus imperative. This paper introduces LLM4Fuzz to optimize automated smart contract security analysis by leveraging large language models (LLMs) to intelligently guide and prioritize fuzzing campaigns. While traditional fuzzing suffers from low efficiency in exploring the vast state space, LLM4Fuzz employs LLMs to direct fuzzers towards high-value code regions and input sequences more likely to trigger vulnerabilities. Additionally, LLM4Fuzz can leverage LLMs to guide fuzzers based on user-defined invariants, reducing blind exploration overhead. Evaluations of LLM4Fuzz on real-world DeFi projects show substantial gains in efficiency, coverage, and vulnerability detection compared to baseline fuzzing. LLM4Fuzz also uncovered five critical vulnerabilities that can lead to a loss of more than $247k.

cs.CR↗

arXiv2023

CodeScholar: Growing Idiomatic Code Examples

Programmers often search for usage examples for API methods. A tool that could generate realistic, idiomatic, and contextual usage examples for one or more APIs would be immensely beneficial to developers. Such a tool would relieve the need for a deep understanding of the API landscape, augment existing documentation, and help discover interactions among APIs. We present CodeScholar, a tool that generates idiomatic code examples demonstrating the common usage of API methods. It includes a novel neural-guided search technique over graphs that grows the query APIs into idiomatic code examples. Our user study demonstrates that in 70% of cases, developers prefer CodeScholar generated examples over state-of-the-art large language models (LLM) like GPT3.5. We quantitatively evaluate 60 single and 25 multi-API queries from 6 popular Python libraries and show that across-the-board CodeScholar generates more realistic, diverse, and concise examples. In addition, we show that CodeScholar not only helps developers but also LLM-powered programming assistants generate correct code in a program synthesis setting.

cs.SE↗