arXiv subjects

Duccio Pappadopulo

Publications and source records attributed to Duccio Pappadopulo.

At least 19 recordsLinked to original sources

arXiv2025

Improving Instruct Models for Free: A Study on Partial Adaptation

Instruct models, obtained from various instruction tuning or post-training steps, are commonly deemed superior and more usable than their base counterpart. While the model gains instruction following ability, instruction tuning may lead to forgetting the knowledge from pre-training or it may encourage the model to become overly conversational or verbose. This, in turn, can lead to degradation of in-context few-shot learning performance. In this work, we study the performance trajectory between base and instruct models by scaling down the strength of instruction-tuning via the partial adaption method. We show that, across several model families and model sizes, reducing the strength of instruction-tuning results in material improvement on a few-shot in-context learning benchmark covering a variety of classic natural language tasks. This comes at the cost of losing some degree of instruction following ability as measured by AlpacaEval. Our study shines light on the potential trade-off between in-context learning and instruction following abilities that is worth considering in practice.

cs.CL↗

arXiv2023

Non-contrastive sentence representations via self-supervision

Sample contrastive methods, typically referred to simply as contrastive are the foundation of most unsupervised methods to learn text and sentence embeddings. On the other hand, a different class of self-supervised loss functions and methods have been considered in the computer vision community and referred to as dimension contrastive. In this paper, we thoroughly compare this class of methods with the standard baseline for contrastive sentence embeddings, SimCSE. We find that self-supervised embeddings trained using dimension contrastive objectives can outperform SimCSE on downstream tasks without needing auxiliary loss functions.

cs.CL↗

arXiv2023

Distillation of encoder-decoder transformers for sequence labelling

Driven by encouraging results on a wide range of tasks, the field of NLP is experiencing an accelerated race to develop bigger language models. This race for bigger models has also underscored the need to continue the pursuit of practical distillation approaches that can leverage the knowledge acquired by these big models in a compute-efficient manner. Having this goal in mind, we build on recent work to propose a hallucination-free framework for sequence tagging that is especially suited for distillation. We show empirical results of new state-of-the-art performance across multiple sequence labelling datasets and validate the usefulness of this framework for distilling a large model in a few-shot learning scenario.

cs.CL↗

arXiv2021

Disentangling Online Chats with DAG-Structured LSTMs

Many modern messaging systems allow fast and synchronous textual communication among many users. The resulting sequence of messages hides a more complicated structure in which independent sub-conversations are interwoven with one another. This poses a challenge for any task aiming to understand the content of the chat logs or gather information from them. The ability to disentangle these conversations is then tantamount to the success of many downstream tasks such as summarization and question answering. Structured information accompanying the text such as user turn, user mentions, timestamps, is used as a cue by the participants themselves who need to follow the conversation and has been shown to be important for disentanglement. DAG-LSTMs, a generalization of Tree-LSTMs that can handle directed acyclic dependencies, are a natural way to incorporate such information and its non-sequential nature. In this paper, we apply DAG-LSTMs to the conversation disentanglement task. We perform our experiments on the Ubuntu IRC dataset. We show that the novel model we propose achieves state of the art status on the task of recovering reply-to relations and it is competitive on other disentanglement metrics.

cs.CL↗

arXiv2021

Reframing Jet Physics with New Computational Methods

We reframe common tasks in jet physics in probabilistic terms, including jet reconstruction, Monte Carlo tuning, matrix element - parton shower matching for large jet multiplicity, and efficient event generation of jets in complex, signal-like regions of phase space. We also introduce Ginkgo, a simplified, generative model for jets, that facilitates research into these tasks with techniques from statistics, machine learning, and combinatorial optimization. We review some of the recent research in this direction that has been enabled with Ginkgo. We show how probabilistic programming can be used to efficiently sample the showering process, how a novel trellis algorithm can be used to efficiently marginalize over the enormous number of clustering histories for the same observed particles, and how dynamic programming, A* search, and reinforcement learning can be used to find the maximum likelihood clustering in this enormous search space. This work builds bridges with work in hierarchical clustering, statistics, combinatorial optmization, and reinforcement learning.

hep-ph↗

arXiv2020

Hierarchical clustering in particle physics through reinforcement learning

Particle physics experiments often require the reconstruction of decay patterns through a hierarchical clustering of the observed final-state particles. We show that this task can be phrased as a Markov Decision Process and adapt reinforcement learning algorithms to solve it. In particular, we show that Monte-Carlo Tree Search guided by a neural policy can construct high-quality hierarchical clusterings and outperform established greedy and beam search baselines.

cs.AI↗

arXiv2019

Dialogue Act Classification in Group Chats with DAG-LSTMs

Dialogue act (DA) classification has been studied for the past two decades and has several key applications such as workflow automation and conversation analytics. Researchers have used, to address this problem, various traditional machine learning models, and more recently deep neural network models such as hierarchical convolutional neural networks (CNNs) and long short-term memory (LSTM) networks. In this paper, we introduce a new model architecture, directed-acyclic-graph LSTM (DAG-LSTM) for DA classification. A DAG-LSTM exploits the turn-taking structure naturally present in a multi-party conversation, and encodes this relation in its model structure. Using the STAC corpus, we show that the proposed method performs roughly 0.8% better in accuracy and 1.2% better in macro-F1 score when compared to existing methods. The proposed method is generic and not limited to conversation applications.

cs.CL↗

arXiv2019

Thermal Relic Targets with Exponentially Small Couplings

If dark matter was produced in the early Universe by the decoupling of its annihilations into known particles, there is a sharp experimental target for the size of its coupling. We show that if dark matter was produced by inelastic scattering against a lighter particle from the thermal bath, then its coupling can be exponentially smaller than the coupling required for its production from annihilations. As an application, we demonstrate that dark matter produced by inelastic scattering against electrons provides new thermal relic targets for direct detection and fixed target experiments.

hep-ph↗

arXiv2019

Inferring the quantum density matrix with machine learning

We introduce two methods for estimating the density matrix for a quantum system: Quantum Maximum Likelihood and Quantum Variational Inference. In these methods, we construct a variational family to model the density matrix of a mixed quantum state. We also introduce quantum flows, the quantum analog of normalizing flows, which can be used to increase the expressivity of this variational family. The eigenstates and eigenvalues of interest are then derived by optimizing an appropriate loss function. The approach is qualitatively different than traditional lattice techniques that rely on the time dependence of correlation functions that summarize the lattice configurations. The resulting estimate of the density matrix can then be used to evaluate the expectation of an arbitrary operator, which opens the door to new possibilities.

quant-ph↗

arXiv2018

New Physics from High Energy Tops

Precision measurements of high energy top quarks at the LHC constitute a powerful probe of new physics. We study the effect of four fermion operators involving two tops and two light quarks on the high energy tail of the $t\bar t$ invariant mass distribution. We use existing measurements at a center of mass energy of 13 TeV, and state of the art calculations of the Standard Model contribution, to derive bounds on the coefficients of these operators. We estimate the projected reach of the LHC at higher luminosities and discuss the validity of these limits within the Effective Field Theory description. We find that current measurements constrain the mass scale of these operators to be larger than about 1-2 TeV, while we project that future LHC data will be sensitive to mass scales of about 3-4 TeV. We apply our bounds to constrain composite Higgs models with partial compositeness and models with approximate flavor symmetries. We find our limits to be most relevant to flavor non-universal models with a moderately large coupling of the heavy new physics states to third generation quarks.

hep-ph↗

arXiv2017

A Fourth Exception in the Calculation of Relic Abundances

We propose that the dark matter abundance is set by the decoupling of inelastic scattering instead of annihilations. This coscattering mechanism is generically realized if dark matter scatters against states of comparable mass from the thermal bath. Coscattering points to dark matter that is exponentially lighter than the weak scale and has a suppressed annihilation rate, avoiding stringent constraints from indirect detection. Dark matter upscatters into states whose late decays can lead to observable distortions to the blackbody spectrum of the cosmic microwave background.

hep-ph↗

arXiv2017

Catching a New Force by the Tail

The Large Hadron Collider (LHC) is sensitive to new heavy gauge bosons that produce narrow peaks in the dilepton invariant mass spectrum up to about $m_{Z'}\sim 5$ TeV. $Z'$s that are too heavy to produce directly can reveal their presence through interference with Standard Model dilepton production. We show that the LHC can significantly extend the mass reach for such $Z'$s by performing precision measurements of the shape of the dilepton invariant mass spectrum. The high luminosity LHC can exclude, with 95$\%$ confidence, new gauge bosons as heavy as $m_{Z'} \sim 10-20$ TeV that couple with gauge coupling strength of $g_{Z'} \sim 1-2$.

hep-ph↗

arXiv2017

Precision Probes of QCD at High Energies

New physics, that is too heavy to be produced directly, can leave measurable imprints on the tails of kinematic distributions at the LHC. We use energetic QCD processes to perform novel measurements of the Standard Model (SM) Effective Field Theory. We show that the dijet invariant mass spectrum, and the inclusive jet transverse momentum spectrum, are sensitive to a dimension 6 operator that modifies the gluon propagator at high energies. The dominant effect is constructive or destructive interference with SM jet production. We compare differential next-to-leading order predictions from POWHEG to public 7 TeV jet data, including scale, PDF, and experimental uncertainties and their respective correlations. We constrain a New Physics (NP) scale of 3.5 TeV with current data. We project the reach of future 13 and 100 TeV measurements, which we estimate to be sensitive to NP scales of 8 and 60 TeV, respectively. As an application, we apply our bounds to constrain heavy vector octet colorons that couple to the QCD current. We project that effective operators will surpass bump hunts, in terms of coloron mass reach, even for sequential couplings.

hep-ph↗

arXiv2017

Phases of Cannibal Dark Matter

A hidden sector with a mass gap undergoes an epoch of cannibalism if number changing interactions are active when the temperature drops below the mass of the lightest hidden particle. During cannibalism, the hidden sector temperature decreases only logarithmically with the scale factor. We consider the possibility that dark matter resides in a hidden sector that underwent cannibalism, and has relic density set by the freeze-out of two-to-two annihilations. We identify three novel phases, depending on the behavior of the hidden sector when dark matter freezes out. During the cannibal phase, dark matter annihilations decouple while the hidden sector is cannibalizing. During the chemical phase, only two-to-two interactions are active and the total number of hidden particles is conserved. During the one way phase, the dark matter annihilation products decay out of equilibrium, suppressing the production of dark matter from inverse annihilations. We map out the distinct phenomenology of each phase, which includes a boosted dark matter annihilation rate, new relativistic degrees of freedom, warm dark matter, and observable distortions to the spectrum of the cosmic microwave background.

hep-ph↗

arXiv2017

Energy helps accuracy: electroweak precision tests at hadron colliders

We show that high energy measurements of Drell-Yan at the LHC can serve as electroweak precision tests. Dimension-6 operators, from the Standard Model Effective Field Theory, modify the high energy behavior of electroweak gauge boson propagators. Existing measurements of the dilepton invariant mass spectrum, from neutral current Drell-Yan at 8 TeV, have comparable sensitivity to LEP. We propose measuring the transverse mass spectrum of charged current Drell-Yan, which can surpass LEP already with 8 TeV data. The 13 TeV LHC will elevate electroweak tests to a new precision frontier.

hep-ph↗

arXiv2017

The photo-philic QCD axion

We propose a framework in which the QCD axion has an exponentially large coupling to photons, relying on the "clockwork" mechanism. We discuss the impact of present and future axion experiments on the parameter space of the model. In addition to the axion, the model predicts a large number of pseudo-scalars which can be light and observable at the LHC. In the most favorable scenario, axion Dark Matter will give a signal in multiple axion detection experiments and the pseudo-scalars will be discovered at the LHC, allowing to determine most of the parameters of the model.

hep-ph↗

arXiv2016

Cannibal Dark Matter

A thermally decoupled hidden sector of particles, with a mass gap, generically enters a phase of cannibalism in the early Universe. The Standard Model sector becomes exponentially colder than the hidden sector. We propose the Cannibal Dark Matter framework, where dark matter resides in a cannibalizing sector with a relic density set by 2-to-2 annihilations. Observable signals of Cannibal Dark Matter include a boosted rate for indirect detection, new relativistic degrees of freedom, and warm dark matter.

hep-ph↗

arXiv2015

Freeze-In Dark Matter with Displaced Signatures at Colliders

Dark matter, $X$, may be generated by new physics at the TeV scale during an early matter-dominated (MD) era that ends at temperature $T_R \ll {\rm TeV}$. Compared to the conventional radiation-dominated (RD) results, yields from both Freeze-Out and Freeze-In processes are greatly suppressed by dilution from entropy production, making Freeze-Out less plausible while allowing successful Freeze-In with a much larger coupling strength. Freeze-In is typically dominated by the decay of a particle $B$ of the thermal bath, $B \rightarrow X$. For a large fraction of the relevant cosmological parameter space, the decay rate required to produce the observed dark matter abundance leads to displaced signals at LHC and future colliders, for any $m_X$ in the range ${\rm keV} < m_X < m_B$ and for values of $m_B$ accessible to these colliders. This result applies whether the early MD era arises after conventional inflation, when $T_R$ is the usual reheat temperature, or is a generic MD era with an alternative origin. In the former case, if $m_X$ is sufficiently large to be measured from kinematics, the reheat temperature $T_R$ can be extracted. Our result is independent of the particular particle physics implementation of $B \rightarrow X$, and can occur via any operator of dimension less than 8 (4) for a post-inflation (general MD) cosmology. An interesting example is provided by DFS axion theories with TeV-scale supersymmetry and axino dark matter of mass GeV to TeV, which is typically overproduced in a conventional RD cosmology. If $B$ is the higgsino, $\tilde h$, Higgs, W and Z particles appear at the displaced decays, $\tilde h \rightarrow h \tilde a, Z \tilde a$ and $\tilde h^\pm \rightarrow W^\pm \tilde a$. The scale of axion physics, $f$, is predicted to be in the range $(3\times10^8 - 10^{12})$ GeV and, over much of this range, can be extracted from the decay length.

hep-ph↗