arXiv subjects

Shruti Mishra

Publications and source records attributed to Shruti Mishra.

11 recordsLinked to original sources

arXiv2026

Surface-mediated frequency aging beyond quality-factor saturation in an AlScN-on-silicon resonator

Vacuum package integrity in micro- and nanoelectromechanical resonators is commonly assessed through the quality factor Q, although Q probes residual-gas damping rather than the surface-state evolution that can govern frequency aging. Here, we disentangle the pressure responses of Q and the resonance frequency f0 in a 64.21 MHz Al0.7Sc0.3N-on-silicon cross-sectional Lamé-mode resonator between 0.01 to 760 Torr. Measurements at 25 °C and at the 68.8°C frequency turnover, where first-order thermal sensitivity is suppressed, reveal widely separated equilibration timescales. Following each pressure step, Q reaches a reversible, history-independent steady value on the pressure-control timescale and approaches a fitted pressure-independent ceiling of 8.5e4 below approximately 1 Torr. By contrast, f0 responds measurably down to 1e-5 Torr and remains history-dependent, relaxing for hours at fixed pressure. The transients follow stretched-exponential kinetics, consistent with a broad distribution of surface relaxation rates, and individual pressure steps produce fractional frequency shifts as large as 1.9 ppm. Ten-hour phase-locked measurements show a common short-term time-deviation floor near 1e-10 s from 0.01 to 100 Torr, whereas residual deterministic relaxation dominates at long averaging times; at 760 Torr, gas damping degrades short-term tracking through the reduced Q. These results establish gas-damping equilibrium and frequency equilibrium as distinct states. Quality-factor saturation alone is therefore insufficient to qualify vacuum packaging for precision mechanical frequency references; package specifications must also constrain surface-mediated frequency aging.

cond-mat.mes-hall↗

arXiv2026

Coherent phononic frequency combs in ferroelectric CMOS oxides

Modern electronic systems require tens of clock and carrier frequencies, each synthesized by a dedicated phase-locked loop from a shared reference, imposing routing, power and synchronization burdens that grow with every domain. Optical frequency combs solved this problem in photonics, whereas electronics has lacked an equivalent source in its native radiofrequency domain. Here we report broadband phononic frequency combs in ferroelectric hafnia-zirconia nanoelectromechanical resonators built from complementary metal-oxide-semiconductor (CMOS) oxides. Lithographically defined detuning of a 2:1 internal resonance selects the generation mechanism: two-tone-seeded wave mixing yields more than 170 lines distributed over two octaves, with mutual coherence verified for representative pump and generated lines, whereas an autonomous Hopf route yields hierarchical combs of more than 200 lines through torus and period-doubling dynamics, in agreement with slow-flow bifurcation theory. Geometric scaling extends comb generation across 0.44 GHz to 2.1 GHz. Heterodyne measurements, analyzed using the modified Allan deviation (MDEV), establish a two-timescale law. The mechanism governs short-term stability: seeded combs inherit the white-phase-noise scaling of their pumps, whereas autonomous combs acquire the phase diffusion of a free-running oscillator, with the one-second MDEV increasing from 1e-11 to 1e-8. The material governs long-term stability: in air and without active thermal control, the temperature-compensated stack suppresses the random-walk drift that dominates uncompensated resonators. These results establish mechanism- and material-level design rules for operating a single resonator as chip-scale frequency infrastructure, from multi-clock generation to radiofrequency parallel processing.

cond-mat.mes-hall↗

arXiv2026

Coachable agents for interactive gameplay

Reinforcement learning has proven to be a valuable tool in the creation of advanced AI and robotic systems, contributing to everything from game playing to robotics to foundation models. Through trial-and-error, these AI systems typically learn one, near-optimal behavior to solve their tasks. However, there are many use cases in which one would like to assert some level of control, preferably in real time, over how the task is solved. We refer to these modifications of a core task as styles. We combine universal value function approximators (UVFAs) with carefully selected training scenarios, learning algorithms, and data augmentation to create a framework for coaching agents that exhibit styles in complex domains. We demonstrate the framework's application in the AAA video games Horizon Forbidden West and Gran Turismo, and in an open-source humanoid test domain. Despite the different nature of the domains -- car racing, stylized game combat, and humanoid walking -- each agent shows strong coherence to the style requests while still satisfying the main task in its domain. Importantly, the techniques outlined in this paper allow an end user to choose the final behavior at run time, giving them flexible control over the final executed performance.

cs.AI↗

arXiv2026

A perspective on fluid mechanical environments for challenges in reinforcement learning

We consider the challenge of developing agents that efficiently interact with high-dimensional, evolving environments, towards a view of practical reinforcement learning (RL) agents interacting with open worlds, of which they witness and affect only a small part. We argue that canonical fluid mechanics problems, and their simulations, present a compelling testbed for the development of such methods. These problems arise in nonlinear instabilities, where small disturbances can grow to transform the dynamics of a system. Nonlinear instabilities represent several open scientific challenges with industrial applications -- the droplet breakup of a liquid jet, mixing at an interface between two fluids, and the appearance of unusually tall rogue waves in the ocean. In these settings, agents may leverage preserved representations across the changing dynamics to learn efficiently. We present two problem descriptions of agents interacting with a fluid mechanical environment, and describe the state and action spaces, and reward functions, for these agents. For these examples, we specify the aspects of the environment which are nonstationary and the preserved invariances. We note Dedalus and JAX-CFD as open-source simulators that can be used for the development of reinforcement learning methods (Burns et al., 2016; Kochkov et al., 2021)) We demonstrate the use of Dedalus for environment generation by creating RL agents that learn to navigate in a stationary environment that is simulated using Dedalus. This sets the stage for future development of RL agents that learn to meaningfully interact with simulated environments that represent scientific challenges in natural and industrial flows.

cs.LG↗

arXiv2025

Physical Reinforcement Learning

Digital computers are power-hungry and largely intolerant of damaged components, making them potentially difficult tools for energy-limited autonomous agents in uncertain environments. Recently developed Contrastive Local Learning Networks (CLLNs) - analog networks of self-adjusting nonlinear resistors - are inherently low-power and robust to physical damage, but were constructed to perform supervised learning. In this work we demonstrate success on two simple RL problems using Q-learning adapted for simulated CLLNs. Doing so makes explicit the components (beyond the network being trained) required to enact various tools in the RL toolbox, some of which (policy function and value function) are more natural in this system than others (replay buffer). We discuss assumptions such as the physical safety that digital hardware requires, CLLNs can forgo, and biological systems cannot rely on, and highlight secondary goals that are important in biology and trainable in CLLNs, but make little sense in digital computers.

cs.LG↗

arXiv2023

Policy composition in reinforcement learning via multi-objective policy optimization

We enable reinforcement learning agents to learn successful behavior policies by utilizing relevant pre-existing teacher policies. The teacher policies are introduced as objectives, in addition to the task objective, in a multi-objective policy optimization setting. Using the Multi-Objective Maximum a Posteriori Policy Optimization algorithm (Abdolmaleki et al. 2020), we show that teacher policies can help speed up learning, particularly in the absence of shaping rewards. In two domains with continuous observation and action spaces, our agents successfully compose teacher policies in sequence and in parallel, and are also able to further extend the policies of the teachers in order to solve the task. Depending on the specified combination of task and teacher(s), teacher(s) may naturally act to limit the final performance of an agent. The extent to which agents are required to adhere to teacher policies are determined by hyperparameters which determine both the effect of teachers on learning speed and the eventual performance of the agent on the task. In the humanoid domain (Tassa et al. 2018), we also equip agents with the ability to control the selection of teachers. With this ability, agents are able to meaningfully compose from the teacher policies to achieve a superior task reward on the walk task than in cases without access to the teacher policies. We show the resemblance of composed task policies with the corresponding teacher policies through videos.

cs.LG↗

arXiv2023

On Multi-objective Policy Optimization as a Tool for Reinforcement Learning: Case Studies in Offline RL and Finetuning

Many advances that have improved the robustness and efficiency of deep reinforcement learning (RL) algorithms can, in one way or another, be understood as introducing additional objectives or constraints in the policy optimization step. This includes ideas as far ranging as exploration bonuses, entropy regularization, and regularization toward teachers or data priors. Often, the task reward and auxiliary objectives are in conflict, and in this paper we argue that this makes it natural to treat these cases as instances of multi-objective (MO) optimization problems. We demonstrate how this perspective allows us to develop novel and more effective RL algorithms. In particular, we focus on offline RL and finetuning as case studies, and show that existing approaches can be understood as MO algorithms relying on linear scalarization. We hypothesize that replacing linear scalarization with a better algorithm can improve performance. We introduce Distillation of a Mixture of Experts (DiME), a new MORL algorithm that outperforms linear scalarization and can be applied to these non-standard MO problems. We demonstrate that for offline RL, DiME leads to a simple new algorithm that outperforms state-of-the-art. For finetuning, we derive new algorithms that learn to outperform the teacher policy.

cs.LG↗

arXiv2021

Computing the viscous effect in early-time drop impact dynamics

The impact of a liquid drop on a solid surface involves many intertwined physical effects, and is influenced by drop velocity, surface tension, ambient pressure and liquid viscosity, among others. Experiments by Kolinski et al. (2014b) show that the liquid-air interface begins to deviate away from the solid surface even before contact. They found that the lift-off of the interface starts at a critical time that scales with the square root of the kinematic viscosity of the liquid. To understand this, we study the approach of a liquid drop towards a solid surface in the presence of an intervening gas layer. We take a numerical approach to solve the Navier-Stokes equations for the liquid, coupled to the compressible lubrication equations for the gas, in two dimensions. With this approach, we recover the experimentally captured early time effect of liquid viscosity on the drop impact, but our results show that lift-off time and liquid kinematic viscosity have a more complex dependence than the square root scaling relationship. We also predict the effect of interfacial tension at the liquid-gas interface on the drop impact, showing that it mediates the lift-off behavior.

physics.flu-dyn↗

arXiv2020

Coordinated Crawling via Reinforcement Learning

Rectilinear crawling locomotion is a primitive and common mode of locomotion in slender, soft-bodied animals. It requires coordinated contractions that propagate along a body that interacts frictionally with its environment. We propose a simple approach to understand how these coordinations arise in a neuromechanical model of a segmented, soft-bodied crawler via an iterative process that might have both biological antecedents and technological relevance. Using a simple reinforcement learning algorithm, we show that an initial all-to-all neural coupling converges to a simple nearest-neighbor neural wiring that allows the crawler to move forward using a localized wave of contraction that is qualitatively similar to what is observed in D. melanogaster larvae and used in many biomimetic solutions. The resulting solution is a function of how we weight gait regularization in the reward, with a tradeoff between speed and robustness to proprioceptive noise. Overall, our results, which embed the brain-body-environment triad in a learning scheme, has relevance for soft robotics while shedding light on the evolution and development of locomotion.

physics.bio-ph↗

arXiv2019

Augmenting learning using symmetry in a biologically-inspired domain

Invariances to translation, rotation and other spatial transformations are a hallmark of the laws of motion, and have widespread use in the natural sciences to reduce the dimensionality of systems of equations. In supervised learning, such as in image classification tasks, rotation, translation and scale invariances are used to augment training datasets. In this work, we use data augmentation in a similar way, exploiting symmetry in the quadruped domain of the DeepMind control suite (Tassa et al. 2018) to add to the trajectories experienced by the actor in the actor-critic algorithm of Abdolmaleki et al. (2018). In a data-limited regime, the agent using a set of experiences augmented through symmetry is able to learn faster. Our approach can be used to inject knowledge of invariances in the domain and task to augment learning in robots, and more generally, to speed up learning in realistic robotics applications.

cs.LG↗

arXiv2019

Machine Learning in a data-limited regime: Augmenting experiments with synthetic data uncovers order in crumpled sheets

Machine learning has gained widespread attention as a powerful tool to identify structure in complex, high-dimensional data. However, these techniques are ostensibly inapplicable for experimental systems where data is scarce or expensive to obtain. Here we introduce a strategy to resolve this impasse by augmenting the experimental dataset with synthetically generated data of a much simpler sister system. Specifically, we study spontaneously emerging local order in crease networks of crumpled thin sheets, a paradigmatic example of spatial complexity, and show that machine learning techniques can be effective even in a data-limited regime. This is achieved by augmenting the scarce experimental dataset with inexhaustible amounts of simulated data of rigid flat-folded sheets, which are simple to simulate and share common statistical properties. This significantly improves the predictive power in a test problem of pattern completion and demonstrates the usefulness of machine learning in bench-top experiments where data is good but scarce.

cond-mat.soft↗