arXiv subjects

Jorge Ramírez-Ruiz

Publications and source records attributed to Jorge Ramírez-Ruiz.

5 recordsLinked to original sources

arXiv2024

Complex behavior from intrinsic motivation to occupy action-state path space

Most theories of behavior posit that agents tend to maximize some form of reward or utility. However, animals very often move with curiosity and seem to be motivated in a reward-free manner. Here we abandon the idea of reward maximization, and propose that the goal of behavior is maximizing occupancy of future paths of actions and states. According to this maximum occupancy principle, rewards are the means to occupy path space, not the goal per se; goal-directedness simply emerges as rational ways of searching for resources so that movement, understood amply, never ends. We find that action-state path entropy is the only measure consistent with additivity and other intuitive properties of expected future action-state path occupancy. We provide analytical expressions that relate the optimal policy and state-value function, and prove convergence of our value iteration algorithm. Using discrete and continuous state tasks, including a high--dimensional controller, we show that complex behaviors such as `dancing', hide-and-seek and a basic form of altruistic behavior naturally result from the intrinsic motivation to occupy path space. All in all, we present a theory of behavior that generates both variability and goal-directedness in the absence of reward maximization.

cs.AI↗

arXiv2023

A general Markov decision process formalism for action-state entropy-regularized reward maximization

Previous work has separately addressed different forms of action, state and action-state entropy regularization, pure exploration and space occupation. These problems have become extremely relevant for regularization, generalization, speeding up learning and providing robust solutions at unprecedented levels. However, solutions of those problems are hectic, ranging from convex and non-convex optimization, and unconstrained optimization to constrained optimization. Here we provide a general dual function formalism that transforms the constrained optimization problem into an unconstrained convex one for any mixture of action and state entropies. The cases with pure action entropy and pure state entropy are understood as limits of the mixture.

cs.LG↗

arXiv2021

Optimal allocation of finite sampling capacity in accumulator models of multi-alternative decision making

When facing many options, we narrow down our focus to very few of them. Although behaviors like this can be a sign of heuristics, they can actually be optimal under limited cognitive resources. Here we study the problem of how to optimally allocate limited sampling time to multiple options, modelled as accumulators of noisy evidence, to determine the most profitable one. We show that the effective sampling capacity of an agent increases with both available time and the discriminability of the options, and optimal policies undergo a sharp transition as a function of it. For small capacity, it is best to allocate time evenly to exactly five options and to ignore all the others, regardless of the prior distribution of rewards. For large capacities, the optimal number of sampled accumulators grows sub-linearly, closely following a power law for a wide variety of priors. We find that allocating equal times to the sampled accumulators is better than using uneven time allocations. Our work highlights that multi-alternative decisions are endowed with breadth-depth tradeoffs, demonstrates how their optimal solutions depend on the amount of limited resources and the variability of the environment, and shows that narrowing down to a handful of options is always optimal for small capacities.

q-bio.NC↗

arXiv2017

NMR in an electric field: A bulk probe of the hidden spin and orbital polarizations

Recent theoretical work has established the presence of hidden spin and orbital textures in non-magnetic materials with inversion symmetry. Here, we propose that these textures can be detected by nuclear magnetic resonance (NMR) measurements carried out in the presence of an electric field. In crystals with hidden polarizations, a uniform electric field produces a staggered magnetic field that points to opposite directions at atomic sites related by spatial inversion. As a result, the NMR resonance peak corresponding to inversion partner nuclei is split into two peaks. The magnitude of the splitting is proportional to the electric field and depends on the orientation of the electric field with respect to the crystallographic axes and the external magnetic field. As a case study, we present a theory of electric-field-induced splitting of NMR peaks for $^{77}$Se, $^{125}$Te and $^{209}$Bi in Bi$_2$Se$_3$ and Bi$_2$Te$_3$. In conducting samples with current densities of $\simeq 10^6\, {\rm A/cm}^2$, the splitting for Bi can reach $100\, {\rm kHz}$, which is comparable to or larger than the intrinsic width of the NMR lines. In order to observe the effect experimentally, the peak splitting must also exceed the linewidth produced by the Oersted field. In Bi$_2$Se$_3$, this requires narrow wires of radius $\lesssim 1\, μ{\rm m}$. We also discuss other potentially more promising candidate materials, such as SrRuO$_3$ and BaIr$_2$Ge$_2$, whose crystal symmetry enables strategies to suppress the linewidth produced by the Oersted field.

cond-mat.mtrl-sci↗

arXiv2016

Tight-binding theory of NMR shifts in topological insulators Bi2Se3 and Bi2Te3

Motivated by recent nuclear magnetic resonance (NMR) experiments, we present a microscopic sp3 tight-binding model calculation of the NMR shifts in bulk Bi2Se3, and Bi2Te3. We compute the contact, dipolar, orbital and core polarization contributions to the carrier-density-dependent part of the NMR shifts in Bi209, Te125 and Se77. The spin-orbit coupling and the layered crystal structure result in a contact Knight shift with strong uniaxial anisotropy. Likewise, because of spin-orbit coupling, dipolar interactions make a significant contribution to the isotropic part of the NMR shift. The contact interaction dominates the isotropic Knight shift in Bi209 NMR, even though the electronic states at the Fermi level have a rather weak s-orbital character. In contrast, the contribution from the contact hyperfine interaction to the NMR shift of Se77 and Te125 is weak compared to the dipolar and orbital shifts therein. In all cases, the orbital shift is at least comparable to the contact and dipolar shifts, while the shift due to core polarization is subdominant (except for Te nuclei located at the inversion centers). By artificially varying the strength of spin-orbit coupling, we evaluate the evolution of the NMR shift across a band inversion but find no clear signature of the topological transition.

cond-mat.mes-hall↗