arXiv subjects

David Harel

Publications and source records attributed to David Harel.

26 records · Page 2Linked to original sources

arXiv2022

Constrained Reinforcement Learning for Robotics via Scenario-Based Programming

Deep reinforcement learning (DRL) has achieved groundbreaking successes in a wide variety of robotic applications. A natural consequence is the adoption of this paradigm for safety-critical tasks, where human safety and expensive hardware can be involved. In this context, it is crucial to optimize the performance of DRL-based agents while providing guarantees about their behavior. This paper presents a novel technique for incorporating domain-expert knowledge into a constrained DRL training loop. Our technique exploits the scenario-based programming paradigm, which is designed to allow specifying such knowledge in a simple and intuitive way. We validated our method on the popular robotic mapless navigation problem, in simulation, and on the actual platform. Our experiments demonstrate that using our approach to leverage expert knowledge dramatically improves the safety and the performance of the agent.

cs.RO↗

arXiv2022

Scenario-Assisted Deep Reinforcement Learning

Deep reinforcement learning has proven remarkably useful in training agents from unstructured data. However, the opacity of the produced agents makes it difficult to ensure that they adhere to various requirements posed by human engineers. In this work-in-progress report, we propose a technique for enhancing the reinforcement learning training process (specifically, its reward calculation), in a way that allows human engineers to directly contribute their expert knowledge, making the agent under training more likely to comply with various relevant constraints. Moreover, our proposed approach allows formulating these constraints using advanced model engineering techniques, such as scenario-based modeling. This mix of black-box learning-based tools with classical modeling approaches could produce systems that are effective and efficient, but are also more transparent and maintainable. We evaluated our technique using a case-study from the domain of internet congestion control, obtaining promising results.

cs.LG↗

arXiv2020

Integrating Inter-Object Scenarios with Intra-object Statecharts for Developing Reactive Systems

In all software development projects, engineers face the challenge of translating the requirements layer into a design layer, then into an implementation-code layer, and then validating the correctness of the result. Many methodologies, languages and tools exist for facilitating the process, including multiple back-and-forth `refinement trips' across the requirements, design and implementation layers, by focusing on formalizing the artifacts involved and on automating a variety of tasks throughout. In this paper, we introduce a novel and unique development environment, which integrates scenario-based programming (SBP) via the LSC language and the object-oriented, visual Statecharts formalism, for the development of reactive systems. LSC targets creation of models and systems directly from requirement specifications, and Statecharts is used mainly for specifying final component behavior. Our integration enables semantically-rich joint execution, with the sharing and interfacing of objects and events, and can be used for creating and then gradually enhancing testable models from early in requirements elicitation through detailed design. In some cases, it can be used for generating final system code. We describe the technical details of the integration and its semantics and discuss its significance for future development methodologies.

cs.SE↗

arXiv2020

Expecting the Unexpected: Developing Autonomous-System Design Principles for Reacting to Unpredicted Events and Conditions

When developing autonomous systems, engineers and other stakeholders make great effort to prepare the system for all foreseeable events and conditions. However, these systems are still bound to encounter events and conditions that were not considered at design time. For reasons like safety, cost, or ethics, it is often highly desired that these new situations be handled correctly upon first encounter. In this paper we first justify our position that there will always exist unpredicted events and conditions, driven among others by: new inventions in the real world; the diversity of world-wide system deployments and uses; and, the non-negligible probability that multiple seemingly unlikely events, which may be neglected at design time, will not only occur, but occur together. We then argue that despite this unpredictability property, handling these events and conditions is indeed possible. Hence, we offer and exemplify design principles that when applied in advance, can enable systems to deal, in the future, with unpredicted circumstances. We conclude with a discussion of how this work and a broader theoretical study of the unexpected can contribute toward a foundation of engineering principles for developing trustworthy next-generation autonomous systems.

cs.SE↗

arXiv2019

Autonomics: In Search of a Foundation for Next Generation Autonomous Systems

The potential benefits of autonomous systems have been driving intensive development of such systems, and of supporting tools and methodologies. However, there are still major issues to be dealt with before such development becomes commonplace engineering practice, with accepted and trustworthy deliverables. We argue that a solid, evolving, publicly available, community-controlled foundation for developing next generation autonomous systems is a must. We discuss what is needed for such a foundation, identify a central aspect thereof, namely, decision-making, and focus on three main challenges: (i) how to specify autonomous system behavior and the associated decisions in the face of unpredictability of future events and conditions and the inadequacy of current languages for describing these; (ii) how to carry out faithful simulation and analysis of system behavior with respect to rich environments that include humans, physical artifacts, and other systems,; and (iii) how to engineer systems that combine executable model-driven techniques and data-driven machine learning techniques. We argue that autonomics, i.e., the study of unique challenges presented by next generation autonomous systems, and research towards resolving them, can introduce substantial contributions and innovations in system engineering and computer science.

cs.SE↗

arXiv2019

Deciphering the underlying mechanisms of the pharyngeal motions in Caenorhabditis elegans

The pharynx of the nematode Caenorhabditis elegans is a neuromuscular pump that exhibits two typical motions: pumping and peristalsis. While the dynamics of these motions are well characterized, the underlying mechanisms generating most of them are not known. In this paper, we propose comprehensive and detailed mechanisms that can explain the various observed dynamics of the different pharyngeal areas: the dynamics of the pumping muscles - corpus, anterior isthmus, and terminal bulb - and the peristalsis dynamics of the posterior isthmus muscles. While the suggested mechanisms are consistent with all available relevant data, the assumptions on which they are based and the open questions they raise could point at additional interesting research directions on the C. elegans pharynx. We are hoping that appropriate experiments on the nematode will eventually corroborate our results, and improve our understanding of the functioning of the C. elegans pharynx, and possibly of the mammalian digestive system.

q-bio.NC↗

arXiv2016

Niépce-Bell or Turing: How to Test Odor Reproduction?

In a 1950 article in Mind, decades before the existence of anything resembling an artificial intelligence system, Alan Turing addressed the question of how to test whether machines can think, or in modern terminology, whether a computer claimed to exhibit intelligence indeed does so. The current paper raises the analogous issue for olfaction: how to test the validity of a system claimed to reproduce arbitrary odors artificially, in a way recognizable to humans, in face of the unavailability of a general naming method for odors. Although odor reproduction systems are still far from being viable, the question of how to test candidates thereof is claimed to be interesting and nontrivial, and a novel method is proposed. To some extent, the method is inspired by Turing`s test for AI, in that it involves a human challenger and the real and artificial entities, yet it is very different: our test is conditional, requiring from the artificial no more than is required from the original, and it employs a novel method of immersion that takes advantage of the availability of near-perfect reproduction methods for sight and sound.

cs.OH↗

arXiv2015

Wise Computing: Towards Endowing System Development with True Wisdom

Encouraged by significant advances in algorithms and tools for verification and analysis, high level modeling and programming techniques, natural language programming, etc., we feel it is time for a major change in the way complex software and systems are developed. We present a vision that will shift the power balance between human engineers and the development and runtime environments. The idea is to endow the computer with human-like wisdom - not general wisdom, and not AI in the standard sense of the term - but wisdom geared towards classical system-building, which will be manifested, throughout development, in creativity and proactivity, and deep insights into the system's own structure and behavior, its overarching goals and rationale. Ideally, the computer will join the development team as an equal partner - knowledgeable, concerned, and responsibly active. We present a running demo of our initial efforts on the topic, illustrating on a small example what we feel is the feasibility of the ideas.

cs.SE↗