arXiv subjects

Matthias Mueller

Publications and source records attributed to Matthias Mueller.

6 recordsLinked to original sources

arXiv2024

L-MAGIC: Language Model Assisted Generation of Images with Coherence

In the current era of generative AI breakthroughs, generating panoramic scenes from a single input image remains a key challenge. Most existing methods use diffusion-based iterative or simultaneous multi-view inpainting. However, the lack of global scene layout priors leads to subpar outputs with duplicated objects (e.g., multiple beds in a bedroom) or requires time-consuming human text inputs for each view. We propose L-MAGIC, a novel method leveraging large language models for guidance while diffusing multiple coherent views of 360 degree panoramic scenes. L-MAGIC harnesses pre-trained diffusion and language models without fine-tuning, ensuring zero-shot performance. The output quality is further enhanced by super-resolution and multi-view fusion techniques. Extensive experiments demonstrate that the resulting panoramic scenes feature better scene layouts and perspective view rendering quality compared to related works, with >70% preference in human evaluations. Combined with conditional diffusion models, L-MAGIC can accept various input modalities, including but not limited to text, depth maps, sketches, and colored scripts. Applying depth estimation further enables 3D point cloud generation and dynamic scene exploration with fluid camera motion. Code is available at https://github.com/IntelLabs/MMPano. The video presentation is available at https://youtu.be/XDMNEzH4-Ec?list=PLG9Zyvu7iBa0-a7ccNLO8LjcVRAoMn57s.

cs.CV↗

arXiv2023

Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning

A central question in robotics is how to design a control system for an agile mobile robot. This paper studies this question systematically, focusing on a challenging setting: autonomous drone racing. We show that a neural network controller trained with reinforcement learning (RL) outperformed optimal control (OC) methods in this setting. We then investigated which fundamental factors have contributed to the success of RL or have limited OC. Our study indicates that the fundamental advantage of RL over OC is not that it optimizes its objective better but that it optimizes a better objective. OC decomposes the problem into planning and control with an explicit intermediate representation, such as a trajectory, that serves as an interface. This decomposition limits the range of behaviors that can be expressed by the controller, leading to inferior control performance when facing unmodeled effects. In contrast, RL can directly optimize a task-level objective and can leverage domain randomization to cope with model uncertainty, allowing the discovery of more robust control responses. Our findings allowed us to push an agile drone to its maximum performance, achieving a peak acceleration greater than 12 times the gravitational acceleration and a peak velocity of 108 kilometers per hour. Our policy achieved superhuman control within minutes of training on a standard workstation. This work presents a milestone in agile robotics and sheds light on the role of RL and OC in robot control.

cs.RO↗

arXiv2023

CLNeRF: Continual Learning Meets NeRF

Novel view synthesis aims to render unseen views given a set of calibrated images. In practical applications, the coverage, appearance or geometry of the scene may change over time, with new images continuously being captured. Efficiently incorporating such continuous change is an open challenge. Standard NeRF benchmarks only involve scene coverage expansion. To study other practical scene changes, we propose a new dataset, World Across Time (WAT), consisting of scenes that change in appearance and geometry over time. We also propose a simple yet effective method, CLNeRF, which introduces continual learning (CL) to Neural Radiance Fields (NeRFs). CLNeRF combines generative replay and the Instant Neural Graphics Primitives (NGP) architecture to effectively prevent catastrophic forgetting and efficiently update the model when new data arrives. We also add trainable appearance and geometry embeddings to NGP, allowing a single compact model to handle complex scene changes. Without the need to store historical images, CLNeRF trained sequentially over multiple scans of a changing scene performs on-par with the upper bound model trained on all scans at once. Compared to other CL baselines CLNeRF performs much better across standard benchmarks and WAT. The source code, and the WAT dataset are available at https://github.com/IntelLabs/CLNeRF. Video presentation is available at: https://youtu.be/nLRt6OoDGq0?si=8yD6k-8MMBJInQPs

cs.CV↗

arXiv2020

Is the Juice Worth the Squeeze? Machine Learning (ML) In and For Agent-Based Modelling (ABM)

In recent years, many scholars praised the seemingly endless possibilities of using machine learning (ML) techniques in and for agent-based simulation models (ABM). To get a more comprehensive understanding of these possibilities, we conduct a systematic literature review (SLR) and classify the literature on the application of ML in and for ABM according to a theoretically derived classification scheme. We do so to investigate how exactly machine learning has been utilized in and for agent-based models so far and to critically discuss the combination of these two promising methods. We find that, indeed, there is a broad range of possible applications of ML to support and complement ABMs in many different ways, already applied in many different disciplines. We see that, so far, ML is mainly used in ABM for two broad cases: First, the modelling of adaptive agents equipped with experience learning and, second, the analysis of outcomes produced by a given ABM. While these are the most frequent, there also exist a variety of many more interesting applications. This being the case, researchers should dive deeper into the analysis of when and how which kinds of ML techniques can support ABM, e.g. by conducting a more in-depth analysis and comparison of different use cases. Nonetheless, as the application of ML in and for ABM comes at certain costs, researchers should not use ML for ABMs just for the sake of doing it.

econ.TH↗

arXiv2018

Rock bottom, the world, the sky: Catrobat, an extremely large-scale and long-term visual coding project relying purely on smartphones

Most of the 700 million teenagers everywhere in the world already have their own smartphones, but comparatively few of them have access to PCs, laptops, OLPCs, Chromebooks, or tablets. The free open source non-profit project Catrobat allows users to create and publish their own apps using only their smartphones. Initiated in 2010, with first public versions of our free apps since 2014 and 47 releases of the main coding app as of July 2018, Catrobat currently has more than 700,000 users from 180 countries, is available in 50+ languages, and has been developed so far by almost 1,000 volunteers from around the world ("the world"). Catrobat is strongly inspired by Scratch and indeed allows to import most Scratch projects, thus giving access to more than 30 million projects on our users' phones as of July 2018. Our apps are very intuitive ("rock bottom"), have many accessibility settings, e.g., for kids with visual or cognitive impairments, and there are tons of constructionist tutorials and courses in many languages. We also have created a plethora of extensions, e.g., for various educational robots, including Lego Mindstorms and flying Parrot quadcopters ("the sky"), as well as for controlling arbitrary external devices through Arduino or Raspberry Pi boards, going up to the stratosphere and even beyond to interplanetary space ("the sky"). A TurtleStitch extension allowing to code one's own embroidery patterns for clothes is currently being developed. Catrobat among others intensely focuses on including female teenagers. While a dedicated version for schools is being developed, our apps are meant to be primarily used outside of class rooms, anywhere and in particular outdoors ("rock bottom", "the world"). Catrobat is discovered by our users through various app stores such as Google Play and via social media channels such as YouTube as well as via our presence on Code.org.

cs.PL↗

arXiv2013

Optical Observations of the Transiting Exoplanet GJ 1214b

We observed nine primary transits of the super-Earth exoplanet GJ 1214b in several optical photometric bands from March to August 2012, with the goal of constraining the short-wavelength slope of the spectrum of GJ 1214b. Our observations were conducted on the Kuiper 1.55 m telescope in Arizona and the STELLA-I robotic 1.2 m telescope in Tenerife, Spain. From the derived light curves we extracted transit depths in R (0.65 μm), V (0.55 μm), and g' (0.475 μm) bands. Most previous observations of this exoplanet suggest a flat spectrum varying little with wavelength from the near-infrared to the optical, corresponding to a low-scale-height, high-molecular-weight atmosphere. However, a handful of observations around Ks band (~2.15 μm) and g-band (~0.46 μm) are inconsistent with this scenario and suggest a variation on a hydrogen- or water-dominated atmosphere that also contains a haze layer of small particles. In particular, the g-band observations of de Mooij et al. (2012), consistent with Rayleigh scattering, limit the potential atmosphere compositions of GJ 1214b due to the increasing slope at optical wavelengths (Howe & Burrows 2012). We find that our results overlap within errors the short-wavelength observations of de Mooij et al. (2012), but are also consistent with a spectral slope of zero in GJ 1214b in the optical wavelength region. Our observations thus allow for a larger suite of possible atmosphere compositions, including those with a high-molecular-weight and/or hazes.

astro-ph.EP↗