与多机器人系统的直观与高效互动

INTUITIVE AND EFFECTIVE INTERACTIONWITH MULTI-ROBOT SYSTEMS DISS. ETH NO.30570 Computational Robotics LabDepartment of Computer ScienceETH Zürich8092Switzerland Cover illustration by Beat Reichenbach. ©2024Florian Kennel-Maushart:Intuitive and Effective Interaction with Multi-RobotSystems. All rights reserved. diss.eth no.30570 I N T U I T I V E A N D E F F E C T I V E I N T E R A C T I O NW I T H M U LT I - R O B O T S Y S T E M S A thesis submitted to attain the degree of doctor of sciences(Dr. sc. ETH Zurich) presented by florian kennel-maushart MSc in Microengineering, École Polytechnique Fédérale de Lausanne born on11September1991 accepted on the recommendation of Prof. Dr. Stelian Coros, examinerProf. Dr. Marc Pollefeys, co-examinerProf. Dr. Amanda Prorok, co-examiner Many tasks in our everyday lives require us to interact with others. In-teraction, orcollaboration, allows us to combine our strengths and enablesus to achieve more than what an individual could achieve on their own.Similarly, when trying to automate such tasks, it might be impossible orprohibitively expensive to just employ one single robot. But coordinatingmultiple robots is difficult: From the computational challenges of the largestate-space of a multi-robot system to the question of how human inputfrom one operator can be mapped to multiple robots, there are multiplelevels to an interaction that might be subtle when interacting with humans,but need to be explicitly addressed when working with robots.The goal of this thesis is to tackle some of the most pertinent of these prob- lems. We first take a look at some challenges related to directteleoperationof single and multi-robot systems. Using manipulability optimization and anullspace projection scheme, we improve the ability of multi-arm systemsto avoid singularities and follow operator input more swiftly. We present afast, local optimization scheme as well as a second-order optimization usingNewton’s method and compare the performance of both and show howthe method performs on a real ABB YuMi bimanual robot. We then extendour system to mobile robots. While mobile robots have many advantagesover fixed-base systems, they also introduce additional complexities. Weexamine the driving dynamics for non-holonomic skid-steering robots andhow the manipulator pose and payload influence the steering capabilities.Additionally, we introduce terms to avoid inter-agent collision, tip-overavoidance and the ability to move through highly constrained environ-ments. We deploy our combined method on multiple Clearpath Huskyplatforms with UR5e manipulators and show how it improves the open-loop performance over traditional control schemes. Finally, we investigatein more detail how users can interact with multi-robot systems. We comparea tablet-based Augmented Reality solution to a HoloLens Mixed Realityheadset. Presenting study participants with different touch interactions aswell as hand-tracking enabled gestures, we compare their performance onmultiple tasks with the goal of guiding a team of tiny, differential driverobots. We compare the objective and subjective performance across thedifferent tasks and show that especially spatial interactions benefit from a3-dimensional user interface. In conclusion, we explore and investigate the interplay of humans and robots for effective and intuitive interaction withmulti-robot systems at multiple levels. We improve the ability of robotsto follow user input, present a methodology for trajectory optimizationof a highly complex, non-holonomic multi-robot system and investigateintuitive gesture-based interactions with mobile robots. And we deploy ourmethods on real robots, with the goal of hopefully bringing them to realbuilding sites or warehouses in the future, to improve the workplace oftomorrow. Viele Aufgaben in unserem Alltag lassen sich nur gemeinsam bewältigen.Interaktion, oderKollaboration, ermöglicht es uns, unsere Stärken zu kombi-nieren und mehr zu erreichen, als eine Einzelperson allein könnte. Ebensokönnte es bei der Automatisierung solcher Aufgaben unmöglich oder unver-hältnismässig teuer sein, nur einen einzelnen Roboter einzusetzen. Aber dieKoordination mehrerer Roboter ist anspruchsvoll: Von den mathematischenHerausforderungen des großen Konfigurationsraumes eines Multi-Roboter-Systems bis hin zur Frage, wie menschliche Eingaben eines Nutzers aufmehrere Roboter abgebildet werden können, gibt es mehrere Ebenen derInteraktion, die beim Umgang mit Menschen subtil sein mögen, aber beimArbeiten mit Robotern explizit behandelt werden müssen.Das Ziel dieser Arbeit ist es, einige der drängendsten dieser Proble- me anzugehen. Zunächst betrachten wir einige Herausforderungen imZusammenhang mit der direktenTeleoperationvon Einzel- und Multi-Roboter-Systemen. Durch die Optimierung der Manipulierbarkeit undeinem Nullraumprojektionsschema verbessern wir die Fähigkeit von Multi-Arm-Systemen, Singularitäten zu vermeiden und Eingaben des Opera