A robot learns its way around the lab

A striking blue robot named Rosie has made itself at home for the last couple of years in the Robotics, Perception and Learning lab at KTH, as part of a project in which robots are learning how to perceive three dimensional environments and move about and interact in them — even in the course of routine afternoons where rooms are randomly closed off, students and researchers bustle around, and things like cups, soda cans and lab gear accumulate and, just as unpredictably, get cleared away.

Unlike you or me, robots actually have to learn things like: chairs change positions from hour to hour, or that a book resting on a table is not part of the table.

So Rosie — a Scitos G5 built by MetraLabs — maps the rooms at RPL, revisiting them repeatedly. She documents everything with the help of a depth camera (RGB-D), and dumps billions of points of physical space into a database, from which 3D models of the rooms can be generated.

The system the KTH researchers use detects objects to learn by modeling the static part of the environment and extracting dynamic elements. It then creates and executes a view plan around a dynamic element to gather additional views for learning.

This autonomous learning process enables Rosie distinguish dynamic elements from static ones and to perceive depth and distance. She learns when things are where they are, and how to negotiate physical spaces.

Beyond that, Rosie also is building an understanding of what kind of spaces she is in — be they office spaces, kitchens or corridors.

The project is called STRANDS and I spent an afternoon at RPL with Rosie and some of the researchers working on it, Johan Ekenkrantz, Nils Bore and Hakan Karaoguz. They helped me make this short video.

Here’s a recent paper from the STRANDS project that explains how robots can learn how to adapt to our dynamic, three dimensional world:

“Autonomous Learning of Object Models on a Mobile Robot” In: IEEE Robotics and Automation Letters. Authors: T. Fäulhammer, R. Ambrus, C. Burbridge, M. Zillich, J. Folkesson, N. Hawes, P. Jensfelt, M. Vincze.

David Callahan