3D Modelling: Artificial Intelligence Meets Consumer Electronics

The images are still pretty rough, with fuzzy details, ragged edges and white gaps here and there. But they’ll get better over time, says robotics researcher Rasmus Göransson, as users all over the world come to the Kinect@Home website to upload scans of people, places and things in their surroundings. “Those scans allow us to adapt our algorithms, he says. “We’re teaching robotic software how to interpret digital streams from the eyes of a Kinect camera.”

 And for the price, even rough images look pretty good. Because that price is free, as in gratis, no charge, complimentary, on the cuff. Proof that the Internet loves free—if any was needed—is seen in the rush of uploaders to kinectathome.com since it went live a few days ago. The site has gone from zero to several thousand hits a day in less than two weeks.

The Kinect is not only a video camera, but also an infrared laser motion sensor that lets gamers control screen action with body movements. Göransson and his robotics team are hijacking that novel combination to create screen images that can be turned for viewing from the sides, top and bottom.

“Big data is where it’s at in artificial intelligence,” Göransson says. “Google gets that, but no one is gathering big data yet for 3D spaces. The gaps in the models show how technically hard this is. The human brain fills in the blanks when you look at objects, but computers and cameras can’t do that very well yet. That’s where big data comes in.” 

If robots are going to be able to navigate in homes and workplaces, they need to see and learn from a lot of different scenarios. “We couldn’t possibly pay people to gather all these scans, but we’re crowdsourcing the problem by giving users something in return,” says Alper Aydemir, a PhD student on the Kinect@Home team. “This database of scanned scenes will give robots a library to help them recognise room layouts and furniture, the human form, and even small objects like cups or hand tools.”

Kinect@Home is a cloud-based service, which means the images are not processed in software installed on the user’s computer, but across the Internet on a server dedicated to a single task. User video is uploaded in near-real time on a standard broadband connection, and processing is almost as quick. A typical one-minute Kinect room scan takes about two or three minutes to convert into a 3D model for delivery back to the user, who can then embed it into a web page, send it in an email or use it in a creative production.

The developers say they expect to see an explosion in applications for the service, because these 3D models are not simple bit-mapped images, where each dot is defined with a colour, but vector graphics, where elements are described by mathematical definitions. That means they can be imported into CAD drawing programs for editing and integration with other photos, videos and games. “Within a few years, 3D models embedded in web pages should be as common as still pictures and videos are today,” Göransson says.

For the moment, Aydemir and Göransson are trying to focus on the immediate demands of the project amidst a flood of media interest—Wired magazine, the BBC, Fast Company and NBC are just a few of the dozens of news outlets and blogs that have written about the new site—but they don’t mind speculating about how it might someday be used in interior design, 3D printing, indoor maps and remote inspection of construction projects.

“Google has already mapped almost the entire planet and put it online, and they’ve just launched a new service called Google Indoor Maps to show building layouts,” says Göransson. “We hope our research will contribute to things like that.”

Models aren’t all the researchers are planning to give away. The anonymised data will be available as open source, so anyone else can come along and try to do a better job of interpretation. A researcher teaching robots to recognise household objects or tell the difference between walls, floors and ceilings will be able to download models from the Kinect@Home library—including the underlying data stream—to see how well their artificial intelligence software is able to process the real world.

Answering a question about the commercial possibilities, Aydemir says, “This field is still in its infancy. We’re giving the data away because we want to help drive innovation and we gain a lot from other researchers’ work. That’s how science works. We’re using open source software and giving back to that community. Media interest shows that people are interested and we know there are opportunities here, but this research is our day job for now.”

By Kevin Billinghurst | kb2@kth.se