AI News, Activity simulator could eventually teach robots tasks like making coffee or setting the table

Activity simulator could eventually teach robots tasks like making coffee or setting the table

Recently, computer scientists have been working on teaching machines to do a wider range of tasks around the house.

In a new paper spearheaded by MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) and the University of Toronto, researchers demonstrate 'VirtualHome,' a system that can simulate detailed household tasks and then have artificial 'agents' execute them, opening up the possibility of one day teaching robots to do such tasks.

The team trained the system using nearly 3,000 programs of various activities, which are further broken down into subtasks for the computer to understand.

The team's AI agent can execute 1,000 of these interactions in the Sims-style world, with eight different scenes including a living room, kitchen, dining room, bedroom, and home office.

'Describing actions as computer programs has the advantage of providing clear and unambiguous descriptions of all the steps needed to complete a task,' says PhD student Xavier Puig, who was lead author on the paper.

The team's model successfully demonstrated that, their agents could learn to reconstruct a program, and therefore perform a task, given either a description: 'pour milk into glass,' or a video demonstration of the activity.

In the future, the team hopes to train the robots using actual videos instead of Sims-style simulation videos, which would enable a robot to learn simply by watching a YouTube video.

Teaching chores to an artificial agent

For many people, household chores are a dreaded, inescapable part of life that we often put off or do with little care.

In a new paper spearheaded by MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and the University of Toronto, researchers demonstrate “VirtualHome,” a system that can simulate detailed household tasks and then have artificial “agents” execute them, opening up the possibility of one day teaching robots to do such tasks.

The team’s artificial agent can execute 1,000 of these interactions in the Sims-style world, with eight different scenes including a living room, kitchen, dining room, bedroom, and home office.

For example, one might tell a human to “switch on the TV and watch it from the sofa.” Here, actions like “grab the remote control” and “sit/lie on sofa” have been omitted, since they’re part of the commonsense knowledge that humans have.

The team’s model successfully demonstrated that their agents could learn to reconstruct a program, and therefore perform a task, given either a description: “pour milk into glass” or a video demonstration of the activity.

This allows the robot to do tasks in a personalized way, or even some day invoke an emotional connection as a result of this personalized learning process.” In the future, the team hopes to train the robots using actual videos instead of Sims-style simulation videos, which would enable a robot to learn simply by watching a YouTube video.

VirtualHome simulator could teach robots about household tasks

Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and the University of Toronto are developing a 3D simulator that could eventually teach robots how to complete household tasks like making coffee or setting the table.

“Describing actions as computer programs has the advantage of providing clear and unambiguous descriptions of all the steps needed to complete a task,” said PhD student Xavier Puig, who was lead author on the paper.

Of the 3,000 programs, the team’s AI agent can already execute 1,000 separate sets of actions in eight different scenes, which include a living room, kitchen, dining room, bedroom, and home office.

This allows the robot to do tasks in a personalized way, or even some day invoke an emotional connection as a result of this personalized learning process.” In the future, the team hopes to train the robots using actual videos instead of Sims-style simulation videos, which would enable a robot to learn simply by watching a YouTube video.

Robot See, Robot Do: How Robots Can Learn New Tasks by Observing

It can take weeks to reprogram an industrial robot to perform a complicated new task, which makes retooling a modern manufacturing line painfully expensive and slow.

“We ask an expert to show the robot a task, and let the robot figure out most parts of sequences of things it needs to do, and then fine-tune things to make it work.” At a recent conference in St. Louis, the researchers demonstrated a cocktail-making robot that uses the approaches they’re working on.

The robot—a two-armed industrial machine made by a Boston-based company called Rethink Robotics, watched a person mix a drink by pouring liquid from several bottles into a jug, and would then copy those actions, grasping bottles in the correct order before pouring the right quantities into the jug.

Watching thousands of YouTube videos may sound time-consuming, but the learning approach is more efficient than programming a robot to handle countless different items, and it can enable a robot to deal with a new object.

The learning systems used for the grasping work involved advanced artificial neural networks, which have seen rapid progress in recent years and are now being used in many areas of robotics.

No More Chores: Robot ‘Agents’ Will Do Them All

The researchers have developed a virtual system that can teach robots to perform detailed household chores, potentially turning that fantasy into reality.

Inspired by the Sims video game, the system, called VirtualHome, is a 3-D house environment capable of generating videos and simulating different household tasks that can be performed by a virtual “agent.”

“We use these programs to instruct virtual characters to execute tasks in a simulated household environment and show how these tasks can be reconstructed from either written descriptions or even video demonstrations.”

The virtual agent can complete 1,000 of these actions throughout eight different scenes, including a living room, kitchen, dining room, bedroom, and home office.

“Our model successfully demonstrated that the agents could learn to reconstruct a program, and therefore perform a task, given either a description: ‘pour milk into glass’

Since VirtualHome includes a large database of detailed instructions and videos that can teach a virtual agent, the team now hopes to use this information to teach robots how to perform the tasks.

“You could imagine a robot being able to anticipate what will happen in a home and acting as an assistive technology by anticipating wants and needs, especially for the elderly,”

Kurzweilaccelerating intelligence

Computer scientists at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and the University of Toronto* have created a Sims-inspired “VirtualHome” system that can simulate detailed household tasks.

Using crowdsourcing, the researchers created videos that simulate detailed household activities and sub-tasks in eight different scenes, including a living room, kitchen, dining room, bedroom, and home office.  A simple model can generate a program from either a video or a textual description, allowing robots to be programmed by naive users, either via natural language or video demonstration.

The researchers have trained the system using nearly 3,000 programs for various activities, which are further broken down into subtasks for the computer to understand.

Humans Need Not Apply

Discuss this video:

Top 10 Tips for Getting Into The Video Game Industry

Want to get into the video game industry but don't know where to start? Well with a little help from Smite Developer Hi-Rez Studio, we're here to give you some ...

Day in the Life: Software Engineer

Victoria S. creates aesthetically captivating user interfaces for Homejoy's website. Watch our latest Day in the Life release and find out what it is like to be a ...

RI Seminar : Sven Koenig: Progress on Multi-Robot Path Finding

Sven Koenig Professor, Computer Science Department, University of Southern California (USC) Abstract Teams of robots often have to assign target locations ...

Programming for Robotics (ROS) Course 1

The slides are available here: The ..

Will your job get outsourced to a robot?

It's not just basic tasks anymore: Computers can now do work once deemed possible only by humans. And in some cases, the computers are doing it better.

New AI system can train robots for armies

Scientists develop that will teach robots and computer programmes to interact with a human instructor for the purpose. Scientists have developed an artificial ...

Robot Systems Design Course 2016

The video shows a single production cycle of a simulated factory developed by Robotics and UAS students at the University of Southern Denmark. A production ...

The Rise of the Machines – Why Automation is Different this Time

Automation in the Information Age is different. Books we used for this video: The Rise of the Robots: The Second Machine Age: ..

ROBOTICS AT PLYMOUTH UNIVERSITY

Title: Meng (hons), BEng (hons) Robotics, BSc (Hons) Robotics Course type: Full-time Duration: 3 years (optional placement) Award: Master of Engineering ...