AI News, Teaching chores to an artificial agent

Teaching chores to an artificial agent

For many people, household chores are a dreaded, inescapable part of life that we often put off or do with little care.

In a new paper spearheaded by MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and the University of Toronto, researchers demonstrate “VirtualHome,” a system that can simulate detailed household tasks and then have artificial “agents” execute them, opening up the possibility of one day teaching robots to do such tasks.

The team’s artificial agent can execute 1,000 of these interactions in the Sims-style world, with eight different scenes including a living room, kitchen, dining room, bedroom, and home office.

For example, one might tell a human to “switch on the TV and watch it from the sofa.” Here, actions like “grab the remote control” and “sit/lie on sofa” have been omitted, since they’re part of the commonsense knowledge that humans have.

The team’s model successfully demonstrated that their agents could learn to reconstruct a program, and therefore perform a task, given either a description: “pour milk into glass” or a video demonstration of the activity.

This allows the robot to do tasks in a personalized way, or even some day invoke an emotional connection as a result of this personalized learning process.” In the future, the team hopes to train the robots using actual videos instead of Sims-style simulation videos, which would enable a robot to learn simply by watching a YouTube video.

Activity simulator could eventually teach robots tasks like making coffee or setting the table

Recently, computer scientists have been working on teaching machines to do a wider range of tasks around the house.

In a new paper spearheaded by MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) and the University of Toronto, researchers demonstrate 'VirtualHome,' a system that can simulate detailed household tasks and then have artificial 'agents' execute them, opening up the possibility of one day teaching robots to do such tasks.

The team trained the system using nearly 3,000 programs of various activities, which are further broken down into subtasks for the computer to understand.

The team's AI agent can execute 1,000 of these interactions in the Sims-style world, with eight different scenes including a living room, kitchen, dining room, bedroom, and home office.

'Describing actions as computer programs has the advantage of providing clear and unambiguous descriptions of all the steps needed to complete a task,' says PhD student Xavier Puig, who was lead author on the paper.

The team's model successfully demonstrated that, their agents could learn to reconstruct a program, and therefore perform a task, given either a description: 'pour milk into glass,' or a video demonstration of the activity.

In the future, the team hopes to train the robots using actual videos instead of Sims-style simulation videos, which would enable a robot to learn simply by watching a YouTube video.

VirtualHome simulator could teach robots about household tasks

Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and the University of Toronto are developing a 3D simulator that could eventually teach robots how to complete household tasks like making coffee or setting the table.

“Describing actions as computer programs has the advantage of providing clear and unambiguous descriptions of all the steps needed to complete a task,” said PhD student Xavier Puig, who was lead author on the paper.

Of the 3,000 programs, the team’s AI agent can already execute 1,000 separate sets of actions in eight different scenes, which include a living room, kitchen, dining room, bedroom, and home office.

This allows the robot to do tasks in a personalized way, or even some day invoke an emotional connection as a result of this personalized learning process.” In the future, the team hopes to train the robots using actual videos instead of Sims-style simulation videos, which would enable a robot to learn simply by watching a YouTube video.

This System Teaches Robots to Do Household Chores in a Virtual World

The Massachusetts Institute of Technology (MIT) Computer Science and Artificial Intelligence Laboratory (CSAIL) and the University of Toronto have released a paper demonstrating a new system called VirtualHome that has the potential to teach robots to execute household chores.

To create VirtualHome, the researchers incorporated in the system nearly 3,000 programs of activities, complete with their corresponding broken down subtasks.

The team then devised a way to illustrate the system through a 3D Sims-like world where artificial agents can be seen executing these activities in eight different rooms of a house.

This new database of robot instructions could one day be incorporated into Alexa-like robotic systems to improve the machines' abilities to incorporate and execute new tasks.

This allows the robot to do tasks in a personalized way, or even some day invoke an emotional connection as a result of this personalized learning process.”

This system defines objects, their corresponding uses and the actions robot can undertake with them so that the machines learn to complete tasks through trial and error.

Robot Conquers One of the Hardest Human Tasks: Assembling Ikea Furniture

Their robot was made of custom software, a three-dimensional camera, two robotic arms, grippers and force detectors.

“So this is kind of the genericity that we wanted to mimic.” Also like humans, the robot had a little help to start: It was fed a kind of manual, a set of ordered instructions on how the pieces fit together.

The robot spent a few seconds photographing the scene and matching each part to the one modeled in its “manual.” Then, over more than 11 minutes, the robot devised a plan that would allow it to quickly assemble the chair without its arms knocking into each other or into the various parts.

Kurzweilaccelerating intelligence

Computer scientists at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and the University of Toronto* have created a Sims-inspired “VirtualHome” system that can simulate detailed household tasks.

Using crowdsourcing, the researchers created videos that simulate detailed household activities and sub-tasks in eight different scenes, including a living room, kitchen, dining room, bedroom, and home office.  A simple model can generate a program from either a video or a textual description, allowing robots to be programmed by naive users, either via natural language or video demonstration.

The researchers have trained the system using nearly 3,000 programs for various activities, which are further broken down into subtasks for the computer to understand.

Wisconsin Robotics - URC 2016 Presentation Task

Made possible thanks to our sponsors: Epic: National Instruments: Boeing: Wisconsin Institute for

Wisconsin Robotics Critical Design Review for URC 2017 - Ascent

Wisconsin Robotics presents the video portion of the Critical Design Review for the University Rover Challenge. Featuring our 2017 rover, Ascent, this video ...

Swarm robotics -- from local rules to global behaviors | Magnus Egerstedt | TEDxEmory

This talk was given at a local TEDx event, produced independently of the TED Conferences. How should large-scale robotic teams be designed so that they can ...

Missouri MRDT - 2018 System Acceptance Review

The Missouri S&T Mars Rover Design Team presents our 2018 URC Contender, Atlas. Social Media: @missouriMRDT marsrover.mst.edu Music: Blue Man ...

Maine 4-H Robotics Program

Overview of University of Maine Cooperative Extension 4-H robotics program.

Robot Programming - Team 1, FIU Miami

Robot programming project in Robot Design class, Department of Mechanical Engineering, Florida International University (FIU), Miami, Florida, Spring 2008.

Programming for Robotics (ROS) Course 1

The slides are available here: The ..

Robot Training Academy Collection

Robot Training Academy, Inc. (RTA, ) was founded by a diverse team of computer scientist and robotics specialists. RTA provides ..

Autonomous Mechatronics Examination Session

This is a video of our Lego robot performing in the examination session of the "Autonomous Mechatronics" course at the Halmstad University of Sweden.

Applied Robotics class at Oregon State

The Applied Robotics course at OSU presents an open ended project to teams of mechanical engineers, electrical engineers, and computer scientists. Jonathan ...