AI News, Top 10 Robotic Kinect Hacks

Top 10 Robotic Kinect Hacks

At a mere $150, it's a dirt-cheap way to bring depth sensing and 3D vision to robots, and while open-source USB drivers made it easy, a forthcoming Windows SDK from Microsoft promises to make it even easier.

Kinect, which is actually hardware made by an Israeli company called PrimeSense, works by projecting an infrared laser pattern onto nearby objects.

A dedicated IR sensor picks up on the laser to determine distance for each pixel, and that information is then mapped onto an image from a standard RGB camera.

What you end up with is an RGBD image, where each pixel has both a color and a distance, which you can then use to map out body positions, gestures, motion, or even generate 3D maps.

Bilibot The great thing about Kinect is that it can be used to give complex vision to cheap robots, and Bilibot is a DIY platform that gives you mobility, eyes, and a brain in a package that costs just $650.

Car Navigation Back when DARPA hosted their Grand Challenge for autonomous vehicles, robot cars required all kinds of crazy sensor systems to make it down a road.

Control a 3-D–Mapping Robot With Gestures? Just Add Kinect

Philipp Robbel, a student at MIT's Personal Robotics Group, has used a hacked Xbox Kinect camera and an iRobot Create kit to make a Roomba-esque KinectBot that can recognize human beings and respond to their gestural commands.

Add the right mix of boops and beeps, a computer-hacking interface, jet packs and the ability to serve drinks and fix starships, and we're just a few iterations away from a full-fledged R2-D2 unit.


Based around a webcam-style add-on peripheral, it enables users to control and interact with their console/computer without the need for a game controller, through a natural user interface using gestures and spoken commands.[12] The first-generation Kinect was first introduced in November 2010 in an attempt to broaden Xbox 360's audience beyond its typical gamer base.[13] A version for Microsoft Windows was released on February 1, 2012.[6] A newer version, Kinect 2.0, was released with the Xbox One platform starting in 2013.[14] Microsoft released the first Beta of the Kinect software development kit for Windows 7 on June 16, 2011.[15][16][17] This SDK was meant to allow developers to write Kinecting apps in C++/CLI, C#, or Visual Basic .NET.[18][19] Ultimately, the Kinect for either Xbox console did not have long-term popularity, and the two versions were discontinued by April 2016 and October 2017, respectively.[9] The Windows version had been discontinued by April 2015.[7]

Microsoft indicated that the company considers it to be a significant initiative, as fundamental to Xbox brand as Xbox Live,[32] and with a launch akin to that of a new Xbox console platform.[33] Kinect was even referred to as a 'new Xbox' by Microsoft CEO Steve Ballmer at a speech for Executives' Club of Chicago.[34][35] When asked if the introduction will extend the time before the next-generation console platform is launched (historically about 5 years between platforms),[36] Microsoft corporate vice president Shane Kim reaffirmed that the company believes that the life cycle of Xbox 360 will last through 2015 (10 years).[24] During Kinect's development, project team members experimentally adapted numerous games to Kinect-based control schemes to help evaluate usability.

Instead, processing would be handled by one of the processor cores of Xbox 360's Xenon CPU.[39] According to Alex Kipman, Kinect system consumes about 10-15% of Xbox 360's computing resources.[40] However, in November, Alex Kipman made a statement that 'the new motion control tech now only uses a single-digit percentage of Xbox 360's processing power, down from the previously stated 10 to 15 percent.'[41] A number of observers commented that the computational load required for Kinect makes the addition of Kinect functionality to pre-existing games through software updates even less likely, with concepts specific to Kinect more likely to be the focus for developers using the platform.[39] On March 25, 2010, Microsoft sent out a save the date flier for an event called the 'World Premiere 'Project Natal' for Xbox 360 Experience' at E3 2010.

David Dennis, Product Manager at Microsoft, said, 'There are hundreds of organizations we are working with to help them determine what's possible with the tech'.[48] On February 1, 2012, Microsoft released the commercial version of Kinect for Windows SDK and told that more than 300 companies from over 25 countries are working on Kinect-ready apps.[49] The tabloid the New York Post claimed Microsoft had a $500 million budget for advertising the launch of Kinect.[50] While this claim was widely re-reported,[51][52][53] an examination of the 10-Q from Microsoft reveals a 20% year-over-year increase ($85 million) for sales and marketing the quarter Kinect was launched for all of the Entertainment and Devices division, making the total sales and marketing spend $425 million for the entire division.[54] The marketing campaign You Are the Controller, aiming to reach new audiences, included advertisements on Kellogg's cereal boxes and Pepsi bottles, commercials during shows such as Dancing with the Stars and Glee as well as print ads in various magazines such as People and InStyle.[55] On October 19, Microsoft advertised Kinect on The Oprah Winfrey Show by giving free Xbox 360 consoles and Kinect sensors to the people in the audience.[56] Two weeks later, Kinect bundles with Xbox 360 consoles were also given away to the audience of Late Night with Jimmy Fallon.[57] On October 23, Microsoft held a pre-launch party for Kinect in Beverly Hills.

Kinect sensor's microphone array enables Xbox 360 to conduct acoustic source localization and ambient noise suppression, allowing for things such as headset-free party chat over Xbox Live.[12] The depth sensor consists of an infrared laser projector combined with a monochrome CMOS sensor, which captures video data in 3D under any ambient light conditions.[12][23] The sensing range of the depth sensor is adjustable, and Kinect software is capable of automatically calibrating the sensor based on gameplay and the player's physical environment, accommodating for the presence of furniture or other obstacles.[27] Described by Microsoft personnel as the primary innovation of Kinect,[24][75][76] the software technology enables advanced gesture recognition, facial recognition and voice recognition.[25] According to information supplied to retailers, Kinect is capable of simultaneously tracking up to six people, including two active players for motion analysis with a feature extraction of 20 joints per player.[77] However, PrimeSense has stated that the number of people the device can 'see' (but not process as players) is only limited by how many will fit in the field-of-view of the camera.[78] Reverse engineering[79] has determined that the Kinect's various sensors output video at a frame rate of ≈9 Hz to 30 Hz depending on resolution.

For example, Philipp Robbel of MIT combined Kinect with iRobot Create to map a room in 3D and have the robot respond to human gestures,[124] while an MIT Media Lab team is working on a JavaScript extension for Google Chrome called depthJS that allows users to control the browser with hand gestures.[125] Other programmers, including Robot Locomotion Group at MIT, are using the drivers to develop a motion-controller user interface similar to the one envisioned in Minority Report.[126] The developers of MRPT have integrated open source drivers into their libraries and provided examples of live 3D rendering and basic 3D visual SLAM.[127] Another team has shown an application that allows Kinect users to play a virtual piano by tapping their fingers on an empty desk.[128] Oliver Kreylos, a researcher at University of California, Davis, adopted the technology to improve live 3-dimensional videoconferencing, which NASA has shown interest in.[129] Alexandre Alahi from EPFL presented a video surveillance system that combines multiple Kinect devices to track groups of people even in complete darkness.[130] Companies So touch and Evoluce have developed presentation software for Kinect that can be controlled by hand gestures;

Researchers at the University of Minnesota have used Kinect to measure a range of disorder symptoms in children, creating new ways of objective evaluation to detect such conditions as autism, attention-deficit disorder and obsessive-compulsive disorder.[134] Several groups have reported using Kinect for intraoperative, review of medical imaging, allowing the surgeon to access the information without contamination.[135][136] This technique is already in use at Sunnybrook Health Sciences Centre in Toronto, where doctors use it to guide imaging during cancer surgery.[137] At least one company, GestSure Technologies, is pursuing the commercialization of such a system.[138] NASA's Jet Propulsion Laboratory (JPL) signed up for the Kinect for Windows Developer program in November 2013 to use the new Kinect to manipulate a robotic arm in combination with an Oculus Rift virtual reality headset, creating 'the most immersive interface' the unit had built to date.[139] Another application of Kinect is for multi-touch displays.

IGN gave the device 7.5 out of 10, saying that 'Kinect can be a tremendous amount of fun for casual players, and the creative, controller-free concept is undeniably appealing', though adding that for '$149.99, a motion-tracking camera add-on for Xbox 360 is a tough sell, especially considering that the entry level variation of Xbox 360 itself is only $199.99'.[144] Game Informer rated Kinect 8 out of 10, praising the technology but noting that the experience takes a while to get used to and that the spatial requirement may pose a barrier.[143] Computer and Video Games called the device a technological gem and applauded the gesture and voice controls, while criticizing the launch lineup and Kinect Hub.[142] CNET's review pointed out how Kinect keeps players active with its full-body motion sensing but criticized the learning curve, the additional power supply needed for older Xbox 360 consoles and the space requirements.[145] Engadget, too, listed the large space requirements as a negative, along with Kinect's launch lineup and the slowness of the hand gesture UI.

The review praised the system's powerful technology and the potential of its yoga and dance games.[146] Kotaku considered the device revolutionary upon first use but noted that games were sometimes unable to recognize gestures or had slow responses, concluding that Kinect is 'not must-own yet, more like must-eventually own.'[153] TechRadar praised the voice control and saw a great deal of potential in the device whose lag and space requirements were identified as issues.[148] Gizmodo also noted Kinect's potential and expressed curiosity in how more mainstream titles would utilize the technology.[154] Ars Technica's review expressed concern that the core feature of Kinect, its lack of a controller, would hamper development of games beyond those that have either stationary players or control the player's movement automatically.[155] The mainstream press also reviewed Kinect.

Sometimes is hard to find what you need especially when you’re interested to build a robot based on Kinect sensor.

The gestures of the users are interpreting using a standard video camera, a depth camera, and by projecting an infrared laser grid onto surfaces and create information by scanning the distortions in the digital mesh.

From simple to complex applications, the Kinect sensor is used in applications like flying robots that pilots itself, attached to aquadcopter for 3D mapping, control arm, as a tool to control a robot that drawing, remote-control vehicle, biometric recognition, 3D modeling interface, or to control fast and accurate parallel robots.

Gesture Controlled Robotic Arm Using Kinect Matlab Simulink & Arduino #MATLABHW2k16

This is a Simulink Model to Control a Robotic Arm By Gestures that are captured using Kinect Sensor. A Robot Arm is developed using Servo Motors that replicates the Right Arm Shoulder, Elbow...

The Kinect Sensor in Mobile Robotics: Initial Experiments

Some early experiments with a Microsoft Kinect depth camera on a mobile robot base. Say hello to KinectBot.

SLAM using Microsoft Kinect and ROS

Simultaneous Location And Mapping, this is a technique used for creating a 2D or 3D map of an unknown environment from the sensor's data which will make the task of knowing the position of...

Playing Electronic Music With Gestures (Kinect + Intel)

The inertial data from Intel's wristbands control the arpeggiator, drums and stabs. Scrolling notes are played using the hand position data from a Kinect camera. Music by Zackery Wilson.

Kinect WiFi Robot Arm

We made a robot arm that can be controlled using a Kinect! The motors are powered by our ESP8266-based WiFi controller boards, running our stock firmware. All of the coding required to make...

【PAS Robot】ROS Mapping with Kinect

First attempt at mapping school campus with the Kinect sensor. A recycled paper box was placed over the batteries and controllers so that the laptop and Kinect sensor could be placed on top....

control robot using kinect sensor

La conexión wifi para el funcionamiento del robot se produce mediante la comunicación de el programa en C# para Kinect con el envío de paquetes por Socket al Arduino. La única configuración...

Related researches to Kinect Navigation Contest(IROS 2014)

This video was made for publishing our researches related to Kinect Navigation Contest 2014. The video represents SLAM, SLAM in low-dynamic environments, human detection, and path planning....

3D Input and Output using Microsoft Kinect

This is a prototype of my final year project for uni: This project improves standard 2D input/output computer interface systems, such as touch screens and LCD displays, to give the user a...

Kinect Equipped Robot for Close-Range Control

The robot built by our team was equipped with a Kinect sensor to navigate through areas without WiFi coverage (such as an elevator) by interpreting voice and gesture commands. See more details...