AI News, Artificial Intelligence Archives

Kinect

The technology includes a set of hardware originally developed by PrimeSense, incorporating RGB cameras, infrared projectors and detectors that mapped depth through either structured light or time of flight calculations, and a microphone array, along with software and artificial intelligence from Microsoft to allow the device to perform real-time gesture recognition, speech recognition and body skeletal detection for up to four person, among other capabilities.

Kinect originated as a means to eliminate the game controller from Microsoft's Xbox video game hardware, competing with the Nintendo Wii's own motion-sensing capabilities, hoping to draw a larger audience beyond traditional video game players to the Xbox.

Kinect first launched as an add-on for the Xbox 360 in November 2010, and within a few months more than 10 million units had been sold, making it one of the fastest-selling computer hardware products at the time.

As Microsoft developed the successor console, the Xbox One, the company had planned to make the second generation of Kinect hardware a required component of the console, giving a reason for developers to seek to take advantage of it.

However, the forced inclusion of Kinect raised concerns related to privacy, and among other major changes, Microsoft eventually eliminated the requirement for Kinect to be always connected to the Xbox One, though still bundled Kinect with the console on its release in November 2013.

A market for Kinect-based games still did not emerge after the Xbox One's launch, and Microsoft eventually eliminated the Kinect from the Xbox One bundles, while later hardware revisions eliminated the Kinect-specific ports on the console, requiring a special USB adapter instead.

Microsoft, after initially objecting to the potential security issues raised by these hacks, changed course and endorsed these efforts, and released its own software development kit for non-commercial applications.

In 2005, PrimeSense was founded by tech-savvy mathematicians and engineers from Israel to develop the 'next big thing' for video games, incorporating cameras that were capable of mapping a human body in front of them and sensing hand motions.

Microsoft began discussions with PrimeSense about what would need to be done to make their product more consumer-friendly: not only improvements in the capabilities of depth-sensing cameras, but a reduction in size and cost, and a means to manufacturer the units at scale was required.

A separate path within Microsoft looked to create an equivalent of the Wii Remote, considering that that type of unit may become standardized similar to how two-thumbstick controllers became a standard feature.[12]

Much of the initial work was related to ethnographic research to see how video game players' home environments were layed out, lit, and how those with Wiis used the system to plan how Kinect units would be used.

Nearing the planned release, the problem of widespread testing of Kinect in various room types and different bodies accounting for age, gender, and race among other factors, while keeping the details of the unit confidential.

Microsoft had not committed to a release date for Kinect at E3 2009, but affirmed it would be after 2009, and likely in 2010 to stay competitive with the Wii and the PlayStation Move (Sony Interactive Entertainment's own motion-sensing system using hand-held devices).[22]

Microsoft's vice president Shane Kim said the company did not expect Kinect would extend the anticipated lifetime of the Xbox 360, which had been planned to last ten years through 2015, nor delay the launch of the successor to the Xbox 360.[18][28]

According to Tsunoda, adding Project Natal-based control to pre-existing games involved significant code alterations, and made it unlikely that existing games could be patched through software updates to support the unit.[30]

Although the Kinect's sensor unit was originally planned to contain a microprocessor that would perform operations such as the system's skeletal mapping, Microsoft reported in January 2010 that the sensor would no longer feature a dedicated processor.

While this was a small fraction of the Xbox 360's capabilities, industry observed believed this further pointed to difficulties in adapting pre-existing games to use Kinect, as the motion-tracking would add to a game's high computational load and exceed the Xbox 360's capabilities.

Microsoft also introduced a revision of the Xbox 360 that was slimmer than the existing model, included a larger 250 GB hard drive, and had a Kinect-ready port: this unit later became known as the Xbox 360 S.[36]

The marketing campaign You Are the Controller, aiming to reach new audiences, included advertisements on Kellogg's cereal boxes and Pepsi bottles, commercials during shows such as Dancing with the Stars and Glee as well as print ads in various magazines such as People and InStyle.[43]

While seemingly successful, the Kinect had a number of downsides: its launch titles were primarily family-oriented games which may have drawn new audience but did not have the selling power of major video games like Battlefield and Call of Duty, and thus did not have a high attach rate, with only around 20% of the 55 million Xbox 360 users having purchased the unit.

The Kinect team recognized some of the downsides with more traditional games and Kinect, and continued ongoing development of the unit to be released as a second-generation unit, such as reducing the latency of motion detection and improving speech recognition.

Part of early Xbox One specifications was that the new Kinect hardware would be automatically included with the console, so that developers would know that Kinect hardware would be available for any Xbox One, and hoping to encourage developers to take advantage of that.[12]

This raised concerns across the video game media: privacy advocates argued that Kinect sensor data could be used for targeted advertising, and to perform unauthorized surveillance on users.

In response to these claims, Microsoft reiterated that Kinect voice recognition and motion tracking can be disabled by users, that Kinect data cannot be used for advertising per its privacy policy, and that the console would not redistribute user-generated content without permission.[57][58][59][60][61][62]

In the months after the Xbox One release, Microsoft decided to launch a Kinect-less Xbox One system in March 2014 at the same price as the PlayStation 4, after considering that the Kinect for Xbox One had not gotten the developer support, and sales of the Xbox One were lagging due to the higher price tag of the Kinect-bundled system.

Developers like Harmonix that had been originally targeting games to use the Xbox One had put these games on hold until they knew there was enough of a Kinect install base to justify release, which resulted in a lack of games for the Kinect and reducing any consumer drive to buy the separate unit.[12]

The functionality of the unit along with its low US$150 cost was seen to be a inexpensive means to add depth-sensing to existing applications, offsetting the high cost and unreliability of other 3D camera options at the time.

Around November 2010, after the Kinect's launch, scientists, engineers, and hobbyists had been able to hack into the Kinect to determine what hardware and internal software it had used, leading to users finding how to connect and operate the Kinect with Microsoft Windows and OS X over USB, which has unsecured data from the various camera elements that could be read.

This further led to prototype demos of other possible applications, such as a gesture-based user interface for the operating system similar to that shown in the film Minority Report, as well as pornographic applications.[69]

Adafruit Industries, having envisioned some of the possible applications of the Kinect outside of gaming, issued a security challenge related to the Kinect, offering prize money for the successful development of an open source software development kit (SDK) and hardware drivers for the Kinect, which came to be known as Open Kinect.[70]

The use of cloud computing to offload some of the computational work from Kinect, as well as more powerful features enable by Azure such as artificial intelligence would improve the accuracy of the depth-sensing and reduce the power demand and would lead to more compact units, Microsoft had envisioned.[94]

The original Kinect for Xbox 360 used structured light for this: the unit used a near-infrared pattern projected across the space in front of the kinect, while an infrared sensor captured the reflected light pattern.

While other structure light depth-sensing technologies used multiple light patterns, Kinect used as few as one as to achieve a high rate of 30 frames per second of depth sensing.

Infrared light reflecting off closer objects will have a shorter time of flight than those more distant, so the infrared sensor captures how much the modulation pattern had been deformed from the time of flight, pixel-by-pixel.

Once Kinect has a pixel-by-pixel depth image, Kinect uses a type of edge detection here to delineate closer objects from the background of the shot, incorporating input from the regular visible light camera.

The unit's software, aided by artificial intelligence, performs segmentation of the shapes to try to identify specific body parts, like the head, arms, and hands, and track those segments individually.

The hardware included a range chipset technology by Israeli developer PrimeSense, which developed a system consisting of an infrared projector and camera and a special microchip that generates a grid from which the location of a nearby object in 3 dimensions can be ascertained.[98][99][100]

The sensing range of the depth sensor is adjustable, and Kinect software is capable of automatically calibrating the sensor based on gameplay and the player's physical environment, accommodating for the presence of furniture or other obstacles.[21]

According to information supplied to retailers, Kinect is capable of simultaneously tracking up to six people, including two active players for motion analysis with a feature extraction of 20 joints per player.[109]

The default RGB video stream uses 8-bit VGA resolution (640 × 480 pixels) with a Bayer color filter, but the hardware is capable of resolutions up to 1280x1024 (at a lower frame rate) and other colour formats such as UYVY.

The horizontal field of the Kinect sensor at the minimum viewing distance of ≈0.8 m (2.6 ft) is therefore ≈87 cm (34 in), and the vertical field is ≈63 cm (25 in), resulting in a resolution of just over 1.3 mm (0.051 in) per pixel.

The hardware included better componenty to eliminate noise along the USB and other cabling paths, and improvements in the depth-sensing camera system for detection of objects at close range, as close as 50 centimetres (20 in), in the new 'Near Mode'.[76]

It can also detect a player's heart rate, facial expression, the position and orientation of 25 individual joints (including thumbs), the weight put on each limb, speed of player movements, and track gestures performed with a standard controller.

Microsoft considered the Kinect 2 for Windows equivalent in performance to the Xbox One version, and by April 2015, having difficulty in keeping up manufacturing demand for the Kinect for Xbox One, discontinued the Kinect 2 for Windows, directing commercial users to use the Xbox One version instead.[91][138]

the original Xbox One user interface software had similar support for Kinect features as the Xbox 360 software, such as voice commands, user identification via skeletal or vocal recognition, and gesture-driven commands, though these features could be fully disabled due to privacy concerns.[147]

Xbox 360 games that require Kinect are packaged in special purple cases (as opposed to the green cases used by all other Xbox 360 games), and contain a prominent 'Requires Kinect Sensor' logo on their front cover.

Researchers at the University of Minnesota have used Kinect to measure a range of disorder symptoms in children, creating new ways of objective evaluation to detect such conditions as autism, attention-deficit disorder and obsessive-compulsive disorder.[167]

IGN gave the device 7.5 out of 10, saying that 'Kinect can be a tremendous amount of fun for casual players, and the creative, controller-free concept is undeniably appealing', though adding that for '$149.99, a motion-tracking camera add-on for Xbox 360 is a tough sell, especially considering that the entry level variation of Xbox 360 itself is only $199.99'.[175]

CNET's review pointed out how Kinect keeps players active with its full-body motion sensing but criticized the learning curve, the additional power supply needed for older Xbox 360 consoles and the space requirements.[176]

Ars Technica's review expressed concern that the core feature of Kinect, its lack of a controller, would hamper development of games beyond those that have either stationary players or control the player's movement automatically.[186]

The slight input lag between making a physical movement and Kinect registering it was not considered a major issue with most games, and the review called Kinect 'a good and innovative product,' rating it 3.5 out of 4 stars.[188]

In its Xbox One review, Engadget praised Xbox One's Kinect functionality, such as face recognition login and improved motion tracking, but that whilst 'magical', 'every false positive or unrecognized [voice] command had us reaching for the controller.'[189]

Writing for Time, Matt Peckham described the device as being 'chunky' in appearance, but that the facial recognition login feature was 'creepy but equally sci-fi-future cool', and that the new voice recognition system was a 'powerful, addictive way to navigate the console, and save for a few exceptions that seem to be smoothing out with use'.

However, its accuracy was found to be affected by background noise, and Peckham further noted that launching games using voice recognition required that the full title of the game be given rather than an abbreviated name that the console 'ought to semantically understand', such as Forza Motorsport 5 rather than 'Forza 5'.[191]

critics showed concerns the device could be used for surveillance, stemming from the originally announced requirements that Xbox One's Kinect be plugged in at all times, plus the initial always-on DRM system that required the console to be connected to the internet to ensure continued functionality.

Reports also surfaced regarding recent Microsoft patents involving Kinect, such as a DRM system based on detecting the number of viewers in a room, and tracking viewing habits by awarding achievements for watching television programs and advertising.

While Microsoft stated that its privacy policy 'prohibit[s] the collection, storage, or use of Kinect data for the purpose of advertising', critics did not rule out the possibility that these policies could be changed prior to the release of the console.

In response to the criticism, a Microsoft spokesperson stated that users are 'in control of when Kinect sensing is On, Off or Paused', will be provided with key privacy information and settings during the console's initial setup, and that user-generated content such as photos and videos 'will not leave your Xbox One without your explicit permission.'[57][58][59][60]

Introducing the Intro to Machine Learning with TensorFlow Nanodegree Program

Start Learning The field of machine learning continues to boast incredible job growth, salaries, and skill sets that can be used in many different industries.

Google utilizes this technology in their Cloud product to allow startups to build machine learning models that work on data of any size, while GE utilizes IoT to help detect and prevent anomalies and crashes in their products.

All of the above skills are incorporated into Udacity’s new Intro to Machine Learning with TensorFlow Nanodegree program, which is a great way to get introduced to the fundamentals of machine learning, including areas like manipulating data, supervised &

Netease Youdao: AI Leader in Online Education in China (TF Dev Summit '19)

A panoramic description of Netease Youdao on AI leadership in online education in China, which includes the product line and industrial AI capabilities, and ...

Humanizing AI development - Lili Cheng (Microsoft Research)

Humanizing AI development: Lessons from China and Xiaoice at Microsoft. Lili Cheng is a general manager for Microsoft Research focusing on conversational ...

The Nature of Machine Learning (The Pattern Machine part 2)

Why learning is the key to intelligence action. Part 2 in my AI series. Consider supporting this program: This video ..

2018 Isaac Asimov Memorial Debate: Artificial Intelligence

Isaac Asimov's famous Three Laws of Robotics might be seen as early safeguards for our reliance on artificial intelligence, but as Alexa guides our homes and ...

IDenTV Sample of AI Video Classifiers

IDenTV provides AI capabilities to create video analytics with advanced computer vision content search and recognition. The AI-powered (Video Juicer) enables ...

AI For Good - UN Congress on Artificial Intelligence

New Videos Every Day - Subscribe: Audi CEO Prof. Rupert Stadler has spoken at a congress of the United Nations about the use of ..

Mesh-TensorFlow: Model Parallelism for Supercomputers (TF Dev Summit ‘19)

Batch-splitting (data-parallelism) is the dominant distributed Deep Neural Network (DNN) training strategy, due to its universal applicability and its amenability to ...

Eric Weinstein: Revolutionary Ideas in Science, Math, and Society | Artificial Intelligence Podcast

Eric Weinstein is a mathematician, economist, physicist, and managing director of Thiel Capital. He formed the "intellectual dark web" which is a loosely ...

Webinar: Cracking the EDR Code with AI

Endpoint detection and response (EDR) products allow analysts to discover unknown threats and respond to them in real time. Unfortunately, many modern ...

ARTIFICIAL INTELLIGENCE AT WORK - ENGLISH VERSION

Artificial Intelligence Industrial Applications: what's available today You will see some examples of Anomaly Detection, Predictive Maintenance, advanced ...