AI News, Microsoft Research Blog

Microsoft Research Blog

Microsoft is very proud to be a diamond sponsor of ECCV 2018 and we’re in Munich, Germany from September 8-14 with the global computer vision community to share our research and to learn from our fellow contributors.

Hardware products, like Surface, Xbox, and HoloLens, contain some of the most advanced computer vision systems in the world (the head tracking in HoloLens is still the industry standard two years after its launch, even when compared to devices with orders of magnitude more computational power).

Designed for use in a huge range of applications, the sensor data can be processed locally, on deep learning hardware, or combined with Azure cloud services to integrate with computer vision and machine learning algorithms at all levels.

As one of the senior computer vision people at Microsoft, I work just as happily on topics that could be viewed as research or engineering – exploring new mathematical models of surface geometry, designing software architectures for parallelized nonlinear optimization, or low-level optimization of numerical code.

If you’ve followed computer vision at Microsoft, you may know that, as in many companies – and indeed many universities – our computer vision activities have historically been distributed across the company.

Microsoft Research Blog

Application code can not only access video and audio streams but can also at the same time leverage the results of built-in computer vision algorithms such as SLAM (simultaneous localization and mapping) to obtain the motion of the device as well as the spatial-mapping algorithms to obtain 3D meshes of the environment.

As shown in Figure 1, two of the gray-scale cameras are configured as a stereo rig capturing the area in front of the device so that the absolute depth of tracked visual features can be determined through triangulation.

These synchronized global-shutter cameras are significantly more light-sensitive than the color camera and can be used to capture images at a rate of up to 30 frames per second (FPS).

The first mode enables high-frequency (30 FPS) near-depth sensing, commonly used for hand tracking, while the other is used for lower-frequency (1-5 FPS) far-depth sensing, currently used by spatial mapping.

With the newest release of Windows 10 for HoloLens, researchers now have the option to enable Research Mode on their HoloLens devices to gain access to all these raw image sensors streams, shown in Figure 2.

MR and Azure 302: Computer vision

In this course, you will learn how to recognize visual content within a provided image, using Azure Computer Vision capabilities in a mixed reality application.

The Microsoft Computer Vision is a set of APIs designed to provide developers with image processing and analysis (with return information), using advanced algorithms, all from the cloud.

Developers upload an image or image URL, and the Microsoft Computer Vision API algorithms analyze the visual content, based upon inputs chosen the user, which then can return information, including, identifying the type and quality of an image, detect human faces (returning their coordinates), and tagging, or categorizing images.

Having completed this course, you will have a mixed reality HoloLens application, which will be able to do the following: In your application, it is up to you as to how you will integrate the results with your design.

The first script you need to create is the ResultsLabel class, which is responsible for the following: To create this class: You will notice that from the script you just dragged into the Camera, there are two fields: Cursor and Label Prefab.

The TapHandler() method increments the number of taps captured from the user and uses the current Cursor position to determine where to position a new Label.

To deploy on HoloLens: Congratulations, you built a mixed reality app that leverages the Azure Computer Vision API to recognize real world objects, and display confidence of what has been seen.

News: Microsoft Opens Access to Sensor Data on the HoloLens with 'Research Mode' Update

The addition of a new research mode for Microsoft HoloLens will enable researchers and developers to tap into a wider range of data collected by the device's sensors.

Available in the Windows 10 RS4 update for HoloLens, application code in Research Mode can access computer vision algorithms, such as simultaneous localization and mapping (SLAM) for spatial mapping and motion, as well as audio and video streams.

The data is collected by the device's image sensors, as opposed to the standard video camera that applications usually access.

These sensors work in conjunction with the depth camera that uses infrared light to measure depth by the time-of-flight method more accurately than standard cameras (which are more sensitive to ambient light).

CVPR18: Tutorial: New From HoloLens: Research Mode

Organizers: Marc Pollefeys Georg Klein Shivkumar Swaminathan Larry Wall Sudipta Sinha Johannes Schönberger Description: The HoloLens is the world's ...

Build 2017: Workplace Safety Demonstration

Subscribe to Microsoft on YouTube here: Follow us on social: LinkedIn: .

Microsoft Cognitive Services and Cameras: No Place to Hyde.

Watch how Microsoft uses Azure Stack, Azure Function, Cognitive services and cameras to digitize the environment for security purposes (and more).

Microsoft Ignite 2018 | Vision Keynote

CEO Satya Nadella takes the Microsoft Ignite stage at 6:00AM PT/9:00 AM ET on Monday, September 24. Join us to get the latest insights and skills direct from ...

Microsoft Build 2018 // Vision Keynote

CEO Satya Nadella takes the stage at Microsoft Build, our annual developer conference, at 8:30AM PT Monday, May 7. Join us to learn what's next for ...

Microsoft Build 2018 // Technology Keynote

Joe Belfiore, CVP of Windows, takes the stage at Microsoft Build, our annual developer conference, at 8:30AM PT Tuesday, May 8. Join us to learn what's next ...

Seeing AI 2016 Prototype - A Microsoft research project

Update : To get this technology into the hands of as many people as possible, this research project has evolved to be a free smartphone app, released in 2017.

Microsoft Business Forward 2018 keynote | James Phillips on Digital Transformation with the Cloud

James Phillips, CVP, Business Applications Group at Microsoft shares the vision, strategy for business applications at Microsoft and demonstrates the latest ...

Microsoft Hololens Research Mode rev 2

If you are one of the lucky 50000 "Humans" who received a HoloLens, enabling "Research Mode" is worth it! Especially, if you are interested in computer vision, ...

Bringing Banking into the 21st Century with Clearbank

ClearBank offer unprecedented security, speed and innovation with Microsoft.