AI News, Microsoft Research Lab artificial intelligence

Microsoft Research

It was formed in 1991, with the intent to advance state-of-the-art computing and solve difficult world problems through technological innovation in collaboration with academic, government, and industry researchers.

The Microsoft Research team employs more than 1,000 computer scientists, physicists, engineers, and mathematicians, including Turing Award winners, Fields Medal winners, MacArthur Fellows, and Dijkstra Prize winners.

Between 2010 and 2018, 154,000 AI patents were filed worldwide, with Microsoft having by far the largest percentage of those patents, at 20%.[1]

Andrew Ng

born 1976) is a Chinese-American computer scientist known as one of the most prolific researchers in machine learning and AI, with his work helping incite the recent revolution in deep learning.[2]

Also a business executive and investor in the Silicon Valley, Ng co-founded and led Google Brain and was a former Vice President and Chief Scientist at Baidu, building the company's Artificial Intelligence Group into a team of several thousand people.[3]

Since 2018 he launched and currently heads AI Fund, initially a $175-million investment fund for backing artificial intelligence startups.

In 1997, he earned his undergraduate degree with a triple major in computer science, statistics, and economics at the top of his class from Carnegie Mellon University in Pittsburgh, Pennsylvania.

At MIT he built the first publicly available, automatically-indexed web-search engine for research papers on the web (it was a precursor to CiteSeer/ResearchIndex, but specialized in machine learning).[6]

He became Director of the Stanford Artificial Intelligence Lab, where he taught students and undertook research related to data mining, big data, and machine learning.

His machine learning course CS229 at Stanford is one of the most popular courses offered on campus with over 1000 students enrolling some years.[11][12]

Since joining Stanford in 2002, he has advised dozens of Ph.D and M.Sc students, including Ian Goodfellow, Quoc Le and many other students who have gone on to work for academia or companies like Google, Facebook, Apple, Twitter, and 23andme.[6]

The rationale was that an efficient computation infrastructure could speed up statistical model training by orders of magnitude, ameliorating some of the scaling issues associated with big data.

Ng researches primarily in machine learning, deep learning, machine perception, computer vision, and natural language processing;

In 2011, Ng founded the Google Brain project at Google, which developed large scale artificial neural networks using Google's distributed computer infrastructure.[24]

Among its notable results was a neural network trained using deep learning algorithms on 16,000 CPU cores, which learned to recognize cats after watching only YouTube videos, and without ever having been told what a 'cat' is.[25][26]

Within Stanford, they include Daphne Koller with her 'blended learning experiences' and co-designing a peer-grading system, John Mitchell (Courseware, a Learning Management System), Dan Boneh (using machine learning to sync videos, later teaching cryptography on Coursera), Bernd Girod (ClassX), and others.

It offered a similar experience to MIT's Open Courseware except it aimed at providing a more 'complete course' experience, equipped with lectures, course materials, problems and solutions, etc.

Widom, Ng, and others were ardent advocates of Khan-styled tablet recordings, and between 2009–2011, several hundred hours of lecture videos recorded by Stanford instructors were recorded and uploaded.

The course featured quizzes and graded programming assignments and became one of the first and most successful Massive open online courses (MOOCs) created by a Stanford professor.[33]

One of the students (Frank Chen) claims another one (Jiquan Ngiam) frequently stranded him in the Stanford building and refused to give him a ride back to his dorm until very late at night, so that he no choice but to stick around and keep working.

This is a non-technical course designed to help people understand AI's impact on society and its benefits and costs for companies, as well as how they can navigate through this technological revolution.[36]

Ng has filed patents in sundry things from text-to-speech (TTS) systems, compressed video and audio recordings, rechargable batteries, electronic roll towel dispensers, an energy saving cooker, and a one-size-fits-all T-shirt.[47]

Ng is one of the scientists credited with bringing humanity to AI, and he sees AI as a technology that will improve the lives of people, not an anathema that will 'enslave' the human race.[3]

In 2017, Ng said he supported basic income to provide people tools to learn about AI and spend time studying so that they can re-enter the workforce as productive members.

JD AIR

Focused on cutting edge innovation in AI 80% of research is driven by business needs Offices in China, the US and Europe to attract leading global talent Leading JD’s academic collaboration initiatives in the field of AI

Free Book: Foundations of Data Science (from Microsoft Research Lab)

By Avrim Blum, John Hopcroft, and Ravindran Kannan (2018).  Computer science as an academic discipline began in the 1960s.

Emphasis was on programming languages, compilers, operating systems, and the mathematical theory that supported these areas.

Courses in theoretical computer science covered finite automata, regular expressions, context-free languages, and computability.

In the 1970s, the study of algorithms was added as an important component of theory.

The emphasis was on making computers useful.

Today, a fundamental change is taking place and the focus is more on applications.

There are many reasons for this change.

The merging of computing and communications has played an important role.

The enhanced ability to observe, collect, and store data in the natural sciences, in commerce, and in other fields calls for a change in our understanding of data and how to handle it in the modern setting.

The emergence of the web and social networks as central aspects of daily life presents both opportunities and challenges for theory.

The book is available and freely downloadable here.

For more information, visit this Microsoft webpage.

More free books are available here. 

Contents 1

Introduction 9 2

High-Dimensional Space 12 2.1 Introduction .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

12 2.2 The Law of Large Numbers .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

12 2.3 The Geometry of High Dimensions .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

15 2.4 Properties of the Unit Ball .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

17 2.5 Generating Points Uniformly at Random from a Ball .

.

.

.

.

.

.

.

.

.

.

.

22 2.6 Gaussians in High Dimension .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

23 2.7 Random Projection and Johnson-Lindenstrauss Lemma .

.

.

.

.

.

.

.

.

.

.

25 2.8 Separating Gaussians .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

27 2.9 Fitting a Spherical Gaussian to Data .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

29 2.10 Bibliographic Notes .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

31 2.11 Exercises .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

32 3

Best-Fit Subspaces and Singular Value Decomposition (SVD) 40 3.1 Introduction .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

40 3.2 Preliminaries .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

41 3.3 Singular Vectors .

42 3.4 Singular Value Decomposition (SVD) .

45 3.5 Best Rank-k Approximations .

47 3.6 Left Singular Vectors .

48 3.7 Power Method for Singular Value Decomposition .

51 3.8 Singular Vectors and Eigenvectors .

54 3.9 Applications of Singular Value Decomposition .

54 3.10 Bibliographic Notes .

65 3.11 Exercises .

67 4

Random Walks and Markov Chains 76 4.1 Stationary Distribution .

80 4.2 Markov Chain Monte Carlo .

81 4.3 Areas and Volumes .

86 4.4 Convergence of Random Walks on Undirected Graphs .

88 4.5 Electrical Networks and Random Walks .

97 4.6 Random Walks on Undirected Graphs with Unit Edge Weights .

102 4.7 Random Walks in Euclidean Space .

109 4.8 The Web as a Markov Chain .

112 4.9 Bibliographic Notes .

116 4.10 Exercises .

118 5

Machine Learning 129 5.1 Introduction .

129 5.2 The Perceptron algorithm .

130 5.3 Kernel Functions .

132 5.4 Generalizing to New Data .

134 5.5 Overfitting and Uniform Convergence .

135 5.6 Illustrative Examples and Occam’s Razor .

138 5.7 Regularization: Penalizing Complexity .

141 5.8 Online Learning .

141 5.9 Online to Batch Conversion .

146 5.10 Support-Vector Machines .

147 5.11 VC-Dimension .

148 5.12 Strong and Weak Learning - Boosting .

157 5.13 Stochastic Gradient Descent .

160 5.14 Combining (Sleeping) Expert Advice .

162 5.15 Deep Learning .

164 5.16 Further Current Directions .

171 5.17 Bibliographic Notes .

175 5.18 Exercises .

176 6

Algorithms for Massive Data Problems: Streaming, Sketching, and Sampling 181 6.1 Introduction .

181 6.2 Frequency Moments of Data Streams .

182 6.3 Matrix Algorithms using Sampling .

192 6.4 Sketches of Documents .

201 6.5 Bibliographic Notes .

203 6.6 Exercises .

204 7

Clustering 208 7.1 Introduction .

208 7.3 k-Center Clustering .

215 7.4 Finding Low-Error Clusterings .

216 7.5 Spectral Clustering .

216 7.6 Approximation Stability .

224 7.7 High-Density Clusters .

227 7.8 Kernel Methods .

228 7.9 Recursive Clustering based on Sparse Cuts .

229 7.10 Dense Submatrices and Communities .

230 7.11 Community Finding and Graph Partitioning .

233 7.12 Spectral clustering applied to social networks .

236 7.13 Bibliographic Notes .

239 7.14 Exercises .

240 8

Random Graphs 245 8.1 The G(n, p) Model .

245 8.2 Phase Transitions .

252 8.3 Giant Component .

261 8.4 Cycles and Full Connectivity .

265 8.5 Phase Transitions for Increasing Properties .

270 8.6 Branching Processes .

272 8.7 CNF-SAT .

277 8.8 Nonuniform Models of Random Graphs .

284 8.9 Growth Models .

286 8.10 Small World Graphs .

294 8.11 Bibliographic Notes .

299 8.12 Exercises .

301 9

Topic Models, Nonnegative Matrix Factorization, Hidden Markov Models, and Graphical Models 310 9.1 Topic Models .

310 9.2 An Idealized Model .

313 9.3 Nonnegative Matrix Factorization - NMF .

315 9.4 NMF with Anchor Terms .

317 9.5 Hard and Soft Clustering .

318 9.6 The Latent Dirichlet Allocation Model for Topic Modeling .

320 9.7 The Dominant Admixture Model .

322 9.8 Formal Assumptions .

324 9.9 Finding the Term-Topic Matrix .

327 9.10 Hidden Markov Models .

332 9.11 Graphical Models and Belief Propagation .

337 9.12 Bayesian or Belief Networks .

338 9.13 Markov Random Fields .

339 9.14 Factor Graphs .

340 9.15 Tree Algorithms .

341 9.16 Message Passing in General Graphs .

342 9.17 Graphs with a Single Cycle .

344 9.18 Belief Update in Networks with a Single Loop .

346 9.19 Maximum Weight Matching .

347 9.20 Warning Propagation .

351 9.21 Correlation Between Variables .

351 9.22 Bibliographic Notes .

355 9.23 Exercises .

357 10 Other Topics 360 10.1 Ranking and Social Choice .

360 10.2 Compressed Sensing and Sparse Vectors .

364 10.3 Applications .

368 10.4 An Uncertainty Principle .

370 10.5 Gradient .

373 10.6 Linear Programming .

375 10.7 Integer Optimization .

377 10.8 Semi-Definite Programming .

378 10.9 Bibliographic Notes .

380 10.10 Exercises .

381 11 Wavelets 385 11.1 Dilation .

385 11.2 The Haar Wavelet .

386 11.3 Wavelet Systems .

390 11.4 Solving the Dilation Equation .

390 11.5 Conditions on the Dilation Equation .

392 11.6 Derivation of the Wavelets from the Scaling Function .

394 11.7 Sufficient Conditions for the Wavelets to be Orthogonal .

398 11.8 Expressing a Function in Terms of Wavelets .

401 11.9 Designing a Wavelet System .

402 11.10Applications .

402 11.11 Bibliographic Notes .

402 11.12 Exercises .

403 12 Appendix 406 12.1 Definitions and Notation .

406 12.2 Asymptotic Notation .

406 12.3 Useful Relations .

408 12.4 Useful Inequalities .

413 12.5 Probability .

420 12.6 Bounds on Tail Probability .

430 12.7 Applications of the Tail Bound .

436 12.8 Eigenvalues and Eigenvectors .

437 12.9 Generating Functions .

451 12.10 Miscellaneous .

456 12.11 Exercises .

A sneak peek inside Microsoft's AI research labs - BBC Newsnight

Our technology editor David Grossman has been given exclusive access to Microsoft's AI labs in Seattle to see how the future of Artificial Intelligence is shaping ...

Inside Microsoft's Cambridge AI Lab

May.11 -- Microsoft's annual Build developer conference is underway in Seattle, bringing together thousands from around the world. Bloomberg's Caroline Hyde ...

A tour of the Microsoft AI & IoT Insider Labs

The IoT Show was invited for a tour of the AI and IoT Insider Labs on the Microsoft campus. If your organization is developing an IoT and/or AI solution, you may ...

Eric Horvitz announces new Microsoft Research AI:

Microsoft Technical Fellow/ Director of Microsoft Research Labs Eric Horvitz speaks at a event in London focused on Microsoft's continued investment in Artificial ...

Why I work at Microsoft Research Cambridge

Find out about working at Microsoft Research Cambridge, and meet some of our Researchers. Learn more at ...

Keynote Talk: Microsoft Research Labs - Expand The State of The Art

The Academic Research Summit, co-organized by Microsoft Research and the Association for Computing Machinery, is a forum to foster meaningful discussion ...

The Intern Experience at Microsoft Research Cambridge

Find out what it's like to be an intern at the Microsoft Research lab in Cambridge, UK. Real interns talk about the projects they are working on, the culture of the ...

Machine Learning Advancing Artificial Intelligence

Chair: Will Lewis, Microsoft Research Projects and speakers: Learning Reusable Skills and Behavioral Hierarchies, Bruno Castro da Silva, Institute of ...

Artificial Intelligence – Microsoft Perspectives

Speaker: Evelyne Viegas Given the investment and evidence of progress in Artificial Intelligence (AI) in the last five years, some suggest that it is merely a matter ...

AI and Security

In the future, every company will be using AI, which means that every company will need a secure infrastructure that addresses AI security concerns. At the same ...