AI News, The tensor renaissance in data science

The tensor renaissance in data science

After sitting in on UC Irvine Professor Anima Anandkumar’s Strata + Hadoop World 2015 in San Jose presentation, I wrote a post urging the data community to build tensor decomposition libraries for data science.

That was one of the main reasons that when the computers were not yet very powerful, tensors could not be handled efficiently. However, I think now we are seeing a renaissance of tensors because we have an explosion in computational capabilities, and tensor operations are highly parallelizable — they can be run on the cloud.

On the other hand, what our research and some of our collaborators and other researchers in this field have shown is that there are a lot of tensor-related problems and machine learning that are not hard. We do not encounter the worst-case hard tensors for machine learning applications.

Anandkumar highlights recent contributions of tensor methods in feature learning: The latest set of results we have been looking at is the use of tensors for feature learning as a general concept.

The idea of feature learning is to look at transformations of the input data that can be classified more accurately using simpler classifiers. This is now an emerging area in machine learning that has seen a lot of interest, and our latest analysis is to ask how can tensors be employed for such feature learning.

For instance, another application we’ve been looking at is to use these hierarchical, graphical models and also deep learning framework features that are extracted from deep learning … to have a better detection of multiple objects in the same image. Most of deep learning currently has focused on benchmark data sets where there’s mostly one image in one object, whereas we are now looking at if there are a lot of objects in an image: how can we efficiently learn this by also using the fact that objects tend to co-occur in images?

Gentlest Intro to TensorFlow #3: Matrices Multi-feature Linear Regression

Summary: With concepts of single-feature linear-regression, cost function, gradient descent (from Part 1), epoch, learn-rate, gradient descent variation (from Part 2) under our belt, we are ready to progress to multi-feature linear regression with TensorFlow (TF).

Our TF code for single-feature linear regression consists of 3 parts (see image below): The change to support 2-feature linear regression equation (explained above) in TF code is shown in red.

First, let us generalize representing a 2-feature model to an n-feature one: It turns out that the complex n-feature formula can be simplified in the world of matrices, and matrices are in-built into TF for these reasons: In TF, they would be written as: x

because the feature matrix (the one in the middle) has expanded from representing a single datapoint of n-features (1 row x n columns) to representing m datapoints with n-features (m rows x n columns), so we extended x<n>, e.g., x1, to x<m>.<n>, e.g., x1.1, where n is the feature number and m is the datapoint number.

= tf.matmul(x, W) + b We do a side-by-side comparison to summarize the change from single to multi-feature linear regression: We illustrated the concept of multi-feature linear regression, and showed how we extend our model and TF code from single to 2-feature linear regression models, which is generalizable to n-feature models.

Why the sudden fascination with tensors?

I think your question should be matched with an answer that is equally free flowing and open minded as the question itself.

b)=0$$ And the solution is $$\bar b=(X'X)^{-1}X'y$$ If you're good at linear algebra, you'll stick to the second approach once you've learned it, because it's actually easier than writing down all the sums in the first approach, especially once you get into multivariate statistics.

Hence my analogy is that moving to tensors from matrices is similar to moving from vectors to matrices: if you know tensors some things will look easier this way.

means you can calculate the force vector $F$ by multiplying the pressure $p$ (scalar) by the unit of area $dS$ (normal vector).

Ok, a scalar and a vector are also tensors :) Another place where tensors show up naturally is covariance or correlation matrices.

We noticed that if we pay $x_1$ dollars to $d_1$ and $x_2$ dollars to $d_2$ then $d_1$ sells us $y_1=2x_1-x_2$ pounds of apples, and $d_2$ sells us $y_2=-0.5x_1+2x_2$ oranges.

We can express this relation in the form of a matrix $P$: Then the merchants produce this much apples and oranges if we pay them $x$ dollars: $$y=Px$$

We either pay both 0.71 dollars, or we pay $d_1$ 0.71 dollars and demand 0.71 dollars from $d_2$ back.

In this case, if we start buying in bundles they'll know for sure how much we pay each of them, because we declare our bundles to the bazaar.

Particularly we may notice that we have an orthonormal basis $\bar d_1,\bar d_2$, where $d_i$ means a payment of 1 dollar to a merchant $i$ and nothing to the other.

We may also notice that the bundles also form an orthonormal basis $\bar d_1',\bar d_2'$, which is also a simple rotation of the first basis by 45 degrees counterclockwise.

The tensor is interesting: it can be represented as $$P=\sum_{ij}p_{ij}\bar d_i\bar d_j$$, and the groceries as $y=y_1 \bar d_1+y_2 \bar d_2$.

Now, when we changed the coordinates to bundles the tensor equation stays the same: $$y=Pz$$ That's nice, but the payment vectors are now in the different basis: $$z=z_1 \bar d_1'+z_2\bar d_2'$$, while we may keep the produce vectors in the old basis $y=y_1 \bar d_1+y_2 \bar d_2$.

Deep Learning with Tensorflow - Tensors, Variables and Placeholders

Enroll in the course for free at: Deep Learning with TensorFlow Introduction The majority of data ..

Vectors - The Math of Intelligence #3

We're going to explore why the concept of vectors is so important in machine learning. We'll talk about how they are used to represent both data and models.

Machine Learning Maths | Vectors And Matrix Norms | Part 2 | Eduonix

Hello, welcome to the second class of our new series 'Machine Learning Math'. In our first class, we covered all the basics like scalars, vectors, tensors, matrix ...

Tensor Decompositions for Learning Hidden Variable Models

In many applications, we face the challenge of modeling the interactions between multiple observations. A popular and successful approach in machine learning ...

Dimensionality Reduction for Matrix- and Tensor-Coded Data [Part 1]

Alex Williams, Stanford University In many scientific domains, data is coded in large tables or higher-dimensional arrays. Compressing these data into smaller, ...

How to Make a Simple Tensorflow Speech Recognizer

In this video, we'll make a super simple speech recognizer in 20 lines of Python using the Tensorflow machine learning library. I go over the history of speech ...

What is a Tensor? | Friendly Data Science #1 (Part 1)

This is Episode 1 (Part 1) of the Friendly Data Science Series See Introductory video of the series here: Synopsis: Hi everyone, ..

5.1: Binary Classification, ROC, AUC for Deep Learning, TensorFlow and Keras (Module 5, Part 1)

Keras neural networks for binary classification. Covers ROC and Area Under Curve (AUC). This video is part of a course that is taught in a hybrid format at ...

Sparse Matrices - Intro to Parallel Programming

This video is part of an online course, Intro to Parallel Programming. Check out the course here:

On Compressing Deep Models by Low Rank and Sparse Decomposition | Spotlight 1-1A

Xiyu Yu; Tongliang Liu; Xinchao Wang; Dacheng Tao Deep compression refers to removing the redundancy of parameters and feature maps for deep learning ...