AI News, Careers

Careers

leads the data science efforts, applying deep learning and Intel Nervana technologies into business applications across different industry domains, and driving the development and design of Intel Nervana Platform.

She and the Intel Nervana team developed open-source deep learning frameworks, neon and Intel Nervana Graph, brought state-of-the-art models on image recognition, image localization, natural language processing into the framework and deep learning solutions.

She and the Intel Nervana team developed open-source deep learning frameworks, neon and Intel Nervana Graph, brought state-of-the-art models on image recognition, image localization, natural language processing into the framework and deep learning solutions.

Yinyin Liu

She and the Intel Nervana team developed open-source deep learning frameworks, neon and Intel Nervana Graph, brought state-of-the-art models on image recognition, image localization, natural language processing into the framework and deep learning solutions.

She and the Intel Nervana team developed open-source deep learning frameworks, neon and Intel Nervana Graph, brought state-of-the-art models on image recognition, image localization, natural language processing into the framework and deep learning solutions.

Getting Started with Deep Learning

At SVDS, our R&D team has been investigating different deep learning technologies, from recognizing images of trains to speech recognition.

One way to give back to the open source community that provides us with tools is to help others evaluate and choose those tools in a way that takes advantage of our experience.

In the coming months, our team is excited to checkout DeepLearning4j, Paddle, Chainer, Apache Signa, and Dynet. We explain our scoring of the reviewed tools below: Languages: When getting started with deep learning, it is best to use a framework that supports a language you are familiar with.

Additionally, we’ve found that the engagement of the GitHub community is a strong indicator of not only a tool’s future development, but also a measure of how likely/fast an issue or bug can be solved through searching StackOverflow or the repo’s Git Issues.

A CNN is composed of a set of distinct layers that transform the initial data volume into output scores of predefined class scores (For more information, check out Eugenio Culurciello’s overview of Neural Network architectures).

These features include the opportunity space to define models, the availability of prebuilt layers, and the tools and functions available to connect these layers.

That said, TensorFlow’s easy ability to build upon it’s InceptionV3 model and Torch’s great CNN resources including easy-to-use temporal convolution set these two technologies apart for CNN modeling capability.

RNN Modeling Capability: Recurrent neural networks (RNNs) are used for speech recognition, time series prediction, image captioning, and other tasks that require processing sequential information.

As prebuilt RNN models are not as numerous as CNNs, it is therefore important if you have a RNN deep learning project that you consider what RNN models have been previously implemented and open sourced for a specific technology.

Keras is also a good choice for a high-level library when considering that its author recently expressed that Keras will continue to exist as a front end that can be used with multiple back ends.

For instance, for an image recognition application with a Python-centric team we would recommend TensorFlow given its ample documentation, decent performance, and great prototyping tools.

Distributed Deep Learning with Docker*, Intel® Nervana™ technology, neon™, and Pachyderm*

Docker* and Pachyderm* can help data scientists build, deploy, and update machine learning applications on production clusters, distributed processing across large data sets, and track input and output data throughout a data pipeline.

Intel Nervana technology with neon is the “world’s fastest deep learning framework.” It is open source (see GitHub*), Python* based, and includes a really nice set of libraries for developing deep learning models.

Assuming you have Docker installed and you are running CPU only, you can get Docker-ized neon by pulling the following image (which is endorsed by the neon team): Then, to experiment with neon inside Docker interactively, you could run: This will open a bash shell in a running instance of the neon image (or a container) in which you could create and run Python programs that utilize neon.

The resulting image could then be used to run your neon model on any machine running Docker by running: Running Docker-ized neon as mentioned above is great for portability, but we don’t necessarily want to manually log in to production machines to deploy single instances of our model.

This model pipeline stage will take a labeled training dataset `labeledTrainData.tsv` as input, and output a persisted (or saved to disk) version of the trained model as a set of model parameters, `imdb.p`, and a model vocab, `imdb.vocab`.

To create the above pipeline, we first need to create the training data repository that will be input to our model pipeline stage: We can then confirm that the repository was created by: Next, we can create the model pipeline stage to process the data in the training repository.

In this case, the JSON specification looks like this: This may seem complicated, but it really just says a few things: (1) create a pipeline stage called model, (2) use the kaixhin/neon Docker image for this pipeline stage, (3) run the provided Python cmd to process data, and (4) process any data in the input repository training.

Once we have this JSON file (saved in `train.json`), creating the above pipeline on a production-ready Pachyderm cluster is as simple as: Now, Pachyderm knows that it should perform the above processing, which in our case is training a neon model on the training data in the training repository.

In fact, because Pachyderm is versioning data and knows what data is new, Pachyderm will keep our model output (versioned in a corresponding model repository created by Pachyderm) in sync with the latest updates to our training data.

To confirm that our model has been trained, you should see both `imdb.p` and `imdb.vocab` in an output data repository created for the pipeline stage by Pachyderm: Note, we actually told Pachyderm what to write out to this repository by saving our files to Pachyderm’s magic `/pfs/out` directory, which specifies the output repository corresponding to our processing stage.

This inference stage will take new movie reviews, which the model hasn’t seen yet as input, and output the inferred sentiment of those movie reviews using our persisted model.

To create this inference pipeline stage, we first need to create the Pachyderm repository that will store and version our input reviews: Then we can create another JSON blog that will tell Pachyderm how to perform the processing for the inference stage: This is similar to our last JSON specification except, in this case, we have two input repositories (the reviews and the model) and we are using a different Docker image that contains `auto_inference.py`.

To create the inference stage, we simply run: Now whenever new reviews are pushed into the reviews repository, Pachyderm will see these new reviews, perform the inference, and output the inference results to a corresponding inference repository.

At any time, a data scientist or engineer could update the training data set utilized by the model to trigger the creation of a newly persisted, versioned model in our model repository.

We can scale the Python/neon scripts we develop on our local machine to production-scale data without having to think about data sharding, complicating your code with frameworks such as Dask*, or even transferring your modeling to another language or framework for production use.

He has spoken at conferences around the world (ODSC, Spark Summit, Datapalooza, DevFest Siberia, GopherCon, and more), teaches data science/engineering with Ardan Labs (@ardanlabs), maintains the Go kernel for Jupyter, and is actively helping to organize contributions to various open source data science projects.

A Survey of Deep Learning Frameworks

Currently a limited variety of tools are available in terms of deep learning frameworks since they implement algorithms which are used in bleeding edge applications such as computer vision and machine translation.

Deep learning at its core can be considered to be a sub class of machine learning that has come to prominence in the last few years with the growth of cores available on GPUs for computing.

In the field of deep learning currently there are several tools used by the industry to power applications such as self-driving cars to automatically generating captions for videos and images.

As deep learning is the bleeding edge of current machine learning research most of the deep learning frameworks are open source and there appears to be little to no proprietary frameworks as of today.

[data01] Deep neural networks can be used to model complex non-linear relationships in both supervised (where historical data is used to make prediction about future outcomes) and unsupervised (clustering and finding new patterns and anomalies in data) settings.

[gop01] Compared to traditional machine learning algorithms, deep learning models can provide significant improvement in areas such speech recognition and language translation as evidenced by the significant improvement in Google Translate after switching from Phrase Based Machine Translation (PBMT) to Neural Machine Translation (NMT).

They have been categorized on languages supported, availability of tutorials and training materials, convolutional neural network modeling capability, recurrent neural network modeling capability, ease of use in terms of architecture, speed and support for multiple GPUs.

Theano has been deployed in production in AI services companies such as ParallelDots serving up to a few 1000 concurrent news recommendations and multiple client Deep Learning as a service calls.

[quor01] TensorFlow was originally developed by researchers and engineers whom worked on the Google Brain Team within Google’s Machine Intelligence research organization for the purposes of conducting machine learning and deep neural networks research and over time the framework has been found to be general enough to be applicable in a wide variety of other domains as well.

[torc01] Torch provides faster performance compared to other deep learning frameworks due to the use of the fast scripting language LuaJIT and its underlying C/CUDA implementation.

Furthermore, even after extensive review of literature it was not possible to find any reference of many major players in the AI space deploying Caffe in a production environment.

Due to the variety of toolkits and frameworks available research is necessary to decide the most beneficial deep learning framework also taking into consideration the documentation available, and the supported languages.

Most frameworks provide common deep learning algorithms suitable for a variety of use cases but may not have an easy to use and modular architecture and may also sacrifice speed.

Mayank, “As of September 2015, is it better to use Torch or Theano for writing deep learning applications in production (not prototyping, not research, but production)?,” Quora, 15 09 2015.

Get Started with the Intel® Deep Learning SDK | Intel Software

This video shows the Intel® Deep Learning SDK from Intel. Data scientists and software developers can use the Deep Learning SDK to simplify installation, ...

Intel Nervana Artificial Intelligence 1/31/17

Presentation Details: Title: Hands-on Deep Learning Workshop Abstract: Deep learning is unlocking tremendous economic value across various market sectors.

A Brief Overview of Deep Learning Using Intel® Architecture | Intel Software

Deep learning attempts to model various levels of abstraction within data. There are various tools to train and deploy deep networks, and Intel is actively working ...

AI News | November 2017 | Intel Software

In this episode you'll get an inside look at building a large-scale image-feature extraction framework with BigDL, You'll learn how to get free DevCloud access, ...

Deep learning at scale and use cases - Naveen Rao (Nervana)

Deep learning has made a major impact in the last three years. Imperfect interactions with machines, such as speech, natural language processing, and image ...

TIG 3 7 Predictive Analytics Meet Up

Deep learning is unlocking tremendous economic value across various market sectors. Individual data scientists can draw from several open source frameworks ...

Deep Net Performance - Ep. 24 (Deep Learning SIMPLIFIED)

Training a large-scale deep net is a computationally expensive process, and common CPUs are generally insufficient for the task. GPUs are a great tool for ...

Arjun Bansal, Nervana Systems - Deep Learning Summit, London, 2016 #reworkDL

Catalyzing Deep Learning's Impact in the Enterprise Deep learning is in the early stages of unlocking tremendous economic value outside its impact in the large ...

What is Intel® Optimized Caffe* | Intel Software

Caffe* is a deep learning framework that is useful for convolutional and fully connected networks, and recently recurrent neural networks were added. There are ...

SD Deep Learning Meetup 12/2/15

Speaker: Urs Köster, PhD Urs will join us to dive deep into the field of Deep Learning and focus on Convolutional and Recurrent Neural Networks. The talk will ...