AI News, Moving machine learning from practice to production

Moving machine learning from practice to production

With growing interest in neural networks and deep learning, individuals andcompanies are claiming ever-increasing adoption rates of artificial intelligence into their daily workflows and product offerings.

number of frameworks, tutorials and guides have popped up to democratize machine learning, but the steps that they prescribe often don’t align with the fuzzier problems that need to be solved.

While starting out, most tutorials usually include well-defined datasets.Whether it be MNIST, the Wikipedia corpus or any of the great options fromthe UCI Machine Learning Repository, these datasets are often not representative of the problem that you wish to solve.

In many cases, even if you possess the required data, significant (and expensive) manual labor might be required to categorize, annotate and label your data for training.

While there exist some standard techniques for images, like cropping, scaling, zero-centering and whitening — the final decision is still up to individuals on the level of normalization required for each task.

billion-parameter network might promise great performance with itsterabyte-sized dataset, but most people cannot afford to wait for weeks while the training is still in progress.

Even with simpler models, the infrastructure and tooling required for the build-up, training, collation and tear-down of tasks across instances can be quite daunting.

Spending some time on planning your infrastructure, standardizing setup and defining workflows early-on can save valuable time with each additional model that you build.

model performing a simple task might perform satisfactorily with its weights packaged directly into your application, while more complicated models might require communication with centralized heavy-lifting servers.

Periodically saving production statistics (data samples, predicted results, outlier specifics) has proven invaluable in performing analytics (and error postmortems) over deployments.

Greater discussion about end-to-end deployments is required among practitioners to take this field forward and truly democratize machine learning for the masses.

Moving machine learning from practice to production

With growing interest in neural networks and deep learning, individuals andcompanies are claiming ever-increasing adoption rates of artificial intelligence into their daily workflows and product offerings.

number of frameworks, tutorials and guides have popped up to democratize machine learning, but the steps that they prescribe often don’t align with the fuzzier problems that need to be solved.

While starting out, most tutorials usually include well-defined datasets.Whether it be MNIST, the Wikipedia corpus or any of the great options fromthe UCI Machine Learning Repository, these datasets are often not representative of the problem that you wish to solve.

In many cases, even if you possess the required data, significant (and expensive) manual labor might be required to categorize, annotate and label your data for training.

While there exist some standard techniques for images, like cropping, scaling, zero-centering and whitening — the final decision is still up to individuals on the level of normalization required for each task.

billion-parameter network might promise great performance with itsterabyte-sized dataset, but most people cannot afford to wait for weeks while the training is still in progress.

Even with simpler models, the infrastructure and tooling required for the build-up, training, collation and tear-down of tasks across instances can be quite daunting.

Spending some time on planning your infrastructure, standardizing setup and defining workflows early-on can save valuable time with each additional model that you build.

model performing a simple task might perform satisfactorily with its weights packaged directly into your application, while more complicated models might require communication with centralized heavy-lifting servers.

Periodically saving production statistics (data samples, predicted results, outlier specifics) has proven invaluable in performing analytics (and error postmortems) over deployments.

Greater discussion about end-to-end deployments is required among practitioners to take this field forward and truly democratize machine learning for the masses.

#3: Developing a Machine Learning Model from Start to Finish

At a high level, building a good ML model is like building any other product: You start with ideation, where you align on the problem you’re trying to solve and some potential approaches.

Since data is an integral part of ML, we need to layer data on top of this product development process, so our new process looks as follows: The goal of this phase is to align as a team on the key problem the model solves, the objective function and the potential inputs to the model.

“Apple”), you need to separate the brand chatter from the general chatter (about the fruit) and run it through a sentiment analysis model, and all that before you can begin to build your prototype.

Say your product is a movie recommendation tool: You may want to only open access to a handful of users but provide a complete experience for each user, in which case your model needs to rank every movie in your database by relevance to each of the users.

You may get unsatisfactory results from your data sourcing efforts and have to rethink the approach, or productize the model and see that it works so poorly with production data that you need to go back to prototyping etc.

For example, labeling hundreds or thousands of data points with the right categories as input for a classification algorithm and then testing whether the output of the classification model is correct.

Azure Machine Learning frequently asked questions: Billing, capabilities, limitations, and support

Here are some frequently asked questions (FAQs) and corresponding answers about Azure Machine Learning, a cloud service for developing predictive models and operationalizing solutions through web services.

These FAQs provide questions about how to use the service, which includes the billing model, capabilities, limitations, and support.

Azure Machine Learning is a fully managed service that you can use to create, test, operate, and manage predictive analytic solutions in the cloud.

Drag-and-drop predictive modeling, a large pallet of modules, and a library of starting templates make common machine-learning tasks simple and quick.

Machine Learning Studio hosts a pallet of modules in a visual composition interface that helps you build an end-to-end, data-science workflow in the form of an experiment.

The Machine Learning API service enables you to deploy predictive models, like those that are built into Machine Learning Studio, as scalable, fault-tolerant, web services.

The web services that the Machine Learning API service creates are REST APIs that provide an interface for communication between external applications and your predictive analytics models.

Web services created using the Classic deployment model and web services created using the New Azure Resource Manager deployment model are listed in the Microsoft Azure Machine Learning Web Services portal.

To make a call to a web service, you pass an API key that was created when you deployed the web service.

A Machine Learning web service is based on REST, a popular architecture choice for web programming projects.

For example, you can write an application in C#, R, or Python by using the sample code that's generated for you when you deployed the web service.

The sample code is available on: You can also use the sample Microsoft Excel workbook that's created for you and is available in the web service dashboard in Machine Learning Studio.

You can download data to a Machine Learning Studio experiment in three ways: To learn more about supported file formats, see Import training data into Machine Learning Studio.

The following types of data can expand to larger datasets during feature normalization and are limited to less than 10 GB: The following modules are limited to datasets less than 10 GB: For datasets that are larger than a couple GBs, upload data to Azure Storage or Azure SQL Database, or use Azure HDInsight rather than directly uploading from a local file.

The algorithm, data source, data format, or data transformation operation that I am looking for isn't in Azure Machine Learning Studio.

Yes, you can bring your existing R or Python code into Machine Learning Studio, run it in the same experiment with Azure Machine Learning learners, and deploy the solution as a web service via Azure Machine Learning.

Because large amounts of data might be sent to a browser, data size is limited to prevent slowing down Machine Learning Studio.

Machine Learning Studio supports machine learning algorithms for multiclass and binary classification, regression, and clustering.

The documentation provides some information about the algorithms and parameters for tuning are described to optimize the algorithm for your use.

The Request Response service (RRS) is a low-latency, high-scale web service that is used to provide an interface to stateless models that are created and deployed from the experimentation environment.

The main difference is that BES reads a block of records from a variety of sources, such as Azure Blob storage, Azure Table storage, Azure SQL Database, HDInsight (hive query), and HTTP sources.

To update a predictive model for an already deployed service, modify and rerun the experiment that you used to author and save the trained model.

You can scale this to 200 concurrent requests per endpoint, and you can scale each web service to 10,000 endpoints per web service as described in Scaling a Web Service.

For BES, each endpoint can process 40 requests at a time, and additional requests beyond 40 requests are queued.

You can also sample larger datasets via Hive queries, via Azure SQL Database queries, or by preprocessing with Learning with Counts modules before ingestion.

The following types of data can expand to larger datasets during feature normalization and are limited to less than 10 GB: The following modules are limited to datasets less than 10 GB: For datasets that are larger than a few GBs, upload data to Azure Storage or Azure SQL Database, or use HDInsight rather than directly uploading from a local file.

After the workspace is created, if the storage account is deleted and can no longer be found, the workspace will stop functioning, and all experiments in that workspace will fail.

If you accidentally deleted the storage account, recreate the storage account with the same name in the same region as the deleted storage account.

If the access keys are changed after the workspace is created, the workspace can no longer access the storage account.

If you changed storage account access keys, resync the access keys in the workspace by using the Azure portal.

These step-by-step guides introduce the services and explain the data science life cycle of importing data, cleaning data, building predictive models, and deploying them in production by using Azure Machine Learning.

When you create a plan, you commit to a fixed cost that comes with an included quantity of API compute hours and API transactions.

If you need a lot more included quantities, you can choose a higher tier plan that provides considerably more included quantities and a better discounted rate.

After the included quantities in existing instances are used up, additional usage is charged at the overage rate that's associated with the billing plan tier.

An API compute hour is the billing unit for the time that API calls take to run by using Machine Learning compute resources.

Studio compute hour is the billing unit for the aggregate time that your experiments use compute resources in studio.

The Dev/Test pricing tier provides limited, included quantities that allow you to test your experiment as a web service without incurring costs.

A new plan with the full included quantities of the upgraded/downgraded tier is created for the rest of the period.

If you want to deploy services to regions other than your default region, you may want to define your billing plans before you deploy your service.

If you find that you are using your included quantities rapidly, you can add instances or move to a higher tier and get better discounted rates.

The new billing plans are available in the three production regions in which we support the new web services: I

Your RRS production API transaction costs are calculated as the total number of API calls that you make multiplied by the price per 1,000 transactions (prorated by individual transaction).

Your RRS API production API compute hour costs are calculated as the amount of time required for each API call to run, multiplied by the total number of API transactions, multiplied by the price per production API compute hour.

For example, for Standard S1 overage, 1,000,000 API transactions that take 0.72 seconds each to run would result in (1,000,000 * $0.50/1K API transactions) in $500 in production API transaction costs and (1,000,000 * 0.72 sec * $2/hr) $400 in production API compute hours, for a total of $900.

However, the API transaction costs represent the number of batch jobs that you submit, and the compute costs represent the compute time that's associated with those batch jobs.

Your BES production API transaction costs are calculated as the total number of jobs submitted multiplied by the price per 1,000 transactions (prorated by individual transaction).

Your BES API production API compute hour costs are calculated as the amount of time required for each row in your job to run multiplied by the total number of rows in your job multiplied by the total number of jobs multiplied by the price per production API compute hour.

When you use the Machine Learning calculator, the transaction meter represents the number of jobs that you plan to submit, and the time-per-transaction field represents the combined time that's needed for all rows in each job to run.

For example, assume Standard S1 overage, and you submit 100 jobs per day that each consist of 500 rows that take 0.72 seconds each.

Your monthly overage costs would be (100 jobs per day = 3,100 jobs/mo * $0.50/1K API transactions) $1.55 in production API transaction costs and (500 rows * 0.72 sec * 3,100 Jobs * $2/hr) $620 in production API compute hours, for a total of $621.55.

The Azure Machine Learning Studio monthly fee is billed on a per workspace per month basis and prorated for partial months.

Apart from the preceding charges, in the case of Resource Manager-based web service, charges are aggregated to the selected plan: How do I sign up for Azure Machine Learning Free tier?

You can sign up for a 30-day free trial Azure subscription and later upgrade to a paid Azure subscription, or you can purchase a paid Azure subscription outright.

Other limitations include lack of support for R and Python, lack of staging APIs, and restricted dataset size and storage capacity.

By comparison, users who choose to sign in with a Microsoft account have full access to the Free tier of Machine Learning Studio that's described previously, which includes a persistent workspace and more comprehensive capabilities.

Applying deep learning to real-world problems

Instead, it is even the opposite: in order to compare deep learning techniques with other approaches and ensure that one method outperforms others, the standard procedure is to measure the performance on a standard dataset with the same evaluation procedure.

To elaborate on this issue, let’s consider two cases: Obviously there are tricks to lower these costs, but the overall message is that labeled data for real-world problems can be expensive to obtain.

The basic idea of pre-training is that we first train a neural network (or another machine learning algorithm) on a cheap and large dataset in a related domain or on noisy data in the same domain.

Now, in a second step, the parameters of the neural network are further optimized on a much smaller and expensive dataset of the problem you are actually trying to solve.

When fine-tuning, the number of classes might change: people often pre-train a neural network on a dataset like ImageNet with 1000 classes and then fine-tune it to their specific problem which likely has a different number of classes.

The learning rate is then often set a bit higher on the last layer as it needs to be learned from scratch, whereas the previous layers are trained with a lower learning rate.

For some datasets like ImageNet the features (the last fully connected layer) learned are so generic that they can be taken off-the-shelf and directly be used for some other computer vision problem.

similar logic can be applied to the medical vision problem where it is required to have several doctors independently label the same image in order to be close to 100% sure that the labeling is correct.

In summary, increasing performance doesn’t necessarily mean that you need human annotations which are often expensive but you might be able to get a labeled dataset for free or at substantially lower costs.

Preparing Your Dataset for Machine Learning: 8 Basic Techniques That Make Your Data Better

There’s a good story about bad data told by Martin Goodson, a data science consultant.

It employed machine learning (ML) to automatically sort through patient records to decide who has the lowest death risk and should take antibiotics at home and who’s at a high risk of death from pneumonia and should be in the hospital.

One of the most dangerous conditions that may accompany pneumonia is asthma, and doctors always send asthmatics to intensive care resulting in minimal death rates for these patients.

So, the absence of asthmatic death cases in the data made the algorithm assume that asthma isn’t that dangerous during pneumonia, and in all cases the machine recommended sending asthmatics home, while they had the highest risk of pneumonia complications.

But regardless of your actual terabytes of information and data science expertise, if you can’t make sense of data records, a machine will be nearly useless or perhaps even harmful.

But as we discussed in our story on data science team structures, life is hard for companies that can’t afford data science talent and try to transition existing IT engineers into the field.

Problems with machine learning datasets can stem from the way an organization is built, workflows that are established, and whether instructions are adhered to or not among those charged with recordkeeping.

Yes, you can rely completely on a data scientist in dataset preparation, but by knowing some techniques in advance there’s a way to meaningfully lighten the load of the person who’s going to face this Herculean task.

While those opportunities exist, usually the real value comes from internally collected golden data nuggets mined from the business decisions and activities of your own company.

The companies that started data collection with paper ledgers and ended with .xlsx and .csv files will likely have a harder time with data preparation than those who have a small but proud ML-friendly dataset.

When formulating the problem, conduct data exploration and try to think in the categories of classification, clustering, regression, and ranking that we talked about in our whitepaper on business application of machine learning.

You want an algorithm to answer binary yes-or-no questions (cats or dogs, good or bad, sheep or goats, you get the idea) or you want to make a multiclass classification (grass, trees, or bushes;

Ranking is actively used to recommend movies in video streaming services or show the products that a customer might purchase with a high probability based on his or her previous search and purchase activities.

Hotels know guests’ credit card numbers, types of amenities they choose, sometimes home addresses, room service use, and even drinks and meals ordered during a stay.

For instance, Salesforce provides a decent toolset to track and analyze salespeople activities but manual data entry and activity logging alienates salespeople.

If you’re aggregating data from different sources or your dataset has been manually updated by different people, it’s worth making sure that all variables within a given attribute are consistently written.

If you haven’t employed a unicorn who has one foot in healthcare basics and the other in data science, it’s likely that a data scientist might have a hard time understanding which values are of real significance to a dataset.

The technique can also be used in the later stages when you need a model prototype to understand whether a chosen machine learning method yields expected results.

Data rescaling belongs to a group of data normalization procedures that aim at improving the quality of a dataset by reducing dimensions and avoiding the situation when some of the values overweight others.

Imagine that you run a chain of car dealerships and most of the attributes in your dataset are either categorical to depict models and body styles (sedan, hatchback, van, etc.) or have 1-2 digit numbers, for instance, for years of use.

But the prices are 4-5 digit numbers ($10000 or $8000) and you want to predict the average time for the car to be sold based on its characteristics (model, years of previous use, body style, price, condition, etc.)  While the price is an important criterion, you don’t want it to overweight the other ones with a larger number.

The sets usually contain information about general processes in a wide range of life areas like healthcare records, historical weather records, transportation measurements, text and translation collections, records of hardware use, etc.

If you recommend city attractions and restaurants based on user-generated content, you don’t have to label thousands of pictures to train an image recognition algorithm that will sort through photos sent by users.

So, you still must find data scientists and data engineers if you need to automate data collection mechanisms, set the infrastructure, and scale for complex machine learning tasks.

Lecture 16: Dynamic Neural Networks for Question Answering

Lecture 16 addresses the question ""Can all NLP tasks be seen as question answering problems?"". Key phrases: Coreference Resolution, Dynamic Memory ...

Keras Tutorial TensorFlow | Deep Learning with Keras | Building Models with Keras | Edureka

TensorFlow Training - ** This Edureka Keras Tutorial TensorFlow video (Blog: .

Crossing the Chasm: Patterns to Develop, Operationalize, and Maintain ML Models (Cloud Next '18)

In many organizations, data scientists develop machine learning models and data/ML engineers put them into production. The chasm between the two roles ...

Jeff Dean's Keynote: Deep Learning to Solve Challenging Problems

Jeff Dean, Head of Google Brain talks about using deep learning to solve challenging problems at the AI/ML Workshop on research and practice in India. Find all ...

NIPS 2015 Workshop (Li) 15695 Learning and privacy with incomplete data and weak supervision

Can we learn to locate objects in images, only from the list of objects those images contain? Or the sentiment of a phrase in a review from the overall score?

Lifecycle of a machine learning model (Google Cloud Next '17)

In this video, you'll hear lessons learned from our experience with machine learning and how having a great model is only a part of the story. You'll see how ...

DensePose - 3D Machine Vision

Can machine vision map humans from videos to 3D Models? Yes! DensePose is a new architecture by the team at Facebook AI research that does just that.

Intro to Azure ML: Building a Machine Learning Model

Let's build our first machine learning model in Azure ML. First, we have to go shopping for a machine learning model. We must identify what type of machine ...

The 7 Steps of Machine Learning (AI Adventures)

How can we tell if a drink is beer or wine? Machine learning, of course! In this episode of Cloud AI Adventures, Yufeng walks through the 7 steps involved in ...

Developing a Warning System for Risk Managers from Scratch on GCP, using AI & ML (Cloud Next '18)

In this session we will demonstrate a Proof of Concept Early Warning System for Credit Risk Managers, built on the Google Cloud Platform. The flexibility of the ...