AI News, What I Learned Implementing a Classifier from Scratch in Python

What I Learned Implementing a Classifier from Scratch in Python

In order to demystify some of the magic behind machine learning algorithms, I decided to implement a simple machine learning algorithm from scratch.

classifier is a machine-learning algorithm that determines the class of an input element based on a set of features.

For example, a classifier could be used to predict the category of a beer based on its characteristics, it’s “features”.

A machine learning classifier could potentially be used to predict that a beer with 8% alcohol content, 100 IBU and with strong aromas of oranges is an Indian Pale Ale.

In machine learning, there are three main types of tasks: unsupervised learning, supervised learning and reinforcement learning.

The following tasks can be considered as part of the “preprocessing”: Once you have your datasets ready to be used, the second step is to select an algorithm to perform your desired task.

There are a wide variety of metrics that can be used to measure the performance of a machine learning model.

According to Raschka, “one commonly used metric is classification accuracy, which is defined as the proportion of correctly classified instances”.

In this post, we’ll mainly focus on this part of the machine learning work flow.

If you are interested in the other sections of the machine learning work flow, which you should be, I’ll be linking to a great notebook at the end of this post.

This metric measures “how accurately an algorithm is able to predict outcome values for previously unseen data”.

Given a training dataset, the algorithm automatically learns “the optimal weight coefficients that are then multiplied with the input features in order to make the decision of whether a neuron fires or not”.

This difference is then multiplied by the learning rate eta, an hyperparameter of value between zero and one set by the data scientist.

Simply said, if you are able to draw a straight line to entirely separate the two classes, the algorithm will converge.

Else, the algorithm will keep iterating and will readjust weights until it reaches the maximum number of iterations n_iter.

Parameters such as learning rate and number of iteration can seem very abstract if you jump in straight to using an algorithm from a library like scikit-learn.

As you can see in the snippet below, the fit function will iterate on each observation, call the predict function and then adjust the weights based on the difference between the target and the predicted value and then multiplied by the learning rate.

If the number of iteration was set to one, the algorithm would loop through the dataset only once and update the weights one time for each data point.

For example, if you have a very low learning rate, which means that the algorithm will adjust it’s weight only marginally at each iteration, you will probably need a higher number of iteration.

Back in 2012 when I was learning to code Ruby on Rails, a web application development framework, I realized that typing down all of the code examples from tutorials really helped me memorize and understand the concepts.

How to Make a Prediction - Intro to Deep Learning #1

Welcome to Intro to Deep Learning! This course is for anyone who wants to become a deep learning engineer. I'll take you from the very basics of deep learning ...

Machine Learning in R - Classification, Regression and Clustering Problems

Learn the basics of Machine Learning with R. Start our Machine Learning Course for free: ...

How SVM (Support Vector Machine) algorithm works

In this video I explain how SVM (Support Vector Machine) algorithm works to classify a linearly separable binary data set. The original presentation is available ...

Logistic Regression in R | Machine Learning Algorithms | Data Science Training | Edureka

Data Science Training - ) This Logistic Regression Tutorial shall give you a clear understanding as to how a Logistic ..

How to Predict Stock Prices Easily - Intro to Deep Learning #7

We're going to predict the closing price of the S&P 500 using a special type of recurrent neural network called an LSTM network. I'll explain why we use ...

Pickling and Scaling - Practical Machine Learning Tutorial with Python p.6

In the previous Machine Learning with Python tutorial we finished up making a forecast of stock prices using regression, and then visualizing the forecast with ...

Lecture 15 | Efficient Methods and Hardware for Deep Learning

In Lecture 15, guest lecturer Song Han discusses algorithms and specialized hardware that can be used to accelerate training and inference of deep learning ...

Randomized Weighted Majority Algorithm

Extra credit for 10701 Link for the slides:

Analytics Case Study: Predicting Probability of Churn in a Telecom Firm| Data Science

In this video you will learn how to predict Churn Probability by building a Logistic Regression Model. This is a data science case study for beginners as to how to ...

Machine Learning - Find relationship between data

Hi friends this video will explain basic introduction on machine learning as relation between data. What is Science? We can look at scince from lot of different ...