AI News, Sarcasm Detection with Machine Learning in Spark
Sarcasm Detection with Machine Learning in Spark
This post is inspired by a site I found whilst searching for a way to detect sarcasm within sentences.
This search led me to the above link site where the author Mathieu Cliche cleverly came up with the idea of using tweets as the training set.
However searching for tweets that contain the hastag #sarcasm or #sarcastic would provide me with a vast amount of training data (providing a good percentage of those tweets are actually sarcstic).
Using that approach as the basis, I developed a Spark application using the MlLib api that would use the Naive Bayes classifier to detect sarcasm in sentences - This post will cover the basics and I will be expanding on this next time to utilise sarcastic tweets to train my model!
The file should have 2 “columns”, the first for the label(I used 1 for a sarcastic row and 0 for a non sarcastic row) and the second for the sentence.
Create two data frames: each with 2 columns “label” and “text” - one data frame for the training data, the other for the test data.
This data can then be used by the algorithm to build a model allowing it to predict/guess whether a similar vector is also sarcastic (or not).
This will now build a model that can be used to classify new sentences - that the model has never seen before - as sarcastic or not sarcastic by seeing if the new sentence (when converted to a vector) is more similar to sarcastic vectors, or non sarcastic vectors.
We want to create a tuple containing the predicted value, and the original label that we gave the data so we can see how accurate it performed.
When you (or anyone else) wishes to then predict the level of sarcasm within a sentence, they can simply write a Spark application that loads your model and can then use it as shown previously.
Training and Testing Data Sets
Typically, when you separate a data set into a training set and testing set, most of the data is used for training, and a smaller portion of the data is used for testing.
The information about the size of the training and testing data sets, and which row belongs to which set, is stored with the structure, and all the models that are based on that structure can use the sets for training and testing.
By default, after you have defined the data sources for a mining structure, the Data Mining Wizard will divide the data into two sets: one with 70 percent of the source data, for training the model, and one with 30 percent of the source data, for testing the model.
You can also configure the wizard to set a maximum number of training cases, or you can combine the limits to allow a maximum percentage of cases up to a specified maximum number of cases.
For example, if you specify 30 percent holdout for the testing cases, and the maximum number of test cases as 1000, the size of the test set will never exceed 1000 cases.
If you use the same data source view for different mining structures, and want to ensure that the data is divided in roughly the same way for all mining structures and their models, you should specify the seed that is used to initialize random sampling.
If you want to determine the number of cases used for training or for testing, or if you want to find additional details about the cases included in the training and test sets, you can query the model structure by creating a DMX query.
Training, test, and validation sets
Most approaches that search through training data for empirical relationships tend to overfit the data, meaning that they can identify apparent relationships in the training data that do not hold in general.
For example, if the most suitable classifier for the problem is sought, the training dataset is used to train the candidate algorithms, the validation dataset is used to compare their performances and decide which one to take and, finally, the test dataset is used to obtain
Since our goal is to find the network having the best performance on new data, the simplest approach to the comparison of different networks is to evaluate the error function using data which is independent of that used for training.
The performance of the networks is then compared by evaluating the error function using an independent validation set, and the network having the smallest error with respect to the validation set is selected.
Since this procedure can itself lead to some overfitting to the validation set, the performance of the selected network should be confirmed by measuring its performance on a third independent set of data called a test set.
An application of this process is in early stopping, where the candidate models are successive iterations of the same network, and training stops when the error on the validation set grows, choosing the previous model (the one with minimum error).
These repeated partitions can be done in various ways, such as dividing into 2 equal datasets and using them as training/validation, and then validation/training, or repeatedly selecting a random subset as a validation dataset.
For example, for an image classifier, if you have a series images in the dataset that are similar and you put half in test and half in training, you will end up inflating the performance metrics of your classifier.
Another example of parameter adjustment is hierarchical classification (sometimes referred to as instance space decomposition ), which splits a complete multi-class problem into a set of smaller classification problems.
on the validation set one can see which classes are most frequently mutually confused by the system and then the instance space decomposition is done as follows: firstly, the classification is done among well recognizable classes, and the difficult to separate classes are treated as a single joint class, and finally, as a second classification step the joint class is classified into the two initially mutually confused classes.
Creating training and test data sets and preparing the data
if it is time-series data about sales, the training and test dataset ought to represent a reasonable business cycle that covers peak and off-peak times, weekends, etc.
Coming back to natural language processing, of which sentiment analysis is a subset, it’s important to cover as many words as possible that express sentiment and represent the lexicon used in the target texts.
When we speak about supervised learning for natural language processing, the labeling has to be performed manually to guarantee correct classification of the training and test data.
Here is a typical viewers’ movie review: This dataset was used in a research paper titled Learning Word Vectors for Sentiment Analysis published by Stanford University, which also includes additional data related to the research.
Tweets tend to be much shorter than typical IMDB reviews, emojis are used in tweets much much often than in IMDB reviews, And there tweets often contain less rigorous grammar and spelling than IMDB reviews.
We also combined positive- and neutral-labeled tweets in one category as our task is to identify (specifically)negative tweets so we must be able to separate negative from neutral tweets, but separating neutral from positive is beyond our scope.
Any Twitter user’s tweet or retweet produces light or heavy social impact directly linked to the number of the tweeter’s followers as that is the number of people who will see the user’s post.
Unfortunately there is no more recent information, but we can guess that now, in 2016, an average Twitter user has more followers than that, but not a huge increase, so we will call users who have fewer than 500 followers regular users.
Here is a list of the basic principles we will follow while building the solution: In order to train and later apply the models to the Twitter data it is first important to clean and prepare the data.
First the text is extracted (assume we got a text filtered by the keyword “Doctor Strange”):Then several normalization steps are performed— removing capitalization, punctuation, keywords used for tweet filtering, and unicode emoticons.
We do it here in a very straightforward way: unnecessary symbols are removed with a regular expression, the text is separated spaces considering exceptions (like “cool stuff”).Now we are almost ready to search for word matches in texts and dictionary entries.
In fact, this transformation is a type of dimensionality reduction that solves two challenges: It increases matching rate and reduces the computational complexity of the machine learning algorithm.
So summarizing the simplified process of data preparation: Now we have all data necessary and properly prepared to perform the modeling or, in other words, build a classifier that will tell whether a tweet is negative or positive.
- On Thursday, September 19, 2019
Boomerang Trick Shots | Dude Perfect
Time to take boomerangs to the next level! ▻ Click HERE to subscribe to Dude Perfect! ▻ Click HERE to watch our most recent ..
Professional Learning Communities.
funny communication skills
how to communicate with others
Funny School Teacher - Such an intelligent qualified teacher
Funny School Teacher - Such an intelligent qualified teacher.
Sentiment Analysis in 4 Minutes
Link to the full Kaggle tutorial w/ code: Sentiment Analysis in 5 lines of ..
Adding Machine Learning to your applications
Google provides infrastructure, services, and APIs for you to create your own machine learning models. They also have pretrained machine learning APIs that ...
Life is easy. Why do we make it so hard? | Jon Jandai | TEDxDoiSuthep
Never miss a talk! SUBSCRIBE to the TEDx channel: Jon is a farmer from northeastern Thailand. He founded the Pun Pun Center for ..
How to Do Sentiment Analysis - Intro to Deep Learning #3
In this video, we'll use machine learning to help classify emotions! The example we'll use is classifying a movie review as either positive or negative via TF Learn ...
Funny! Orchestra plays Microsoft Windows™ - the waltz
Rainer Hersch performs his waltz based on the sounds used in Windows XP. CHECK out Rainer Hersch: BUY STUFF ..