AI News, Data Science Blog
- On Wednesday, August 22, 2018
- By Read More
Data Science Blog
What I realized was that as I was writing the post, I found that I kept struggling with inconsistent data across different seasons.
In particular I'm going to show you how you can use normalization techniques to compare seemlingly incomparable data!
The data I'm using is a collection of MLB standings and attendance data from the past 70 years.
After running my data collection script in R, I sent the output to a .feather file using the feather R package.
First off (and probably most obviously) is that the value of the dollar has changed over the past 70 years.
Despite having been retired for 15 years, the Mets still pay him over $1M per year, thanks to an interesting negotiation and Mets owner Fred Wilpon's involvement in Bernie Madoff's Ponzi scheme.
For each year we're going to calculate the mean salary for the league as whole, and then create a derived field which compares a given team's payroll to the mean payroll for the entire league.
for instance, what if you wanted the norm_payroll to bet a standardized value between 0 and 1 (instead of a uncapped scale as in the previous example)?
Many machine learning algorithms perform much better using scaled data (support vector machine comes to mind).
To do this we'll use the same approach as before (as in, normalizing by year) but instead of using the mean, we're going to use the max and min values for each year.
By no means is this the end all be all of data normalization (there are many books on the subject), but hopefully this gives you a quick intro to this very important topic.
- On Wednesday, October 23, 2019
930: Self-Reported by Colin Wright of Exile Lifestyle (Using Data & Self Assessment to Evaluate...
Colin Wright of Exile Lifestyle shares how he uses personal data tracking practically. Episode 930: Self-Reported by Colin Wright of Exile Lifestyle (Using Data ...
National Assembly for Wales Plenary 20.03.18
Plenary is the meeting of the whole Assembly which takes place in the Siambr, the Senedd's debating chamber. Plenary is chaired by the Presiding Officer and ...