AI News, Machine Learning Kaggle Competition Part One: Getting Started

Machine Learning Kaggle Competition Part One: Getting Started

Hit the leftward facing arrow in the upper right to expand the kernel control panel which brings up three tabs (if the notebook is not in fullscreen, then these three tabs may already be visible next to the code).

We can view changes to the code, look at log files of a run, see the notebook generated by a run, and download the files that are output from a run.

For this problem, there is 1 main training data file (with the labels included), 1 main testing data file, and 6 additional data files.

To understand the data, it’s best to take a couple minutes away from the keyboard and read through the problem documentation, such as the column descriptions of each data file.

Once we understand the data and the problem, we can start structuring it for a machine learning task This means dealing with categorical variables (through one-hot encoding), filling in the missing values (imputation), and scaling the variables to a range.


By bringing the interactive style of Light Table to the rock-solid usability of Atom, Hydrogen makes it easy to write code the way you want to.

You also may be interested in our latest project – nteract – a desktop application that wraps up the best of the web based Jupyter notebook.

Feel free to add your own to the list: If you are interested in building a plugin take a look at our plugin API documentation.

Jupyter Notebook Tutorial: The Definitive Guide

(To practice pandas dataframes in Python, try this course on Pandas foundations.) In this case, "notebook"

Because of the mix of code and text elements, these documents are the ideal place to bring together an analysis description and its results as well as they can be executed perform the data analysis in real time.

Unfortunately, the prototype had never become fully usable.  Fast forward two years: the IPython team had kept on working, and in 2007, they formulated another attempt at implementing a notebook-type system.

Since the layout of the Sage notebook was based on the layout of Google notebooks, you can also conclude that also Google used to have a notebook feature around that time.  For what concerns the idea of the notebook, it seems that Fernando Pérez, as well as William Stein, one of the creators of the Sage notebook, have confirmed that they were avid users of the Mathematica notebooks and Maple worksheets.

The Mathematica notebooks were created as a front end or GUI in 1988 by Theodore Gray.  The concept of a notebook, which contains ordinary text and calculation and/or graphics, was definitely not new.  Also, the developers had close contact with one another and this, together with other failed attempts at GUIs for IPython and the use of "AJAX"

= web applications, which didn't require users to refresh the whole page every time you do something, were two other motivations for the team of William Stein to start developing the Sage notebooks.  If you want to know more details, check out the personal accounts of Fernando Pérez and William Stein about the history of their notebooks.

The general recommendation is that you use the Anaconda distribution to install both Python and the notebook application.  The advantage of Anaconda is that you have access to over 720 packages that can easily be installed with Anaconda's conda, a package, dependency, and environment manager.

Also, if the application shows python [conda root] and python [default] as kernel names instead of Python 3, you can try executing conda remove _nb_ext_conf or read up on the following Github issue and make the necessary adjustments.

You can also manually register your kernels, for example: To configure the Python 3.5 environment, you can just use the same commands but replace py27 by py35 and the version number by 3.5.  Alternatively, if you're working with Python 3 and you want to set up a Python 2 kernel, you can also do this: As the explanation of the kernels in the first section already suggested, you can also run other languages besides Python in your notebook!

If you don't want to install the essentials in your current environment, you can use the following command to create a new environment just for the R essentials: Open up the notebook application to start working with R with the usual command.  If you now want to install additional R packages to elaborate your data science project, you can either build a Conda R package by running, for example: Or you can install the package from inside of R via install.packages or devtools::install_github (from GitHub).

To see which magic commands you have available in your interpreter, you can simply run the following: Tip: the regular Python help() function also still works and you can use the magic command %quickref to show a quick reference sheet for IPython.

If you're looking for more information on the magics commands or on functions, you can always use the ?, just like this: Note that if you want to start a single-line expression to run with the magics command, you can do this by using % .

The following example illustrates the difference between the two: Stated differently, the magic commands are either line-oriented or cell-oriented. In the first case, the commands are prefixed with the % character and they work as follows: they get as an argument the rest of the line.

You can also use magics to mix languages in your notebook with the IPython kernel without setting up extra kernels: there is rmagics to run R code, SQL for RDBMS or Relational Database Management System access and cythonmagic for interactive work with cython,...

But there is so much more!  To make use of these magics, you first have to install the necessary packages: Tip: if you want to install packages, you can also execute these commands as shell commands from inside your notebook by placing a ! in front of the commands, just like this: Only then, after a successful install, can you load in the magics and start using them: Let's demonstrate how the magics exactly work with a small example: This is just an initial not nearly everything you can do with R magics, though.

  See an example of a text input widget below: This example was taken from a wonderful tutorial on building interactive dashboards in Jupyter, which you can find on this page.

In practice, you might want to share your notebooks with colleagues or friends to show them what you have been up to or as a data science portfolio for future employers. However, the notebook documents are JSON documents that contain text, source code, rich media output, and metadata.

But don't forget to import nbconvert first if you don't have it yet!  Then, you can give in something like the following command to convert your notebooks:  With nbconvert, you can make sure that you can calculate an entire notebook non-interactively, saving it in place or to a variety of other formats.

You probably already know the drill, but these principles include the following: In addition to these general best practices for programming, you could also consider the following tips to make your notebooks the best source for other users to learn: Jonathan Whitmore wrote in his article some practices for using notebooks for data science and specifically addresses the fact that working with the notebook on data science problems in a team can prove to be quite a  challenge.  That is why Jonathan suggests some best practices: This section is meant to give you a short list with some of the best notebooks that are out there so that you can get started on learning from these examples.  Note that this list is definitely not exhaustive.

Introduction to Kaggle Kernels

In this episode of AI Adventures, Yufeng explains how to use Kaggle Kernels to do data science in your browser without any downloads! Associated Medium ...

My First Kaggle Submission

To prepare you for Data Science Dojo's day two homework we will explain what Kaggle is and show you how to create a Kaggle account and submit your model ...

Python Code from Jupyter

This is how to use python code .py file from your Jupyter notebook.

Configuring Your Atom Editor to Work With Python

I love, and I was reluctant to use it for coding Python projects, until I realized that the hackable nature of the editor meant I could probably do everything I ...

8 How do I submit to Kaggle

In this video I walk you through the instructions for submission. These may be different to each competition on Kaggle. The website generally offers decent ...

Windows\System32\Config\System missing or corrupt fix

Video tutorial on how to fix the "Windows could not start because the following file is missing or corrupt: Windows\System32\Config\System" error message in ...


Learn easy way how to fix repair Windows 7/8/8.1/10 NOT booting to start up screen Recovery error code "0xc000000f" aka blue screen of death 2016 solved, ...

Learn Jupyter Notebooks (pt.1a) Hydrogen with Atom

In this video I show a plugin called Hydrogen, which adds Jupyter like functionality to Atom.

How to Fix "0xc000000e: Windows Failed to Start" boot error

How to fix the bootmgr Windows Boot Manager error "0xc000000e Windows Failed to Start: The boot selection failed because a required device is inaccessible" ...

How to fix "Your PC/Device needs to be repaired" error (0xc0000428)

PC was booted from Windows 10 Technical Preview DVD. Displayed error message: Recovery, Your PC/Device needs to be repaired, Digital Signature couldn't ...