AI News, Advantages of containerizing your data pipeline:

Advantages of containerizing your data pipeline:

(This is part 1 of a two part series of blog posts about doing data science and engineering in a containerized world) I'm sure you have been hearing at least some of the hype over "containers"

Here are some Docker images and Dockerfiles to help you quickly spin up a data pipeline: Also, if you are really interested in containerized data pipelines, here are some interesting open source projects that you may want to follow, contribute to, or utilize: Stay tuned for Part 2, where I will discuss doing data science in containers!

#BDAM: Data Pipelines in Kubernetes, by Sean Suchter, Pepperdata

Speaker: Sean Suchter, Pepperdata Big Data Applications Meetup, 07/19/2017 Palo Alto, CA More info here: Link to ..

Container management and deployment: from development to production (Google Cloud Next '17)

There are common questions around container management and deployment. What does a development and deployment workflow look like in a containerized ...

Building GPU-Accelerated Workflows with TensorFlow and Kubernetes [I] - Daniel Whitenack

Building GPU-Accelerated Workflows with TensorFlow and Kubernetes [I] - Daniel Whitenack, Pachyderm GPUs are critical to some artificial intelligence ...

JuliaCon 2017 | Integrating Julia in Real-World, Distributed Pipelines | Daniel Whitenack

DANIEL WHITENACK, PACHYDERM After attending this workshop, you will have the skills needed to integrate Julia in real-world environments. Not only that ...

Apache Beam: Portable and Parallel Data Processing (Google Cloud Next '17)

Apache Beam provides a portable standard for expressing robust, out-of-order data processing pipelines in a variety of languages across a variety of platforms.

Cloud 66 Container Stack

A step-by-step demo to build, run and maintain containerized apps in production with Skycap and Maestro. Skycap is a container native CI/CD pipeline to build ...

Microsoft Data Platform – SQL Server 2017 and Azure Data Services

All around us, data is driving digital transformation. Companies that invest heavily in cloud, data and AI have nearly double the operating margin of those that do ...

snakemake analysis example

A simple walk-through example of using snakemake as a workflow management tool for engineering analysis. All files are here: ...

Azure: “Kubernetes the Easy Way” Managed Kubernetes on Azure AKS | E101

We'll get hands-on with Azure Managed Kubernetes (AKS) and walk you through how to manage your containers using AKS—from provisioning your first cluster ...

Red Hat OpenShift 3 Hadoop Demo

With the power of Docker and Kubernetes, OpenShift 3 can easily and rapidly deploy, provision and scale out complicated cluster systems, e.g. Hadoop. Here is ...