AI News, OptNet - reducing memory usage in torch neural networks

OptNet - reducing memory usage in torch neural networks

memory used, memory used for the outputs, memory used for the internal buffers, memory used for the parameters and grad parameters): Note that for most of the models, for a batch size of 1 most of the memory is spent with the weights and gradWeights of the network (and the latter can be safely freed during inference). More

The memory usage is shown in the following table (for float type), following (total memory used, memory used for the outputs, memory used for the parameters and grad parameters) as cudnn almost don't use internal buffers: We currently support a basic algorithm for training mode. Using

cudnn with batch size of 64, we currently obtain the following savings, in the format (total memory used, memory used for the outputs, memory used for the gradInputs, memory used for the parameters and gradParameters): Note that the relative save of the gradInput stays constant for different batch sizes, meaning that the total relative savings will be more important for bigger batch sizes (as the parameters doesn't depend on the batch size).

The Logstash defaults are chosen to provide fast, safe performance for most users.

The Monitor pane in particular is useful for checking whether your heap allocation is sufficient for the current workload.

Note that the specific batch sizes used here are most likely not applicable to your specific workload, as the memory demands of Logstash vary in large part based on the type of messages you are sending.

Examining the in-depth GC statistics with a tool similar to the excellent VisualGC plugin shows that the over-allocated VM spends very little time in the efficient Eden GC, compared to the time spent in the more resource-intensive Old Gen “Full” GCs.


For single-node setups Flink is ready to go out of the box and you don’t need to change the default configuration to get started.

You can manually set the environment variable JAVA_HOME or the configuration key in conf/flink-conf.yaml if you want to manually override the Java runtime to use.

By default, Flink allocates a fraction of 0.7 of the free memory (total memory configured via taskmanager.heap.mb minus memory used for network buffers) for its managed memory.

These options are useful for debugging a Flink application for memory and garbage collection related issues, such as performance and out-of-memory process kills or exceptions.

Flink supports Kerberos authentication for the following services: Configuring Flink for Kerberos security involves three aspects, explained separately in the following sub-sections.

For Hadoop components, Flink will automatically detect if the configured Kerberos credentials should be used when connecting to HDFS, HBase, and other Hadoop components depending on whether Hadoop security is enabled (in core-site.xml).

For any connector or component that uses a JAAS configuration file, make the Kerberos credentials available to them by configuring JAAS login contexts for each one respectively, using the following configuration: This allows enabling Kerberos authentication for different connectors or components independently.

You may also provide a static JAAS configuration file using the mechanisms described in the Java SE Documentation, whose entries will override those produced by the above configuration option.

Below is a list of currently first-class supported connectors or components by Flink for Kerberos authentication: For more information on how Flink security internally setups Kerberos authentication, please see here.

Setups that do not specify a HDFS configuration have to specify the full path to HDFS files (hdfs://address:port/path/to/files) Files will also be written with default HDFS parameters (block size, replication factor).

The configuration keys in this section are independent of the used resource management framework (YARN, Mesos, Standalone, …) Previously this key was named recovery.mode and the default value was standalone.

You have to configure jobmanager.archive.fs.dir in order to archive terminated jobs and add it to the list of monitored directories via historyserver.archive.fs.dir if you want to display them via the HistoryServer’s web frontend.

The number and size of network buffers can be configured with the following parameters: Although Flink aims to process as much data in main memory as possible, it is not uncommon that more data needs to be processed than memory is available.

If the taskmanager.tmp.dirs parameter is not explicitly specified, Flink writes temporary data to the temporary directory of the operating system, such as /tmp in Linux systems.

Lecture 7 | Training Neural Networks II

Lecture 7 continues our discussion of practical issues for training neural networks. We discuss different update rules commonly used to optimize neural networks during training, as well as...

Lecture 10 | Recurrent Neural Networks

In Lecture 10 we discuss the use of recurrent neural networks for modeling sequence data. We show how recurrent neural networks can be used for language modeling and image captioning, and how...

Lecture 11 | Detection and Segmentation

In Lecture 11 we move beyond image classification, and show how convolutional networks can be applied to other core computer vision tasks. We show how fully convolutional networks equipped...

How to Do Sentiment Analysis - Intro to Deep Learning #3

In this video, we'll use machine learning to help classify emotions! The example we'll use is classifying a movie review as either positive or negative via TF Learn in 20 lines of Python. ...

Sequence Models and the RNN API (TensorFlow Dev Summit 2017)

In this talk, Eugene Brevdo discusses the creation of flexible and high-performance sequence-to-sequence models. He covers reading and batching sequence data, the RNN API, fully dynamic calculation...

Storage Devices and Video Post-Production (Part 01: Problems)

If you are also experiencing a choppy playback, this video might be able to help you. In this episode we discuss about the essential knowledge of storage devices, that is handy for everyone...

Lecture 2 | Word Vector Representations: word2vec

Lecture 2 continues the discussion on the concept of representing words as numeric vectors and popular approaches to designing word vectors. Key phrases: Natural Language Processing. Word...

Lecture 9 | CNN Architectures

In Lecture 9 we discuss some common architectures for convolutional neural networks. We discuss architectures which performed well in the ImageNet challenges, including AlexNet, VGGNet, GoogLeNet,...

Lecture 15 | Efficient Methods and Hardware for Deep Learning

In Lecture 15, guest lecturer Song Han discusses algorithms and specialized hardware that can be used to accelerate training and inference of deep learning workloads. We discuss pruning, weight...

Lecture 13: Convolutional Neural Networks

Lecture 13 provides a mini tutorial on Azure and GPUs followed by research highlight "Character-Aware Neural Language Models." Also covered are CNN Variant 1 and 2 as well as comparison between...