Comparing deep learning frameworks: TensorFlow, PyTorch, Keras, MXNet, Microsoft Cognitive Toolkit, Caffe, etc
Hello. On this Friday day, we are sharing with you the first publication on the launch of the Data Scientist course. Enjoy your reading.
A great number of organizations were inspired to use artificial intelligence in their decisions to expand the scope of activities or develop their start-up, but one important thing to be understood: the chosen development technology should go along with a good framework for deep learning, especially because each framework serves some kind of separate goal. The search for the perfect tandem in this case is a necessary condition for the smooth and rapid development of business and the successful deployment of projects.
')
The following list of deep learning frameworks can be helpful in the process of choosing the appropriate means for solving specific problems that you encounter while working on a project. You will be able to compare the pros and cons of various solutions, evaluate their limits of possibilities and learn about the best use cases for each solution!
1. Tensorflow
Created by Google and written in Python and C ++, TensorFlow is one of the best open source libraries for numerical computation. It simply has to be good, because even such giants as DeepMind, Uber, AirBnB or Dropbox have chosen this framework for their needs.
TensorFlow is good for complex projects, such as creating multi-layer neural networks. It is used to recognize voice or images and text applications, such as Google Translate, for example.
Of course, the experts called many of its advantages:
And cons:
It loses in performance in benchmark tests , in comparison with CNTK and MXNet, for example;
It has a higher entry threshold for beginners than PyTorch or Keras. Naked Tensorflow is quite low-level and requires a lot of generic code, and the “define and run” mode for Tensorflow makes the debug process much more complicated.
There is another significant limitation: the only fully supported language is Python.
2. PyTorch
PyTorch is the successor to Python for the Torch library written in Lua, and a great competitor to TensorFlow. It was developed by Facebook and used by Twitter, Salesforce, Oxford University and many other companies.
PyTorch is mainly used to train models quickly and efficiently, therefore it is the choice of a large number of developers.
It has many important advantages:
Distributed learning is available from version 0.4.
But there is a framework and several undeniable minuses:
3. Keras
Keras is a minimalist Python-based library that can run on top of TensoFlow, Theano, or CNTK. It was developed by Google’s engineer, François Chollet, to speed up experiments. Keras supports a wide range of layers of neural networks, such as convolutional, recurrent or dense layers.
This framework is good in cases for translation, recognition of images, speech, etc.
Benefits:
And a few flaws:
Keras is not as functional as TensorFlow and offers fewer options for managing a network connection, which can be a serious limitation if you are going to create some kind of specialized deep learning model.
4. MXNet
This is a deep learning framework created by Apache that supports an abundance of languages, such as Python, Julia, C ++, R or JavaScript. It is used in Microsoft, Intel, and Amazon web services.
The MXNet framework is known for its high scalability, which is why it is used by large companies mainly for speech recognition and handwriting, NLP and forecasting.
It has many advantages:
And now about the disadvantages of MXNet:
Based on the above arguments, MXNet is a good framework for large industrial projects, but it is still quite young, so remember that you may not receive technical support in resolving your issue as quickly as you would like.
5. CNTK
Now this framework is called Microsoft Cognitive Toolkit. It is an open framework for deep learning, designed to work with really large datasets with support for Python, C ++, C # and Java.
CNTK provides effective learning models for voice recognition, handwriting and pictures, also supports CNN and RNN. It is used in Skype, Xbox and Cortana.
As always, experts have already appreciated its benefits:
And one single flaw:
6. Caffe and Caffe2
Caffe is a C ++ framework that has a very useful Python interface. It supports CNN and direct distribution networks, and is also good for training models (without writing extra lines of code), image processing, and improvements to existing neural networks. On the other hand, it is not so well documented and difficult to compile. There is no information about the use of Caffe by any large companies. In 2017, Facebook released Caffe 2, which became the real successor to Caffe, and was created for both mobile and large-scale development in a production environment.
On Facebook, it is known as, “A production-ready platform (...), operating on more than 1 billion smartphones, covering 8 generations of an IPhone and six generations of Android processor architectures.”
This framework is good for several reasons:
And again the main disadvantage:
Despite the fact that Caffe2 is well tested for speed and memory, it is relatively new and quite limited in the deployment of complex networks that need at least handy tools for debugging and technical support.
7. Deeplearning4j
If your main programming language is Java, then you should definitely take a closer look at DL4J. It is an open source commercial platform, written primarily for Java and Scala, which provides good support for various types of neural networks (such as CNN, RNN, RNTN, or LTSM).
This framework will be a good choice because it has great potential in the field of image recognition, natural language processing, vulnerability search and text analysis.
Pros:
Surprisingly, speaking of this framework, the experts do not focus on any specific flaws, as is the case in the general debate about the suitability of Java for machine learning in general. Since Java is not so often used in machine learning projects, the framework cannot rely on the growing bases of ready-made code to solve specific problems. As a result, the cost of your project can be much higher and the solution of one task can significantly slow down the overall progress of the project.
8. Chainer
Another Python deep learning framework supported by giants such as Intel, IBM, NVIDIA and AWS. He can easily use multiple GPUs.
Chainer is mainly used for voice recognition, machine translation and tonality analysis. It supports various network architectures, such as CNN, fast-forward, RNN, and has several important advantages over its competitors:
On the other hand:
Since other Python-centric frameworks are much more popular, you may not get such quick help, as is the case with other frameworks, such as TensorFlow or PyTorch.
Summing up
Choosing the best framework for a deep learning project can be extremely difficult. You need to consider the following factors:
I hope that this comparison of frameworks will shed a little light on this problem and help you choose the most suitable solution for your project.
Write comments and see you at the open day , which Alexander Nikitin will conduct on March 19.
A great number of organizations were inspired to use artificial intelligence in their decisions to expand the scope of activities or develop their start-up, but one important thing to be understood: the chosen development technology should go along with a good framework for deep learning, especially because each framework serves some kind of separate goal. The search for the perfect tandem in this case is a necessary condition for the smooth and rapid development of business and the successful deployment of projects.
')
The following list of deep learning frameworks can be helpful in the process of choosing the appropriate means for solving specific problems that you encounter while working on a project. You will be able to compare the pros and cons of various solutions, evaluate their limits of possibilities and learn about the best use cases for each solution!
1. Tensorflow
Created by Google and written in Python and C ++, TensorFlow is one of the best open source libraries for numerical computation. It simply has to be good, because even such giants as DeepMind, Uber, AirBnB or Dropbox have chosen this framework for their needs.
TensorFlow is good for complex projects, such as creating multi-layer neural networks. It is used to recognize voice or images and text applications, such as Google Translate, for example.
Of course, the experts called many of its advantages:
- For her, written a large number of manuals and documentation;
- It offers powerful tools for monitoring the process of learning models and visualization (Tensorboard);
- It is supported by a large community of developers and tech companies;
- It provides service models;
- It supports distributed learning;
- TensorFlow Lite provides low latency output for mobile devices;
And cons:
It loses in performance in benchmark tests , in comparison with CNTK and MXNet, for example;
It has a higher entry threshold for beginners than PyTorch or Keras. Naked Tensorflow is quite low-level and requires a lot of generic code, and the “define and run” mode for Tensorflow makes the debug process much more complicated.
There is another significant limitation: the only fully supported language is Python.
2. PyTorch
PyTorch is the successor to Python for the Torch library written in Lua, and a great competitor to TensorFlow. It was developed by Facebook and used by Twitter, Salesforce, Oxford University and many other companies.
PyTorch is mainly used to train models quickly and efficiently, therefore it is the choice of a large number of developers.
It has many important advantages:
- Thanks to the architecture of the framework, the process of creating a model is quite simple and transparent;
- The default mode of define-by-run is a reference to traditional programming. The framework supports popular debugging tools such as pdb, ipdb, or the PyCharm debugger;
- It supports declarative data concurrency ;
- It has many pre-trained models and ready-made modular parts that are easy to combine;
Distributed learning is available from version 0.4.
But there is a framework and several undeniable minuses:
- Insufficient model support;
- It’s not yet ready for full production, but the roadmap for version 1.0 looks really impressive;
- There is a lack of interfaces for monitoring and visualization, like TensorBoard - however, it has an external connection to the Tensorboard.
3. Keras
Keras is a minimalist Python-based library that can run on top of TensoFlow, Theano, or CNTK. It was developed by Google’s engineer, François Chollet, to speed up experiments. Keras supports a wide range of layers of neural networks, such as convolutional, recurrent or dense layers.
This framework is good in cases for translation, recognition of images, speech, etc.
Benefits:
- Prototyping is really quick and easy;
- It is lightweight enough to build deep learning models for multiple layers;
- It has fully configurable modules;
- It has a simple and intuitive interface, respectively, good for beginners;
- It has built-in support for multi-GPU training;
- Can be configured as evaluators for TensorFlow and trained on GPU clusters on the Google Cloud platform;
- Runs on Spark;
- Supports GPU from NVIDIA, TPU from Google, GPU with Open-CL, such as AMD.
And a few flaws:
- It may be too high-level and not always easy to customize;
- It is limited to Tensorflow, CNTK and Theano backends.
Keras is not as functional as TensorFlow and offers fewer options for managing a network connection, which can be a serious limitation if you are going to create some kind of specialized deep learning model.
4. MXNet
This is a deep learning framework created by Apache that supports an abundance of languages, such as Python, Julia, C ++, R or JavaScript. It is used in Microsoft, Intel, and Amazon web services.
The MXNet framework is known for its high scalability, which is why it is used by large companies mainly for speech recognition and handwriting, NLP and forecasting.
It has many advantages:
- It is quite fast, flexible and effective in working with deep learning algorithms;
- It provides advanced GPU support;
- It can run on any device;
- It has a high-performance imperative API;
- It provides easy support for models;
- It is extremely scalable;
- It provides good support for many programming languages, such as Python, R, Scala, JavaScript and C ++, and many others.
And now about the disadvantages of MXNet:
- Around him, there is no more community than TensorFlow;
- Not so popular in the scientific community.
Based on the above arguments, MXNet is a good framework for large industrial projects, but it is still quite young, so remember that you may not receive technical support in resolving your issue as quickly as you would like.
5. CNTK
Now this framework is called Microsoft Cognitive Toolkit. It is an open framework for deep learning, designed to work with really large datasets with support for Python, C ++, C # and Java.
CNTK provides effective learning models for voice recognition, handwriting and pictures, also supports CNN and RNN. It is used in Skype, Xbox and Cortana.
As always, experts have already appreciated its benefits:
- Shows good performance and scalability;
- Provides many well optimized components;
- Offers Apache Spark support;
- Efficient at using resources;
- It has easy integration with Azure Cloud;
And one single flaw:
- Limited community support.
6. Caffe and Caffe2
Caffe is a C ++ framework that has a very useful Python interface. It supports CNN and direct distribution networks, and is also good for training models (without writing extra lines of code), image processing, and improvements to existing neural networks. On the other hand, it is not so well documented and difficult to compile. There is no information about the use of Caffe by any large companies. In 2017, Facebook released Caffe 2, which became the real successor to Caffe, and was created for both mobile and large-scale development in a production environment.
On Facebook, it is known as, “A production-ready platform (...), operating on more than 1 billion smartphones, covering 8 generations of an IPhone and six generations of Android processor architectures.”
This framework is good for several reasons:
- It offers pre-trained models for building demo applications;
- It is fast, scalable and takes up little space;
- It works well with other frameworks, such as PyTorch, and will eventually be integrated into PyTorch 1.0;
- It provides optimized work with the server.
And again the main disadvantage:
- Limited community support.
Despite the fact that Caffe2 is well tested for speed and memory, it is relatively new and quite limited in the deployment of complex networks that need at least handy tools for debugging and technical support.
7. Deeplearning4j
If your main programming language is Java, then you should definitely take a closer look at DL4J. It is an open source commercial platform, written primarily for Java and Scala, which provides good support for various types of neural networks (such as CNN, RNN, RNTN, or LTSM).
This framework will be a good choice because it has great potential in the field of image recognition, natural language processing, vulnerability search and text analysis.
Pros:
- Reliable, flexible and efficient;
- It can process large amounts of data without losing speed;
- Works with Apache Hadoop and Spark, on distributed CPUs or GPUs;
- Has good documentation;
- It has a community and enterprise version.
Surprisingly, speaking of this framework, the experts do not focus on any specific flaws, as is the case in the general debate about the suitability of Java for machine learning in general. Since Java is not so often used in machine learning projects, the framework cannot rely on the growing bases of ready-made code to solve specific problems. As a result, the cost of your project can be much higher and the solution of one task can significantly slow down the overall progress of the project.
8. Chainer
Another Python deep learning framework supported by giants such as Intel, IBM, NVIDIA and AWS. He can easily use multiple GPUs.
Chainer is mainly used for voice recognition, machine translation and tonality analysis. It supports various network architectures, such as CNN, fast-forward, RNN, and has several important advantages over its competitors:
- It is much faster than any other Python framework;
- It is incredibly flexible and intuitive;
- Existing networks can be modified right at runtime;
On the other hand:
- Very complex in terms of debag;
- Around him is a relatively small community;
Since other Python-centric frameworks are much more popular, you may not get such quick help, as is the case with other frameworks, such as TensorFlow or PyTorch.
Summing up
Choosing the best framework for a deep learning project can be extremely difficult. You need to consider the following factors:
- The type of neural networks that you will develop;
- The programming language you use;
- The number of tools and additional options that you may need;
- Project's budget;
- The nature and goals pursued by the project itself.
I hope that this comparison of frameworks will shed a little light on this problem and help you choose the most suitable solution for your project.
Write comments and see you at the open day , which Alexander Nikitin will conduct on March 19.
Source: https://habr.com/ru/post/443874/
All Articles