Why Swift can be a great event in depth learning
Hello, Habr! I present to you the translation of Max Pechyonkin’s article “Why Swift May Be the Next Big Thing in Deep Learning” , in which the author discusses the language in which deep learning will be carried out in the coming years.
If you are engaged in deep learning, then most likely you should start learning Swift
If you are engaged in programming, then most likely when you hear Swift, you are thinking about developing applications on iOS and MacOS. If you are doing deep learning, then you must have heard about Swift for Tensorflow (S4TF). Then you may ask yourself: “Why did Google create a version of TensorFlow for Swift? After all, there are already versions for Python and C ++, then why another language? ”In this post, I will try to answer this question and state the reasons why you should closely follow S4TF, as well as Swift itself. I will not try to do a detailed analysis, just try to describe the big picture with a lot of links, if you are interested, you can dig deeper.
Swift was created by Chris Lattner when he worked at Apple. Now Chris is working in Google Brain (one of the best research teams in the field of artificial intelligence in the world). The mere fact that the creator of the Swift language is now working in the laboratory, which is engaged in in-depth training, speaks about the seriousness of this project.
Some time ago, people in Google realized that even if Python is a wonderful language, there are still a lot of limitations in it that are hard to overcome. It took a new language for TensorFlow, and after long deliberation, Swift was chosen as a candidate. I will not go into details here, but here is a document that describes the disadvantages of Python, as well as which other languages were considered and how it all came together on Swift.
')
Swift for TensorFlow is not just TF for another language. This is essentially another branch (in the git sense) of the Swift language itself. This means that S4TF is not a library; This is a language with its own rules, with its built-in functions that support all the functionality required by TensorFlow. For example, in S4TF there is a very powerful system of automatic differentiation : it is one of the fundamental things in depth learning, it is necessary for calculating gradients. Compare this with Python, in which automatic differentiation is not the main component of a language. Some functions originally developed as part of S4TF were later integrated into the Swift language itself.
When I first found out that Swift is as fast as C, I was amazed. I knew that C was highly optimized and allowed me to achieve very high speed, but this was due to micro memory management, and therefore C had problems with memory security. In addition, C is not so easy to learn.
Now Swift works as fast as C in numerical calculations, it has no problems with memory security, and it is much easier to learn. Swift's LLVM compiler is very powerful and has very effective optimization, it will provide a very fast code speed.
Since Swift for machine learning is at a very early stage of its development, so many libraries for MO are not created for it. You don't have to worry about that much, since Swift has amazing Python compatibility with Python. You simply import any Python library into Swift, and it works. You can also import C and C ++ libraries into Swift (for C ++, you must make sure that the header files are written in pure C, without C ++ functions).
To summarize, if you need specific functionality, but it is not yet implemented in Swift, you can import the appropriate Python, C, or C ++ packages.
Impressive!
If you used TensorFlow once, you most likely did it with the Python package. Under the hood, the Python version of the TensorFlow library uses C code at a very low level. So when you call a function in TensorFlow, at some level you use some C code. This means that there is a limit to how deeply you can check the source code. For example, if you want to see how convolutions are implemented, you will not see the Python code, because they are implemented in C.
Swift is different. Chris Lattner called Swift “ syntactic sugar for LLVM [assembly language].” This means that, in fact, Swift works almost directly with hardware, and there are no other lines of code written in C between. This also means that Swift is very fast, as I described above. This all leads to the fact that you, as a developer, can check the code both at very high and at very low levels, without the need to use C.
Swift is only one part of the innovation in depth learning that is being prepared at Google. There is another component that is also very closely related: MLIR , which means multi-level intermediate representation (Multi-level intermediate representation). MLIR will be the unifying compiler infrastructure that allows you to write code in Swift (or any other language) and compile it on any supporting hardware. Now there are many compilers for different target equipment, but MLIR is capable of changing this, will allow not only to reuse the code, but also to write its own low-level compiler components. It will also allow researchers to use machine learning to optimize low-level algorithms.
While MLIR acts as a compiler for machine learning, we also see the possibility of using machine learning methods in compilers. This is especially important since the number of engineers who develop numerical libraries is not growing as fast as the variety of models or hardware for machine learning is growing.
Imagine being able to use in-depth training to optimize low-level data partitioning algorithms ( Halide is trying to solve a similar problem). And this is just the beginning, we still have many other creative applications using machine learning in compilers.
If you are in the field of deep learning, then you probably should start learning Swift. This will bring many advantages over Python. Google is seriously investing in Swift to become the main component of their TensorFlow ML infrastructure, and Swift is likely to become the language of in-depth learning. If you start learning and using Swift right now, this will give you the first mover advantage.
If you are engaged in deep learning, then most likely you should start learning Swift
Introduction
If you are engaged in programming, then most likely when you hear Swift, you are thinking about developing applications on iOS and MacOS. If you are doing deep learning, then you must have heard about Swift for Tensorflow (S4TF). Then you may ask yourself: “Why did Google create a version of TensorFlow for Swift? After all, there are already versions for Python and C ++, then why another language? ”In this post, I will try to answer this question and state the reasons why you should closely follow S4TF, as well as Swift itself. I will not try to do a detailed analysis, just try to describe the big picture with a lot of links, if you are interested, you can dig deeper.
Swift is supported by very strong support.
Swift was created by Chris Lattner when he worked at Apple. Now Chris is working in Google Brain (one of the best research teams in the field of artificial intelligence in the world). The mere fact that the creator of the Swift language is now working in the laboratory, which is engaged in in-depth training, speaks about the seriousness of this project.
Some time ago, people in Google realized that even if Python is a wonderful language, there are still a lot of limitations in it that are hard to overcome. It took a new language for TensorFlow, and after long deliberation, Swift was chosen as a candidate. I will not go into details here, but here is a document that describes the disadvantages of Python, as well as which other languages were considered and how it all came together on Swift.
')
Swift for TensorFlow is much more than just a library.
Swift for TensorFlow is not just TF for another language. This is essentially another branch (in the git sense) of the Swift language itself. This means that S4TF is not a library; This is a language with its own rules, with its built-in functions that support all the functionality required by TensorFlow. For example, in S4TF there is a very powerful system of automatic differentiation : it is one of the fundamental things in depth learning, it is necessary for calculating gradients. Compare this with Python, in which automatic differentiation is not the main component of a language. Some functions originally developed as part of S4TF were later integrated into the Swift language itself.
Swift fast
When I first found out that Swift is as fast as C, I was amazed. I knew that C was highly optimized and allowed me to achieve very high speed, but this was due to micro memory management, and therefore C had problems with memory security. In addition, C is not so easy to learn.
Now Swift works as fast as C in numerical calculations, it has no problems with memory security, and it is much easier to learn. Swift's LLVM compiler is very powerful and has very effective optimization, it will provide a very fast code speed.
You can use Python, C and C ++ in Swift
Since Swift for machine learning is at a very early stage of its development, so many libraries for MO are not created for it. You don't have to worry about that much, since Swift has amazing Python compatibility with Python. You simply import any Python library into Swift, and it works. You can also import C and C ++ libraries into Swift (for C ++, you must make sure that the header files are written in pure C, without C ++ functions).
To summarize, if you need specific functionality, but it is not yet implemented in Swift, you can import the appropriate Python, C, or C ++ packages.
Impressive!
Swift can go very low
If you used TensorFlow once, you most likely did it with the Python package. Under the hood, the Python version of the TensorFlow library uses C code at a very low level. So when you call a function in TensorFlow, at some level you use some C code. This means that there is a limit to how deeply you can check the source code. For example, if you want to see how convolutions are implemented, you will not see the Python code, because they are implemented in C.
Swift is different. Chris Lattner called Swift “ syntactic sugar for LLVM [assembly language].” This means that, in fact, Swift works almost directly with hardware, and there are no other lines of code written in C between. This also means that Swift is very fast, as I described above. This all leads to the fact that you, as a developer, can check the code both at very high and at very low levels, without the need to use C.
What's next?
Swift is only one part of the innovation in depth learning that is being prepared at Google. There is another component that is also very closely related: MLIR , which means multi-level intermediate representation (Multi-level intermediate representation). MLIR will be the unifying compiler infrastructure that allows you to write code in Swift (or any other language) and compile it on any supporting hardware. Now there are many compilers for different target equipment, but MLIR is capable of changing this, will allow not only to reuse the code, but also to write its own low-level compiler components. It will also allow researchers to use machine learning to optimize low-level algorithms.
While MLIR acts as a compiler for machine learning, we also see the possibility of using machine learning methods in compilers. This is especially important since the number of engineers who develop numerical libraries is not growing as fast as the variety of models or hardware for machine learning is growing.
Imagine being able to use in-depth training to optimize low-level data partitioning algorithms ( Halide is trying to solve a similar problem). And this is just the beginning, we still have many other creative applications using machine learning in compilers.
Results
If you are in the field of deep learning, then you probably should start learning Swift. This will bring many advantages over Python. Google is seriously investing in Swift to become the main component of their TensorFlow ML infrastructure, and Swift is likely to become the language of in-depth learning. If you start learning and using Swift right now, this will give you the first mover advantage.
Source: https://habr.com/ru/post/457954/
All Articles