Deep Learning Course Overview

Hi, Habr! Recently, more and more achievements in the field of artificial intelligence are connected with the tools of deep learning or deep learning. We decided to figure out where you can learn the necessary skills to become an expert in this field.

1. “Deep Learning” from Google

Lecturer: Vincent Vanhoucke, Google Research Fellow, Google Brain Technology Manager

Platform: Udacity

Cost: Free

Language: English

Duration: approximately 3 months (can be held at your own pace)

Dates: free, permanent course

Link to the course: www.udacity.com/course/deep-learning--ud730

')

Program



Machine learning: basics of machine learning, logistic regression, stochastic optimization, data loading and preprocessing, parameter selection, cross-validation, quality assessment.



Deep neural networks: an introduction to neural networks, architecture of deep neural networks, selection of hyperparameters, regularization.



Convolutional neural networks: introduction and basic principles of operation, architecture of convolutional neural networks, regularization and parameter selection, image processing and other applications.



Deep neural networks for working with texts: the main approaches to word processing in machine learning, word bags, word2vec, recurrent neural networks, LSTM, regularization.



Benefits

• The course from the industry leaders Google, in which they widely use their library of deep learning TensorFlow - actively developing and one of the best tools available today.
• A large number of interactive surveys and interesting practical tasks in iPython notebook format + course project: development of a mobile application that recognizes numbers in images from a camera phone in real time.
• Active course forums, the opportunity to discuss assignments with teachers and other students.
• The program is not overloaded, it covers both the general theory of neural networks and their main types used in deep learning: convolutional and recurrent.

disadvantages

• The course is in English.
• High enough complexity: a rich advanced course, has a limited introductory part, which covers only part of the basic machine learning topics, does not include mathematics and the basics of Python.
• In practical tasks, standard training tasks of image recognition and word processing are used, practical applications of the studied models to business problems in tasks are absent and are not discussed in the course.
• In the free version, all practical tasks are checked by the students themselves on the basis of the results of the work of the models and quality metrics; receiving comments from teachers or other students is not provided.

2. Neural Networks for Machine Learning from the University of Toronto

Lecturer: Geoffrey Hinton, a professor at the University of Toronto, a well-known British computer scientist and cognitive psychologist who made a great contribution to the theory of artificial neural networks.

Platform: Coursera

Cost: Free

Language: English

Duration: 4 months

Dates: the restart starts now after a break from 2012

Link to the course: www.coursera.org/learn/neural-networks



Program



Introduction, learning perceptrons, backpropagation procedure, obtaining feature vectors for words, object recognition using neural networks, optimization: accelerating the learning process, recurrent neural networks, improving the generalizing ability of neural networks, combining neural networks, Hopfield networks and Boltzmann machines, limited Boltzmann machines, Deep Belief neural networks, deep neural networks with generative pre-training, modeling hierarchical structures, new applications of deep neural networks Tei.



Benefits

• The author of the course is an outstanding scientist, one of the pioneers in the field of artificial neural networks.
• The course covers both general principles of construction and operation of neural networks, as well as their modern development, including deep learning with applications for speech recognition and visual objects, image segmentation, modeling of natural languages ​​and human movements, etc. Neural networks are studied very deeply here, many topics in other courses are not covered.
• After reviewing each of the algorithms, practical tips are given on how to use it for solving machine learning problems, building architecture, and selecting parameters.
• Laboratory assignments and tests are available.
• This course has a very large number of students and is held not for the first time, it has extensive forums where you can ask questions and discuss assignments.

disadvantages

• High complexity: a lot of mathematics, quite complex tasks for independent work, an extensive theoretical part, a rich program.
• Tasks are performed in Octave, not in Python, modern deep learning libraries are not used.
• High entry threshold: a good knowledge of mathematics will be enough to complete the course, but before this course it is recommended to first take a course on machine learning from Andrew Ng on the same platform, otherwise there will be too much new information and the course will be hard to take.

3. “Neural networks” from the Institute of Bioinformatics

Lecturers: Anatoly Karpov, a graduate student at the Department of General Psychology at St. Petersburg State University, reads a course in mathematical statistics and data analysis in R for biologists at the Institute of Bioinformatics; Arseny Moskvichev, research engineer, graduate of the Faculty of Biology, St. Petersburg State University; Anastasia Miller, Faculty of Mathematics and Mechanics, St. Petersburg State University, JetBrains.

Platform: Stepik

Cost: Free

Language: Russian

Duration: approximately 3 months (can be held at your own pace)

Dates: the course is completed, but free access to materials is preserved

Course link: stepik.org/course/NeuronNetworks-401



Program



Fundamentals of linear algebra, perceptron and gradient descent, algorithm for back propagation of errors, monitoring network status, conclusion.



Benefits

• Course in Russian.
• Not very high complexity, it will be clear not only to graduates of technical and physical and mathematical specialties, but even to schoolchildren.
• Low threshold of entry: requires only knowledge of Python and school math.
• The lecture contains a large number of practical tasks and tests, the course also contains a number of full-fledged practical tasks in iPython Notebook format.

disadvantages

• Modern libraries and deep learning frameworks are not used.
• Introductory course: is dedicated to neural networks proper, which form the basis of deep learning, but there are no specific types of neural networks used in deep learning and their applications in the course program.
• The course is focused on the academic community, including schoolchildren and students, business applications are not discussed.

4. “Convolutional Neural Networks for Visual Recognition” by Stanford

Lecturers: Professor and Head of Computer Vision and Artificial Intelligence Laboratories at Stanford University Fei-Fei Li and her students Justin Johnson and Andrej Karpathy (PhD students)

Platform: Stanford University

Cost: Free

Language: English

Duration: approximately 3 months (can be held at your own pace)

Dates: free

Course link: cs231n.stanford.edu



Program



Introduction to computer vision, linear image classification, optimization, stochastic gradient descent, introduction to neural networks, training of neural networks, introduction to convolutional neural networks, convolutional neural networks for localizing objects, visualization of convolutional neural networks and imaging, recurrent neural networks, long short memory networks, deep learning libraries review, network learning practice: multiprocessing, GPU / CPU utilization, effective convolutions, course project.



Benefits

• High academic reputation of the university.
• Completeness of the program: the program covers both the general principles of neural networks and their training, as well as the main types of neural networks used not only in computer vision, but also in deep learning in general.
• Practical issues of learning neural networks on GPU / CPU, modern libraries of deep learning are discussed.
• For external course participants, a community has been created in reddit, where you can discuss assignments and ask questions.

disadvantages

• The course is in English.
• High complexity, focus on the academic community: this is the original course at Stanford University, aimed at its students.
• More theory than practice, specific libraries and software implementations are given less time than the theoretical part, the course is not tied to a specific library.
• High input requirements: strong knowledge of mathematics, machine learning, Python and its libraries is required.
• This is not a massive online course, but published in the open access materials of the university course.

5. “Stanford Deep Learning for Natural Language Processing”

Lecturers: Richard Socher, Professor at Stanford University and Leading Researcher at Salesforce.

Platform: Stanford University

Cost: Free

Language: English

Duration: approximately 3 months (can be held at your own pace)

Dates: free, but not all lectures are

Course link: cs224d.stanford.edu



Program



Introduction to natural language processing and deep learning, simple vector word representations: word2vec, GloVe, advanced vector word representations, neural networks for recognizing named entities, practical issues of network design, training and parameter selection, an introduction to TensorFlow, recurrent neural networks, GRU networks and LSTM and their use in machine translation, recursive neural networks, their application in parsing and tonality analysis, convolutional neural networks in text classification, speech recognition, machine translation Euodia, seq2seq model, the future of neural networks for natural language processing: dynamic memory network course project.



Benefits

• High academic reputation of the university.
• Completeness of the program: all main models and types of neural networks used in deep learning for analyzing texts, including promising and developing, are covered.
• The modern library of deep learning TensorFlow is used, the practical issues of learning networks, building their architecture and selecting parameters are covered.
• The course has a separate forum on the Piazza platform, where you can discuss tasks and ask questions.

disadvantages

• The course is in English.
• High complexity, focus on the academic community: this is the original course at Stanford University, aimed at its students.
• More theory than practice, specific libraries and software implementations are given less time than the theoretical part.
• High input requirements: strong knowledge of mathematics, machine learning, Python and its libraries is required.
• This is not a massive online course, but published in the open access materials of the university course.

6. “Data Science: Deep Learning in Python” by Lazy Programmer

Course author: data analyst, developer and big data engineer with extensive academic experience (taught mathematical analysis, statistics, machine learning, algorithms, computer graphics and physics at Columbia University, NYU, Humber College and The New School) and experience with online advertising and digital media, hiding under the name Lazy Programmer. In addition to this course leads a number of courses on deep learning .

Platform: Udemy

Cost: $ 120

Language: English

Duration: not specified, the course contains 37 lectures, combined in 7 sections

Dates: free, permanent course

Course link: www.udemy.com/data-science-deep-learning-in-python



Program



Introduction to neural networks, multi-class classification, training of neural networks, setting up hyper parameters, cross-validation, regularization, working with TensorFlow, projects: facial expression recognition and prediction of user behavior on the site.



Benefits

• A large number of tests and practical tasks, including 2 course projects.
• Low threshold of entry: requires only some information from higher mathematics and the basics of Python.
• Practice-oriented course, not overloaded with mathematics and complex theory, but containing all the necessary theoretical information.
• The course covers both the basics of the theory of neural networks and specific types of neural networks used in deep learning.
• The course is also aimed at practical applications in business, contains a course project in the field in which students will predict user behavior on the website of the online store.
• Modern libraries of data analysis and deep learning, including TensorFlow, are used.

disadvantages

• The course is in English.
• Not free, has a fairly high price for online courses.
• Limited program: the course covers only part of the main topics of deep learning, for example, word processing and recurrent neural networks are left out of this course, but they can be found in other author courses.

7. “Reinforcement Learning” by Georgia Institute of Technology

Lecturers: Charles L. Isbell, Georgia Institute of Technology, professor, expert in artificial intelligence. Michael L. Littman, Brown University, professor, specialist in the field of training with reinforcement.

Platform: Udacity

Cost: Free

Language: English

Duration: approximately 4 months (can be held at your own pace)

Dates: free, permanent course

Link to the course: www.udacity.com/course/reinforcement-learning--ud600



Program (incomplete)



Introduction to reinforcement learning, Markov decision-making processes, incl. generalized and partially observable, rewards and their sequence, policies and their search, behavioral structures, assessment of policies and agents, TD-training (temporal difference), Q-training, convergence, advanced algorithmic analysis, research strategy (intelligence), game theory and its relationship with machine learning.



Benefits

• The course is prepared by leading researchers in the field of training with reinforcement.
• Very broad coverage of topics, including related fields, such as game theory.
• A large number of interesting practical tasks.
• Despite the fact that in the field of deep learning with reinforcement, outstanding results were obtained (for example, AlphaGo from Google DeepMind) and this trend is quite promising, in other courses presented in this review, the topic of learning with reinforcement is not disclosed.

disadvantages

• The course is in English.
• High entry threshold: knowledge of machine learning and mathematics at the graduate level of a western university is required. You also need to know the Java language, there is no introduction to Java in the course.
• High complexity: a lot of mathematics, an extensive theoretical part.
• Covers only one direction rather specific direction of machine learning, applied in the field of artificial intelligence.
• Not all sections of the course are dedicated to deep learning. For deep reinforcement learning, you can additionally use Deep Reinforcement Learning, Berkley course materials.
• The course is focused on the academic community, business applications are not considered.

A list of other online courses on deep learning is in the Eclass.cc compilation .



TL: DR In the end, the advantages of most online courses are in their cost, convenience (you can learn from any place at any time), good coverage of theoretical topics. The main disadvantages are: orientation towards the academic community and insufficient attention to practical issues, most of the programs are in English.



Recently at Data Science Week we announced our full-time deep learning program.

“Deep learning” from New Professions Lab

Lecturer: Grigory Sapunov, CTO and co-founder Intento, ex-head of development of the Yandex.News service. He has been programming for more than 20 years, of which for 15 years he has been engaged in data analysis, artificial intelligence and machine learning, since 2011 he has been engaged in Deep Learning, participated in the projects RoadAR (neural network object recognition on the road), Icon8 (neural network filters), etc.

Site: Moscow, m. Krasnopresnenskaya

Cost: 60 thousand rubles.

Language: Russian

Duration: day + week laboratory work + day

Dates: from November 26 to December 3

Link to the course



Program



Day 1

An overview of the current capabilities of neural networks

Basics of Neural Networks

Principles of image classification. Convolution networks (CNN)

Case studies. Analysis of famous models: LeNet, AlexNet, ...

Practice: Caffe Library. Creating your own neural network classifier from scratch

Using convolutional networks for other tasks (style transfer, detection / segmentation, text classification, ...)

Case studies: Transferring image style. How do algorithms behind services like Prisma work?

Weekly laboratory work on the classification of images.



Day 2

Analysis of laboratory work and award winners

Basics of Recurrent Networks (RNN)

Classification of texts using neural networks. Word2vec, doc2vec. Fully connected networks, convolutional networks, recurrent networks for classification

Practice: Keras / Theano Library. Work on sentimental analysis of texts using RNN

Sequence Learning and the seq2seq paradigm. Examples of problems solved with the help of seq2seq: translation, text generation, speech recognition, ...

Case study: “Create a chat bot”. Text generation in dialogs

Multimodal training. Connection of convolutional and recurrent networks. Case study: generation of image descriptions

Master class in deep learning in business

Features of the program:

• Emphasis on practical solutions
• Business case analysis
• There are just enough theories to start becoming a specialist in deep learning.
• Laboratory work on the issued GPU-machine in the cloud
• The ability to ask questions to the teacher
• Lack of remote participation (currently)
• Theoretical issues are not given the same attention as in online courses.

You can read more about our program here, and with the HABR-DL code (you need to tell it to the manager), you will receive a 10% discount.



We welcome your comments and questions.