36 materials on neural networks: books, articles and recent research
What if you want to learn more about neural networks, pattern recognition methods, computer vision and deep learning? One of the obvious options is to find courses for yourself and start actively studying the theory and solving practical problems. However, this will have to allocate a significant portion of personal time. There is another way - to turn to the “passive” source of knowledge: choose literature for yourself and immerse yourself in the subject, devoting it only half an hour per hour.
Therefore, wanting to make life easier for ourselves and our readers, we made a short selection of books, articles and texts in the direction of neural networks and in-depth training recommended for residents to read GitHub, Quora, Reddit and other platforms. It includes materials both for those who are just starting to get acquainted with neurotechnologies, and for colleagues who want to expand their knowledge in this area or just pick up “easy reading” for the evening.
/ Flickr / Giuseppe Milo / CC
')
Any proposed list, no matter how long it is, will have a main and defining feature - incompleteness. Because life does not stand still: both scientific thought and technology are developed, many problem statements are described, and the solutions obtained are disclosed in the reporting publications of conferences, in journals and collections. For those who are wondering what is happening at the moment and how the community lives, it is recommended to follow the materials of the relevant events - ICML and NIPS .
Neural Networks and Deep Learning
This is a free online book by scientist and programmer Michael Nielsen. The author reveals the topic of deep learning of neural networks and answers such questions as: “Why is the neural network difficult to train?”, “How does the back-propagation error algorithm work?”.
Book author: Tariq Rashid
Make Your Own Neural Network
The book reveals the mathematical principles underlying neural networks, and suggests writing your own neural network in Python. The network will recognize handwritten numbers. The purpose of the book is to give the reader a clear understanding of how neural systems work, to make information more accessible.
A Brief Introduction to Neural Networks
The author of the book, a specialist in data analysis and machine learning, explains the principles of neural networks in simple language. After reading, you can start working with neural systems yourself and understand someone else's code. The book is constantly improving, in updated versions, based on feedback from readers.
An Introduction to Statistical Learning
The book is an introduction to the methods of statistical training. Target audience - students and graduates of universities, including non-mathematical specialties. Everything is very accessible and with tutorials on R.
Programming Collective Intelligence
The book tells how to analyze user experience and human behavior based on the information we receive daily. The proposed algorithms are accompanied by code that can be immediately used on a website or in an application. Each chapter includes practical exercises, the task of which is to strengthen and polish the algorithms.
Neural Networks: A Systematic Introduction
The general theory of the creation of artificial neural networks. Each chapter contains examples, illustrations and a bibliography. The book is suitable for those who want to deepen their knowledge in this area, but can also serve as a good base for courses on neuro-computation.
Deep Learning: Methods and Applications
A book from Microsoft Research with basic deep learning methodologies. The authors talk about how neural networks are used in signal processing and information processing. It considers areas in which deep learning has already found active application, as well as areas where it can have a significant impact in the long term.
Deep learning tutorial
The publication of the University of Montreal (Canada). Here are collected guidelines for the most important algorithms for deep learning. The book shows how to implement them using the Theano library. As the authors note, the reader should have an idea about Python and NumPy, as well as take a course on how to handle Theano.
Pattern Recognition and Machine Learning
This is the first tutorial on pattern recognition, representing the Bayesian method. The book contains approximate inference algorithms for situations in which exact answers cannot be obtained. Information is supported by graphic models for describing the probability distribution. The book is suitable for everyone, because for its free reading does not require a thorough knowledge of the concepts of machine learning and pattern recognition.
Author of the book: Simon S Haykin
Neural Networks and Learning Machines
The book understands the concepts and principles of the neural networks and self-learning machines. To date, the third edition has been released.
Hands-On Machine Learning
With the help of illustrative examples, a minimum of theory and two production-ready frameworks for Python, the author helps to understand how intelligent systems are built. You will learn about different techniques: from simple linear regression to deep learning. Each chapter provides exercises to consolidate the acquired knowledge.
Hacker's Guide to Neural Networks
Andrej Karpati, head of AI development at Tesla, offers a glimpse into the past of neural networks and begin acquaintance with real-valued circuits technology. The author is also a lecturer at CS231 at Stanford, whose materials are closely related to this article. Slides can be found at the link . And notes - here .
Deep learning, natural language processing and data presentation
How to use deep neural networks for natural language processing (NLP). The author also tries to answer the question why neural networks work.
Deep Learning: A Guide
Java-developer Ivan Vasilyev presents key concepts and algorithms behind deep learning, using the Java programming language. The Java library for deep learning is here .
The origin of deep learning
This publication is a historical overview of the development of deep learning models. The authors begin the story with how neural networks appeared, and smoothly move to the technologies of the last decade: deep trust networks, convolutional and recurrent neural networks.
Deep Learning with Reinforcement: An Overview
The material is dedicated to the latest achievements in the field of deep learning with reinforcement (RL). First, the authors turn to the principles of deep learning and training with reinforcement, and then proceed to the problems of their real applicability: games (AlphaGo), robotics, chat bots, etc.
/ Flickr / Brandur Øssursson / PD
Neural Networks for Applied Sciences and Engineering
Overview of neural network architectures for direct data analysis. In separate chapters, the authors discuss the applicability of self-organizing maps for clustering nonlinear data, as well as the use of recurrent networks in science.
Neural networks. Full course
The book discusses the paradigms of artificial neural networks with illustrations and examples of specific tasks. The role of neural networks in solving problems of pattern recognition, control and signal processing is analyzed. The book will be useful for engineers, experts in the field of computer science, physicists, as well as for anyone interested in artificial neural networks.
Self-organizing cards
Self-organizing cards, along with their varieties, represent one of the most popular neural network architectures oriented to learning without a teacher. The book provides a detailed presentation of the mathematical apparatus and applications for self-organizing maps. Suitable for specialists in the field of neuro-modeling, as well as undergraduate and graduate students of universities.
Book author: Ian Goodfellow
Deep Learning (Adaptive Computation and Machine Learning series)
Deep Learning is the only comprehensive book in this area, ”these are the words of Ilona Mask, co-founder of Tesla and SpaceX. The text has accumulated mathematical background, discusses important concepts of linear algebra, probability theory, information theory and machine learning.
Neural Networks for Pattern Recognition
The book provides techniques for modeling probability density functions. Algorithms to minimize the error function, as well as the Bayesian method and its application are considered. In addition, the authors have collected over a hundred useful exercises under this cover.
Fast learning algorithm for deep trust networks
The authors of the article propose an algorithm capable of teaching deep trust networks (DBM) one layer at a time. Also pay attention to the video guide on deep trust networks from one of the authors - Geoffrey Hinton (GE Hinton).
Learning representations by back propagation error
It is considered the basis of the concept of learning neural networks. Historical excursion and implementation. Recommended for reading.
Learning to generate chairs, tables and cars using convolutional networks
The article shows that generative networks can find similarities between objects, having a higher productivity, compared with competitive solutions. The concept presented in this article can also be used to generate individuals.
Completion of images with deep learning in TensorFlow
The article tells how to use deep learning to complete images using DCGAN. The post is designed for a technical audience with a background in machine learning. The author has laid out all source code on GitHub .
Torch Face Generator
The author implements a generative model that transforms random "noise" into images of faces. This is done using a generative contention network (GAN).
Practical guide to training limited Boltzmann machines
Review of limited Boltzmann machines. The authors cite many recipes for debugging and improving the system: assigning weights, monitoring, choosing the number of hidden nodes.
Improving neural networks by preventing co-adaptation of trait detectors
When a large neural network learns on a small training dataset, it usually produces poor results. The authors propose a method that should solve the problem of "retraining" by teaching neurons to identify signs that help generate the correct answer.
YOLO: real-time object detection
The authors demonstrate the approach to the recognition of objects - YOLO (You Only Look Once). According to their idea, one neural network works with the image, which divides it into regions. Regions are outlined by boundary frames and “weighed” based on predicted probabilities. You can learn how to implement the “mini-version” of YOLO to work on mobile devices under iOS.
How to predict unrecognizable images
One of recent studies has shown that changing the image (imperceptible to humans) can deceive deep neural networks, forcing the latter to set the wrong marker. This work sheds light on the interesting differences between human and machine vision.
Deep Voice: real-time text-to-speech conversion
The authors represent the Deep Voice system for text-to-speech, built on deep neural networks. According to scientists, each component has its own neural network, so their system is much faster than traditional solutions. It is worth to feel.
PixelNet: Representation of pixels, and for pixels
The authors explore the principles of generalization at the pixel level, suggesting an algorithm that adequately demonstrates itself in such tasks as semantic segmentation, selection of boundaries and estimation of surface normals.
Generative models from OpenAI
This post describes four projects that adapt generative models. The authors tell what it is, where they are used and why they are important.
Learning to generate chairs using convolutional neural networks
It describes the process of training the generative convolutional neural network to generate images of objects by type and color. The network is able to interpolate rows of images and fill in “empty spaces” with missing elements.
Generative-contention network in 50 lines of code
How to train the generative consensual network (GAN)? Just take PyTorch and write 50 lines of code. Let's try at leisure.
Which book is on the tables of many employees of the Neurodata Lab and can be considered one of the favorites?
Authors of the book: Amit Konar , Aruna Chakraborty
Emotion Recognition. A Pattern Analysis Approach
Excellent material, well-structured and based on a large amount of sources and data. The book is suitable for anyone who is passionate about the problems of detecting and recognizing emotions from a technical point of view, and for those who are just looking for an exciting reading.
PS We understand that it is impossible to highlight all available materials on this topic in a single article. So if you're interested, you can devote a fraction of your attention to collections on GitHub and other platforms. Here are some of them:
Therefore, wanting to make life easier for ourselves and our readers, we made a short selection of books, articles and texts in the direction of neural networks and in-depth training recommended for residents to read GitHub, Quora, Reddit and other platforms. It includes materials both for those who are just starting to get acquainted with neurotechnologies, and for colleagues who want to expand their knowledge in this area or just pick up “easy reading” for the evening.
/ Flickr / Giuseppe Milo / CC')
Current Context
Any proposed list, no matter how long it is, will have a main and defining feature - incompleteness. Because life does not stand still: both scientific thought and technology are developed, many problem statements are described, and the solutions obtained are disclosed in the reporting publications of conferences, in journals and collections. For those who are wondering what is happening at the moment and how the community lives, it is recommended to follow the materials of the relevant events - ICML and NIPS .
And yet, where to start?
Neural Networks and Deep Learning
This is a free online book by scientist and programmer Michael Nielsen. The author reveals the topic of deep learning of neural networks and answers such questions as: “Why is the neural network difficult to train?”, “How does the back-propagation error algorithm work?”.
Book author: Tariq Rashid
Make Your Own Neural Network
The book reveals the mathematical principles underlying neural networks, and suggests writing your own neural network in Python. The network will recognize handwritten numbers. The purpose of the book is to give the reader a clear understanding of how neural systems work, to make information more accessible.
A Brief Introduction to Neural Networks
The author of the book, a specialist in data analysis and machine learning, explains the principles of neural networks in simple language. After reading, you can start working with neural systems yourself and understand someone else's code. The book is constantly improving, in updated versions, based on feedback from readers.
An Introduction to Statistical Learning
The book is an introduction to the methods of statistical training. Target audience - students and graduates of universities, including non-mathematical specialties. Everything is very accessible and with tutorials on R.
Programming Collective Intelligence
The book tells how to analyze user experience and human behavior based on the information we receive daily. The proposed algorithms are accompanied by code that can be immediately used on a website or in an application. Each chapter includes practical exercises, the task of which is to strengthen and polish the algorithms.
Neural Networks: A Systematic Introduction
The general theory of the creation of artificial neural networks. Each chapter contains examples, illustrations and a bibliography. The book is suitable for those who want to deepen their knowledge in this area, but can also serve as a good base for courses on neuro-computation.
Deep Learning: Methods and Applications
A book from Microsoft Research with basic deep learning methodologies. The authors talk about how neural networks are used in signal processing and information processing. It considers areas in which deep learning has already found active application, as well as areas where it can have a significant impact in the long term.
Deep learning tutorial
The publication of the University of Montreal (Canada). Here are collected guidelines for the most important algorithms for deep learning. The book shows how to implement them using the Theano library. As the authors note, the reader should have an idea about Python and NumPy, as well as take a course on how to handle Theano.
Pattern Recognition and Machine Learning
This is the first tutorial on pattern recognition, representing the Bayesian method. The book contains approximate inference algorithms for situations in which exact answers cannot be obtained. Information is supported by graphic models for describing the probability distribution. The book is suitable for everyone, because for its free reading does not require a thorough knowledge of the concepts of machine learning and pattern recognition.
Author of the book: Simon S Haykin
Neural Networks and Learning Machines
The book understands the concepts and principles of the neural networks and self-learning machines. To date, the third edition has been released.
Hands-On Machine Learning
With the help of illustrative examples, a minimum of theory and two production-ready frameworks for Python, the author helps to understand how intelligent systems are built. You will learn about different techniques: from simple linear regression to deep learning. Each chapter provides exercises to consolidate the acquired knowledge.
Hacker's Guide to Neural Networks
Andrej Karpati, head of AI development at Tesla, offers a glimpse into the past of neural networks and begin acquaintance with real-valued circuits technology. The author is also a lecturer at CS231 at Stanford, whose materials are closely related to this article. Slides can be found at the link . And notes - here .
Deep learning, natural language processing and data presentation
How to use deep neural networks for natural language processing (NLP). The author also tries to answer the question why neural networks work.
Deep Learning: A Guide
Java-developer Ivan Vasilyev presents key concepts and algorithms behind deep learning, using the Java programming language. The Java library for deep learning is here .
The origin of deep learning
This publication is a historical overview of the development of deep learning models. The authors begin the story with how neural networks appeared, and smoothly move to the technologies of the last decade: deep trust networks, convolutional and recurrent neural networks.
Deep Learning with Reinforcement: An Overview
The material is dedicated to the latest achievements in the field of deep learning with reinforcement (RL). First, the authors turn to the principles of deep learning and training with reinforcement, and then proceed to the problems of their real applicability: games (AlphaGo), robotics, chat bots, etc.
/ Flickr / Brandur Øssursson / PDAdvanced reading
Neural Networks for Applied Sciences and Engineering
Overview of neural network architectures for direct data analysis. In separate chapters, the authors discuss the applicability of self-organizing maps for clustering nonlinear data, as well as the use of recurrent networks in science.
Neural networks. Full course
The book discusses the paradigms of artificial neural networks with illustrations and examples of specific tasks. The role of neural networks in solving problems of pattern recognition, control and signal processing is analyzed. The book will be useful for engineers, experts in the field of computer science, physicists, as well as for anyone interested in artificial neural networks.
Self-organizing cards
Self-organizing cards, along with their varieties, represent one of the most popular neural network architectures oriented to learning without a teacher. The book provides a detailed presentation of the mathematical apparatus and applications for self-organizing maps. Suitable for specialists in the field of neuro-modeling, as well as undergraduate and graduate students of universities.
Book author: Ian Goodfellow
Deep Learning (Adaptive Computation and Machine Learning series)
Deep Learning is the only comprehensive book in this area, ”these are the words of Ilona Mask, co-founder of Tesla and SpaceX. The text has accumulated mathematical background, discusses important concepts of linear algebra, probability theory, information theory and machine learning.
Neural Networks for Pattern Recognition
The book provides techniques for modeling probability density functions. Algorithms to minimize the error function, as well as the Bayesian method and its application are considered. In addition, the authors have collected over a hundred useful exercises under this cover.
Fast learning algorithm for deep trust networks
The authors of the article propose an algorithm capable of teaching deep trust networks (DBM) one layer at a time. Also pay attention to the video guide on deep trust networks from one of the authors - Geoffrey Hinton (GE Hinton).
Learning representations by back propagation error
It is considered the basis of the concept of learning neural networks. Historical excursion and implementation. Recommended for reading.
Learning to generate chairs, tables and cars using convolutional networks
The article shows that generative networks can find similarities between objects, having a higher productivity, compared with competitive solutions. The concept presented in this article can also be used to generate individuals.
Completion of images with deep learning in TensorFlow
The article tells how to use deep learning to complete images using DCGAN. The post is designed for a technical audience with a background in machine learning. The author has laid out all source code on GitHub .
Torch Face Generator
The author implements a generative model that transforms random "noise" into images of faces. This is done using a generative contention network (GAN).
Practical guide to training limited Boltzmann machines
Review of limited Boltzmann machines. The authors cite many recipes for debugging and improving the system: assigning weights, monitoring, choosing the number of hidden nodes.
Improving neural networks by preventing co-adaptation of trait detectors
When a large neural network learns on a small training dataset, it usually produces poor results. The authors propose a method that should solve the problem of "retraining" by teaching neurons to identify signs that help generate the correct answer.
YOLO: real-time object detection
The authors demonstrate the approach to the recognition of objects - YOLO (You Only Look Once). According to their idea, one neural network works with the image, which divides it into regions. Regions are outlined by boundary frames and “weighed” based on predicted probabilities. You can learn how to implement the “mini-version” of YOLO to work on mobile devices under iOS.
How to predict unrecognizable images
One of recent studies has shown that changing the image (imperceptible to humans) can deceive deep neural networks, forcing the latter to set the wrong marker. This work sheds light on the interesting differences between human and machine vision.
Deep Voice: real-time text-to-speech conversion
The authors represent the Deep Voice system for text-to-speech, built on deep neural networks. According to scientists, each component has its own neural network, so their system is much faster than traditional solutions. It is worth to feel.
PixelNet: Representation of pixels, and for pixels
The authors explore the principles of generalization at the pixel level, suggesting an algorithm that adequately demonstrates itself in such tasks as semantic segmentation, selection of boundaries and estimation of surface normals.
Generative models from OpenAI
This post describes four projects that adapt generative models. The authors tell what it is, where they are used and why they are important.
Learning to generate chairs using convolutional neural networks
It describes the process of training the generative convolutional neural network to generate images of objects by type and color. The network is able to interpolate rows of images and fill in “empty spaces” with missing elements.
Generative-contention network in 50 lines of code
How to train the generative consensual network (GAN)? Just take PyTorch and write 50 lines of code. Let's try at leisure.
And last but not least
Which book is on the tables of many employees of the Neurodata Lab and can be considered one of the favorites?
Authors of the book: Amit Konar , Aruna Chakraborty
Emotion Recognition. A Pattern Analysis Approach
Excellent material, well-structured and based on a large amount of sources and data. The book is suitable for anyone who is passionate about the problems of detecting and recognizing emotions from a technical point of view, and for those who are just looking for an exciting reading.
PS We understand that it is impossible to highlight all available materials on this topic in a single article. So if you're interested, you can devote a fraction of your attention to collections on GitHub and other platforms. Here are some of them:
Source: https://habr.com/ru/post/336218/
All Articles