Lectures of the Technosphere. 1 semester Algorithms for intelligent processing of large amounts of data
We continue to publish the materials of our educational projects. This time we offer to get acquainted with the lectures of Tekhnosfera on the course "Algorithms for intelligent processing of large amounts of data." The goal of the course is the study by students of both classical and modern approaches to solving Data Mining problems based on machine learning algorithms. Course instructors: Nikolai Anokhin (@anokhinn), Vladimir Gulin (@vgulin) and Pavel Nesterov (@mephistopheies).
The amount of data generated daily by the services of a large Internet company is truly enormous. The goal of Data Mining’s dynamically developing discipline in recent years is to develop approaches that effectively process such data to extract business-friendly information. This information can be used when creating referral and search engines, optimizing advertising services, or making key business decisions.
Overview of data mining tasks. Standardization of the approach to solving problems of Data Mining. CRISP-DM process. Types of data. Clustering, classification, regression. The concept of a model and learning algorithm.
')
Setting the clustering problem. Distance functions. Clustering quality criteria. EM algorithm. K-means and modifications.
Hierarchical clustering. Agglomerative and Divisive algorithms. Different kinds of distances between clusters. Stepwise-optimal algorithm. The case of non-Euclidean spaces. Criteria for choosing the number of clusters: rand, silhouette. DBSCAN.
Setting the tasks of classification and regression. Decision making theory. Types of models. Examples of loss functions. Retraining Classification quality metrics. MDL. Decisive trees. Algorithm CART.
Conditional probability and Bayes theorem. Normal distribution. Naive Bayes: multinomial, binomial, gaussian. Smoothing. Generative NB model and Bayesian inference. Graphic models.
Generalized linear models. Setting the optimization problem. Examples of criteria. Gradient descent. Regularization. Maximum Likelihood method. Logistic regression.
Splitting surface with maximum clearance. The formulation of the optimization problem for the cases of linearly separable and linearly non-separable classes. The conjugate problem. Reference vectors. KKT conditions. SVM for classification and regression tasks. Kernel trick. Mercer's theorem. Examples of kernel functions.
The problem of curse dimension. Selection and selection of signs. Methods for feature extraction. Principal Component Method (PCA). Independent Component Method (ICA). Methods based on autoencoders. Methods for feature selection. Methods based on the mutual correlation of signs. Methods of maximum relevance and minimum redundancy (mRMR). Methods based on decision trees.
Combinations of classifiers. Model decision trees. A mixture of experts. Stacking Stochastic methods for building classifier ensembles. Bagging. RSM. RandomForest Algorithm.
Key ideas of boosting. Differences between boosting and bagging. Algorithm AdaBoost. Gradient boosting. Meta-algorithms over algorithmic compositions. BagBoo algorithm.
Biological neuron and neural networks. McCulloch-Pitts artificial neuron and artificial neural network. Perceptron Rosenblatt and Rumelhart. Error propagation algorithm. The moment of learning, regularization in the neural network, local learning rate, softmax layer. Various training modes.
Neural network autoencoder. Stochastic and recurrent neural networks. Boltzmann machine and Boltzmann limited machine. Gibbs distribution. The contrastive divergence algorithm for learning the BSR. Sampling data from the BSR. Binary BSR and Gaussian Binary BSR. Influence of regularization, nonlinear compression of dimensions, feature extraction. Semantic hashing.
The difficulties of learning a multilayer perceptron. Pre-training using RBM. Deep autoencoder, deep multilayered neural network. Deep belief network and deep Boltzmann machine. The device of the human eye and visual cortex. Convolution networks.
Previous issues Technopark:
Subscribe to the youtube channel Technopark and Technosphere!
The amount of data generated daily by the services of a large Internet company is truly enormous. The goal of Data Mining’s dynamically developing discipline in recent years is to develop approaches that effectively process such data to extract business-friendly information. This information can be used when creating referral and search engines, optimizing advertising services, or making key business decisions.
Lecture 1. Tasks of Data Mining (Nikolai Anokhin)
Overview of data mining tasks. Standardization of the approach to solving problems of Data Mining. CRISP-DM process. Types of data. Clustering, classification, regression. The concept of a model and learning algorithm.
')
Lecture 2. The clustering problem and the EM-algorithm (Nikolai Anokhin)
Setting the clustering problem. Distance functions. Clustering quality criteria. EM algorithm. K-means and modifications.
Lecture 3. Various clustering algorithms (Nikolai Anokhin)
Hierarchical clustering. Agglomerative and Divisive algorithms. Different kinds of distances between clusters. Stepwise-optimal algorithm. The case of non-Euclidean spaces. Criteria for choosing the number of clusters: rand, silhouette. DBSCAN.
Lecture 4. The task of classification (Nikolai Anokhin)
Setting the tasks of classification and regression. Decision making theory. Types of models. Examples of loss functions. Retraining Classification quality metrics. MDL. Decisive trees. Algorithm CART.
Lecture 5. Naive Bayes (Nikolai Anokhin)
Conditional probability and Bayes theorem. Normal distribution. Naive Bayes: multinomial, binomial, gaussian. Smoothing. Generative NB model and Bayesian inference. Graphic models.
Lecture 6. Linear models (Nikolai Anokhin)
Generalized linear models. Setting the optimization problem. Examples of criteria. Gradient descent. Regularization. Maximum Likelihood method. Logistic regression.
Lecture 7. Support vector machine (Nikolai Anokhin)
Splitting surface with maximum clearance. The formulation of the optimization problem for the cases of linearly separable and linearly non-separable classes. The conjugate problem. Reference vectors. KKT conditions. SVM for classification and regression tasks. Kernel trick. Mercer's theorem. Examples of kernel functions.
Lecture 8. Reducing the dimension of space (Vladimir Gulin)
The problem of curse dimension. Selection and selection of signs. Methods for feature extraction. Principal Component Method (PCA). Independent Component Method (ICA). Methods based on autoencoders. Methods for feature selection. Methods based on the mutual correlation of signs. Methods of maximum relevance and minimum redundancy (mRMR). Methods based on decision trees.
Lecture 9. Algorithmic compositions 1 (Vladimir Gulin)
Combinations of classifiers. Model decision trees. A mixture of experts. Stacking Stochastic methods for building classifier ensembles. Bagging. RSM. RandomForest Algorithm.
Lecture 10. Algorithmic compositions 2 (Vladimir Gulin)
Key ideas of boosting. Differences between boosting and bagging. Algorithm AdaBoost. Gradient boosting. Meta-algorithms over algorithmic compositions. BagBoo algorithm.
Lecture 11. Neural networks, training with a teacher (Pavel Nesterov)
Biological neuron and neural networks. McCulloch-Pitts artificial neuron and artificial neural network. Perceptron Rosenblatt and Rumelhart. Error propagation algorithm. The moment of learning, regularization in the neural network, local learning rate, softmax layer. Various training modes.
Lecture 12. Neural networks, learning without a teacher (Pavel Nesterov)
Neural network autoencoder. Stochastic and recurrent neural networks. Boltzmann machine and Boltzmann limited machine. Gibbs distribution. The contrastive divergence algorithm for learning the BSR. Sampling data from the BSR. Binary BSR and Gaussian Binary BSR. Influence of regularization, nonlinear compression of dimensions, feature extraction. Semantic hashing.
Lecture 13. Neural networks, deep networks (Pavel Nesterov)
The difficulties of learning a multilayer perceptron. Pre-training using RBM. Deep autoencoder, deep multilayered neural network. Deep belief network and deep Boltzmann machine. The device of the human eye and visual cortex. Convolution networks.
Previous issues Technopark:
- 1 semester Web technologies
- 1 semester Algorithms and data structures
- 1 semester C / C ++
- Semester 2 Database
- Term 3 Design of high-load systems
Subscribe to the youtube channel Technopark and Technosphere!
Source: https://habr.com/ru/post/254897/
All Articles