Is there a lot to start up with big data and AI?
Suppose you want to create an application that will predict, recommend, recognize images or voice, understand natural language text ... To do this, you need knowledge of machine learning, including its complex and advanced sections, such as deep learning, large training samples and complex algorithms, servers for receiving and processing data from users, means for storing and processing large data. Sounds too hard? If you do not have a Stanford University diploma, you are not ready to hire a data scientist team and deploy Hadoop clusters, but you have a good business idea, there is a simpler and less expensive solution - use the machine learning and artificial intelligence APIs.
Fortunately, many machine learning tasks are fairly standard and arise in many areas. For example, recognition of emotions from photographs. With the help of emotion recognition, you can solve many tasks: track students who are asleep or distracted in the classroom, test commercials for the emotional reaction of focus groups, identify suspicious behavior of customers in the shop to combat theft, determine how best to communicate with a client or candidate admission to work, personalize the look of applications. The task of recognizing emotions, like many others, has already been solved at a fairly good level. For this task alone, more than 20 API services are available: Emotient , Affectiva , EmoVu , Nviso , Kairos Emotion Analysis API , Microsoft Project Oxford , Noldus Face Reader API , Sightcorp FACE , SkyBiometry , Face ++ and many others. The total number of machine learning APIs and AIs is difficult to calculate, they appear like mushrooms after rain, and many of them have high recognition / prediction quality. So for any standard machine learning task, you can easily find a suitable API. Instead of reinventing the wheel every time, you can use ready-made services along with all the necessary infrastructure in your projects.
Existing machine learning and artificial learning APIs can be divided into major groups. The first is giant services that include entire API packages for solving various tasks. Such services also provide data storage and preprocessing infrastructure, constantly running servers for processing incoming data and generating responses online, have convenient and intuitive interfaces for building models and evaluating their quality, for visualizing data, model structures and the results obtained. This group includes solutions from the largest companies: Amazon Machine Learning , Microsoft Azure Machine Learning and Microsoft Cognitive Services , Google Cloud Prediction API and Google Cloud Machine Learning , IBM Watson Cloud and AlchemyAPI , BigML and others. The undoubted advantages of such services are access to high-performance computers, servers and infrastructure for working with big data, high quality documentation and the availability of ready-made libraries for working with API from various programming languages.
')
Many of these services, for example, Amazon Machine Learning, Azure Machine Learning and BigML, emphasize simple and intuitive use: the user does not need knowledge of machine learning algorithms and data preprocessing, he will receive all the necessary hints and explanations, visualizations, everything will be done for him. At the same time, these services use fairly simple binary and multiclass classification and regression algorithms, which can be easily implemented using Python libraries and R packages, which do not require large computational powers. The algorithms used are often not flexible enough, but they are still able to give very good results , for example, when applied to Kaggle tasks. These services, while building models, do not select all possible algorithms. For example, BigML focuses on decision trees, and Amazon Machine Learning uses only classifiers based on stochastic gradient descent, leaving random forests and other algorithms out of consideration. Such solutions can be categorized as “machine learning for dummies” - they are useful for developers and analysts who want to get a constantly working service and infrastructure for working with big data and, at the same time, may not have sufficient knowledge in the field of machine learning. The services of this group provide their own data warehouses; if the data is too large to load, they automatically build the necessary statistical aggregates. A number of services allows you to connect third-party services and data sources, including Hadoop and Spark, to integrate R and Python scripts (for example, Azure Machine Learning), which makes them more flexible and allows you to use them for a wider range of tasks.
The rest of the largest APIs for machine learning and AI allows for the use of models and algorithms that are rather complex for independent implementation and are based on deep learning, mainly related to image processing and natural languages. The tasks of deep learning are often quite complex, require advanced theoretical knowledge and significant computational power for teaching and using models. In this case, the use of the API is particularly appropriate even for those who are well acquainted with machine learning. The range of capabilities of such API services is very wide. For example, Microsoft Cognitive Services capabilities in video and image analysis include: image description (who or what is depicted and what is happening), categorization, type definition (photo or picture), identifying adult content, determining dominant colors, searching photos certain people, for example, celebrities, logos and objects, analyzing what is happening on the video in almost real time, recognizing text on images, generating thumbnails, identifying and tracking movements. Identifying persons, determining gender, age, posture, presence of glasses, beard and other appearance features for people in the photo and video, verification by photo (comparing the image from the base and the camera image), grouping the photo of the same people, stabilizing fuzzy videos creating thumbnail frames. The possibilities of this service in the field of speech and natural language analysis are still limited to English, but many other services support Russian, for example, wit.ai , completely free, acquired by Facebook, and its Russian competitor api.ai (understanding text and voice commands and questions on natural languages, speech to text conversion), IBM AlchemyAPI (text tonality analysis, identification of entities and keywords), Google Natural Language API (text classification, connection graphs, text extraction, tone analysis, us research, extraction of insights; supports the Russian language using the technology of machine translation Google Translate, uses deep learning and word2vec). The list of available APIs is constantly growing, some services provide very interesting tools that are not represented in other packages. For example, IBM Watson offers a Personality Insights tool that allows you to define a person’s personality traits, needs and values, intentions, and other characteristics from his tweets, social networks, or other text sources. Unfortunately, Russian is not yet supported.
In addition to the largest packaged API services discussed above, there are many APIs for machine learning and artificial intelligence aimed at solving individual problems. For example, Diffbot allows you to automatically scan the pages of sites, extract the necessary information from them: texts, images, videos, information about products, comments, etc., in a cleaned out in a structured form, and also allows you to classify pages. A wide range of technologies are used: page structure analysis, machine learning, artificial intelligence, processing of natural languages ​​and machine vision. On the site of the service more than 35 libraries are available for working with its API from various programming languages. Restb provides an API for solving a wide range of computer vision tasks, including in the field of disease diagnostics, content filtering, biological research, determining the brand, year of production, and type of vehicle trim. The clarifai service allows you to form a set of tags for images and videos. Deepomatic can classify images, and also allows you to create chat bots with image recognition and visual search functions. For example, you can take a picture in the fitting shop in the clothes you chose there, and the chat bot will automatically select other models from the range that are successfully combined with this clothes. Decisions based on Deepomatic allow you to find information about the film by its poster, information about a painting or sculpture at an exhibition from her photo taken on the camera of the phone, allow you to download music, take a picture of the album cover on the disk, etc.
Based on the machine learning API and artificial intelligence, many commercially successful products have been created. For example, Ford uses the Google Prediction API to predict the route that a driver chooses, focusing on time, speed, trip schedule, and the history of typical driver’s routes to optimize fuel consumption. The LifeLearn Sofie app uses the IBM Watson services and helps veterinarians determine the disease and treatment regimen by scanning medical literature collections based on examination results. Information is transmitted to the application in natural language through speech recognition. In veterinary medicine, the use of such applications is especially important, since patients cannot express their symptoms, veterinarians often do not have time to study medical books and journals, and sometimes they have to work with breeds and animal species that the veterinarian has not previously encountered. VRad has successfully applied deep learning through the MetaMind service API, which is now part of the Salesforce Salesforce Einstein CRM module, to improve the speed and accuracy of identifying anomalies, which are dangerous for patients' lives, primarily brain hemorrhages.
There are, of course, much more examples of successful application of machine learning API and AI in business, in application development and creation of startups, so nothing prevents you from implementing your business ideas, even if you do not own all the necessary technologies yet, and capitalize on their implementation. With a small monthly number of requests (several thousand), most of the considered services can be used for free, so you have the opportunity to try everything in practice and test it during development. Further, the payment will depend only on the actually used capacity (pay only for what you use), while the tariffs for using the services, especially at the initial stages, when your application users have a little more, will be zero or very small. No large upfront investment is required, so costs can quickly pay for themselves. So, if you have a business idea that involves the use of machine learning and artificial intelligence, the use of API is a great opportunity to implement it!
Fortunately, many machine learning tasks are fairly standard and arise in many areas. For example, recognition of emotions from photographs. With the help of emotion recognition, you can solve many tasks: track students who are asleep or distracted in the classroom, test commercials for the emotional reaction of focus groups, identify suspicious behavior of customers in the shop to combat theft, determine how best to communicate with a client or candidate admission to work, personalize the look of applications. The task of recognizing emotions, like many others, has already been solved at a fairly good level. For this task alone, more than 20 API services are available: Emotient , Affectiva , EmoVu , Nviso , Kairos Emotion Analysis API , Microsoft Project Oxford , Noldus Face Reader API , Sightcorp FACE , SkyBiometry , Face ++ and many others. The total number of machine learning APIs and AIs is difficult to calculate, they appear like mushrooms after rain, and many of them have high recognition / prediction quality. So for any standard machine learning task, you can easily find a suitable API. Instead of reinventing the wheel every time, you can use ready-made services along with all the necessary infrastructure in your projects.
Existing machine learning and artificial learning APIs can be divided into major groups. The first is giant services that include entire API packages for solving various tasks. Such services also provide data storage and preprocessing infrastructure, constantly running servers for processing incoming data and generating responses online, have convenient and intuitive interfaces for building models and evaluating their quality, for visualizing data, model structures and the results obtained. This group includes solutions from the largest companies: Amazon Machine Learning , Microsoft Azure Machine Learning and Microsoft Cognitive Services , Google Cloud Prediction API and Google Cloud Machine Learning , IBM Watson Cloud and AlchemyAPI , BigML and others. The undoubted advantages of such services are access to high-performance computers, servers and infrastructure for working with big data, high quality documentation and the availability of ready-made libraries for working with API from various programming languages.
')
Many of these services, for example, Amazon Machine Learning, Azure Machine Learning and BigML, emphasize simple and intuitive use: the user does not need knowledge of machine learning algorithms and data preprocessing, he will receive all the necessary hints and explanations, visualizations, everything will be done for him. At the same time, these services use fairly simple binary and multiclass classification and regression algorithms, which can be easily implemented using Python libraries and R packages, which do not require large computational powers. The algorithms used are often not flexible enough, but they are still able to give very good results , for example, when applied to Kaggle tasks. These services, while building models, do not select all possible algorithms. For example, BigML focuses on decision trees, and Amazon Machine Learning uses only classifiers based on stochastic gradient descent, leaving random forests and other algorithms out of consideration. Such solutions can be categorized as “machine learning for dummies” - they are useful for developers and analysts who want to get a constantly working service and infrastructure for working with big data and, at the same time, may not have sufficient knowledge in the field of machine learning. The services of this group provide their own data warehouses; if the data is too large to load, they automatically build the necessary statistical aggregates. A number of services allows you to connect third-party services and data sources, including Hadoop and Spark, to integrate R and Python scripts (for example, Azure Machine Learning), which makes them more flexible and allows you to use them for a wider range of tasks.
The rest of the largest APIs for machine learning and AI allows for the use of models and algorithms that are rather complex for independent implementation and are based on deep learning, mainly related to image processing and natural languages. The tasks of deep learning are often quite complex, require advanced theoretical knowledge and significant computational power for teaching and using models. In this case, the use of the API is particularly appropriate even for those who are well acquainted with machine learning. The range of capabilities of such API services is very wide. For example, Microsoft Cognitive Services capabilities in video and image analysis include: image description (who or what is depicted and what is happening), categorization, type definition (photo or picture), identifying adult content, determining dominant colors, searching photos certain people, for example, celebrities, logos and objects, analyzing what is happening on the video in almost real time, recognizing text on images, generating thumbnails, identifying and tracking movements. Identifying persons, determining gender, age, posture, presence of glasses, beard and other appearance features for people in the photo and video, verification by photo (comparing the image from the base and the camera image), grouping the photo of the same people, stabilizing fuzzy videos creating thumbnail frames. The possibilities of this service in the field of speech and natural language analysis are still limited to English, but many other services support Russian, for example, wit.ai , completely free, acquired by Facebook, and its Russian competitor api.ai (understanding text and voice commands and questions on natural languages, speech to text conversion), IBM AlchemyAPI (text tonality analysis, identification of entities and keywords), Google Natural Language API (text classification, connection graphs, text extraction, tone analysis, us research, extraction of insights; supports the Russian language using the technology of machine translation Google Translate, uses deep learning and word2vec). The list of available APIs is constantly growing, some services provide very interesting tools that are not represented in other packages. For example, IBM Watson offers a Personality Insights tool that allows you to define a person’s personality traits, needs and values, intentions, and other characteristics from his tweets, social networks, or other text sources. Unfortunately, Russian is not yet supported.
In addition to the largest packaged API services discussed above, there are many APIs for machine learning and artificial intelligence aimed at solving individual problems. For example, Diffbot allows you to automatically scan the pages of sites, extract the necessary information from them: texts, images, videos, information about products, comments, etc., in a cleaned out in a structured form, and also allows you to classify pages. A wide range of technologies are used: page structure analysis, machine learning, artificial intelligence, processing of natural languages ​​and machine vision. On the site of the service more than 35 libraries are available for working with its API from various programming languages. Restb provides an API for solving a wide range of computer vision tasks, including in the field of disease diagnostics, content filtering, biological research, determining the brand, year of production, and type of vehicle trim. The clarifai service allows you to form a set of tags for images and videos. Deepomatic can classify images, and also allows you to create chat bots with image recognition and visual search functions. For example, you can take a picture in the fitting shop in the clothes you chose there, and the chat bot will automatically select other models from the range that are successfully combined with this clothes. Decisions based on Deepomatic allow you to find information about the film by its poster, information about a painting or sculpture at an exhibition from her photo taken on the camera of the phone, allow you to download music, take a picture of the album cover on the disk, etc.
Based on the machine learning API and artificial intelligence, many commercially successful products have been created. For example, Ford uses the Google Prediction API to predict the route that a driver chooses, focusing on time, speed, trip schedule, and the history of typical driver’s routes to optimize fuel consumption. The LifeLearn Sofie app uses the IBM Watson services and helps veterinarians determine the disease and treatment regimen by scanning medical literature collections based on examination results. Information is transmitted to the application in natural language through speech recognition. In veterinary medicine, the use of such applications is especially important, since patients cannot express their symptoms, veterinarians often do not have time to study medical books and journals, and sometimes they have to work with breeds and animal species that the veterinarian has not previously encountered. VRad has successfully applied deep learning through the MetaMind service API, which is now part of the Salesforce Salesforce Einstein CRM module, to improve the speed and accuracy of identifying anomalies, which are dangerous for patients' lives, primarily brain hemorrhages.
There are, of course, much more examples of successful application of machine learning API and AI in business, in application development and creation of startups, so nothing prevents you from implementing your business ideas, even if you do not own all the necessary technologies yet, and capitalize on their implementation. With a small monthly number of requests (several thousand), most of the considered services can be used for free, so you have the opportunity to try everything in practice and test it during development. Further, the payment will depend only on the actually used capacity (pay only for what you use), while the tariffs for using the services, especially at the initial stages, when your application users have a little more, will be zero or very small. No large upfront investment is required, so costs can quickly pay for themselves. So, if you have a business idea that involves the use of machine learning and artificial intelligence, the use of API is a great opportunity to implement it!
Source: https://habr.com/ru/post/313162/
All Articles