Telegram bot and use of Google Cloud Vision
Hello! Recently, I already wrote an article about the integration of my bot with IBM Watson, and in this article I will consider integration with Google Cloud Vision to recognize cats and describe in more detail the insides of my bot.
A little background:
My bot worked quite successfully for a couple of months using IBM Watson recognition, but then a google cloud vision article appeared on Habré and it turned out that Google recognized images better than IBM. On the same day, I logged into the Google cloud platform developer console and began to rewrite the cat moderation unit in my bot.
With a bit of searching, I found a suitable C # example on the GoogleCloudPlatform github . I changed the authentication from the example and made it from the json file with the private key, which I took in the “service accounts” section of the console.
')
Next, I redid the image moderation (label detection). In the example on the githaba, DetectLabels works with the file, and I needed to work with the link I received from the Telegram servers in order not to store image files in myself. I save only file_id in the database, which gives a good increase in the speed of work.
Then I search for Responses if there is a label with a description of the cat, with a rating of more than 0.6, and thus, I determine if there is a cat in the image transferred to the bot:
Here is the code to work with the Telegram API to get a link to the image from file_id, I used the library on the C # telegram bot :
And when I send the image to the user using sendPhoto, I simply pass the saved file_id to the second parameter.
Thus, it turns out that when a user sends his photo of a cat for moderation (or uses thecatapi.com for this), I save only file_id in the database and later use it to get a link to the picture on Telegram servers and to send to users using sendPhoto. And image recognition using Google cloud vision works more accurately than IBM Watson
A little background:
My bot worked quite successfully for a couple of months using IBM Watson recognition, but then a google cloud vision article appeared on Habré and it turned out that Google recognized images better than IBM. On the same day, I logged into the Google cloud platform developer console and began to rewrite the cat moderation unit in my bot.
With a bit of searching, I found a suitable C # example on the GoogleCloudPlatform github . I changed the authentication from the example and made it from the json file with the private key, which I took in the “service accounts” section of the console.
')
authorization code on c #
private VisionService service; private string _JsonPath = @"C:\BOTS\fcatsbot\json.json"; private VisionService CreateAuthorizedClient(string JsonPath) { GoogleCredential credential = GoogleCredential.FromStream(new FileStream(JsonPath, FileMode.Open)); // Inject the Cloud Vision scopes if (credential.IsCreateScopedRequired) { credential = credential.CreateScoped(new[] { VisionService.Scope.CloudPlatform }); } var res = new VisionService(new BaseClientService.Initializer { HttpClientInitializer = credential, GZipEnabled = false }); return res; } Next, I redid the image moderation (label detection). In the example on the githaba, DetectLabels works with the file, and I needed to work with the link I received from the Telegram servers in order not to store image files in myself. I save only file_id in the database, which gives a good increase in the speed of work.
label detection code
private async Task<IList<AnnotateImageResponse>> DetectLabels( VisionService vision, string imageUrl) { // Convert image to Base64 encoded for JSON ASCII text based request MemoryStream ms = new MemoryStream(); using (var client = new HttpClient()) { Stream imageBytes = await client.GetStreamAsync(imageUrl); imageBytes.CopyTo(ms); } byte[] imageArray = ms.ToArray(); string imageContent = Convert.ToBase64String(imageArray); // Post label detection request to the Vision API // [START construct_request] var responses = vision.Images.Annotate( new BatchAnnotateImagesRequest() { Requests = new[] { new AnnotateImageRequest() { Features = new [] { new Feature() { Type = "LABEL_DETECTION"} }, Image = new Image() { Content = imageContent } } } }).Execute(); ms.Dispose(); return responses.Responses; } Then I search for Responses if there is a label with a description of the cat, with a rating of more than 0.6, and thus, I determine if there is a cat in the image transferred to the bot:
work with labels
foreach (var response in responses.Responses) { foreach (var label in response.LabelAnnotations) { double _score = label.Score == null ? 0 : Convert.ToDouble(label.Score.Value); var class = label.Description.Trim(); if (class .Contains("kitten") || class .Contains("cat") ) && (_score > 0.60)) { HasCatOrKittenClass = true;//moderation OK } } } Here is the code to work with the Telegram API to get a link to the image from file_id, I used the library on the C # telegram bot :
telegram getFile
var file = await MainParams.TGBot.GetFileAsync(fileid); var file_path = file.FilePath; var urlImage = "https://api.telegram.org/file/bot" + MainParams.bot_token + "/" + file_path; And when I send the image to the user using sendPhoto, I simply pass the saved file_id to the second parameter.
Thus, it turns out that when a user sends his photo of a cat for moderation (or uses thecatapi.com for this), I save only file_id in the database and later use it to get a link to the picture on Telegram servers and to send to users using sendPhoto. And image recognition using Google cloud vision works more accurately than IBM Watson
Source: https://habr.com/ru/post/399877/
All Articles