Google Cloud Storage with Java: images and other files in the clouds
In the continuation of a series of articles on Java web development on the Google App Engine / Google Cloud Endpoints platform, we consider the Google Cloud Storage file storage service.
In general, the scheme is as follows: the server on the backend generates a temporary link (address) to transfer the file to a specific container (bucket) of our storage, which is inserted into the frontend form for file transfer. The user sends an HTTP-request POST to the specified address with one or more files in the request body, files are received and placed in the repository, and HTTP-request along with the data about the placed files is received by the servlet, which after processing the information about the placed files, returns the HTTP response to the user: JSON or text / html, or in general what we wish.
Files are stored in the repository, the servlet has a key that allows access to the file, in particular, you can give the file to the user using another servlet or create a “static” link (https: //).
Access to the repository is also available via the web interface, and from the command line using the gsutil utility.
As an example, we will integrate Google Cloud Storage with an application on GAE: hello-habrahabr-api.appspot.com + hello-habrahabr-webapp.appspot.com used in the previous examples .
')
To start, go to the developer console (App Engine Developer console): appengine.google.com/dashboard?&app_id=hello-habrahabr-api (https://appengine.google.com/dashboard?&app_id={proekt ID})
Go to the Application Settings> Cloud Integration menu and click 'Create' at the bottom of the page:
We receive the message "Cloud integration tasks have started".
Notice now that the Google Developer Console exists in two versions of the “old” and “new”, functions are gradually transferred from the “old” to the “new”. We are turning on Cloud Integration from the old developer console (we should expect that this function will soon appear in the new console).
We are reloading the page, below in the Cloud Integration section, instead of the 'Create' button, we see the message “The project was created successfully. See a bit higher in the Basics section, we see a link to the connected Google Cloud Storage Bucket, by default it is given the same name as the GAE project, in my case hello-habrahabr-api.appspot.com:
We click on the link, it leads us to the address console.developers.google.com/storage/browser {Bucket name} / D in my case: console.developers.google.com/storage/browser/hello-habrahabr-api.appspot.com (of course, requires authorization) and we get into the Storage browser:
Here we can create new folders, upload and delete files, manage file access rights, including we can make the file access public and get a permanent link to the file for the web (for example, if we want to use an image or another static file for the web). site), search and filter.
Cloud Storage provides a free bucket for each application on the Google App Engine, but in this case, the Storage Browser web interface only provides the ability to view the contents of the bucket. In order to activate all the functions of the Storage browser and to create additional buckets, you must enable billing and enter your credit card details (click on “Sign for free trial” and enter credit card details, for 60 days we get a free, or rather, within $ 300, trial period ).
Required Imports:
Team to create links:
For example, if we create an API on Cloud Endpoints, then an API that returns a link to download a file will look like:
Frontend form:
The same form in the form of JSP:
The link will look like this:
But the HTTP response will be generated by the servlet located at
This servlet will look like this:
and will issue in HTTP response JSON of the following type:
That is, the downloaded file we can then give the user using any type of link
It should be noted that BlobKey does not give direct access to the file bypassing the servlet, and the servlet may or may not transfer the file depending on the criteria we set, including we can use the OAuth2.0 authentication provided by Google App Engine, use additional parameters in the request, etc.
The servlet rendering file may look like this:
As already shown above, using the Images Java API, we can
or by
get the URL providing the image file. This method of downloading a file is faster than using a servlet, but accordingly we have a link to a “static” file and cannot process the request as in the case of using a servlet.
But such a created link to a file can be, just as it was created, deleted using the .deleteServingUrl (BlobKey blobKey) method while the file itself is not deleted from the repository, and a new link can be created for it. Those. we can make such links “disposable” by creating and deleting them if necessary.
In addition to the links to images created using getServingUr (), you can add image-changing parameters in the format
s640 - generates an image of 640 pixels on the largest face
s0 - the original image size (by default, the output image is reduced!)
w100 - generates an image width of 100 pixels
h100 - generates an image height of 100 pixels
c - cuts the image to the specified size (s200, for example)
p - “smart” image cropping, trying to cut to the face (not very successful)
pp - an alternative method to do the same as in the previous paragraph (it works the same way)
cc - generates a round image
fv - flips vertically
fh - flips horizontally
r {90} - rotates by the specified number of degrees clockwise
rj - gives the image in JPG format
rp - gives an image in PNG format
rw - produces an image in WebP format
rg - gives an image in GIF format
b10 - adds a frame of the specified width (in this case 10px)
c0xffff0000 - sets the frame color (in this case, red)
d - adds the header to start the download in the browser
h - displays the HTML page containing the image
For example, from the source image:
with parameters:
with parameters:
Unlike using CSS, in this case, the image will be downloaded from the server already reduced to the desired size.
For more options, see stackoverflow.com/questions/25148567/list-of-all-the-app-engine-images-service-get-serving-url-uri-options
gsutil is written in Python (requires Python 2.6.x or 2.7.x) and works from the command line on Linux / Unix, Mac OS, and Windows (XP and above).
Installation Instructions: cloud.google.com/storage/docs/gsutil_install
After installation, run:
and log in (as described on habrahabr.ru/post/268863 )
gsutil represents access to storage containers using commands similar to the usual Linux / Unix console commands, files in the storage are indicated by the “path” of the form gs: // {container name}, for example gs: //hello-habrahabr-api.appspot.com
So, to display information about the files in the container, enter the command
for all files in all containers available to the current user (Google account):
for output by the ls command, specify the
Accordingly, you can use the commands
Thus, using the gsutil capabilities, you can organize and automate work with files in the storage.
In general, the scheme is as follows: the server on the backend generates a temporary link (address) to transfer the file to a specific container (bucket) of our storage, which is inserted into the frontend form for file transfer. The user sends an HTTP-request POST to the specified address with one or more files in the request body, files are received and placed in the repository, and HTTP-request along with the data about the placed files is received by the servlet, which after processing the information about the placed files, returns the HTTP response to the user: JSON or text / html, or in general what we wish.
Files are stored in the repository, the servlet has a key that allows access to the file, in particular, you can give the file to the user using another servlet or create a “static” link (https: //).
Access to the repository is also available via the web interface, and from the command line using the gsutil utility.
As an example, we will integrate Google Cloud Storage with an application on GAE: hello-habrahabr-api.appspot.com + hello-habrahabr-webapp.appspot.com used in the previous examples .
')
Connect Google Cloud Storage to a project on Google App Engine / Google Cloud Endpoints
To start, go to the developer console (App Engine Developer console): appengine.google.com/dashboard?&app_id=hello-habrahabr-api (https://appengine.google.com/dashboard?&app_id={proekt ID})
Go to the Application Settings> Cloud Integration menu and click 'Create' at the bottom of the page:
We receive the message "Cloud integration tasks have started".
Notice now that the Google Developer Console exists in two versions of the “old” and “new”, functions are gradually transferred from the “old” to the “new”. We are turning on Cloud Integration from the old developer console (we should expect that this function will soon appear in the new console).
We are reloading the page, below in the Cloud Integration section, instead of the 'Create' button, we see the message “The project was created successfully. See a bit higher in the Basics section, we see a link to the connected Google Cloud Storage Bucket, by default it is given the same name as the GAE project, in my case hello-habrahabr-api.appspot.com:
We click on the link, it leads us to the address console.developers.google.com/storage/browser {Bucket name} / D in my case: console.developers.google.com/storage/browser/hello-habrahabr-api.appspot.com (of course, requires authorization) and we get into the Storage browser:
Here we can create new folders, upload and delete files, manage file access rights, including we can make the file access public and get a permanent link to the file for the web (for example, if we want to use an image or another static file for the web). site), search and filter.
Cloud Storage provides a free bucket for each application on the Google App Engine, but in this case, the Storage Browser web interface only provides the ability to view the contents of the bucket. In order to activate all the functions of the Storage browser and to create additional buckets, you must enable billing and enter your credit card details (click on “Sign for free trial” and enter credit card details, for 60 days we get a free, or rather, within $ 300, trial period ).
Creating a temporary file download link
Required Imports:
import com.google.appengine.api.blobstore.BlobstoreService; import com.google.appengine.api.blobstore.BlobstoreServiceFactory; import com.google.appengine.api.blobstore.UploadOptions; Team to create links:
String uploadUrl = BlobstoreServiceFactory.getBlobstoreService().createUploadUrl( "/upload", // path to upload handler (servlet) UploadOptions.Builder.withGoogleStorageBucketName("hello-habrahabr-api.appspot.com") // bucket name ) For example, if we create an API on Cloud Endpoints, then an API that returns a link to download a file will look like:
package com.appspot.hello_habrahabr_api; import com.google.api.server.spi.config.Api; import com.google.api.server.spi.config.ApiMethod; import com.google.appengine.api.blobstore.BlobstoreServiceFactory; import com.google.appengine.api.blobstore.UploadOptions; import java.io.Serializable; @Api(name = "uploadAPI", version = "ver.1.0", scopes = {Constants.EMAIL_SCOPE}, clientIds = { Constants.WEB_CLIENT_ID, Constants.API_EXPLORER_CLIENT_ID }, description = "uploads API") public class UploadAPI { // add this class to <init-param> of <servlet-name>SystemServiceServlet</servlet-name> in web.xml /* API methods can return JavaBean Objects only, so we use this as a wrapper for String */ class StringWrapperObject implements Serializable { private String string; public StringWrapperObject() { } public StringWrapperObject(String string) { this.string = string; } public String getString() { return string; } public void setString(String string) { this.string = string; } } // end of StringWrapperObject class @ApiMethod( name = "getCsUploadURL", path = "getCsUploadURL", httpMethod = ApiMethod.HttpMethod.POST ) @SuppressWarnings("unused") public StringWrapperObject getCsUploadURL() { String uploadURL = BlobstoreServiceFactory.getBlobstoreService().createUploadUrl( "/cs-upload", // upload handler servlet address UploadOptions.Builder.withGoogleStorageBucketName( "hello-habrahabr-api.appspot.com" // Cloud Storage bucket name ) ); return new StringWrapperObject(uploadURL); } } Frontend form:
<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <script src="https://ajax.googleapis.com/ajax/libs/jquery/2.2.0/jquery.min.js"></script> <title>File Upload Form</title> </head> <body> <hr> One File: <hr> <form action="" method="post" enctype="multipart/form-data"> <input type="file" name="file"> <input type="submit" value="Upload"> </form> <hr> Multiple Files: <hr> <form action="" method="post" enctype="multipart/form-data"> <input type="file" name="files[]" multiple> <input type="submit" value="Upload"> </form> <!-- JavaScript --> <script> 'use strict'; $(document).ready(function () { var url = "https://hello-habrahabr-api.appspot.com/_ah/api/uploadAPI/ver.1.0/getCsUploadURL"; $.ajax(url, { method: "POST", // (default: 'GET') async: false, // default: true processData: true, // (default: true) (By default, data passed in to the 'data' option as an object) success: function (data) { console.log("responce received:"); console.log(data); $("form").attr("action", data.string); } }) }) </script> </body> </html> The same form in the form of JSP:
<%@ page contentType="text/html;charset=UTF-8" language="java" %> <%@ page import="com.google.appengine.api.blobstore.BlobstoreService" %> <%@ page import="com.google.appengine.api.blobstore.BlobstoreServiceFactory" %> <%@ page import="com.google.appengine.api.blobstore.UploadOptions" %> <% BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService(); %> <html> <head> <title>File Upload Form</title> </head> <body> <hr> One File: <hr> <form action="<%= blobstoreService.createUploadUrl("/cs-upload", UploadOptions.Builder.withGoogleStorageBucketName("hello-habrahabr-api.appspot.com")) %>" method="post" enctype="multipart/form-data"> <input type="file" name="file"> <input type="submit" value="Upload"> </form> <hr> Multiple Files: <hr> <form action="<%= blobstoreService.createUploadUrl("/cs-upload", UploadOptions.Builder.withGoogleStorageBucketName("hello-habrahabr-api.appspot.com")) %>" method="post" enctype="multipart/form-data"> <input type="file" name="files[]" multiple> <input type="submit" value="Upload"> </form> </body> </html> The link will look like this:
https://hello-habrahabr-api.appspot.com/_ah/upload/AMmfu6YJ0ci-sKP5k98sKaJEUjYwBFbkVfQ7iylXTJV52_gy5HIECKNG52IPUCJ9PB3wpL2wxgX82GkGkzetHt-6fuu4yzAzFFhD8HGOcD7eJ48KJLnKnb2EqbuoFEdyuc8r_FTR7779IIaf42rf_jhkl7Hju3GxWDmxh2WtmcPR2AbB9OWlQhYxBIWtZgBW9OsHO50pI21/ALBNUaYAAAAAVp2DRSZYST46t2kPmrGrrBoY3AFjyOiD/
But the HTTP response will be generated by the servlet located at
/cs-uploadHTTP response response upload (upload handler)
This servlet will look like this:
package com.appspot.hello_habrahabr_api; import com.google.appengine.api.blobstore.BlobKey; import com.google.appengine.api.blobstore.BlobstoreServiceFactory; import com.google.appengine.api.blobstore.FileInfo; import com.google.appengine.api.images.ImagesServiceFactory; import com.google.appengine.api.images.ServingUrlOptions; import com.google.appengine.repackaged.com.google.gson.Gson; import com.google.appengine.repackaged.com.google.gson.GsonBuilder; import javax.servlet.ServletException; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import java.io.IOException; import java.io.PrintWriter; import java.util.ArrayList; import java.util.logging.Logger; public class CSUploadHandlerServlet extends HttpServlet { private final static Logger LOG = Logger.getLogger(CSUploadHandlerServlet.class.getName()); private final static String HOST = "https://hello-habrahabr-api.appspot.com"; /* Object to be returned as JSON in HTTP-response (and can be stored in data base) */ class UploadedFileData { FileInfo fileInfo; String BlobKey; String fileServeServletLink; String servingUrlFromgsObjectName; String servingUrlFromGsBlobKey; } // end of uploadedFileData @Override public void doPost(HttpServletRequest req, HttpServletResponse res) throws ServletException, IOException { // Returns the FileInfo for any files that were uploaded, keyed by the upload form "name" field. // This method should only be called from within a request served by the destination of a createUploadUrl call. // https://cloud.google.com/appengine/docs/java/javadoc/com/google/appengine/api/blobstore/BlobstoreService#getFileInfos-HttpServletRequest- java.util.Map<java.lang.String, java.util.List<FileInfo>> fileInfoListsMap = BlobstoreServiceFactory.getBlobstoreService().getFileInfos(req); LOG.warning("[LOGGER]: " + new Gson().toJson(fileInfoListsMap)); ArrayList<UploadedFileData> uploadedFilesDataList = new ArrayList<>(); for (java.util.List<FileInfo> fileInfoList : fileInfoListsMap.values()) { for (FileInfo fileInfo : fileInfoList) { UploadedFileData uploadedFileData = new UploadedFileData(); uploadedFileData.fileInfo = fileInfo; LOG.warning("uploadedFileData created:" + new Gson().toJson(uploadedFileData)); BlobKey blobKey = BlobstoreServiceFactory.getBlobstoreService().createGsBlobKey(fileInfo.getGsObjectName()); uploadedFileData.BlobKey = blobKey.getKeyString(); uploadedFileData.fileServeServletLink = HOST + "/serve?blob-key=" + blobKey.getKeyString(); // Use Images Java API to create serving URL // works only for images (PNG, JPEG, GIF, TIFF, BMP, ICO, WEBP) for (com.google.appengine.api.images.Image.Format type : com.google.appengine.api.images.Image.Format.values()) { LOG.warning("com.google.appengine.api.images.Image.Format type: " + type.toString()); LOG.warning("fileInfo.getContentType(): " + fileInfo.getContentType()); if (fileInfo.getContentType().toLowerCase().contains(type.toString().toLowerCase())) { uploadedFileData.servingUrlFromgsObjectName = ImagesServiceFactory.getImagesService().getServingUrl(ServingUrlOptions.Builder.withGoogleStorageFileName(fileInfo.getGsObjectName())); // should be the same as servingUrlFromGsBlobKey uploadedFileData.servingUrlFromGsBlobKey = ImagesServiceFactory.getImagesService().getServingUrl(ServingUrlOptions.Builder.withBlobKey(blobKey)); // should be the same as servingUrlFromgsObjectName } } uploadedFilesDataList.add(uploadedFileData); } } res.setContentType("application/json"); res.setCharacterEncoding("UTF-8"); PrintWriter pw = res.getWriter(); //get the stream to write the data Gson gson = new GsonBuilder().disableHtmlEscaping().create(); pw.println( gson.toJson(uploadedFilesDataList) ); LOG.warning("uploadedFilesDataMap" + new Gson().toJson(uploadedFilesDataList)); pw.close(); //closing the stream } // doPost End } and will issue in HTTP response JSON of the following type:
Json
[{ "fileInfo": { "contentType": "image/svg+xml", "creation": "Jan 18, 2016 7:16:18 PM", "filename": "Sun_symbol.svg", "size": 188, "md5Hash": "YWZmM2UzMzk2ZDk2NTc0ZWM3NDI0YjYyMDMxZGIxYTM=", "gsObjectName": "/gs/hello-habrahabr-api.appspot.com/L2FwcGhvc3RpbmdfcHJvZC9ibG9icy9BRW5CMlVxM3RWVU9nMWVxdmxfQ1dlSk5HaXIwNHpwamE3Y2FhVzlwRjF3dk4zQnFlU3JBMVkxaERGT2NRbXpLZ0pBOW0yTjZiaXhsZjFGWElmdFRNX1p2WXF4WTJnTlF1Zy42LWZnTzcybk1xNG03X1pw" }, "BlobKey": "AMIfv968eMcYEHQml68MAl4NVtOQGKjXUWadeyP7njbaVhHXq_1xDAnRQgHeHrOv4RPLm-KdmqEHP5nb1zNuCFFszRxOVUV4Z97B9slNi7SSGWZ1qKbYcbJi2nl5Z7JX9g1xN4RclYpmLPLfh5k2jAULi6p9g84JSyh5uP3RDkNnPuXkBjxSuBTOWCVxOmpRS-xsB1YedYAgF6cRYLq0hVpm_bOY3Cbl3Ai0W-_req9jxcuPWkoguhHiZ2SSBRF9NlvgG_hCf3vouYtYS2O9DBbioeOL_p1Ck8gfvhQiiK6XpXM4S7vAYqYZCQKJ_9T4tswy075-e6NlsdtXGj9zhSxCy_GfSSBrnvbwcQUDA7lN_IYIfm0QWs-XgzBl9izizUeE46jOI-1O", "fileServeServletLink": "https://hello-habrahabr-api.appspot.com/serve?blob-key=AMIfv968eMcYEHQml68MAl4NVtOQGKjXUWadeyP7njbaVhHXq_1xDAnRQgHeHrOv4RPLm-KdmqEHP5nb1zNuCFFszRxOVUV4Z97B9slNi7SSGWZ1qKbYcbJi2nl5Z7JX9g1xN4RclYpmLPLfh5k2jAULi6p9g84JSyh5uP3RDkNnPuXkBjxSuBTOWCVxOmpRS-xsB1YedYAgF6cRYLq0hVpm_bOY3Cbl3Ai0W-_req9jxcuPWkoguhHiZ2SSBRF9NlvgG_hCf3vouYtYS2O9DBbioeOL_p1Ck8gfvhQiiK6XpXM4S7vAYqYZCQKJ_9T4tswy075-e6NlsdtXGj9zhSxCy_GfSSBrnvbwcQUDA7lN_IYIfm0QWs-XgzBl9izizUeE46jOI-1O" }, { "fileInfo": { "contentType": "image/jpeg", "creation": "Jan 18, 2016 7:16:18 PM", "filename": "world_map_04.jpg", "size": 44680, "md5Hash": "MzQyMzliZGQ4NmYyNmZiNzc3ZjAyMzBhNmM4NDVmNWE=", "gsObjectName": "/gs/hello-habrahabr-api.appspot.com/L2FwcGhvc3RpbmdfcHJvZC9ibG9icy9BRW5CMlVxM3RWVU9nMWVxdmxfQ1dlSk5HaXIwNHpwamE3Y2FhVzlwRjF3dk4zQnFlU3JBMVkxaERGT2NRbXpLZ0pBOW0yTjZiaXhsZjFGWElmdFRNX1p2WXF4WTJnTlF1Zy5Ld1pSRTQ2M3J3ZWYxa3Bm" }, "BlobKey": "AMIfv95nBw0rYnC39nCATxvyecFw0JEe64eTm-OhpsSsrR3Idv_rPbO2c6xTDx3q1xkulXfUyapqtEXdeQQur7FcppXa9rRcnlF7QnU8jur7a7AP3T5Ze_-bdD_F6F5mGP9Tteo7p7cN4UccqoYhnAyabAIsJBq3pZIwX2NlHhqcK_aelnu1tl3aszZU4cVmhLiZGE8hFvgDQyt-2oB4DurXUKTwGC56cZykCdYONO0EDETgkImiytbtk1iV_muyYZzfd7on3OS0LSmY8ls7QIcm1IMgl5jDPJANlsk_iWtnRJfEiYAC9pZ7DfhSPxTeYzko0b1TXrKuGjpG8cYMcxiA0Cmeya8y-7SCQuWQLlKCX8WFpIVOr26UguDaq8SFYplALbxgQUiB", "fileServeServletLink": "https://hello-habrahabr-api.appspot.com/serve?blob-key=AMIfv95nBw0rYnC39nCATxvyecFw0JEe64eTm-OhpsSsrR3Idv_rPbO2c6xTDx3q1xkulXfUyapqtEXdeQQur7FcppXa9rRcnlF7QnU8jur7a7AP3T5Ze_-bdD_F6F5mGP9Tteo7p7cN4UccqoYhnAyabAIsJBq3pZIwX2NlHhqcK_aelnu1tl3aszZU4cVmhLiZGE8hFvgDQyt-2oB4DurXUKTwGC56cZykCdYONO0EDETgkImiytbtk1iV_muyYZzfd7on3OS0LSmY8ls7QIcm1IMgl5jDPJANlsk_iWtnRJfEiYAC9pZ7DfhSPxTeYzko0b1TXrKuGjpG8cYMcxiA0Cmeya8y-7SCQuWQLlKCX8WFpIVOr26UguDaq8SFYplALbxgQUiB", "servingUrlFromgsObjectName": "http://lh3.googleusercontent.com/biRXwDZgclmYJa4hDUwOqBMK--VDNwj-9kZ27vzachWAGBunKVDelImXC9S5EZIhDm1T4xbyq8djFqNKkTzkSpcVkgbPO2ovxg", "servingUrlFromGsBlobKey": "http://lh3.googleusercontent.com/biRXwDZgclmYJa4hDUwOqBMK--VDNwj-9kZ27vzachWAGBunKVDelImXC9S5EZIhDm1T4xbyq8djFqNKkTzkSpcVkgbPO2ovxg" }] -xsB1YedYAgF6cRYLq0hVpm_bOY3Cbl3Ai0W-_req9jxcuPWkoguhHiZ2SSBRF9NlvgG_hCf3vouYtYS2O9DBbioeOL_p1Ck8gfvhQiiK6XpXM4S7vAYqYZCQKJ_9T4tswy075-e6NlsdtXGj9zhSxCy_GfSSBrnvbwcQUDA7lN_IYIfm0QWs-XgzBl9izizUeE46jOI-1O", [{ "fileInfo": { "contentType": "image/svg+xml", "creation": "Jan 18, 2016 7:16:18 PM", "filename": "Sun_symbol.svg", "size": 188, "md5Hash": "YWZmM2UzMzk2ZDk2NTc0ZWM3NDI0YjYyMDMxZGIxYTM=", "gsObjectName": "/gs/hello-habrahabr-api.appspot.com/L2FwcGhvc3RpbmdfcHJvZC9ibG9icy9BRW5CMlVxM3RWVU9nMWVxdmxfQ1dlSk5HaXIwNHpwamE3Y2FhVzlwRjF3dk4zQnFlU3JBMVkxaERGT2NRbXpLZ0pBOW0yTjZiaXhsZjFGWElmdFRNX1p2WXF4WTJnTlF1Zy42LWZnTzcybk1xNG03X1pw" }, "BlobKey": "AMIfv968eMcYEHQml68MAl4NVtOQGKjXUWadeyP7njbaVhHXq_1xDAnRQgHeHrOv4RPLm-KdmqEHP5nb1zNuCFFszRxOVUV4Z97B9slNi7SSGWZ1qKbYcbJi2nl5Z7JX9g1xN4RclYpmLPLfh5k2jAULi6p9g84JSyh5uP3RDkNnPuXkBjxSuBTOWCVxOmpRS-xsB1YedYAgF6cRYLq0hVpm_bOY3Cbl3Ai0W-_req9jxcuPWkoguhHiZ2SSBRF9NlvgG_hCf3vouYtYS2O9DBbioeOL_p1Ck8gfvhQiiK6XpXM4S7vAYqYZCQKJ_9T4tswy075-e6NlsdtXGj9zhSxCy_GfSSBrnvbwcQUDA7lN_IYIfm0QWs-XgzBl9izizUeE46jOI-1O", "fileServeServletLink": "https://hello-habrahabr-api.appspot.com/serve?blob-key=AMIfv968eMcYEHQml68MAl4NVtOQGKjXUWadeyP7njbaVhHXq_1xDAnRQgHeHrOv4RPLm-KdmqEHP5nb1zNuCFFszRxOVUV4Z97B9slNi7SSGWZ1qKbYcbJi2nl5Z7JX9g1xN4RclYpmLPLfh5k2jAULi6p9g84JSyh5uP3RDkNnPuXkBjxSuBTOWCVxOmpRS-xsB1YedYAgF6cRYLq0hVpm_bOY3Cbl3Ai0W-_req9jxcuPWkoguhHiZ2SSBRF9NlvgG_hCf3vouYtYS2O9DBbioeOL_p1Ck8gfvhQiiK6XpXM4S7vAYqYZCQKJ_9T4tswy075-e6NlsdtXGj9zhSxCy_GfSSBrnvbwcQUDA7lN_IYIfm0QWs-XgzBl9izizUeE46jOI-1O" }, { "fileInfo": { "contentType": "image/jpeg", "creation": "Jan 18, 2016 7:16:18 PM", "filename": "world_map_04.jpg", "size": 44680, "md5Hash": "MzQyMzliZGQ4NmYyNmZiNzc3ZjAyMzBhNmM4NDVmNWE=", "gsObjectName": "/gs/hello-habrahabr-api.appspot.com/L2FwcGhvc3RpbmdfcHJvZC9ibG9icy9BRW5CMlVxM3RWVU9nMWVxdmxfQ1dlSk5HaXIwNHpwamE3Y2FhVzlwRjF3dk4zQnFlU3JBMVkxaERGT2NRbXpLZ0pBOW0yTjZiaXhsZjFGWElmdFRNX1p2WXF4WTJnTlF1Zy5Ld1pSRTQ2M3J3ZWYxa3Bm" }, "BlobKey": "AMIfv95nBw0rYnC39nCATxvyecFw0JEe64eTm-OhpsSsrR3Idv_rPbO2c6xTDx3q1xkulXfUyapqtEXdeQQur7FcppXa9rRcnlF7QnU8jur7a7AP3T5Ze_-bdD_F6F5mGP9Tteo7p7cN4UccqoYhnAyabAIsJBq3pZIwX2NlHhqcK_aelnu1tl3aszZU4cVmhLiZGE8hFvgDQyt-2oB4DurXUKTwGC56cZykCdYONO0EDETgkImiytbtk1iV_muyYZzfd7on3OS0LSmY8ls7QIcm1IMgl5jDPJANlsk_iWtnRJfEiYAC9pZ7DfhSPxTeYzko0b1TXrKuGjpG8cYMcxiA0Cmeya8y-7SCQuWQLlKCX8WFpIVOr26UguDaq8SFYplALbxgQUiB", "fileServeServletLink": "https://hello-habrahabr-api.appspot.com/serve?blob-key=AMIfv95nBw0rYnC39nCATxvyecFw0JEe64eTm-OhpsSsrR3Idv_rPbO2c6xTDx3q1xkulXfUyapqtEXdeQQur7FcppXa9rRcnlF7QnU8jur7a7AP3T5Ze_-bdD_F6F5mGP9Tteo7p7cN4UccqoYhnAyabAIsJBq3pZIwX2NlHhqcK_aelnu1tl3aszZU4cVmhLiZGE8hFvgDQyt-2oB4DurXUKTwGC56cZykCdYONO0EDETgkImiytbtk1iV_muyYZzfd7on3OS0LSmY8ls7QIcm1IMgl5jDPJANlsk_iWtnRJfEiYAC9pZ7DfhSPxTeYzko0b1TXrKuGjpG8cYMcxiA0Cmeya8y-7SCQuWQLlKCX8WFpIVOr26UguDaq8SFYplALbxgQUiB", "servingUrlFromgsObjectName": "http://lh3.googleusercontent.com/biRXwDZgclmYJa4hDUwOqBMK--VDNwj-9kZ27vzachWAGBunKVDelImXC9S5EZIhDm1T4xbyq8djFqNKkTzkSpcVkgbPO2ovxg", "servingUrlFromGsBlobKey": "http://lh3.googleusercontent.com/biRXwDZgclmYJa4hDUwOqBMK--VDNwj-9kZ27vzachWAGBunKVDelImXC9S5EZIhDm1T4xbyq8djFqNKkTzkSpcVkgbPO2ovxg" }] -bdD_F6F5mGP9Tteo7p7cN4UccqoYhnAyabAIsJBq3pZIwX2NlHhqcK_aelnu1tl3aszZU4cVmhLiZGE8hFvgDQyt-2oB4DurXUKTwGC56cZykCdYONO0EDETgkImiytbtk1iV_muyYZzfd7on3OS0LSmY8ls7QIcm1IMgl5jDPJANlsk_iWtnRJfEiYAC9pZ7DfhSPxTeYzko0b1TXrKuGjpG8cYMcxiA0Cmeya8y-7SCQuWQLlKCX8WFpIVOr26UguDaq8SFYplALbxgQUiB", [{ "fileInfo": { "contentType": "image/svg+xml", "creation": "Jan 18, 2016 7:16:18 PM", "filename": "Sun_symbol.svg", "size": 188, "md5Hash": "YWZmM2UzMzk2ZDk2NTc0ZWM3NDI0YjYyMDMxZGIxYTM=", "gsObjectName": "/gs/hello-habrahabr-api.appspot.com/L2FwcGhvc3RpbmdfcHJvZC9ibG9icy9BRW5CMlVxM3RWVU9nMWVxdmxfQ1dlSk5HaXIwNHpwamE3Y2FhVzlwRjF3dk4zQnFlU3JBMVkxaERGT2NRbXpLZ0pBOW0yTjZiaXhsZjFGWElmdFRNX1p2WXF4WTJnTlF1Zy42LWZnTzcybk1xNG03X1pw" }, "BlobKey": "AMIfv968eMcYEHQml68MAl4NVtOQGKjXUWadeyP7njbaVhHXq_1xDAnRQgHeHrOv4RPLm-KdmqEHP5nb1zNuCFFszRxOVUV4Z97B9slNi7SSGWZ1qKbYcbJi2nl5Z7JX9g1xN4RclYpmLPLfh5k2jAULi6p9g84JSyh5uP3RDkNnPuXkBjxSuBTOWCVxOmpRS-xsB1YedYAgF6cRYLq0hVpm_bOY3Cbl3Ai0W-_req9jxcuPWkoguhHiZ2SSBRF9NlvgG_hCf3vouYtYS2O9DBbioeOL_p1Ck8gfvhQiiK6XpXM4S7vAYqYZCQKJ_9T4tswy075-e6NlsdtXGj9zhSxCy_GfSSBrnvbwcQUDA7lN_IYIfm0QWs-XgzBl9izizUeE46jOI-1O", "fileServeServletLink": "https://hello-habrahabr-api.appspot.com/serve?blob-key=AMIfv968eMcYEHQml68MAl4NVtOQGKjXUWadeyP7njbaVhHXq_1xDAnRQgHeHrOv4RPLm-KdmqEHP5nb1zNuCFFszRxOVUV4Z97B9slNi7SSGWZ1qKbYcbJi2nl5Z7JX9g1xN4RclYpmLPLfh5k2jAULi6p9g84JSyh5uP3RDkNnPuXkBjxSuBTOWCVxOmpRS-xsB1YedYAgF6cRYLq0hVpm_bOY3Cbl3Ai0W-_req9jxcuPWkoguhHiZ2SSBRF9NlvgG_hCf3vouYtYS2O9DBbioeOL_p1Ck8gfvhQiiK6XpXM4S7vAYqYZCQKJ_9T4tswy075-e6NlsdtXGj9zhSxCy_GfSSBrnvbwcQUDA7lN_IYIfm0QWs-XgzBl9izizUeE46jOI-1O" }, { "fileInfo": { "contentType": "image/jpeg", "creation": "Jan 18, 2016 7:16:18 PM", "filename": "world_map_04.jpg", "size": 44680, "md5Hash": "MzQyMzliZGQ4NmYyNmZiNzc3ZjAyMzBhNmM4NDVmNWE=", "gsObjectName": "/gs/hello-habrahabr-api.appspot.com/L2FwcGhvc3RpbmdfcHJvZC9ibG9icy9BRW5CMlVxM3RWVU9nMWVxdmxfQ1dlSk5HaXIwNHpwamE3Y2FhVzlwRjF3dk4zQnFlU3JBMVkxaERGT2NRbXpLZ0pBOW0yTjZiaXhsZjFGWElmdFRNX1p2WXF4WTJnTlF1Zy5Ld1pSRTQ2M3J3ZWYxa3Bm" }, "BlobKey": "AMIfv95nBw0rYnC39nCATxvyecFw0JEe64eTm-OhpsSsrR3Idv_rPbO2c6xTDx3q1xkulXfUyapqtEXdeQQur7FcppXa9rRcnlF7QnU8jur7a7AP3T5Ze_-bdD_F6F5mGP9Tteo7p7cN4UccqoYhnAyabAIsJBq3pZIwX2NlHhqcK_aelnu1tl3aszZU4cVmhLiZGE8hFvgDQyt-2oB4DurXUKTwGC56cZykCdYONO0EDETgkImiytbtk1iV_muyYZzfd7on3OS0LSmY8ls7QIcm1IMgl5jDPJANlsk_iWtnRJfEiYAC9pZ7DfhSPxTeYzko0b1TXrKuGjpG8cYMcxiA0Cmeya8y-7SCQuWQLlKCX8WFpIVOr26UguDaq8SFYplALbxgQUiB", "fileServeServletLink": "https://hello-habrahabr-api.appspot.com/serve?blob-key=AMIfv95nBw0rYnC39nCATxvyecFw0JEe64eTm-OhpsSsrR3Idv_rPbO2c6xTDx3q1xkulXfUyapqtEXdeQQur7FcppXa9rRcnlF7QnU8jur7a7AP3T5Ze_-bdD_F6F5mGP9Tteo7p7cN4UccqoYhnAyabAIsJBq3pZIwX2NlHhqcK_aelnu1tl3aszZU4cVmhLiZGE8hFvgDQyt-2oB4DurXUKTwGC56cZykCdYONO0EDETgkImiytbtk1iV_muyYZzfd7on3OS0LSmY8ls7QIcm1IMgl5jDPJANlsk_iWtnRJfEiYAC9pZ7DfhSPxTeYzko0b1TXrKuGjpG8cYMcxiA0Cmeya8y-7SCQuWQLlKCX8WFpIVOr26UguDaq8SFYplALbxgQUiB", "servingUrlFromgsObjectName": "http://lh3.googleusercontent.com/biRXwDZgclmYJa4hDUwOqBMK--VDNwj-9kZ27vzachWAGBunKVDelImXC9S5EZIhDm1T4xbyq8djFqNKkTzkSpcVkgbPO2ovxg", "servingUrlFromGsBlobKey": "http://lh3.googleusercontent.com/biRXwDZgclmYJa4hDUwOqBMK--VDNwj-9kZ27vzachWAGBunKVDelImXC9S5EZIhDm1T4xbyq8djFqNKkTzkSpcVkgbPO2ovxg" }] That is, the downloaded file we can then give the user using any type of link
http://lh3.googleusercontent.com/biRXwDZgclmYJa4hDUwOqBMK--VDNwj-9kZ27vzachWAGBunKVDelImXC9S5EZIhDm1T4xbyq8djFqNKkTzkSpcVkgbPO2ovxg
http://lh3.googleusercontent.com/biRXwDZgclmYJa4hDUwOqBMK--VDNwj-9kZ27vzachWAGBunKVDelImXC9S5EZIhDm1T4xbyq8djFqNKkTzkSpcVkgbPO2ovxg
- if it is an image file, or (in any case) the servlet to which reference will appear as https://hello-habrahabr-api.appspot.com/serve?blob-key=AMIfv95nBw0rYnC39nCATxvyecFw0JEe64eTm-OhpsSsrR3Idv_rPbO2c6xTDx3q1xkulXfUyapqtEXdeQQur7FcppXa9rRcnlF7QnU8jur7a7AP3T5Ze_-bdD_F6F5mGP9Tteo7p7cN4UccqoYhnAyabAIsJBq3pZIwX2NlHhqcK_aelnu1tl3aszZU4cVmhLiZGE8hFvgDQyt-2oB4DurXUKTwGC56cZykCdYONO0EDETgkImiytbtk1iV_muyYZzfd7on3OS0LSmY8ls7QIcm1IMgl5jDPJANlsk_iWtnRJfEiYAC9pZ7DfhSPxTeYzko0b1TXrKuGjpG8cYMcxiA0Cmeya8y-7SCQuWQLlKCX8WFpIVOr26UguDaq8SFYplALbxgQUiB
https://hello-habrahabr-api.appspot.com/serve?blob-key=AMIfv95nBw0rYnC39nCATxvyecFw0JEe64eTm-OhpsSsrR3Idv_rPbO2c6xTDx3q1xkulXfUyapqtEXdeQQur7FcppXa9rRcnlF7QnU8jur7a7AP3T5Ze_-bdD_F6F5mGP9Tteo7p7cN4UccqoYhnAyabAIsJBq3pZIwX2NlHhqcK_aelnu1tl3aszZU4cVmhLiZGE8hFvgDQyt-2oB4DurXUKTwGC56cZykCdYONO0EDETgkImiytbtk1iV_muyYZzfd7on3OS0LSmY8ls7QIcm1IMgl5jDPJANlsk_iWtnRJfEiYAC9pZ7DfhSPxTeYzko0b1TXrKuGjpG8cYMcxiA0Cmeya8y-7SCQuWQLlKCX8WFpIVOr26UguDaq8SFYplALbxgQUiB
where to serve
serve
- servlet path, blob-key
blob-key
- the parameter with which we can find the required file, in the most obvious variant its value will be BlobKey.It should be noted that BlobKey does not give direct access to the file bypassing the servlet, and the servlet may or may not transfer the file depending on the criteria we set, including we can use the OAuth2.0 authentication provided by Google App Engine, use additional parameters in the request, etc.
The servlet rendering file may look like this:
package com.appspot.hello_habrahabr_api; import com.google.appengine.api.blobstore.BlobKey; import com.google.appengine.api.blobstore.BlobstoreService; import com.google.appengine.api.blobstore.BlobstoreServiceFactory; import com.google.appengine.api.users.User; import com.google.appengine.api.users.UserService; import com.google.appengine.api.users.UserServiceFactory; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import java.io.IOException; import java.util.logging.Logger; public class FileServeServlet extends HttpServlet { private final static Logger LOG = Logger.getLogger(CSUploadHandlerServlet.class.getName()); private BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService(); // works both for Bloobstore and Cloud Storage public void doGet( HttpServletRequest req, HttpServletResponse res ) throws IOException { // --- check user: UserService userService = UserServiceFactory.getUserService(); User user = userService.getCurrentUser(); if (user == null) { LOG.warning("[LOGGER] User not logged in"); } else { LOG.warning("[LOGGER] user: " + user.getEmail()); } // get parameter from url constructed with: // "/serve?blob-key=" + blobKey.getKeyString() BlobKey blobKey = new BlobKey(req.getParameter("blob-key")); blobstoreService.serve(blobKey, res); } } Images Java API
As already shown above, using the Images Java API, we can
ImagesServiceFactory.getImagesService().getServingUrl( ServingUrlOptions.Builder.withGoogleStorageFileName(fileInfo.getGsObjectName()) ); or by
ImagesServiceFactory.getImagesService().getServingUrl( ServingUrlOptions.Builder.withBlobKey(blobKey) ); get the URL providing the image file. This method of downloading a file is faster than using a servlet, but accordingly we have a link to a “static” file and cannot process the request as in the case of using a servlet.
But such a created link to a file can be, just as it was created, deleted using the .deleteServingUrl (BlobKey blobKey) method while the file itself is not deleted from the repository, and a new link can be created for it. Those. we can make such links “disposable” by creating and deleting them if necessary.
In addition to the links to images created using getServingUr (), you can add image-changing parameters in the format
http://[image-url]=s200-fh-p-b10-c0xFFFF0000
http://[image-url]=s200-fh-p-b10-c0xFFFF0000
:s640 - generates an image of 640 pixels on the largest face
s0 - the original image size (by default, the output image is reduced!)
w100 - generates an image width of 100 pixels
h100 - generates an image height of 100 pixels
c - cuts the image to the specified size (s200, for example)
p - “smart” image cropping, trying to cut to the face (not very successful)
pp - an alternative method to do the same as in the previous paragraph (it works the same way)
cc - generates a round image
fv - flips vertically
fh - flips horizontally
r {90} - rotates by the specified number of degrees clockwise
rj - gives the image in JPG format
rp - gives an image in PNG format
rw - produces an image in WebP format
rg - gives an image in GIF format
b10 - adds a frame of the specified width (in this case 10px)
c0xffff0000 - sets the frame color (in this case, red)
d - adds the header to start the download in the browser
h - displays the HTML page containing the image
For example, from the source image:
with parameters:
=w100-h100-cc
=w100-h100-cc
- You can generate a round avatar;with parameters:
=s200-b3-c0xffff0000
=s200-b3-c0xffff0000
- thumbnail the size of the maximum face in 200px with a red frame width of 3px:Unlike using CSS, in this case, the image will be downloaded from the server already reduced to the desired size.
For more options, see stackoverflow.com/questions/25148567/list-of-all-the-app-engine-images-service-get-serving-url-uri-options
Access to the repository from the command line ( gsutil utility)
gsutil is written in Python (requires Python 2.6.x or 2.7.x) and works from the command line on Linux / Unix, Mac OS, and Windows (XP and above).
Installation Instructions: cloud.google.com/storage/docs/gsutil_install
After installation, run:
gcloud auth login and log in (as described on habrahabr.ru/post/268863 )
gsutil represents access to storage containers using commands similar to the usual Linux / Unix console commands, files in the storage are indicated by the “path” of the form gs: // {container name}, for example gs: //hello-habrahabr-api.appspot.com
So, to display information about the files in the container, enter the command
gsutil ls gs://hello-habrahabr-api.appspot.com for all files in all containers available to the current user (Google account):
gsutil ls gs://* for output by the ls command, specify the
-l parameter for more detailed information; for complete information about the files, specify the -L parameter:Accordingly, you can use the commands
cp , mv , rm as the file addresses in the container using gs://{ } /{ }
gs://{ } /{ }
and normal paths for files on the local OS, wildcard characters are also supported ( gs://*
gs://*
) Learn more about gsutil commands: cloud.google.com/storage/docs/gsutilThus, using the gsutil capabilities, you can organize and automate work with files in the storage.
Links
Source: https://habr.com/ru/post/275211/
All Articles