Introduction
Today (approx. Translator 24-07-2018), the Go team at Google released the new Open Source project
Go Cloud , a library and tools for developing in
an open cloud . With this project, we aim to ensure that developers choose the Go language to create cross-cloud applications.
This post explains why we started this project, the details of how Go Cloud works, and how to get involved and start using it.
Why cross-cloud programming? Why now?
According to our estimates, there are
more than one million developers in the world using Go. Go serves many of the most critical cloud-oriented projects, including Kubernetes, Istio and Docker. Companies such as Lyft, Capital One, Netflix and
many others depend on Go in production. Over the years, we found that developers love using Go for cloud development because of its efficiency, performance, built-in concurrency and low latency.
As part of our work to support the rapid development of Go, we interviewed teams who work with Go and understand how they use the language and how the Go ecosystem can be improved in the future. One of the main voiced themes among many of the organizations surveyed is the need for cross-cloud development. These teams (respondents) want to be able to deploy their applications in
multi-cloud and
hybrid cloud environments, and distribute the load between cloud providers without significant changes in the application code.
')
To achieve this goal, some teams are trying to untie their applications from a provider-specific API in order to write simpler and more portable (cross-cloud) code. However, short-term functional delivery requirements mean that teams have to sacrifice long-term cross-cloud requirements. As a result, most Go applications running in the cloud are tightly tied to the cloud provider that was originally selected.
Alternatively, teams can use Go Cloud, a set of open public cloud APIs, to program simpler and more portable cloud applications. Go Cloud is also a foundation for the portable cloud library ecosystem. Go Cloud allows teams to focus on the functionality during application development, while still retaining long-term flexibility to deploy the application in multi-cloud and hybrid-cloud architectures. Go Cloud applications can also be migrated to the cloud provider that best meets the needs of the application.
What is Go Cloud?
We identified common services used by cloud applications and created a common API to work between cloud providers. Today, Go Cloud can work with blob storage, MySQL databases, runtime settings (configuration), and an HTTP server configured with query logging, monitoring and health checking. Go Cloud works with the Google Cloud Platform (GCP) and Amazon Web Services (AWS). We will continue to work with partners in the cloud industry and the Go community to add support for other cloud providers in the near future.
Go Cloud aims to develop a vendor-neutral general API for most of the services used, for example, a simple and easy deployment of Go applications to another cloud. Go Cloud can also be used as the basis for developing other open source libraries for work between cloud providers. Feedback from all types of developers and at all levels of development will influence the priority of implementing / adding this or that functionality in future versions of the Go Cloud API.
How it works?
Go Cloud is based on a set of common APIs for cross-cloud programming. Let's look at an example of using blob storage. You can use the generic
* blob.Storage type to copy a file from a local disk to the cloud. Let's start by opening S3 storage using the bundled
s3blob package :
From the moment
* blob.Bucket appears in the application, you get the opportunity to create
* blob.Writer , which in turn implements the
io.Writer interface. From this point on, the application (program) can use
* blob.Writer to write data to the cloud storage, checking that
Close does not return an error.
ctx := context.Background() b, err := setupBucket(ctx) if err != nil { log.Fatalf("Failed to open bucket: %v", err) } data, err := ioutil.ReadFile("gopher.png") if err != nil { log.Fatalf("Failed to read file: %v", err) } w, err := b.NewWriter(ctx, "gopher.png", nil) if err != nil { log.Fatalf("Failed to obtain writer: %v", err) } _, err = w.Write(data) if err != nil { log.Fatalf("Failed to write to bucket: %v", err) } if err := w.Close(); err != nil { log.Fatalf("Failed to close: %v", err) }
Notice that the logic of working with the storage (bucket) does not refer to the specifics of AWS S3 operation. Go Cloud turns the replacement of cloud storage, essentially a replacement for the function used to open
* blob.Bucket . An application can easily switch to using Google Cloud Storage by creating a
blob.Bucket instance using
gcsblob.OpenBucket without changing the code that copies the file:
So far, however, different steps are required to access the storage for different cloud providers, the final type used by the application remains the same
* blob.Bucket . Thus, the application code remains isolated from the cloud-specific code. To increase compatibility with existing Go libraries, Go Cloud uses existing interfaces supplied in the standard Go library, such as
io.Writer ,
io.Reader, and
* sql.DB.The code required to access cloud services (the
setupBucket () function from the example above) follows the following pattern: higher abstractions are constructed using more basic (lower) abstractions. While you can write such code with pens, Go Cloud automates this with
Wire , a tool that generates cloud-specific code for you.
The Wire documentation explains how to install and use it, and
examples show Wire in action.
How to get involved in the project and learn more?
To begin with, we recommend the following
guide , and then we recommend to try building the application using Go Cloud yourself. If you are already using AWS or GCP, you can try migrating parts of existing applications to use Go Cloud. If you use other cloud providers or an on-premise subscription service, you can expand Go Cloud to support this by implementing driver interfaces (for example,
driver.Bucket ).
We will appreciate any and all sorts of experiences with Go Cloud.
Go Cloud development is managed on GitHub. We will be glad to make any contribution to the project, including the pool rekuest. Create an
issue to tell us what you think should be improved or which APIs the library should support first. To follow updates and news, join the project
mailing list .
The project requires contributors to sign the same Contributor License Agreement adopted in the Go project. Read
the contributor's manual for more details.
Thank you for your time spent exploring Go Cloud, we are happy to work with you to make the Go language the choice of developers for building cross-cloud (portable) applications.