From the epicenter of Google Cloud Next '19: CEO field notes
If you want to know where the wind is blowing from, go to the heart of the hurricane.
Approximately the feeling came from attending the Google Cloud Next '19 conference, where for three days developers, product managers, data experts and other bright minds shared the hottest news in application development and architecture, security, cost management, data analysis, hybrid cloud technologies, ML and I, off-server data processing, etc.… 30,000 people, flows in industry and technology, almost non-stop lectures. This is a stir!
Let me tell you some important news about Google Cloud in 2019-2020. and its 122 updates are not to be missed, based on the notes of Vlad Flax, CEO of OWOX , who attended the conference.
')
While the product is in a closed beta, but the developers are already waiting for an acquaintance with Anthos , which allows them to write and run hybrid applications regardless of the development environment. Locally or in the cloud, you are guaranteed the same level of security and ease of access.
Thanks to the system of containers and microservices, your virtual machines, servers and services can be launched from various points of the planet (and possibly the Moon)
In Anthos:
Although all the technical stuffing is based on open source, this luxury in combination is still worth the money: from $ 10,000 / month. See Anthos video presentation here .
Another joy for developers, which will finally answer the question of how to run applications from half a ping without hating the choice between local or server data processing.
Synthesizing a product that solves the problems of scaling, supporting and configuring servers, and the limitations of standard procedures, libraries, and local solution languages ​​(read, crossing bulldogs and rhinos) for web application developers, Cloud Run has appeared, so far beta.
The point is: to take everything useful from these two approaches. Place the HTTP container without saving the state, working in any language for which all the necessary libraries and dependencies will be automatically pulled, in a full-fledged “cloud”, where the server's capacity will be automatically tightened if the traffic load changes. Sounds hard?
But it really brings more time-to-value in the development process: no more thinking, choosing-asking the price for server-side, container development is easily managed and adapted using the Cloud Code . Created, launched, got the result.
Apigee hybrid (beta) is a new feature of the API management system that provides the ability to manage the API in hybrid cross-cloud environments: in your data center or in a public cloud environment.
It also includes many updates to expand access to the basic functions of Google Cloud. As well as the necessary security alert APIs .
This again simplifies the execution of business tasks that want to go everywhere in the shortest and safest way.
Data management in Google Cloud is based on the principle: no matter how much data you have, let it be easy to manage, transfer and access this data. Therefore, this year the following service improvements were announced:
And a lot of security and access issues got their updates. For example, improved management of the file system of the Google Filestore cloud , a unified system for coordinating access to the database is now more than convenient, and full access via AWS V4 keys will become fully available after beta testing.
For business, this is again good news, because it speeds up and simplifies access to managing your data from anywhere in the world and expands the operating and software and technical base for Google Cloud.
To start using this service you will not need to do anything superhuman soon. After all, another precept of Google - to simplify everything for a non-technical specialist, while maintaining an adequate level of security and privacy.
Depending on the type of organization, you have the option to choose one of four schemes that provides the fastest traffic to internal or external IP addresses, from local to global, to your Google Cloud, BigQuery services or third-party SaaS services via VPN or Cloud Interconnect. By the way, is 100 Gbps a lot or a little for Interconnect? Such a connection capacity will delight any giants of the data world!
In addition, there is already a Traffic Director in beta that fully covers the SLA and is designed to cope with hundreds of products and services at your disposal.
If you came up with what kind of application you would like to have, more than likely, Google has already released its beta.
More than 17 updates designed to save you from even accidental data loss, information vandalism, phishing, etc. Additional protection of individual perimeters, the definition of malicious software, crypto-mining or DDoS attacks based on proprietary computational models.
Who knows more about security flaws than a company that professionally collects data?
Data Fusion (Beta) is a fully managed, native cloud data aggregation service designed to accept and integrate data from various sources into BigQuery. All ETL processes and data structures, regardless of complexity and type, will be presented in a single interface with a high level of protection, ease of data visualization in real time and integration with other services.
And all this happiness is “for businesses of all sizes” and “without technical knowledge”.
Without high-tech expertise, thanks to standardized procedures, Data Fusion creates and maintains your data in a data lake format - the data lake, the ideal “universal soup” for analytics.
But from the point of view of the OWOX practice, data lake is still a lot of work to build an analytical structure for the daily business needs.
BigQuery Data Transfer Service now supports more than 100+ SaaS applications, automating the collection of data on a schedule without a single line of code.
But if you are looking for personal solutions, you still have to sweat.
Google makes it easy to migrate databases of any size from platforms such as Teradata, Amazon Redshift, Amazon S3, and more ...
Cloud Dataflow SQL (already in full alpha version) and Dataflow Flexible Resource Scheduling (FlexRS, in beta) allows you to configure flows using the familiar Standard SQL on a schedule for equal batch and stream processing. The system itself “decides” for you what method to apply in a particular case.
But still there is no better than collecting data with an understanding of the specifics of marketing data.
Despite the fact that Google defines many of its products as those that do not need “technical” specialists, updates, solutions and services for data specialists are the air that a corporation breathes.
The next three updates (all - beta) relate to just these toothy technologies.
The Cloud Dataproc autoscalling service will solve the pain of data experts by connecting and disconnecting Hadoop and Spark clusters on the Google Cloud Platform. Dataproc Kerberos TLC enables Hadoop protection mode on Dataproc using the Kerberos protocol support API.
And also, Presto lovers will have the opportunity to work with the Dataproc Presto job type through the native Dataproc API and write simple requests to disparate sources like Cloud Storage and Hive meta-storage.
Meet the BigQuery BI Engine beta, promising a big data analytics presentation service with controlled access to RAM data, less than a second response to queries, high stability, simplified BI architecture, and smart performance tuning.
Despite so many years of development, Google continues to cover many “simplified” models, creating niches for companies such as OWOX BI, who are not afraid to come up with more accurate solutions for the needs of the business. Also, the cost of the BigQuery BI Engine so far is a bit repulsive compared to Amazon Kinesis and ClickHouse.
A new type of table Connected sheets is specialized for the needs of data experts working with BigQuery. This is a really handy thing, which hundreds of specialists all over the world have dreamed of, because previously it would have been necessary to at least set up a separate API and tinker with requests. And now everything is beautifully inscribed in a single interface.
The full library of BigQuery ML is now available along with new types of models called using SQL queries.
The alpha version of import of TensorFlow models now allows you to call them directly from BigQuery to create a classifier and predictive models directly from the repository. You can also teach, configure, and run the DNN model directly from the standard BigQuery SQL interface. In general, much attention was paid to this technology.
Google has prepared more than 14 updates that simplify the development and customization of applications for AI using Google Cloud services. And among them there are 4 actually developed APIs that deal with a variety of tasks: from navigating through auto-marking video with object tracking and automatic annotation of streaming video to libraries of Russian and Japanese languages ​​available for NLP text processing on ML-capacities from Google.
But the most attractive is the beta version of Recommendations AI .
This future service candy promises to become a personalized adviser to your customers without limiting the scope of the business, while remaining fully GDPR-compliant.
Here are the promised achievements, possible with Recommendations AI:
+ 90% CTR from recommendations
+ 40% conversion from recommendations
+ 50% profit from recommendations
+ 5% of the total profit from the visit
Here is how it will work:
What can go wrong? The fact that if a business does not know its customers, it will not be able to give the necessary recommendation (at any step). The fact that the power of recommendations multiplied by marketing pressure can also cause a strictly reverse result.
But in general, any instrument needs to be tested before judging, and we look forward to the alpha version!
The trend “without technical specialists” and the trend “only for technical specialists” mixed together, and now it is important for business to be as close as possible to all relevant technologies in order to select those that meet business goals.
This is evidenced by the number of visitors to the C-level of the corporate world who attended the conference. Because the demands, pains, and needs of these professionals constitute the driving force of advanced business and technology around the world.
Most of all, Google Cloud invests in the simplicity of implementing complex tasks and simplifying the lives of its main users - large corporations and large companies, understanding the non-triviality and non-linearity of the challenges of modern work with data. And it is increasingly turning programming and working with data into a virtuosic non-technical art performed by the orchestra of the Google Cloud products.
Approximately the feeling came from attending the Google Cloud Next '19 conference, where for three days developers, product managers, data experts and other bright minds shared the hottest news in application development and architecture, security, cost management, data analysis, hybrid cloud technologies, ML and I, off-server data processing, etc.… 30,000 people, flows in industry and technology, almost non-stop lectures. This is a stir!
Let me tell you some important news about Google Cloud in 2019-2020. and its 122 updates are not to be missed, based on the notes of Vlad Flax, CEO of OWOX , who attended the conference.
')
Products: Anthos (formerly Cloud Services Platform)
While the product is in a closed beta, but the developers are already waiting for an acquaintance with Anthos , which allows them to write and run hybrid applications regardless of the development environment. Locally or in the cloud, you are guaranteed the same level of security and ease of access.
Thanks to the system of containers and microservices, your virtual machines, servers and services can be launched from various points of the planet (and possibly the Moon)
In Anthos:
- GKE On-Prem, the key element, is responsible for the local orchestration of container infrastructure.
- For microservice management - Istio
- For cloud capabilities - GCP
Although all the technical stuffing is based on open source, this luxury in combination is still worth the money: from $ 10,000 / month. See Anthos video presentation here .
Products: serverless, but still slightly cloudy Cloud Run (BETA)
Another joy for developers, which will finally answer the question of how to run applications from half a ping without hating the choice between local or server data processing.
Synthesizing a product that solves the problems of scaling, supporting and configuring servers, and the limitations of standard procedures, libraries, and local solution languages ​​(read, crossing bulldogs and rhinos) for web application developers, Cloud Run has appeared, so far beta.
The point is: to take everything useful from these two approaches. Place the HTTP container without saving the state, working in any language for which all the necessary libraries and dependencies will be automatically pulled, in a full-fledged “cloud”, where the server's capacity will be automatically tightened if the traffic load changes. Sounds hard?
But it really brings more time-to-value in the development process: no more thinking, choosing-asking the price for server-side, container development is easily managed and adapted using the Cloud Code . Created, launched, got the result.
API Management: Hybridization
Apigee hybrid (beta) is a new feature of the API management system that provides the ability to manage the API in hybrid cross-cloud environments: in your data center or in a public cloud environment.
It also includes many updates to expand access to the basic functions of Google Cloud. As well as the necessary security alert APIs .
This again simplifies the execution of business tasks that want to go everywhere in the shortest and safest way.
Data Management: From Anywhere, From Anything.
Data management in Google Cloud is based on the principle: no matter how much data you have, let it be easy to manage, transfer and access this data. Therefore, this year the following service improvements were announced:
- Cloud SQL under Microsoft SQL Server (in special display mode)
- CloudSQL under PostgreSQL, now version 11 is supported
- Multiregional copy of Cloud Bigtable is available.
And a lot of security and access issues got their updates. For example, improved management of the file system of the Google Filestore cloud , a unified system for coordinating access to the database is now more than convenient, and full access via AWS V4 keys will become fully available after beta testing.
For business, this is again good news, because it speeds up and simplifies access to managing your data from anywhere in the world and expands the operating and software and technical base for Google Cloud.
To start using this service you will not need to do anything superhuman soon. After all, another precept of Google - to simplify everything for a non-technical specialist, while maintaining an adequate level of security and privacy.
Networking
Depending on the type of organization, you have the option to choose one of four schemes that provides the fastest traffic to internal or external IP addresses, from local to global, to your Google Cloud, BigQuery services or third-party SaaS services via VPN or Cloud Interconnect. By the way, is 100 Gbps a lot or a little for Interconnect? Such a connection capacity will delight any giants of the data world!
In addition, there is already a Traffic Director in beta that fully covers the SLA and is designed to cope with hundreds of products and services at your disposal.
If you came up with what kind of application you would like to have, more than likely, Google has already released its beta.
Security and Identification
More than 17 updates designed to save you from even accidental data loss, information vandalism, phishing, etc. Additional protection of individual perimeters, the definition of malicious software, crypto-mining or DDoS attacks based on proprietary computational models.
Who knows more about security flaws than a company that professionally collects data?
Analytics: the most important and interesting
Data Fusion (Beta) is a fully managed, native cloud data aggregation service designed to accept and integrate data from various sources into BigQuery. All ETL processes and data structures, regardless of complexity and type, will be presented in a single interface with a high level of protection, ease of data visualization in real time and integration with other services.
And all this happiness is “for businesses of all sizes” and “without technical knowledge”.
Without high-tech expertise, thanks to standardized procedures, Data Fusion creates and maintains your data in a data lake format - the data lake, the ideal “universal soup” for analytics.
But from the point of view of the OWOX practice, data lake is still a lot of work to build an analytical structure for the daily business needs.
BigQuery Data Transfer Service now supports more than 100+ SaaS applications, automating the collection of data on a schedule without a single line of code.
But if you are looking for personal solutions, you still have to sweat.
Google makes it easy to migrate databases of any size from platforms such as Teradata, Amazon Redshift, Amazon S3, and more ...
Cloud Dataflow SQL (already in full alpha version) and Dataflow Flexible Resource Scheduling (FlexRS, in beta) allows you to configure flows using the familiar Standard SQL on a schedule for equal batch and stream processing. The system itself “decides” for you what method to apply in a particular case.
But still there is no better than collecting data with an understanding of the specifics of marketing data.
Despite the fact that Google defines many of its products as those that do not need “technical” specialists, updates, solutions and services for data specialists are the air that a corporation breathes.
The next three updates (all - beta) relate to just these toothy technologies.
The Cloud Dataproc autoscalling service will solve the pain of data experts by connecting and disconnecting Hadoop and Spark clusters on the Google Cloud Platform. Dataproc Kerberos TLC enables Hadoop protection mode on Dataproc using the Kerberos protocol support API.
And also, Presto lovers will have the opportunity to work with the Dataproc Presto job type through the native Dataproc API and write simple requests to disparate sources like Cloud Storage and Hive meta-storage.
Meet the BigQuery BI Engine beta, promising a big data analytics presentation service with controlled access to RAM data, less than a second response to queries, high stability, simplified BI architecture, and smart performance tuning.
Despite so many years of development, Google continues to cover many “simplified” models, creating niches for companies such as OWOX BI, who are not afraid to come up with more accurate solutions for the needs of the business. Also, the cost of the BigQuery BI Engine so far is a bit repulsive compared to Amazon Kinesis and ClickHouse.
A new type of table Connected sheets is specialized for the needs of data experts working with BigQuery. This is a really handy thing, which hundreds of specialists all over the world have dreamed of, because previously it would have been necessary to at least set up a separate API and tinker with requests. And now everything is beautifully inscribed in a single interface.
The full library of BigQuery ML is now available along with new types of models called using SQL queries.
The alpha version of import of TensorFlow models now allows you to call them directly from BigQuery to create a classifier and predictive models directly from the repository. You can also teach, configure, and run the DNN model directly from the standard BigQuery SQL interface. In general, much attention was paid to this technology.
AI & ML: towards the target client
Google has prepared more than 14 updates that simplify the development and customization of applications for AI using Google Cloud services. And among them there are 4 actually developed APIs that deal with a variety of tasks: from navigating through auto-marking video with object tracking and automatic annotation of streaming video to libraries of Russian and Japanese languages ​​available for NLP text processing on ML-capacities from Google.
But the most attractive is the beta version of Recommendations AI .
This future service candy promises to become a personalized adviser to your customers without limiting the scope of the business, while remaining fully GDPR-compliant.
Here are the promised achievements, possible with Recommendations AI:
+ 90% CTR from recommendations
+ 40% conversion from recommendations
+ 50% profit from recommendations
+ 5% of the total profit from the visit
Here is how it will work:
What can go wrong? The fact that if a business does not know its customers, it will not be able to give the necessary recommendation (at any step). The fact that the power of recommendations multiplied by marketing pressure can also cause a strictly reverse result.
But in general, any instrument needs to be tested before judging, and we look forward to the alpha version!
Main findings
The trend “without technical specialists” and the trend “only for technical specialists” mixed together, and now it is important for business to be as close as possible to all relevant technologies in order to select those that meet business goals.
This is evidenced by the number of visitors to the C-level of the corporate world who attended the conference. Because the demands, pains, and needs of these professionals constitute the driving force of advanced business and technology around the world.
Most of all, Google Cloud invests in the simplicity of implementing complex tasks and simplifying the lives of its main users - large corporations and large companies, understanding the non-triviality and non-linearity of the challenges of modern work with data. And it is increasingly turning programming and working with data into a virtuosic non-technical art performed by the orchestra of the Google Cloud products.
Source: https://habr.com/ru/post/452322/
All Articles