Oracle Storage Cloud Services - everything your enterprise data warehouse needs
Corporate data storage on the basis of a public cloud and related services have already been sufficiently tested by customers. They meet their data storage, access and protection needs. In the performance of Oracle, the secure, resilient and simple use of this data is available at any time and in any IT-related environment associated with the Internet.
Oracle develops and delivers data warehouses that reduce storage costs, as well as provide minimal access time to data stored at any level ─ by optimizing their storage and a single interface to access them.
This article describes data storage services, their differences from other systems of this class, the structure of the solution architecture, and a practical example of using the Oracle Database Cloud Service.
')
1. Oracle Storage Cloud Service ─ Object Storage
This solution provides storage of data sets of any type, including structured and unstructured data. A copy of the data can be stored in local (on-premise) systems, moved to the cloud, or shared.
This approach provides simplification of the data center, scaling without the need to purchase new hardware, reducing capital investments and operating costs due to lower power consumption and requirements for cooling systems.
The overall infrastructure takes into account the possibility of almost infinite scalability. This eliminates the need for constant forecasting and long cycles of acquiring new capacity.
Simple management based on the OpenStack and RESTful API simplifies integration by freeing resources for other cloud projects. Pay-as-you-Go and subscription models reduce costs compared to long-term contracts.
Client data encryption, combined with additional encryption in the Oracle Information Center, provides multilevel information protection. On the client side, encryption is performed by the Oracle Storage Cloud Software Appliance and the Java SDK. All transferred data is encrypted at the SSL (Secure Sockets Layer) level. The level of access of users and administrators to the data is carried out at the container level.
Integration with the main backup and archive applications simplifies the deployment and recovery of data with minimal damage. It is possible to connect both via the public Internet and FastConnect, ─ for the most demanding workloads.
Geo-replication technology is supported ─ automatically creating a copy of data in a geographically remote information center. At the same time, data privacy rules are observed ─ stored data cannot be moved from the specified geographic areas by the administrator.
Development Standards ─ RestAPI on Open Stack Swift, Java Applications (File Transfer Manager and Java API), Shell scripting (Upload CLI).
The solution has “eleven reliability nines”, i.e., 99.999999999% reliability is ensured by supporting multiple copies of each object on different devices. Periodic integrity of data is verified by means of self-healing.
Oracle Storage Cloud Software Appliance is an easy-to-use cloud-based data storage with an NFS-compatible POSIX interface. Is the NAS gateway to the cloud.
2. Oracle Storage Cloud Service ─ Archive Storage
This is the most profitable data warehouse in the industry. They are specifically designed for data that does not require frequent access, with enterprise-level security and resilient scalability.
Typical applications ─ archived in the cloud for large data sets and their long-term storage, as well as heterogeneous multimedia content, research results and digital copies of the cultural heritage.
The main advantages of the solution include the maximum simplification of data center operations, low capital investment and operating costs due to lower power consumption and requirements for cooling systems. Pay-as-You-Go payment and subscription models are also practiced, and the total storage fee for archives is the lowest among its peers ($ 12 / TB per year).
Client data is stored in encrypted form, SSL traffic is encrypted, role-based access is carried out at the container level. Access, development standards, connectivity, reliability, geo-replication are the same as for Object Storage.
3. Oracle Database Backup Service
The data protection service is designed for the unique needs of Oracle Database customers, with direct integration with RMAN (Oracle Recovery Manager). Thus, you can take advantage of cloud-based data protection in IT processes.
Backups are encrypted directly in the data source, securely transmitted and stored in the cloud. Data is always stored in specified geographic regions. Built-in redundancy is used ─ mirroring is carried out in three ways, which ensures continuous data availability. Backups are easily encrypted from RMAN. Data is constantly monitored, which prevents their degradation.
Backups are compressed before moving to the cloud. Data storage capacity is changed on demand by a simple click on the button. Data management includes role-based access control and automatic deletion of data by the specified criteria (after storage and so on).
4. Oracle Storage Cloud Software Appliance
The solution provides role-based encryption, ensures data integrity by checking checksums, performs automatic conversion between files and objects, has performance at the local NAS level due to data caching and docker technology.
The solution has a familiar, well-understood interface. It provides connection of applications with cloud-based data storage based on the NFS protocol, as well as compatibility with POSIX and Unix / Linux NFS clients. Implemented automatic data conversion, ─ the interface allows you to work with the data as files, while they are stored in the cloud as objects.
Information security tools are well developed. Data is encrypted before transmission. Client-side encryption is possible, including granular (selective). A layered architecture with symmetric and asymmetric keys is used to maximize information security. The user can bring his own keys, or generate them immediately before encryption.
Data availability is ensured by regularly backing up metadata to the cloud, and performance at the local NAS level — buffer caching. The cache can be configured according to workloads. The algorithm Least Recently Used (LRU) caches actual data and deletes unnecessary data.
The management uses the N-Way Management technology (N-Way ─ telecommunication protocol for managing Ethernet network devices and user data). It is possible to work from the Admin UI based on the browser, or the command line interface (Command Line Interface). Scripts can be used for automation. REST APIs that provide multiple deployments from a single central location.
5. Oracle Public Cloud Data Transfer Services
Services provide fast transfer of the initial data set to the cloud and the formation of a data warehouse in the public Oracle cloud. This is the fastest way to start working with data in the cloud. The Storage Appliance Import service securely transfers large data sets (historical archives, data lakes, large databases of inherited data). Up to 400 TB of data can be transferred at one time.
There is no need to pay for building a faster network for a one-time “wholesale” data transfer. The transmission is secure because traffic is encrypted with the AES-256 standard. A simple standard NFSv3 interface is used to load object or archive data. At the same time a multi-row structure is being formed. It is also possible to copy data inside the cloud to create a replica of the working base.
Oracle Storage Cloud Service and Traditional Solutions
Traditional storage solutions tend to have certain scalability, performance, and management issues. Oracle Storage Cloud Service helps to overcome them.
1. In a system with a directly attached storage device (such as a regular hard disk in a desktop or laptop), data storage, retrieval and sequencing through the file system is controlled by the OS.
Such a device provides low latency and fairly reliable long-term data storage. However, if you need a large capacity drive, it is distributed between individual disk devices, which makes scaling difficult and reduces performance.
2. In network-attached storage devices (Network-Attached Storage, NAS) storage hardware is physically separated from the servers on which applications run. Storage devices are available as network drives. Data storage, retrieval and ordering is managed by a network file system (Network File System, NFS). Applications running on different servers share a NAS for data storage. Managing storage resources on a local network is centralized.
As with directly attached storage devices, NAS applications also rely on the primary OS, ─ but also on the network file system. But this architecture can be implemented only in a limited geographic area, and has limited scalability.
3. Block storage is used in applications such as OLTP databases with high input / output operations (Input / Output Operations per Second, IOPS). This allows you to efficiently save and restore data bypassing the OS, directly interacting with virtual block devices.
Data fragments are stored in blocks, each with its own address, but without other metadata. Where data is stored, decide applications. They recover data at the corresponding block addresses.
The block storage principle optimizes storage systems for IOPS and provides POSIX compatible file systems for the Oracle Compute Cloud Service. However, this approach has limitations in terms of scalability and does not support the granulation of metadata.
4. Object data storage provides the optimal combination of performance, scalability and manageability when storing large amounts of unstructured data. Multiple storage nodes (nodes) form a single, common, horizontally scalable pool in which data is stored as objects (“data drops in a data pool”).
All object-stored data is associated with metadata and has a unique identifier. Thus, you can assign custom metadata to containers and objects, which greatly facilitates the search, analysis and management of data. Applications use object identifiers to directly access data through REST API calls.
Object storage is easy to use, does its job well, and can virtually virtually limitlessly scale.
Summarizing the above, it should be noted that Oracle Storage Cloud Service provides an inexpensive, reliable, secure, and scalable storage solution. This allows you to store unstructured data and access it at any time and from anywhere.
This approach is ideal for backing up and archiving data, sharing files, and also for storing large amounts of unstructured data, such as logs, data received from sensors, and virtual machine images.
Architecture Overview
The Oracle Storage Cloud Service architecture is very accessible and redundant. It is well suited for external access, including user applications, Java SDK and REST clients.
When objects are stored in the Oracle Storage Cloud Service, data is replicated across multiple storage nodes in the datacenter. This strategy ensures that the stored object data can withstand a hardware failure.
Typical Oracle Cloud Storage Service Architecture
Oracle develops and delivers data warehouses that reduce storage costs, as well as provide minimal access time to data stored at any level ─ by optimizing their storage and a single interface to access them.
This article describes data storage services, their differences from other systems of this class, the structure of the solution architecture, and a practical example of using the Oracle Database Cloud Service.
')
1. Oracle Storage Cloud Service ─ Object Storage
This solution provides storage of data sets of any type, including structured and unstructured data. A copy of the data can be stored in local (on-premise) systems, moved to the cloud, or shared.
This approach provides simplification of the data center, scaling without the need to purchase new hardware, reducing capital investments and operating costs due to lower power consumption and requirements for cooling systems.
The overall infrastructure takes into account the possibility of almost infinite scalability. This eliminates the need for constant forecasting and long cycles of acquiring new capacity.
Simple management based on the OpenStack and RESTful API simplifies integration by freeing resources for other cloud projects. Pay-as-you-Go and subscription models reduce costs compared to long-term contracts.
Client data encryption, combined with additional encryption in the Oracle Information Center, provides multilevel information protection. On the client side, encryption is performed by the Oracle Storage Cloud Software Appliance and the Java SDK. All transferred data is encrypted at the SSL (Secure Sockets Layer) level. The level of access of users and administrators to the data is carried out at the container level.
Integration with the main backup and archive applications simplifies the deployment and recovery of data with minimal damage. It is possible to connect both via the public Internet and FastConnect, ─ for the most demanding workloads.
Geo-replication technology is supported ─ automatically creating a copy of data in a geographically remote information center. At the same time, data privacy rules are observed ─ stored data cannot be moved from the specified geographic areas by the administrator.
Development Standards ─ RestAPI on Open Stack Swift, Java Applications (File Transfer Manager and Java API), Shell scripting (Upload CLI).
The solution has “eleven reliability nines”, i.e., 99.999999999% reliability is ensured by supporting multiple copies of each object on different devices. Periodic integrity of data is verified by means of self-healing.
Oracle Storage Cloud Software Appliance is an easy-to-use cloud-based data storage with an NFS-compatible POSIX interface. Is the NAS gateway to the cloud.
2. Oracle Storage Cloud Service ─ Archive Storage
This is the most profitable data warehouse in the industry. They are specifically designed for data that does not require frequent access, with enterprise-level security and resilient scalability.
Typical applications ─ archived in the cloud for large data sets and their long-term storage, as well as heterogeneous multimedia content, research results and digital copies of the cultural heritage.
The main advantages of the solution include the maximum simplification of data center operations, low capital investment and operating costs due to lower power consumption and requirements for cooling systems. Pay-as-You-Go payment and subscription models are also practiced, and the total storage fee for archives is the lowest among its peers ($ 12 / TB per year).
Client data is stored in encrypted form, SSL traffic is encrypted, role-based access is carried out at the container level. Access, development standards, connectivity, reliability, geo-replication are the same as for Object Storage.
3. Oracle Database Backup Service
The data protection service is designed for the unique needs of Oracle Database customers, with direct integration with RMAN (Oracle Recovery Manager). Thus, you can take advantage of cloud-based data protection in IT processes.
Backups are encrypted directly in the data source, securely transmitted and stored in the cloud. Data is always stored in specified geographic regions. Built-in redundancy is used ─ mirroring is carried out in three ways, which ensures continuous data availability. Backups are easily encrypted from RMAN. Data is constantly monitored, which prevents their degradation.
Backups are compressed before moving to the cloud. Data storage capacity is changed on demand by a simple click on the button. Data management includes role-based access control and automatic deletion of data by the specified criteria (after storage and so on).
4. Oracle Storage Cloud Software Appliance
The solution provides role-based encryption, ensures data integrity by checking checksums, performs automatic conversion between files and objects, has performance at the local NAS level due to data caching and docker technology.
The solution has a familiar, well-understood interface. It provides connection of applications with cloud-based data storage based on the NFS protocol, as well as compatibility with POSIX and Unix / Linux NFS clients. Implemented automatic data conversion, ─ the interface allows you to work with the data as files, while they are stored in the cloud as objects.
Information security tools are well developed. Data is encrypted before transmission. Client-side encryption is possible, including granular (selective). A layered architecture with symmetric and asymmetric keys is used to maximize information security. The user can bring his own keys, or generate them immediately before encryption.
Data availability is ensured by regularly backing up metadata to the cloud, and performance at the local NAS level — buffer caching. The cache can be configured according to workloads. The algorithm Least Recently Used (LRU) caches actual data and deletes unnecessary data.
The management uses the N-Way Management technology (N-Way ─ telecommunication protocol for managing Ethernet network devices and user data). It is possible to work from the Admin UI based on the browser, or the command line interface (Command Line Interface). Scripts can be used for automation. REST APIs that provide multiple deployments from a single central location.
5. Oracle Public Cloud Data Transfer Services
Services provide fast transfer of the initial data set to the cloud and the formation of a data warehouse in the public Oracle cloud. This is the fastest way to start working with data in the cloud. The Storage Appliance Import service securely transfers large data sets (historical archives, data lakes, large databases of inherited data). Up to 400 TB of data can be transferred at one time.
There is no need to pay for building a faster network for a one-time “wholesale” data transfer. The transmission is secure because traffic is encrypted with the AES-256 standard. A simple standard NFSv3 interface is used to load object or archive data. At the same time a multi-row structure is being formed. It is also possible to copy data inside the cloud to create a replica of the working base.
Oracle Storage Cloud Service and Traditional Solutions
Traditional storage solutions tend to have certain scalability, performance, and management issues. Oracle Storage Cloud Service helps to overcome them.
1. In a system with a directly attached storage device (such as a regular hard disk in a desktop or laptop), data storage, retrieval and sequencing through the file system is controlled by the OS.
Such a device provides low latency and fairly reliable long-term data storage. However, if you need a large capacity drive, it is distributed between individual disk devices, which makes scaling difficult and reduces performance.
2. In network-attached storage devices (Network-Attached Storage, NAS) storage hardware is physically separated from the servers on which applications run. Storage devices are available as network drives. Data storage, retrieval and ordering is managed by a network file system (Network File System, NFS). Applications running on different servers share a NAS for data storage. Managing storage resources on a local network is centralized.
As with directly attached storage devices, NAS applications also rely on the primary OS, ─ but also on the network file system. But this architecture can be implemented only in a limited geographic area, and has limited scalability.
3. Block storage is used in applications such as OLTP databases with high input / output operations (Input / Output Operations per Second, IOPS). This allows you to efficiently save and restore data bypassing the OS, directly interacting with virtual block devices.
Data fragments are stored in blocks, each with its own address, but without other metadata. Where data is stored, decide applications. They recover data at the corresponding block addresses.
The block storage principle optimizes storage systems for IOPS and provides POSIX compatible file systems for the Oracle Compute Cloud Service. However, this approach has limitations in terms of scalability and does not support the granulation of metadata.
4. Object data storage provides the optimal combination of performance, scalability and manageability when storing large amounts of unstructured data. Multiple storage nodes (nodes) form a single, common, horizontally scalable pool in which data is stored as objects (“data drops in a data pool”).
All object-stored data is associated with metadata and has a unique identifier. Thus, you can assign custom metadata to containers and objects, which greatly facilitates the search, analysis and management of data. Applications use object identifiers to directly access data through REST API calls.
Object storage is easy to use, does its job well, and can virtually virtually limitlessly scale.
Summarizing the above, it should be noted that Oracle Storage Cloud Service provides an inexpensive, reliable, secure, and scalable storage solution. This allows you to store unstructured data and access it at any time and from anywhere.
This approach is ideal for backing up and archiving data, sharing files, and also for storing large amounts of unstructured data, such as logs, data received from sensors, and virtual machine images.
Architecture Overview
The Oracle Storage Cloud Service architecture is very accessible and redundant. It is well suited for external access, including user applications, Java SDK and REST clients.
When objects are stored in the Oracle Storage Cloud Service, data is replicated across multiple storage nodes in the datacenter. This strategy ensures that the stored object data can withstand a hardware failure.
Typical Oracle Cloud Storage Service Architecture
Source: https://habr.com/ru/post/334578/
All Articles