NewSQL - a new milestone in the evolution of BigData, taking the best from SQL and NoSQL

NewSQL

Start

Today it is very easy to observe the rapid growth of data on the Internet. According to one estimate, the data created in 2010 is approximately 1,200 DL (10 ¹⁸ bytes) and will grow to almost 8,000 DL by 2015 on the Internet, which is the main supplier of data to the consumer.



This growth outpaces the growth in capacity, leading to the emergence of information management systems, where data is stored in a distributed way, but accessed and analyzed as if they were on the same machine.



While programmers around the world are arranging global Holywars on the topic: “SQL vs NoSQL”, large companies such as Google and Facebook with their billions of audiences are struggling with the lack of capacity and the marginal work of the DBMS. Despite the emergence of NoSQL technology, which allowed data to be easily scaled, it did not resolve issues related to compliance of operations with ACID requirements (atomicity, consistency, isolation, durability - “atomicity, consistency, isolation, durability”) - a standard that guarantees accuracy execution of operational transactions by means of the DBMS, even if the system was interrupted. Against this background, the company VoltDB, with the support of several other companies, began to develop from scratch a new opensource project called NewSQL, combining the best aspects of SQL and NoSQL.



')

SQL

Currently, in order to cope with the load created by 1 billion users, Facebook operates four thousand MySQL instances (using sharding, i.e., spreading data across servers, starting from a specific feature, such as the first letter of the login) and nine thousand memcached installations . Facebook even maintains a special MySQL @ Facebook page, where work on maintaining the company's databases is monitored.



MySQL's well-known problem is that this DBMS was never intended to handle huge amounts of data and a large number of transactions. Stonebreaker adds that MySQL, like other SQL-based databases, consumes too much resources for additional database operations (for example, to support multithreading and to support correct execution of queries within ACID). These requirements and costs do not interfere with work with small amounts of data, but quickly begin to interfere with the normal functioning of their increase.

NoSQL

Many emerging NoSQL systems, such as MongoDB and Cassandra, are viewed by many as an alternative, free from the limitations inherent in conventional relational DBMS.



To solve problems, large companies have adopted the NoSQL paradigm, however NoSQL databases are poorly suited for storing ordinary structured data, and the ACID logic with NoSQL has to be embedded in user code, thereby complicating the work. In addition, according to Stonebreaker, NoSQL has not much increased performance relative to traditional SQL-oriented DBMS.

Welcome NewSQL

Technical Specifications of NewSQL Solutions

1. SQL as the main mechanism for interaction.
2. ACID transaction support.
3. The control mechanism without the use of locks, thus reading real-time data will not be in conflict with the recording, which eliminates the conflict.
4. An architecture that provides much higher node performance than that available from traditional RDBMS solutions.
5. Convenient scaling, able to manage a large number of nodes, without transferring bottlenecks.

Project developers claim that NewSQL systems are about 50 times faster than traditional OLTP RDBMS.



An architectural example of one of the solutions NewSQL (dbShards).

NewSQL classification

The classification is based on various approaches taken to maintain the SQL interface, as well as to solve the scalability and performance that are the problems of traditional OLTP solutions.

• New databases : NewSQL system is developed completely from scratch in order to achieve scalability and performance. One of the key factors in improving performance is the use of RAM or new types of disks (flash memory / SSD), which are the primary data storage. This solution can be implemented in software (VoltDB, NuoDB) or at the iron level (Clustrix, Translattice). Examples of developments are Clustrix, NuoDB and Translattice (commercial) and VoltDB, (Open Source).
• A new MySQL database engine : MySQL is part of the LAMP stack and is used in OLTP. To overcome the problems of MySQL scalability, a number of MySQL-based engines were created. The positive side is the use of the MySQL interface, but there is a bad side — data migration from other databases (including the old MySQL) is not supported. Implementation examples - Xeround, GenieDB (commercial) TokuTek; and Akiban, MySQL NDB Group and others (opensource).
• Transparent clustering : These solutions retain the OLTP databases in their original form, but provide an expansion feature, with transparent grouping and guaranteeing scalability. Another approach is to provide transparent sharding to also improve scalability. Schooner MySQL, Continuent Tungsten and ScalArc BDs follow the first approach, while ScaleBase and dbShards follow the second approach. Both approaches allow the reuse of existing sets and ecosystems, and avoid the need to rewrite code or perform any data migrations. Examples of implementations - ScalArc, Schooner MySQL, dbShards (commercial) ScaleBase; and Continuent Tungsten (opensource).

Conclusion

The new generation of information management systems, called NewSQL, is in line with this trend and limitations. NewSQL is inclined for companies that are planning:

1. migration of existing applications to adapt to new data growth trends
2. developing new applications on well-scalable OLTP systems
3. relying on existing knowledge of using OLTP

According to the creators of NewSQL, traditional SQL is outdated, too complicated and has many problems, and object-oriented DBMS is no longer the future, but the present. To simplify the migration, SQL to NewSQL and NewSQL to SQL converters will be developed, and they will be able to translate queries on the fly, providing the ability to run old applications without modification.



Sources:

The NewSQL project aims to solve the problems Facebook has encountered using MySQL

NewSQL takes all the best from SQL and NoSQL

http://www.linuxforu.com/2012/01/newsql-handle-big-data/

NoSQL is Out and NewSQL is In - Google Says (Google Spanner)



Translation in some places may not be correct.

We look forward to your comments on this introductory article.