Hi, Habr! Last week, the Percona Live Data Performance Conference 2016 was held. As usual, there are tons of new information, and I am still sorting out my notes and scrolling through the slides of the reports that we could not get. The MySQL community is open and friendly towards “parallel” communities and projects. Naturally, there were reports on Oracle MySQL, MariaDB and Percona Server, but there were also reports on MongoDB, Redis, Tarantool, Hadoop, Cassandra, Scylla, Ignite, HBase, ActorDB, SQLite, ToroDB, Tempesta DB and even the PostgreSQL optimization report, somewhat generalized for MySQL and other DBMSs.

Oracle employees shared plans about MySQL 8 unexpectedly generously, both in reports and on the sidelines. I found this information quite interesting, so I decided to make a brief summary of what I heard.

Yes, with the next version of MySQL, the release numbering scheme is changed. The next major release will be called MySQL 8, and all subsequent major releases will be numbered 9, 10, etc. This is done firstly because MySQL will be merged with MySQL Cluster (which currently has a release number of 7.5), and secondly to simplify the numbering so that the main releases (5.5, 5.6, 5.7) do not look as corrective.

Rapid plugins

According to the MySQL development team, users want a more flexible development cycle so that the new functionality does not have to wait until the next major release. Many users are ready to test the new functionality, even if it is damp, but so that if problems arise it can be turned off.

The proposed solution to this problem is called the difficult plug-in term rapid plugins. The bottom line is that, whenever possible, the new functionality will be released as plug-ins. Versions of modules will coincide with the version of the server, and as I understand it, no compatibility of these modules between different versions of the server is promised. I.e,. this is done solely to isolate new functions from the old ones, and so that new functions can be connected / disconnected at will.

As a matter of fact, this new feature is already being tested on the current stable release 5.7. In MySQL 5.7.12, a new module with a slightly mystical name X Plugin appeared and it is very strange that an article on this topic has not yet appeared on Habré, because the functionality it provides is impressive: the new protocol for communicating with the server with the ability to perform asynchronous queries , the new NoSQL API with a focus on CRUD, JSON and document store with bindings for Python, JavaScript, Node.js, .Net and Java, as well as a new command utility for interactive work with the server .

However, all this functionality should be considered rather as a prototype for testing . The developers report that for the first release, performance and completeness were not priorities and work will continue in future releases.

X protocol / DevAPI / Shell

For MySQL 8, developers have extensive plans for a new protocol and API:

• performance
• API extension and bindings to other languages
• using X protocol for replication and sharding (streaming protocol + more stringent query structure simplifies these tasks greatly)
• consolidation of all administrative utilities and front-end to the server in MySQL Shell

Query optimizer

The query optimizer in MySQL 8 will receive, perhaps, the most significant changes in my memory:

• execution plan caching for prepared statements
• histograms
• window functions (window functions)
• common table expressions (including recursive)

As I understand it, this list is not final, other “advanced” SQL features are also considered, as well as improvements in JSON functionality and indexes.

By the way, MariaDB users can use histograms starting from version 10.0, while window functions and CTEs (albeit non-recursive) are scheduled for MariaDB 10.2, which will most likely come out earlier than MySQL 8.

Group replication

Analogue Galera Cluster in the performance of Oracle is called Group Replication . Pre-releases have already been available as a plugin. As I understand it, the stable release is planned for the emergence of MySQL 8. The developers didn’t report any details about the planned features, besides that it would be even faster, more convenient and more stable than the preliminary releases.

Bye, MyISAM!

Finally all system tables will be translated to InnoDB. MyISAM, therefore, becomes absolutely unnecessary from the point of view of the server, but will be available as an option for those who love cacti.

Data Dictionary and Atomic DDL

One of the most frequently mentioned shortcomings in MySQL is the lack of a transactional data dictionary. As I already wrote , this disadvantage has many rather unpleasant consequences: the lack of transactional DDLs, expensive queries to INFORMATION_SCHEMA , problems with the extensibility of the metadata format, problems with physical backups and table locks.

In MySQL 8, at least some of these problems will be fixed. .frm files will no longer .frm , the table metadata will be stored only in the general data dictionary (and not .frm and separately in InnoDB, as it is now). That is, problems with slow access via INFORMATION_SCHEMA and out of sync metadata in case of failures are a thing of the past. Storage engines will be able to store extended attributes for their tables, which opens up possibilities for some interesting functionality.

And most importantly, the DDL will become, although not transactional, but atomic. That is, the ability to wrap the DDL in a transaction and roll back to MySQL 8 will not be yet. But we will no longer see partially executed DDL in case of server crash or problems with replication.

And further...

Surely this is an incomplete list of planned functionality, and it will expand to the release. Among other things, the next revision of default values ​​for configuration parameters is planned. In particular, UTF-8 will become the default encoding instead of latin1.

Developers also vaguely hint at improved support for cloud platforms and a new storage engine based on Log Structured Merge trees . In any case, MySQL 8 already looks interesting.