Doctrine: Experience with migrations to symfony
For those who do not know, migration is a way to make changes to the database structure.
You can manage changes in different ways, but it all comes down to working with instructions for changing the structure.
Why migrations do it in the best way:
1. Automation . You can store instructions in sql-files, roll them if necessary. But it becomes wildly inconvenient when the question arises of switching between different revisions (database versions), for team development, when all developers need to roll changes, to deploy a test environment.
2. Rollback (as a continuation of the first paragraph). We can roll back any migration and get a database version at any time. What is convenient, see below.
3. The identity of DEV and PROD versions of the database. This is very important, at least for me, to be sure that the DEV, PROD and TEST versions are exactly the same. Yes, this can be achieved in other ways. But when exactly migrations are carriers of information about the structure of the database, together with automation, this task becomes much more convenient and easier.
I will not describe the basic things, you can see:
When we start a new project (and we don’t have a base yet), in the development process, I prefer to create tables using migrations. On the one hand, it can be done a little easier - to create a database from a schema. And, accordingly, roll out the first release on PROD in a similar way. But, on the other hand, the work on the first release does not stop and a moment when there is a need to make new changes in the structure of the database necessarily arises. In this case, we MUST have to fix the original structure in the migrations (see below). Therefore, than to switch between the two approaches, I chose one.
')
Further, if we already have a database (inherited or developed without migrations), we fix its structure in migrations.
If we do not do this and write the first migration for an already existing database, in the future we will get many problems, trust me. This is an unpleasant problem with the deployment of the project, with the creation of a clean base for the tests, and switching between revisions.
In general, in no case can you mix 2 approaches: managing the structure through migration and other options.
If the base is inherited, then, with high probability, the doctrine will not be able to generate a schema and migration to create an identical database. In declarations of tables, engine-specific instructions may be used, reserved words may occur.
Therefore, in order to fix the identical state, I am writing the first migration, which contains the raw-dump structure. Something like: $ this-> rawQuery ("CREATE TABLE ...")
With this, I achieve 100% identity and I am always confident that when a new migration version of the database is rolled into DEV, PROD and TEST they will be the same, that the tests will work with the very structure that will be downloaded to PROD.
I put all the instructions in one migration in order to separate it from everyone else and not produce a bunch of files, as the doctrine does when exporting.
The result is one simple starting point that takes everything into account.
Migrations must be atomic
One edit is one migration. Without fanaticism, of course.
Why? If we have written one big migration (tables, keys, data edits), and it suddenly falls with us (and this is easy, for example, creating FK on keys with different ints in an inherited database or creating FK when referential integrity is broken). Then we will have a situation where some of the changes will be made, and some will not. What part exactly should be looked for. We cannot roll back because the migration number has not changed. Therefore, it will be necessary to search where it has fallen, what has already been changed, and what has not, roll it all over with pens. Or you can manually change the migration number if the rolbek will be able to roll everything back. It is still possible, if conditions allow us, to crash the database and roll up the migration to an erroneous one and already there to figure out what fell.
In general, what to talk about probabilities and possible approaches, do atomic migrations and grief do not know. If something falls, it is quickly diagnosed and corrected without dancing with a tambourine.
Titles
In the name of the file and migration class, I indicate the version number, model, action and description:
It is convenient to find a similar migration in order to copy it to create a new, similar migration.
I use sequence numbers. The doctrine doesn't care what we call migration.
The migration generator substitutes a timestamp, for example. The main thing is alphabetical order.
I use sequence numbers to match the version number of the DB.
Because if you write 2 migrations with the names 1 and 100, then the version number of the database will be 2.
I do not want to be confused.
However, this approach creates certain inconveniences when working simultaneously with migrations in different branches. If we have 3 migrations, and 2 developers add one by one in their branch, then at the merge we will have 2 migrations with the name "4".
So far I have only 1 solution:
* When merging branches, rename migrations as a separate commit, + for repo purity, merge edits for specific migrations into one commit.
Therefore, I always recommend that each migration be issued as a separate commit,
so that you can then merge all commits for one migration (as part of solving one task)
* Also, migrations can be given temporary names with obviously large indices, given that they will be renamed during the merge.
Ps I have short branches and I merge them with a rebase (git help rebase)
Migrate ()
If possible, use a short entry with migrate (). This is convenient and reliable - no one will forget to write down () and will not make a mistake there.
Rollback
I always leave the opportunity to roll back the migration. Even if it is “irreversible” migration. I have never reached a rollback to PRDO (or I did not bring it), but kickbacks are primarily important for switching between revisions.
Theoretically, when the base is empty, you can switch to any revision and raise the current structure of the migrations. But it happens that in the database for development, there is valuable data that can not be lifted from the fixtures. In this case, rollerback is indispensable. And I don’t want to mess around with the dump every time, especially when I switch between different branches with a different database structure 10 times a day.
Therefore, I have no irreversible migrations. If I delete a table or column, I create it in rolbeck. In this case, the deleted data is no longer important, the main thing is not to break the chain of migrations, so that you can always go back to the very beginning and return to the starting point.
Therefore, I always write and test kickbacks (--up; --down; --up). And it happens that they fall.
Data migration
By data migrations, I mean changes not related to the database structure. Those. add a couple of lines to reference tables, delete lines, update information after changing the structure.
At first, I started writing these changes as part of normal migrations and ran into the same rake as Ryan Weaver (see the link at the end). If during migration from 50 to 100 migrations, 51 migrations use models that have already been deleted or changed (that is, the current revision has a different state), then the migration drops. And to roll it, you need to roll back to the desired revision, rebuild the model, and then it will pass.
All this seemed to me wrong and I decided to refuse to change the data in the framework of migrations. I decided that migrations are only a change in the database schema and no more. I thought that data migration is needed only on PROD, where there is data, and the tests and developers do not need it. And fixtures references can also be uploaded via yml.
But as practice has shown, this turned out to be inconvenient. Migrations of data should also be fixed somewhere, rolled by hand. Developers also found it inconvenient to work.
As a result, I came to the conclusion that migration is one convenient entry point for managing the structure and data of the database. Especially with team development.
Therefore, now I am writing data migrations in regular migrations in pre / post hooks and most importantly in pure SQL, in order not to use models that depend on the current revision.
Plus, it turned out to be very convenient to store / control fixture references in migrations.
We roll migration on one
In general, the doctrine allows you to specify the version number of the database to which you need to update. If the migrations are verified and you have already trodden the path, then there is nothing terrible about it. But if they are not yet run-in or you roll on the PROD, then they should be launched one by one:
And you are very unlucky if the fallen migration was not atomic, and there is valuable data in the database that cannot be deleted.
Therefore, on PROD I always roll one by one with a sinking heart. And I also test it on a similar copy of the database.
Tests
To run a full set of tests, I create a clean database from migrations:
Reset migrations
I was interested in this idea - from time to time to delete all migrations, most of which have already become redundant and it takes a lot of time to create a test environment. You can again write the first migration for the initial import and change the migration number to the database.
True, I have not tried it yet and I don’t know how it will affect the work with different revisions of the project.
Similar thoughts on the topic: Ryan Weaver
http://www.slideshare.net/weaverryan/the-art-of-doctrine-migrations
You can manage changes in different ways, but it all comes down to working with instructions for changing the structure.
Why migrations do it in the best way:
1. Automation . You can store instructions in sql-files, roll them if necessary. But it becomes wildly inconvenient when the question arises of switching between different revisions (database versions), for team development, when all developers need to roll changes, to deploy a test environment.
2. Rollback (as a continuation of the first paragraph). We can roll back any migration and get a database version at any time. What is convenient, see below.
3. The identity of DEV and PROD versions of the database. This is very important, at least for me, to be sure that the DEV, PROD and TEST versions are exactly the same. Yes, this can be achieved in other ways. But when exactly migrations are carriers of information about the structure of the database, together with automation, this task becomes much more convenient and easier.
I will not describe the basic things, you can see:
- Doctrine Migration Doc 1.2
- Working with symfony migrations
- Description of console commands for working with migrations in symfony
Getting started on the project
When we start a new project (and we don’t have a base yet), in the development process, I prefer to create tables using migrations. On the one hand, it can be done a little easier - to create a database from a schema. And, accordingly, roll out the first release on PROD in a similar way. But, on the other hand, the work on the first release does not stop and a moment when there is a need to make new changes in the structure of the database necessarily arises. In this case, we MUST have to fix the original structure in the migrations (see below). Therefore, than to switch between the two approaches, I chose one.
')
Further, if we already have a database (inherited or developed without migrations), we fix its structure in migrations.
If we do not do this and write the first migration for an already existing database, in the future we will get many problems, trust me. This is an unpleasant problem with the deployment of the project, with the creation of a clean base for the tests, and switching between revisions.
In general, in no case can you mix 2 approaches: managing the structure through migration and other options.
If the base is inherited, then, with high probability, the doctrine will not be able to generate a schema and migration to create an identical database. In declarations of tables, engine-specific instructions may be used, reserved words may occur.
Therefore, in order to fix the identical state, I am writing the first migration, which contains the raw-dump structure. Something like: $ this-> rawQuery ("CREATE TABLE ...")
With this, I achieve 100% identity and I am always confident that when a new migration version of the database is rolled into DEV, PROD and TEST they will be the same, that the tests will work with the very structure that will be downloaded to PROD.
I put all the instructions in one migration in order to separate it from everyone else and not produce a bunch of files, as the doctrine does when exporting.
The result is one simple starting point that takes everything into account.
How to write migrations
Migrations must be atomic
One edit is one migration. Without fanaticism, of course.
Why? If we have written one big migration (tables, keys, data edits), and it suddenly falls with us (and this is easy, for example, creating FK on keys with different ints in an inherited database or creating FK when referential integrity is broken). Then we will have a situation where some of the changes will be made, and some will not. What part exactly should be looked for. We cannot roll back because the migration number has not changed. Therefore, it will be necessary to search where it has fallen, what has already been changed, and what has not, roll it all over with pens. Or you can manually change the migration number if the rolbek will be able to roll everything back. It is still possible, if conditions allow us, to crash the database and roll up the migration to an erroneous one and already there to figure out what fell.
In general, what to talk about probabilities and possible approaches, do atomic migrations and grief do not know. If something falls, it is quickly diagnosed and corrected without dancing with a tambourine.
Titles
In the name of the file and migration class, I indicate the version number, model, action and description:
001_Article_CreateTable.php
002_Article_AddColumn_AuthorId.php
003_Article_AddFk_Authors.php
004_Article_UpdateColumn_Title.php
005_Article_DropTable.php
class:
class Migration001_Article_CreateTable
This approach allows you to quickly find / filter all migrations for a given model.It is convenient to find a similar migration in order to copy it to create a new, similar migration.
I use sequence numbers. The doctrine doesn't care what we call migration.
The migration generator substitutes a timestamp, for example. The main thing is alphabetical order.
I use sequence numbers to match the version number of the DB.
Because if you write 2 migrations with the names 1 and 100, then the version number of the database will be 2.
I do not want to be confused.
However, this approach creates certain inconveniences when working simultaneously with migrations in different branches. If we have 3 migrations, and 2 developers add one by one in their branch, then at the merge we will have 2 migrations with the name "4".
So far I have only 1 solution:
* When merging branches, rename migrations as a separate commit, + for repo purity, merge edits for specific migrations into one commit.
Therefore, I always recommend that each migration be issued as a separate commit,
so that you can then merge all commits for one migration (as part of solving one task)
* Also, migrations can be given temporary names with obviously large indices, given that they will be renamed during the merge.
Ps I have short branches and I merge them with a rebase (git help rebase)
Migrate ()
If possible, use a short entry with migrate (). This is convenient and reliable - no one will forget to write down () and will not make a mistake there.
Rollback
I always leave the opportunity to roll back the migration. Even if it is “irreversible” migration. I have never reached a rollback to PRDO (or I did not bring it), but kickbacks are primarily important for switching between revisions.
Theoretically, when the base is empty, you can switch to any revision and raise the current structure of the migrations. But it happens that in the database for development, there is valuable data that can not be lifted from the fixtures. In this case, rollerback is indispensable. And I don’t want to mess around with the dump every time, especially when I switch between different branches with a different database structure 10 times a day.
Therefore, I have no irreversible migrations. If I delete a table or column, I create it in rolbeck. In this case, the deleted data is no longer important, the main thing is not to break the chain of migrations, so that you can always go back to the very beginning and return to the starting point.
Therefore, I always write and test kickbacks (--up; --down; --up). And it happens that they fall.
Data migration
By data migrations, I mean changes not related to the database structure. Those. add a couple of lines to reference tables, delete lines, update information after changing the structure.
At first, I started writing these changes as part of normal migrations and ran into the same rake as Ryan Weaver (see the link at the end). If during migration from 50 to 100 migrations, 51 migrations use models that have already been deleted or changed (that is, the current revision has a different state), then the migration drops. And to roll it, you need to roll back to the desired revision, rebuild the model, and then it will pass.
All this seemed to me wrong and I decided to refuse to change the data in the framework of migrations. I decided that migrations are only a change in the database schema and no more. I thought that data migration is needed only on PROD, where there is data, and the tests and developers do not need it. And fixtures references can also be uploaded via yml.
But as practice has shown, this turned out to be inconvenient. Migrations of data should also be fixed somewhere, rolled by hand. Developers also found it inconvenient to work.
As a result, I came to the conclusion that migration is one convenient entry point for managing the structure and data of the database. Especially with team development.
Therefore, now I am writing data migrations in regular migrations in pre / post hooks and most importantly in pure SQL, in order not to use models that depend on the current revision.
Plus, it turned out to be very convenient to store / control fixture references in migrations.
Start Migrations
We roll migration on one
In general, the doctrine allows you to specify the version number of the database to which you need to update. If the migrations are verified and you have already trodden the path, then there is nothing terrible about it. But if they are not yet run-in or you roll on the PROD, then they should be launched one by one:
./symfony doctrine: migrate --upAnd all because when you update in one fell swoop, for example, from 2 to 10 version, and you have some kind of migration (which one is unknown), then the version number will not update and 2 will remain, although the actual number may be 5. And you have to find a fallen migration, edit the version number in the database in order to roll back and repair the migration.
And you are very unlucky if the fallen migration was not atomic, and there is valuable data in the database that cannot be deleted.
Therefore, on PROD I always roll one by one with a sinking heart. And I also test it on a similar copy of the database.
Tests
To run a full set of tests, I create a clean database from migrations:
$ task = new sfDoctrineBuildTask (new sfEventDispatcher, new sfFormatter);
$ task-> run ($ args = array (), $ options = array (
'env' => 'test',
'no-confirmation' => true
'db' => true
'and-migrate' => true
))
By this I achieve 100% identity TEST and PROD. Plus, I indirectly test the work of the application taking into account new migrations.Other
Reset migrations
I was interested in this idea - from time to time to delete all migrations, most of which have already become redundant and it takes a lot of time to create a test environment. You can again write the first migration for the initial import and change the migration number to the database.
True, I have not tried it yet and I don’t know how it will affect the work with different revisions of the project.
Similar thoughts on the topic: Ryan Weaver
http://www.slideshare.net/weaverryan/the-art-of-doctrine-migrations
Source: https://habr.com/ru/post/97940/
All Articles