Entity Framework Code First in teamwork
From the translator: Excellent article on understanding the mechanism of migrations in the Entity Framework 6 and ways to solve conflicts of migrations when working in a team. Original article: Code First Migrations in Team Environments .
This article assumes that you are familiar with the Entity Framework and the basics of working with it. Otherwise, you first need to read Code First Migrations before continuing.
In team work, the problems are mainly related to the merging of migrations created by two developers in their local databases. While the steps to solve these problems are fairly simple, they require a clear understanding of how migrations work. Please take your time to read the whole article carefully.
Before we dive into how to manage the merge of migrations created by several developers, here are some general principles.
')
The migration mechanism uses the __MigrationsHistory table to store the list of migrations that have been applied to the database. If you have several developers in a team that generate migrations, then when you try to work with the same database (and, consequently, with the same __MigrationsHistory table), the migration mechanism may have difficulty.
Of course, if you have team members that do not create migrations, then problems with working with a central database for development will not arise.
The bottom line is that automatic migrations initially look good in teamwork, but in reality they just don't work. If you want to know the reason, keep reading the article, otherwise you can skip to the next chapter.
Automatic migrations allow updating the database schema in accordance with the current model without the need to create files with a migration code (code-based migration).
Automatic migrations will work well in team work only if you have ever used them and never created any code-based migrations. The problem is that automatic migrations are limited and cannot cope with a number of operations - renaming a property / column, transferring data from one table to another, etc. To process such scenarios, you will eventually create code-based migrations (and edit the generated code), which leads to a mix of changes that are processed using automatic migrations. This makes it almost impossible to merge the changes of the two developers.
From the translator: in the original article 2 screencasts were posted, I recommend to read
The key to successful use of migrations in a team is the basic understanding of how the migration engine tracks and uses model information to detect changes.
When you add the first migration to your project, you run something like Add-Migration First in the Package Manager Console. Below are the steps that this command performs.
Based on the code, the current model is calculated (1). Then, using the model differ, the required database objects are calculated (2) - since this is the first migration, the model differ uses an empty model for comparison. The necessary changes are transmitted to the code generator to create the necessary migration code (3), which is then added to the Visual Studio solution (4).
In addition to the migration code that is stored in the main file, the migration mechanism also creates additional code-behind files. These are the metadata files that are used by the migration engine and you should not modify them. One of these files is a resource file (.resx) that contains a snapshot of the model at the time the migration was created. In the next section, you will see how it is used.
At this point, you can run Update-Database to apply the changes to the database, and then start implementing the rest of your application.
Let's make some changes to the model - in our example we will add the Url property to the Blog class. Then you need to run the Add-Migration AddUrl command to create a migration to apply the corresponding changes in the database. Below are the steps that this command performs.
Just like last time, the current model is calculated from code (1). However, this time there are existing migrations, and the previous model is extracted from the last migration (2). These two models are compared to find the necessary changes in the database (3), and then the process ends as before.
The same process is used for all subsequent migrations that are added to the project.
You may be wondering why EF uses snapshots of a model for comparison - why not just look at the database.
There are a number of reasons why EF saves model states:
The process described in the previous section works great if you are the only developer working on the application. He also works well in a team if you are the only person who makes changes to the model. In this case, you can make changes to the model, generate migrations and send them to the version control system. Other developers may receive such changes and run the Update-Database to update the schema.
Problems start to occur when you have several developers who make changes to the EF model and send them to the version control system. What EF lacks is a first-class way to combine local migrations with migrations, other developers sent to the version control system after the last synchronization.
First, let's look at a specific example of the merging of such a conflict. We will continue with an example that was considered earlier. As a starting point, let's assume that the changes from the previous section have been verified by this developer. We will track two developers who make code changes.
We will track the EF model and the migration through a series of changes. Both developers are synchronized through the repository in the version control system, as shown in the following figure.
Developer # 1 and Developer # 2 make some changes to the EF model in the local code base. Developer # 1 adds the Rating property to the Blog class, creates an AddRating migration for applying changes to the database. Developer # 2 adds the Readers property to the Blog class, creates an AddReaders migration. Both developers run Update-Database to apply changes to their local databases, and then continue to develop the application.
Note: Migrations start with a timestamp, so our drawing shows that AddReaders migration from developer # 2 comes after migration AddRating from developer # 1. From the point of view of working in a team, to us without a difference in what order these changes were created, we will consider the process of their merging in the next section.
Today, lucky developer # 1, as he first sends his changes to the version control system. Since no one else sent changes to the repository, he can simply submit his changes without performing any merging.
Now it's time for developer # 2 to submit your changes. He's not so lucky. Since someone posted changes after the last sync, the developer must pick them up and merge them. The version control system will most likely be able to automatically merge changes at the code level, as they are very simple. The state of the local code base of developer # 2 after synchronization is shown in the following figure.
At this stage, developer # 2 can run the Update-Database , which allows you to discover the new AddRating migration (which has not been applied to developer database # 2), and apply it. Now the Rating column is added to the Blogs table, and the database is synchronized with the model.
There are several problems, for example:
The good news is that it is not very difficult to merge migrations manually, if you have an idea how migration works. So if you missed the beginning of this article ... sorry, you need to go back and read the first part of the article first!
There are two options, the simplest is to create an empty migration that has the correct snapshot of the model. The second option is to update the snapshot in the last migration in order to have the correct snapshot of the model. This is a bit more complicated and this option cannot be used in each case. Its advantage is that it does not involve the addition of additional migration.
In this option, we generate an empty migration only to ensure that the last migration has the correct snapshot of the model stored in it.
This feature can be used regardless of who created the last migration. In the example, we watched developer # 2, which generated the latest migration. But these same steps can be used if developer # 1 has generated the latest migration. The steps also apply if there are multiple migrations.
The following algorithm can be used from the moment when changes appear, which should be synchronized with the version control system.
This is the state of the developer model # 2 after applying this approach.
This option is very similar to option 1, but removes the extra empty migration.
This approach is possible if the last migration exists only locally and has not yet been sent to the version control system (that is, the last migration was created by the user performing the merge). Editing migration metadata that is applied by other developers, or even worse - applied to the combat database, can lead to unexpected side effects. In the process, we will roll back the last migration in our local database and reapply it with updated metadata.
As long as the last migration is local, there are no restrictions on the number or order of migrations.
The same steps apply for multiple migrations from several different developers.
The following algorithm can be used when changes appear that must be synchronized with the version control system.
This is the state of the local developer code base # 2 after applying this approach.
There are some problems when using Code First migrations in a team. However, a general idea of how migrations work, and some simple approaches to resolving merge conflicts make it easy to overcome these problems.
The fundamental problem is erroneous metadata that is stored in the last migration. This allows Code First to incorrectly determine that the current model and database schema do not match and generate the wrong code for the next migration. This situation can be corrected by generating a blank migration with the correct model, or by updating the metadata in the last migration.
This article assumes that you are familiar with the Entity Framework and the basics of working with it. Otherwise, you first need to read Code First Migrations before continuing.
Pour a cup of coffee, you need to read the entire article.
In team work, the problems are mainly related to the merging of migrations created by two developers in their local databases. While the steps to solve these problems are fairly simple, they require a clear understanding of how migrations work. Please take your time to read the whole article carefully.
Some general principles
Before we dive into how to manage the merge of migrations created by several developers, here are some general principles.
')
Each team member must have a local development database.
The migration mechanism uses the __MigrationsHistory table to store the list of migrations that have been applied to the database. If you have several developers in a team that generate migrations, then when you try to work with the same database (and, consequently, with the same __MigrationsHistory table), the migration mechanism may have difficulty.
Of course, if you have team members that do not create migrations, then problems with working with a central database for development will not arise.
Avoid automatic migrations
The bottom line is that automatic migrations initially look good in teamwork, but in reality they just don't work. If you want to know the reason, keep reading the article, otherwise you can skip to the next chapter.
Automatic migrations allow updating the database schema in accordance with the current model without the need to create files with a migration code (code-based migration).
Automatic migrations will work well in team work only if you have ever used them and never created any code-based migrations. The problem is that automatic migrations are limited and cannot cope with a number of operations - renaming a property / column, transferring data from one table to another, etc. To process such scenarios, you will eventually create code-based migrations (and edit the generated code), which leads to a mix of changes that are processed using automatic migrations. This makes it almost impossible to merge the changes of the two developers.
From the translator: in the original article 2 screencasts were posted, I recommend to read
Principles of operation of the migration mechanism
The key to successful use of migrations in a team is the basic understanding of how the migration engine tracks and uses model information to detect changes.
First migration
When you add the first migration to your project, you run something like Add-Migration First in the Package Manager Console. Below are the steps that this command performs.
Based on the code, the current model is calculated (1). Then, using the model differ, the required database objects are calculated (2) - since this is the first migration, the model differ uses an empty model for comparison. The necessary changes are transmitted to the code generator to create the necessary migration code (3), which is then added to the Visual Studio solution (4).
In addition to the migration code that is stored in the main file, the migration mechanism also creates additional code-behind files. These are the metadata files that are used by the migration engine and you should not modify them. One of these files is a resource file (.resx) that contains a snapshot of the model at the time the migration was created. In the next section, you will see how it is used.
At this point, you can run Update-Database to apply the changes to the database, and then start implementing the rest of your application.
Subsequent migrations
Let's make some changes to the model - in our example we will add the Url property to the Blog class. Then you need to run the Add-Migration AddUrl command to create a migration to apply the corresponding changes in the database. Below are the steps that this command performs.
Just like last time, the current model is calculated from code (1). However, this time there are existing migrations, and the previous model is extracted from the last migration (2). These two models are compared to find the necessary changes in the database (3), and then the process ends as before.
The same process is used for all subsequent migrations that are added to the project.
Why bother with pictures of the model?
You may be wondering why EF uses snapshots of a model for comparison - why not just look at the database.
There are a number of reasons why EF saves model states:
- This allows your database to differ from the EF model. These changes can be made directly in the database, or you can change the scaffolded code in your migrations to make changes. Here are some examples of this in practice:
- You want to add Inserted and Updated columns to one or more tables, but you do not want to include them in the EF model. If the migration engine looked at the database, it would constantly try to delete these columns each time you generate a migration code. Using a snapshot of the model, the EF will detect only the desired changes in the model.
- You want to change the body of the stored procedure used to update certain debug information. If the migration mechanism looked at the stored procedure in the database, he would constantly try to reset it back to the definition. When using a snapshot of the model, EF will generate code to change the stored procedure when you change the procedure in the EF model
- The same principles apply when adding additional indexes, including additional tables in the database, EF mapping to DB View, etc.
- You want to add Inserted and Updated columns to one or more tables, but you do not want to include them in the EF model. If the migration engine looked at the database, it would constantly try to delete these columns each time you generate a migration code. Using a snapshot of the model, the EF will detect only the desired changes in the model.
- The EF model contains more than just a database image. Having a model, the migration mechanism allows you to view information about the properties and classes of your model and how they are mapped to columns and tables. This information allows the migration mechanism to be smarter with automatic code generation. For example, if you rename a column, the migration mapping will detect renaming, seeing that it is the same property. This would be impossible if we only looked at the database.
What causes questions during teamwork
The process described in the previous section works great if you are the only developer working on the application. He also works well in a team if you are the only person who makes changes to the model. In this case, you can make changes to the model, generate migrations and send them to the version control system. Other developers may receive such changes and run the Update-Database to update the schema.
Problems start to occur when you have several developers who make changes to the EF model and send them to the version control system. What EF lacks is a first-class way to combine local migrations with migrations, other developers sent to the version control system after the last synchronization.
Conflict Merger Example
First, let's look at a specific example of the merging of such a conflict. We will continue with an example that was considered earlier. As a starting point, let's assume that the changes from the previous section have been verified by this developer. We will track two developers who make code changes.
We will track the EF model and the migration through a series of changes. Both developers are synchronized through the repository in the version control system, as shown in the following figure.
Developer # 1 and Developer # 2 make some changes to the EF model in the local code base. Developer # 1 adds the Rating property to the Blog class, creates an AddRating migration for applying changes to the database. Developer # 2 adds the Readers property to the Blog class, creates an AddReaders migration. Both developers run Update-Database to apply changes to their local databases, and then continue to develop the application.
Note: Migrations start with a timestamp, so our drawing shows that AddReaders migration from developer # 2 comes after migration AddRating from developer # 1. From the point of view of working in a team, to us without a difference in what order these changes were created, we will consider the process of their merging in the next section.
Today, lucky developer # 1, as he first sends his changes to the version control system. Since no one else sent changes to the repository, he can simply submit his changes without performing any merging.
Now it's time for developer # 2 to submit your changes. He's not so lucky. Since someone posted changes after the last sync, the developer must pick them up and merge them. The version control system will most likely be able to automatically merge changes at the code level, as they are very simple. The state of the local code base of developer # 2 after synchronization is shown in the following figure.
At this stage, developer # 2 can run the Update-Database , which allows you to discover the new AddRating migration (which has not been applied to developer database # 2), and apply it. Now the Rating column is added to the Blogs table, and the database is synchronized with the model.
There are several problems, for example:
- Although the Update-Database operation will apply AddRating migrations, it will also show a warning:
Unable to update the database for the current model because there are pending changes ...
The problem is that the model snapshot is stored in the last migration (AddReader), which skips the Rating property in the Blog class (since it was not part of the model when the migration was created).
Code First detects that the model in the past migration does not match the current model and displays a warning. - Running the application will result in an InvalidOperationException "The model backing the created bloggingContext ". Consider using the code to update the database ... "
Again, the problem is that the snapshot of the model is stored in the last migration that does not match the current model. - Finally, one would expect the launch of Add-Migration to now generate an empty migration (since there are no changes to apply to the database). But since the migrations compare the current model in one of the latest migrations (in which there is no Rating property), this will generate another AddColumn call to add a Rating column.
Of course, this migration will fail with Update-Database , because the Rating column already exists.
Resolving Merge Conflicts
The good news is that it is not very difficult to merge migrations manually, if you have an idea how migration works. So if you missed the beginning of this article ... sorry, you need to go back and read the first part of the article first!
There are two options, the simplest is to create an empty migration that has the correct snapshot of the model. The second option is to update the snapshot in the last migration in order to have the correct snapshot of the model. This is a bit more complicated and this option cannot be used in each case. Its advantage is that it does not involve the addition of additional migration.
Option 1: Adding an empty migration "merge"
In this option, we generate an empty migration only to ensure that the last migration has the correct snapshot of the model stored in it.
This feature can be used regardless of who created the last migration. In the example, we watched developer # 2, which generated the latest migration. But these same steps can be used if developer # 1 has generated the latest migration. The steps also apply if there are multiple migrations.
The following algorithm can be used from the moment when changes appear, which should be synchronized with the version control system.
- Ensure that all model changes in your local codebase have been saved in the migration. This step ensures that you don’t miss any important changes when it comes to creating a clean migration.
- Synchronize code with version control system
- Run Update-Database to apply any new migrations made by other developers.
Note: if you do not receive any warnings during the execution of the Update-Database, then there are no new migrations from other developers and there is no need to perform additional merges. - Run Add-Migration <pick_a_name> -ignorechanges (for example, add-migration merge -ignorechanges ). This command creates a migration with all the metadata (including a snapshot of the current model), but will ignore any changes it finds when comparing the current model with a snapshot of the last migration (that is, you get empty Up and Down methods).
- Continue development, or send changes to the version control system (after running the unit tests, of course).
This is the state of the developer model # 2 after applying this approach.
Option 2: Update the latest migration model snapshot
This option is very similar to option 1, but removes the extra empty migration.
This approach is possible if the last migration exists only locally and has not yet been sent to the version control system (that is, the last migration was created by the user performing the merge). Editing migration metadata that is applied by other developers, or even worse - applied to the combat database, can lead to unexpected side effects. In the process, we will roll back the last migration in our local database and reapply it with updated metadata.
As long as the last migration is local, there are no restrictions on the number or order of migrations.
The same steps apply for multiple migrations from several different developers.
The following algorithm can be used when changes appear that must be synchronized with the version control system.
- Ensure that all model changes in your local codebase have been saved in the migration.
This step ensures that you don’t miss any important changes when it comes to creating a clean migration. - Synchronize the code with the version control system.
- Run Update-Database to apply any new migrations made by other developers.
Note: if you do not receive any warnings during the execution of the Update-Database , then there are no new migrations from other developers and there is no need to perform additional merges. - Run Update-Database -TargetMigration <second_last_migration> (in the example, this would be Update-Database -TargetMigration AddRating ).
This action rolls the database back to the state of the penultimate migration — in fact,
where the last migration to the database was not applied.
Note: This step is necessary to make editing migration metadata safe, because the metadata is also stored in the __MigrationsHistory table in the database. That is why you should use this function only if the last migration is only local. If you need to apply the latest migration on other databases, you also need to roll it back and reapply the latest migration to update the metadata. - Run Add-Migration <full_name_including_timestamp_of_last_migration> (in the example, this would be something like Add-Migration 201311062215252_AddReaders migration add -ons ).
Note: You must specify a label so that the migration mechanism understands that you want to change an existing migration, and not create a new one. This will update the metadata for the last migration to fit the current model. You will receive the following warning when completing the command, but this is exactly what you want. “Only the Designer Code for Migration 201311062215252_AddReaders' was re-scaffolded. To re-scaffold the for migration, use the -Force parameter. ” - Run Update-Database to reapply the latest migration with updated metadata.
- Continue development, or send changes to the version control system (after running the unit tests, of course).
This is the state of the local developer code base # 2 after applying this approach.
Total
There are some problems when using Code First migrations in a team. However, a general idea of how migrations work, and some simple approaches to resolving merge conflicts make it easy to overcome these problems.
The fundamental problem is erroneous metadata that is stored in the last migration. This allows Code First to incorrectly determine that the current model and database schema do not match and generate the wrong code for the next migration. This situation can be corrected by generating a blank migration with the correct model, or by updating the metadata in the last migration.
Source: https://habr.com/ru/post/277915/
All Articles