Practical aspects of using svn: externals
Despite the fact that distributed version control systems (Git, Mercurial, Bazaar) are gaining increasing popularity, the good old Subversion is still widely used. In this article, I will discuss the pros and cons of using external dependencies (svn: externals) in practice in SVN repositories.
Let's look at an example of what svn: externals is. Suppose we have a project that uses third-party open-source code, for example, the well-known PDF library iText. Usually we completely copy all the necessary code of this library to our repository. Subsequently, when iText updates are released, manually replace the old files with new ones.
svn: externals provide a more convenient way. Select some folder of our repository and set the svn: externals property for it in this form ( how to do it with the help of TortoiseSVN ):
The first parameter determines where to download the changes, the second - from where. Now, every time we take the latest updates from the repository, the latest version of iText is automatically downloaded. Conveniently.
')
Consider the benefits of using svn: externals.
The principle of DRY (Don't Repeat Yourself) - the cornerstone of high-quality software development - extends to source code management. If the same files are duplicated in several repositories - this is not very good.
For example, in established teams, as a rule, a base of common code is formed, which is constantly used in various projects. All projects will not necessarily be in one large repository. Most likely - in different. In this case, without svn: externals, we are doomed to copy the same files between different repositories, as well as to manually update them everywhere in case of changes.
A direct consequence of the previous paragraph - the source code is stored only in the original repository, we only download it from there to our working copy. The size of the repository can be very critical if you use paid SVN hosting with a limit on the total size of the repository (s).
It seems reasonable to divide projects according to the rule “1 project <-> 1 repository”. With this approach, using svn: externals helps to simplify the separation of dependent projects.
Of course, we can work on all projects / modules within one repository, but this approach is not without flaws. For example, all developers can download many unnecessary changes to their working copies. It will not be possible to restrict access to developers to those projects to which they should not have access. Nearby are unrelated projects. And generally keep everything in a heap does not look like the right approach.
However, if we still work in one large repository, svn: externals can also be useful. Example: let in our repository (in the trunk) the following folder structure: / Common / C # and / OurCustomer / Windows / HelloCSharp /. If in the HelloCSharp project we want to use code from Common / C #, then we have three ways:
The second is normal, but with minor flaws. For example, when transferring the Common / C # folder, we will be forced to change all relative paths in HelloCSharp. Or - to look at the shared files will have to walk through the folders in the repository.
The third way through svn: externals looks the most ideologically correct. We just load the common code in the specified place, in fact - just refer to it. A kind of symbolic link within SVN.
What was told in the very first example. If we want to pour the whole boost into our repository, it’s not worth it. Using svn: externals for this purpose would be more appropriate.
Pros look very convincing, but do not rush to the widespread introduction of svn: externals in their repositories. Let us evaluate the problems that svn: externals brings:
If we use external dependencies, then we can forget about the previous quick updates. Each dependency is at least one additional HTTP connection during the update. Even if nothing has changed in the external repository, we will still wait. In real work, it seems that the case is not limited to additional connections, since the update time is growing very significantly.
svn: externals is really slow. At least in the current Subversion implementation.
If you use svn: externals in this form:
this can never be guaranteed that our previously running code still works. Any change in the external repository can break it. The code may stop compiling, or hidden errors will appear, and without any of our participation.
Such an approach can hardly be recommended for use at all. Even if externals are within the same repository. Even if we completely control the external repository. With this approach, the history of the repository ceases to be unchanged. Having rolled back to some revision, we get an unpredictable result, since the code loaded via svn: externals can change in the time elapsed since the revision was created. It is very likely that previously trouble-free code simply won't compile.
However, you should not put an end to svn: externals right now, there is a solution to this problem - you need to refer to a specific revision in the external repository, like this:
Now we will always receive a specific version of the code for this revision. When we want to get updates, we update the revision number for the one that suits us and download the updates. We check that our code compiles and works as before (we run the tests). If everything is ok, we commit changes to svn: externals.
If the external server (for example, itext.svn.sourceforge.net) is unavailable, then we will not be able to upgrade to either fresh or previous revisions (if externals is also used there).
If we, of course, want all the old revisions to remain workable, otherwise we can restrict ourselves only to changes for the current revision. But this, of course, is a pernicious way.
In this case, it is not very difficult to completely modify the repository;
Both cases entail extremely negative consequences - our repository will become completely inoperable.
Example: we have the iText folder in our repository. Then, after reading one clever article, we decide to implement svn: externals instead. We successfully implement, it works, everyone is happy. However, if now we want to roll back from the current revision to some earlier revision, it will not work.
This is due to the fact that when the svn: externals installation is rolled back, the target folder (iText) is not physically deleted. And when you need to replace it with the usual tracked folder in the repository - it turns out a conflict that the current version of SVN cannot resolve.
In this case, the only solution is to make a complete checkout from the zero revision to the necessary one.
If we decide to transfer the code from the SVN repository to the repository, for example, to Git or Mercurial, then svn: externals is a serious hindrance. There are three solutions: either not to move at all, or to ignore this problem and transfer it as it will (in this case, the old revisions will not work), or completely get rid of svn: externals by hand.
The last path is thorny, but the result is worth it - we get a completely independent repository, which we can import anywhere. How specifically to get rid of svn: externals is a topic for a future article.
Use svn: externals or not - everyone decides for himself. We initially worked without external dependencies, then when decomposing projects into several repositories, we began to actively use svn: externals, but over time we realized that the disadvantages outweigh the advantages. In the end, we generally decided to migrate to Mercurial, and for this we had to completely get rid of svn: externals.
And finally, some tips on using svn: externals:
Let's look at an example of what svn: externals is. Suppose we have a project that uses third-party open-source code, for example, the well-known PDF library iText. Usually we completely copy all the necessary code of this library to our repository. Subsequently, when iText updates are released, manually replace the old files with new ones.
svn: externals provide a more convenient way. Select some folder of our repository and set the svn: externals property for it in this form ( how to do it with the help of TortoiseSVN ):
iText https://itext.svn.sourceforge.net/svnroot/itext/trunk
The first parameter determines where to download the changes, the second - from where. Now, every time we take the latest updates from the repository, the latest version of iText is automatically downloaded. Conveniently.
')
About the pros
Consider the benefits of using svn: externals.
Avoid duplication
The principle of DRY (Don't Repeat Yourself) - the cornerstone of high-quality software development - extends to source code management. If the same files are duplicated in several repositories - this is not very good.
For example, in established teams, as a rule, a base of common code is formed, which is constantly used in various projects. All projects will not necessarily be in one large repository. Most likely - in different. In this case, without svn: externals, we are doomed to copy the same files between different repositories, as well as to manually update them everywhere in case of changes.
Do not occupy a place in the repository
A direct consequence of the previous paragraph - the source code is stored only in the original repository, we only download it from there to our working copy. The size of the repository can be very critical if you use paid SVN hosting with a limit on the total size of the repository (s).
Easier to structure projects and modules
It seems reasonable to divide projects according to the rule “1 project <-> 1 repository”. With this approach, using svn: externals helps to simplify the separation of dependent projects.
Of course, we can work on all projects / modules within one repository, but this approach is not without flaws. For example, all developers can download many unnecessary changes to their working copies. It will not be possible to restrict access to developers to those projects to which they should not have access. Nearby are unrelated projects. And generally keep everything in a heap does not look like the right approach.
However, if we still work in one large repository, svn: externals can also be useful. Example: let in our repository (in the trunk) the following folder structure: / Common / C # and / OurCustomer / Windows / HelloCSharp /. If in the HelloCSharp project we want to use code from Common / C #, then we have three ways:
- copy the necessary code in HelloCSharp
- use relative paths
- use svn: externals to load code from our_repository / trunk / Common / C # into HelloCSharp / Common, for example.
The second is normal, but with minor flaws. For example, when transferring the Common / C # folder, we will be forced to change all relative paths in HelloCSharp. Or - to look at the shared files will have to walk through the folders in the repository.
The third way through svn: externals looks the most ideologically correct. We just load the common code in the specified place, in fact - just refer to it. A kind of symbolic link within SVN.
Easier to use open source code in our repository
What was told in the very first example. If we want to pour the whole boost into our repository, it’s not worth it. Using svn: externals for this purpose would be more appropriate.
About cons
Pros look very convincing, but do not rush to the widespread introduction of svn: externals in their repositories. Let us evaluate the problems that svn: externals brings:
Slow updates (svn update)
If we use external dependencies, then we can forget about the previous quick updates. Each dependency is at least one additional HTTP connection during the update. Even if nothing has changed in the external repository, we will still wait. In real work, it seems that the case is not limited to additional connections, since the update time is growing very significantly.
svn: externals is really slow. At least in the current Subversion implementation.
The code may stop working by itself.
If you use svn: externals in this form:
iText https://itext.svn.sourceforge.net/svnroot/itext/trunk
this can never be guaranteed that our previously running code still works. Any change in the external repository can break it. The code may stop compiling, or hidden errors will appear, and without any of our participation.
Such an approach can hardly be recommended for use at all. Even if externals are within the same repository. Even if we completely control the external repository. With this approach, the history of the repository ceases to be unchanged. Having rolled back to some revision, we get an unpredictable result, since the code loaded via svn: externals can change in the time elapsed since the revision was created. It is very likely that previously trouble-free code simply won't compile.
However, you should not put an end to svn: externals right now, there is a solution to this problem - you need to refer to a specific revision in the external repository, like this:
iText –r 247 https://itext.svn.sourceforge.net/svnroot/itext/trunk
Now we will always receive a specific version of the code for this revision. When we want to get updates, we update the revision number for the one that suits us and download the updates. We check that our code compiles and works as before (we run the tests). If everything is ok, we commit changes to svn: externals.
If the server we are referring to is unavailable - we will not be able to upgrade
If the external server (for example, itext.svn.sourceforge.net) is unavailable, then we will not be able to upgrade to either fresh or previous revisions (if externals is also used there).
If the server we are referring to has moved, the repository must be completely modified.
If we, of course, want all the old revisions to remain workable, otherwise we can restrict ourselves only to changes for the current revision. But this, of course, is a pernicious way.
In this case, it is not very difficult to completely modify the repository;
- Dump repository using “svnadmin dump"
- In the resulting dump, find all the lines of this type:
V 66 iText –r 247 https://itext.svn.sourceforge.net/svnroot/itext/trunk PROPS-END
- Replace the path to the repository and update the number in the first line - this is the number of bytes in the property description, for example:
V 123 iText -r 155 http://new_server.com/svnroot/itext/trunk/ iTextSharp -r 155 http://new_server.com/svnroot/itextsharp/trunk/ PROPS-END
Here, for clarity, I added the second line to make it clearer how the number V is calculated. - Delete our old repository and roll out a new dump using “svnadmin load”
If the server we are referring to, changed Subversion to something else or simply stopped working, everything was gone
Both cases entail extremely negative consequences - our repository will become completely inoperable.
Update to revision may stop working
Example: we have the iText folder in our repository. Then, after reading one clever article, we decide to implement svn: externals instead. We successfully implement, it works, everyone is happy. However, if now we want to roll back from the current revision to some earlier revision, it will not work.
This is due to the fact that when the svn: externals installation is rolled back, the target folder (iText) is not physically deleted. And when you need to replace it with the usual tracked folder in the repository - it turns out a conflict that the current version of SVN cannot resolve.
In this case, the only solution is to make a complete checkout from the zero revision to the necessary one.
Impossibility of correct migration to another version control system
If we decide to transfer the code from the SVN repository to the repository, for example, to Git or Mercurial, then svn: externals is a serious hindrance. There are three solutions: either not to move at all, or to ignore this problem and transfer it as it will (in this case, the old revisions will not work), or completely get rid of svn: externals by hand.
The last path is thorny, but the result is worth it - we get a completely independent repository, which we can import anywhere. How specifically to get rid of svn: externals is a topic for a future article.
Conclusion
Use svn: externals or not - everyone decides for himself. We initially worked without external dependencies, then when decomposing projects into several repositories, we began to actively use svn: externals, but over time we realized that the disadvantages outweigh the advantages. In the end, we generally decided to migrate to Mercurial, and for this we had to completely get rid of svn: externals.
And finally, some tips on using svn: externals:
- Use only references to specific revisions in the external repository;
- Try not to use too many external dependencies, otherwise the updates will last for a very long time;
- Use svn: externals for only one root folder (for example, trunk), and do not set this property for multiple folders scattered across the repository. Since the target folder can be set using relative paths (for example, Common / Java / iText), it is much more convenient to define all dependencies in one place than in several;
- If you install / update svn: externals for a folder - do not mix these changes with code changes, file transfers and other actions. It is better to make a separate commit for just changing the svn: externals property. This will greatly simplify life if you ever decide to completely get rid of svn: externals;
- Try too often not to change svn: externals, it will also simplify life when getting rid of external dependencies.
Source: https://habr.com/ru/post/109291/
All Articles