Project refactoring in SVN using ANT
The article describes a method for separating logic and implementing logic in ant scripts, used to solve one practical task of refactoring a large project in the SVN repository.
There is a project in SVN of 15,000 files and 5,000 folders. The project is almost 10 years old; several generations of developers with different qualifications worked on it. At some point, a couple of years ago, and maybe earlier, the architecture of the project “flowed”. Different modules and layers began to be written in different standards for organizing code, and cyclical dependencies between modules arose. As a result, a landfill has been formed in SVN over the years. The project is being assembled, but in a completely shamanic way.
Bring the code to a single storage format. At the same time save the history of changes for each file and do not stop the development process.
')
Saving history by file or folder in SVN is quite simple with the svn copy command . With a small number of files, everything can be done manually.
With the analysis of a large project is difficult. While you manually disassemble 15,000 files, the developers will commit new changes and they will also need to be copied. Vicious circle.
Need automation. Skriptik, which time! - and translates the project into a new structure.
The task was completed, and the by-product was the approach to writing ANT scripts, which in large programming is called encapsulation. I want to share the received approach.
Source codes (significantly shortened, without company and product names) are available in Google Code .
Made a plan of action:
The description of the new structure from point one is beyond the scope of this article.
The script for copying can be written in bat, bash.
Since we use the ANT assembly tool, I started writing it on it.
In my case, it took a couple of weeks to write the ANT script and get the first successful result.
The script consisted of commands of this type:
It turned out very long, poorly readable and modifiable.
He had one virtue - he worked. But I had to write a similar file for svn copy ... And it was a bit annoying.
After intense thought, I decided that I should write differently. Get rid of copy-paste, make a mapping into a separate file and hide the implementation in include files. But how to do it in ANT? Using macros for ANT!
Using the macrodef directive, we define a new task in ant.macros
Task copy_by_pattern copies files by include mask.
The code above is converted to
We want it to work as before.
And then we write the implementation copy_by_pattern for svn copy in svn.macros
This task gets the fileset and converts its local variable (appeared in ANT 1.8) out.script using pathconvert and regexpmapper . Then it displays a fragment of the script with svn copy in the file $ {script.filename}
Note: Before ANT 1.8, there were no local variables . Having written down in variable value, it cannot be changed then.
An example of a fragment of a script with svn copy obtained at the output:
key --parents will create all the necessary new directories
She has 4 options for copying:
The most convenient option is WC -> WC :, because everyone else commits to the repository immediately. Shuffling 15,000 project files will create 15,000 commits. The WC -> WC option allows you to make one pending commit.
Review your ANT scripts.
If you see many duplicate fragments in your ANT scripts,
They can be combined into a new task using macrodef.
And at the same time you can separate the logic from the implementation.
Prehistory
There is a project in SVN of 15,000 files and 5,000 folders. The project is almost 10 years old; several generations of developers with different qualifications worked on it. At some point, a couple of years ago, and maybe earlier, the architecture of the project “flowed”. Different modules and layers began to be written in different standards for organizing code, and cyclical dependencies between modules arose. As a result, a landfill has been formed in SVN over the years. The project is being assembled, but in a completely shamanic way.
Task
Bring the code to a single storage format. At the same time save the history of changes for each file and do not stop the development process.
')
Difficulties
Saving history by file or folder in SVN is quite simple with the svn copy command . With a small number of files, everything can be done manually.
With the analysis of a large project is difficult. While you manually disassemble 15,000 files, the developers will commit new changes and they will also need to be copied. Vicious circle.
Need automation. Skriptik, which time! - and translates the project into a new structure.
Result
The task was completed, and the by-product was the approach to writing ANT scripts, which in large programming is called encapsulation. I want to share the received approach.
Source codes (significantly shortened, without company and product names) are available in Google Code .
- mapping.xml - mapping, from where and where to copy, according to what rule
- old2new.properties - global config.
- macros.ant.xml - the included ant file with the file system copying implementation
- macros.svn.xml - an included ant file with copying implementation with svn copy.
The story of how I came to such scripts.
Made a plan of action:
- come up with a new structure
- write a script to copy in the project file system from the old structure to the new
- debug on a new structure, write build scripts, etc. - debugging can take a lot of time and dozens (if not hundreds) of starting copy operations in the file system
- write a script that prepares a script with svn copy
- debug on new structure in svn
- declare the hour X, run the script and transfer all developers to the new structure.
Implementation of the action plan:
The description of the new structure from point one is beyond the scope of this article.
The script for copying can be written in bat, bash.
Since we use the ANT assembly tool, I started writing it on it.
In my case, it took a couple of weeks to write the ANT script and get the first successful result.
The script consisted of commands of this type:
<mkdir dir="@{to}"/> <copy todir="@{to}" overwrite="false"> <fileset dir="@{from}"> <include name="@{include}"/> <include name="@{include}"/> <exclude name="@{include}"/> </fileset> </copy> It turned out very long, poorly readable and modifiable.
He had one virtue - he worked. But I had to write a similar file for svn copy ... And it was a bit annoying.
After intense thought, I decided that I should write differently. Get rid of copy-paste, make a mapping into a separate file and hide the implementation in include files. But how to do it in ANT? Using macros for ANT!
Using the macrodef directive, we define a new task in ant.macros
<macrodef name="copy_by_pattern"> <attribute name="from"/> <attribute name="to"/> <attribute name="include"/> <sequential> <echo message="copy from dir @{from} to dir @{to} by pattern @{include}"/> <mkdir dir="@{to}"/> <copy todir="@{to}" overwrite="false"> <fileset dir="@{from}"> <include name="@{include}"/> </fileset> </copy> </sequential> </macrodef> Task copy_by_pattern copies files by include mask.
The code above is converted to
<copy_by_pattern from="${dir.input.base}/module1" to="${dir.output.base}" include="**"/> <copy_by_pattern from="${dir.input.base}/module2" to="${dir.output.base}" include="**"/> <copy_by_pattern from="${dir.input.base}/module3" to="${dir.output.base}" include="**"/> We want it to work as before.
And then we write the implementation copy_by_pattern for svn copy in svn.macros
<macrodef name="copy_by_pattern"> <attribute name="from"/> <attribute name="to"/> <attribute name="include"/> <sequential> <fileset id="localfs" dir="@{from}"> <include name="@{include}"/> </fileset> <local name="out.script"/> <pathconvert property="out.script" refid="localfs" pathsep="${line.separator}"> <chainedmapper> <identitymapper/> <regexpmapper from="@{from}/(.*)" to="${cli.command} @{from}/\1 @{to}/\1" handledirsep="yes"/> </chainedmapper> </pathconvert> <echo file="${script.filename}" message="${out.script}${line.separator}" append="true"/> </sequential> </macrodef> This task gets the fileset and converts its local variable (appeared in ANT 1.8) out.script using pathconvert and regexpmapper . Then it displays a fragment of the script with svn copy in the file $ {script.filename}
Note: Before ANT 1.8, there were no local variables . Having written down in variable value, it cannot be changed then.
An example of a fragment of a script with svn copy obtained at the output:
svn copy --parents D:\path_old\app.xml D:\path_new\app.xml
svn copy --parents D:\path_old\Cls.java D:\path_new\Cls.java
svn copy --parents D:\path_old\page.jsp D:\path_new\page.jsp
svn copy --parents D:\path_old\js.js D:\path_new\js.js
key --parents will create all the necessary new directories
Svn copy command notice
She has 4 options for copying:
WC -> WC: copy and schedule for addition (with history)
WC -> URL: immediately commit a copy of WC to URL
URL -> WC: check out URL into WC, schedule for addition
URL -> URL: complete server-side copy; used to branch and tag
The most convenient option is WC -> WC :, because everyone else commits to the repository immediately. Shuffling 15,000 project files will create 15,000 commits. The WC -> WC option allows you to make one pending commit.
Conclusion
Review your ANT scripts.
If you see many duplicate fragments in your ANT scripts,
They can be combined into a new task using macrodef.
And at the same time you can separate the logic from the implementation.
Source: https://habr.com/ru/post/119013/
All Articles