Practice refactoring in large projects

Some time ago I got into game dev, where I ran into projects of 2 million lines of code written by dozens of programmers. At such a scale of the code base, problems of a previously unknown nature arise. I want to tell you about one of them now.

So imagine the following situation. It just so happens that you need to refactor a very large piece of code, the whole subsystem. Rows, commercials, at 200K. Moreover, refactoring clearly looks very large, affecting the basic concepts by which your subsystem is built. In fact, it is necessary to rewrite the entire architecture, preserving the business logic. This happens if, for example, you have done one project and you have a new one ahead, and you want to correct all the mistakes of the past in it. Suppose, according to first estimates, it is necessary to refactor month 2, no less. In the process of refactoring, everything should work, including not to prevent other programmers from adding new features and fixing bugs in the subsystem. Often, such refactoring is how complicated, that it is absolutely impossible to freeze the old code into a new one, and also it is impossible to roll out the result in parts. In fact, you need to replace the engine of the aircraft on the fly.

Examples from the practice, both mine and my colleagues:

• Redo all database work from pure JDBC to Hibernate.
• Transform the service architecture from sending-receiving messages to remote procedure call (RPC).
• Completely rewrite the XML file translation subsystem into runtime objects.

')
What to do? Which way to approach the problem? Below is a set of tips and practices that help us deal with this problem. First, more general words, and then specific techniques. In general, nothing supernatural, but someone can help.

Preparing for refactoring

• Break refactoring apart if possible. If possible, no problem. All other tips about what to do if it failed.
• Try to choose the period when the activity of adding new features to the subsystem will be minimal. For example, it is convenient to remake the backend, while all the efforts of the team are focused on the frontend.
• Well read the code, why is it even needed? What architecture is embedded in the base of the subsystem, what are the standard approaches that were used there. Clearly define for yourself what the new concept is and what exactly you want to change. There must be a measurable final goal.
• Determine the area of ​​code that refactoring will capture. If possible, isolate it in a separate module / in a separate directory. This will be very useful in the future.
• Refactoring without tests is very dangerous. You must have tests. How to live without them, I do not know.
• Run the tests with the calculation of coverage, it will give a lot of information for reflection.
• Fix broken tests that relate to the desired subsystem.
• Analyzing information about the coverage of methods you can find and delete unused code. Oddly enough, this often happens up to 10-15%. The more code you manage to delete, the less refactoring. Profit!
• By coverage, determine which parts of the code are not covered. It is necessary to add the missing tests. If unit tests are long and tedious, write at least high-level smke tests.
• Try to bring coverage up to 80-90% of the meaningful code. Do not try to cover everything. Kill a lot of time with little benefit. Cover at least the main execution path.

Refactoring

• Wrap your subsystem interface. Translate all external code to use this interface. This not only boosts the use of good programming practices, but also simplifies refactoring.
• Make sure your tests test the interface, not the implementation.
• Make it possible at startup to indicate which implementation of this interface to use. This opportunity needs to be supported both in tests and in production.
• Record the revision of the version control system on which the writing of the new implementation began. From this second on, every commit to your old subsystem is your enemy.
• Write a new implementation of your subsystem interface. Sometimes you can start from scratch, sometimes you can use a favorite method - copy & paste. You need to write it in a separate module. Periodically drive tests on the new implementation. Your goal is to ensure that all tests pass successfully on a new and old implementation.
• No need to wait for you to write everything completely. Fill the code in the repository as often as possible, leaving the old implementation enabled. If you keep the code in store for a long time, you may experience problems with the fact that other people will refactor the modules you are using. Simply renaming a method can cause you a lot of problems.
• Do not forget to write tests specific to the new implementation, if any.
• After all write, see the history of SVN in the folder with the old code to find what has changed there during your refactoring. It is necessary to transfer these changes to the new code. In theory, if you forgot to move something, the tests should catch it.
• After that, all tests must pass with both the old and the new subsystem.
• Immediately after you are convinced of the stability of the new version of your module, switch tests and production to it. Block commits to the old subsystem. Try by all means to minimize the time of existence of the two subsystems in parallel.
• Wait a week or two, collect all the baglo, start it and safely remove the old subsystem.

Additional tips

• All new features created in parallel with refactoring should be covered by tests at 100%. This is necessary so that when switching to a new implementation, the tests drop and signal that the new implementation lacks the code from the old one.
• Any fix bug should be done according to the principle - first we write a test that will reproduce the problem and fall, then fix it. The reasons are the same.
• If you use TeamCity a la system, for the time being refactoring make a separate build, where all the tests on the new subsystem will run. Automatic build makes your new, not yet used, code "official". All the same policies and rules are beginning to apply to it as to everything else.
• It often happens that you don’t know if you fixed everything that you wanted in the old code for a new architecture. For example, you don’t know if your code uses a direct JDBC connection somewhere, instead of Hibernate. Or suddenly a message slipped somewhere, not an RPC call. To find such places you need to think of a way to make the old method inoperative. Those. break it in tests. For example, break the message delivery system or slip the system into a non-working JDBC driver. Practice shows that in this way there are usually at least 5 forgotten and uncorrected places.
• Talk to other programmers, keep them informed of your progress. If they know that you have a week left, they can sometimes move their tasks before the release of the new version of your subsystem. No need to merge changes.

Experience suggests that even scary and large subsystems can be refactored with relatively little blood. Your main assistants are tests and systematic.