RealSync - one-way real-time source synchronization for web developers

I present to the public the RealSync utility (GPL). Her vocation is to facilitate the work of those who periodically suffer from the lags of the Samba network folder when searching / editing web project files. The idea of ​​RealSync is that you now work with site files on a local machine in a familiar IDE, and you still see the result on a remote development web server where RealSync copies changes in real time. As a result, you can, for example, run a search on all files in the IDE - they are local, and not connected via a network folder via Samba, so the search works very quickly; while your Ctrl + S continues to get to the server instantly, as when working through a network folder.

RealSync is a utility for Windows, MacOS and Linux that allows you to keep an exact copy of files on a remote server (for example, scripts in PHP, Python, Ruby, etc.) from a folder on your local computer, even in conditions of poor communication when you working from home. All changes made in the local folder get to the server almost instantly (a delay of about 0.2 s), regardless of how many of these changes and exactly how they were made (at least through IDE, at least through Notepad or Far).

The main difference between RealSync and its analogs is that it is extremely resistant to instability of the Internet connection, reconnects and timeouts. It uses an SSH connection, the access through which is configured automatically when the utility is first run (i.e., you don’t have to mess around with the keys — the setting is done interactively).
')
In fact, accidentally "kill" RealSync is almost impossible. You can keep it constantly minimized and forget about its existence (it almost does not eat the CPU). If the utility sees that the connection is broken for a long time, the RSYNC algorithm, familiar to many, is automatically launched to quickly copy a large number of differences. In real-time mode, your own protocol is used over SSH, so that when you press Ctrl + S in the editor, you will immediately see the changes on the server. The file transfer is accompanied by a pleasant “clutter” (turned off if necessary in the config), and the temporary loss of communication is due to the icon redness (when the connection is restored, the icon will turn gray again, and RealSync will “catch up” the accumulated changes).

And why this bike, when there is Samba or Denver or XAMPP?

Generally speaking, there are several ways to develop web scripts. The first way is to use a local web server. This method has both a lot of advantages (greater control, improved portability and cross-platform, etc.), and a lot of disadvantages, such as: the potential difference between the configuration of the local server and the configuration of the virgin and production zones, the need to either monitor the synchronism of the local SQL databases, or wait until it slows down access to a single dev base on the Internet, etc. We will not consider this method in this article, although it certainly has the right to life (and, at least, 1 million registered users of the same Denver apply).

The second way is to use a web server installed in the company's office (or on a test machine in a data center), served by full-time admins. There can be several directories on such a web server, one for each developer. Each programmer works in his own directory so as not to interfere with others, and looks at the result on his subdomain (or on his port number). Here we will consider this method.

Using Remote Folders for Web Development

After all, the directory on a remote server needs to be “accessed” somehow. In the most common case, it is simply connected as a network folder to the local machine via Samba. This method has several disadvantages at once:

1. For decent speed, you must be on the same local network as the server.
2. But even in this case, it will be quite difficult for you to run a search on all files of a large project in the IDE - too much delay.
3. In the event of a disconnection (or you, God forbid, work from home via the Internet) - be prepared for strange effects and unpleasant lags.

^{I know many people who go to the server via SSH to get around the inconvenience (2) and use grep to search through files in the console, and they themselves work in the IDE. It seems to me the height of inconvenience. Moreover, in phpStorm, for example, searching all local files of even a very large project works significantly faster than grep and takes 1-2 seconds (as they achieved this is a mystery to me; apparently, they index something in advance). Personally, I constantly and comfortably use phpStorm search in local files - an average of 50 times per day.}

There are, of course, lovers working in VIM in the SSH console right on the server; I respect them, but I hardly ever join their camp: after all, the amenities provided by a good graphical IDE outweigh for me.

By and large RealSync is a solution to IDE speed problems.

So, to solve the problem of search speed, you need to store the sources on a local disk and look for them already. And the question immediately arises how the sources will get to the remote web server when you press Ctrl + S. There are several options.

1. Almost all IDEs support the transfer of a modified file (via FTP or SSH) to a remote server by pressing Ctrl + S. But only here are two problems. First of all, you will not be able to modify the files “by” the IDE (for example, do git pull from the command line on the local machine). Secondly, if you disappear from the reality of the Internet for a few days, and then come back with the changed files, the IDE will not be able to determine which files have changed (especially if they are accidentally changed on the server for some reason independently of you), and the best case will start a full copy, which will take half an hour.
2. Run in the RSYNC perpetual loop. It works more or less when the project is not very large, when the Internet is good and when you are not sitting on Windows (in Windows RSYNC is very slow at startup). In all other cases, be prepared for 2-3-second delays.
3. Use a special synchronization utility, such as RealSync, Unison, WinSCP, etc.

RealSync was conceived as a tool that ensures stable operation in conditions of poor communication and does not require attention from the programmer (“launched and forgotten”). This is exactly what it turned out, unlike numerous analogues.

And if I need two-way synchronization?

This is a very popular question related to the fact that RealSync always overwrites any changes made directly in the folder on the server (if not immediately, then at the next reconnect for sure). Only changes in the local folder have meaning and priority.

But let's think about why we might need two-way synchronization?

• Do you want to work, for example, with Git in the console on the server? But why? After all, you can put Git on your machine and enjoy all the benefits of local work and different GUIs for Git.
• Do your scripts write to some local folder within the document directory? Add this folder to the RealSync exceptions (this is done easily, in a single line in the config file or at the very first launch). In addition, to make the document directory writable by scripts is an antipattern.
• Do you work on a laptop at home and on another desktop - at work and want to synchronize files through the web server folder? But the web server is not a means of synchronization and version control. Use better Dropbox or commit to the version control system.
• Are you actively using symlinks in a working copy on the server? Well ... and did not try to stop using them, will it at least simplify support and deployment in the future?

In general, you need to understand that RealSync is not a synchronizer, it is a replicator . If you stop perceiving a folder on a web server as something independent, and consider it just an automatic “mirror” and stop walking your hands on it over SSH (really, why do it?), Everything falls into place. And in most cases, it turns out that two-way synchronization is simply not needed (but if you still think otherwise, use Unison — if you get it, of course.)

RealSync installation and first launch

Everything is quite simple: first you need to download RealSync and copy it somewhere, and then somewhere (at Startup or on the Desktop) place a shortcut with the following command line:

 "C: / Program Files / dklab_realsync / realsync.exe" Full Path Up to Your FoldersIshodniki
     - or for MacOS and Linux -
 perl / opt / dklab_realsync / realsync Full Path to Your FoldersIshodniki

When you first start, you will get something like this in which you will need to interactively enter the server address, your SSH login and password on it (the password is requested only by the SSH process and is not stored on the local machine, see below), the folder name on the remote server, etc.

_{If you already have passwordless access to the server using an SSH key, the password will not be requested at all, even for the first time. If you don’t have it yet - RealSync will generate the public + private key itself, the public will rely on the server, and the private will retain to your home directory on the local machine (in a special folder perceived by RealSync).}

The rest of the configuration settings will be recorded in the Full Path file, to the Present / .realsync Folders, so that the next time you start, no questions will be asked , but the synchronization will start and the program will be minimized to tray (for Windows).

Now you can edit any source and make sure that they instantly fall on the server. If you click on the icon of the launched RealSync in the tray, then the work process can be observed approximately in this window:

^{The RealSync distribution for Windows is self-contained: it already includes RSYNC, SSH, a mini-version of Perl, and other necessary utilities, so all you need to do is simply run realsync.exe and not think about it. On MacOS and Linux, all these utilities, as a rule, are already in the system, which is why they are used.}

So, RealSync hangs in the tray, you never turn it off, so you have a complete illusion that you are working on a local machine with a remote folder. At the same time, you have all the advantages of local search and IDE performance. That was the goal of developing RealSync .

I would be glad if the utility makes someone else's daily work easier. Comments are welcome. GitHub-e project: dklab_realsync .