Rare features of nginx: mod_zip

Once I started talking about rare and useful modules for nginx - I will tell you about the online ZIP file wrapper for nginx. It allows you to create a ZIP archive on the fly from a given list of files with the possibility of multi-threaded download and file resume, but without compression.

Usage example

There is a great service for storing user files. One of the features of our filesharing is that you can upload a torrent file, and our system will download and upload it to the server itself. Very often, downloaded torrents are divided into dozens of files of 10-100 MB and to download them all - the user will have to click 10 to 100 times on each file to download the archive of the target. But with the help of nginx mod_zip we allow the user to click only once and just enjoy the speed of the jump;) (those people who say that it’s easier to teach the system to unzip such files yourself - let them read about the Zip Bomb )

pros

1. Save space and CPU time for creating a static ZIP file.
2. Downloading such a ZIP file does not take up more resources than downloading a single static file from nginx
')

Minuses

1. You need to know the exact size, path and crc32 file. (If you do not specify crc32 - the resume and multithreading will not be supported)
2. Version 1.1.4 does not support the total archive weight greater than 4 GB . The code that fixes this has already been sent to the author of the plugin for review. For patchik many thanks to vasfed habraiser . This code will soon be included in the main branch.
3. There is a confirmed bug with a multi-threaded download, which was promised to be fixed in version 1.1.5 (in some cases, the file is downloaded broken)

How to start using?

1. Download mod_zip 1.1.4 and recompile nginx with the key --add-module = / path / to / mod_zip-1.x
2. In your script, transfer the header: “X-Archive-Files: zip” (so mod_zip will understand that it’s time to get to work)
3. List the required files of the form:

1034ab38 428 /foo.txt My Document1.txt
83e8110b 100339 / bar.txt My Other Document1.txt

4. Download the ZIP file and enjoy.

Passed rake

1. If at least one of the files transferred by you is not on the disk, the entire archive is downloaded as broken, therefore, you should check the availability of files upon request of the ZIP file.
2. If you incorrectly transferred the file size - the archive will be downloaded as broken.
3. Different utilities generate different crc32 values. We stopped at the “perl-Archive-Zip” (Fedora) package, which includes the crc32 utility written in pearl.

ps about new mod_zip versions - I will report to the nginx mailing list