Web-gui for wget (light)

Earlier there was a description of possible situations where this solution might be necessary for you, but let's omit it. The ability to easily create remote downloads that are performed by the usual wget (you can safely see their list with the help of ps ), with the progress displayed - the idea is not new. And even there are some solutions , but not actual ones, since they have not been supported by anyone for more than 5 years.

For torrents, everything is simple and trivial - we put Transmission or any similar client with a web-muzzle. But for links to simple files / pages you need something different. Here is a short list of tasks that pushed me to write it:

• I watch the movie online using a tablet, but there are some things coming up and I need to save it in order to watch it later;
• You need to download the file to the remote server, and you have to start the terminal every time;
• It would be necessary to download an image of fresh linuxmint, but on a home NAS , not a laptop, working for which this idea came;
• During surfing, there is often a problem to save the file and share it.

If it became interesting to you - welcome under kat:

System requirements

The web interface is built on Javascript + css3 from the client side, and php (selected as the most popular) from the server side. To complete the work will require:

• * nix (at least it was written for this platform, to run under another, you need working ports wget , ps and kill );
• php5.x (most likely it will work on php4.x, but it was not tested at the time of publication);
• Browser with support for Javascript (and very desirable - with support for css3 ).

')

Features and settings of the server part

As already mentioned above - in the role of the server side is a script written in php. It performs the following tasks:

• Getting information about running tasks;
• Cancel running tasks;
• Adding new tasks;
• Return results in JSON format.

For his job he needs ps, wget and kill respectively. To obtain the value of the download status (how many percent is completed), the following algorithm is used:

• Wget tasks run with the " --background " and " --progress = bar: force " flags;
• The output of the boot log is made to the file set in the parameter " --output-file = FILE ";
• When requesting the status of tasks using " ps -ax ", we get the path to the file set to " --output-file = FILE ";
• We read the last line of this file, getting the required value from it by regularity.

The path for temporary log files is set in the line " define ( 'tmp_path' , '/ tmp' ) ; ";

The path to the directory where all downloaded files will be saved is set in the line " define ( 'download_path' , BASEPATH . '/ Downloads ' ) ; ".

A convenient indication of paths to ps, wget and kill is available. To do this, it is enough to remove the comment at the beginning of the line and specify your path, for example: " define ( 'wget' , '/ usr / bin / wget' ) ; ".

It is possible to set limits on the download speed from the settings section. The line define ( 'wget_download_limit' , '1024' ) ; "is responsible for this. If you leave it commented out - there will be no restriction.

To determine the list of tasks launched through any other GUI, a specific flag unique to the GUI is used. It is set in the line define ( 'wget_secret_flag' , '--max-redirect = 4321' ) ; "and it is not necessary to change it unnecessarily. By the way, if you want your other tasks running from the terminal to be displayed in the web interface, just add this parameter to them. But do not forget that there are some other parameters that are no less mandatory (depending on the settings).

I tried to write with the most economical attitude to resources and minimal dependence on the system, but I do not have much experience in php, so I will appreciate the recommendations for optimization.

The script responds to both POST and GET requests. There is no difference between them. Also responds to parameters passed on the command line. For example: php ./rpc.php get_list or php ./rpc.php add_task http://goo.gl/5Qi0Xs .

Example of POST request and Json response:

Request:
192.168.1.2 / wget / rpc. php ? action = add_task & url = http : //mirror.yandex.ru/linuxmint/stable/17/linuxmint-17-cinnamon-dvd-64bit.iso
Answer:
{
status : 1 ,
msg : "Task added" ,
id : 10910
}

Request:
192.168.1.2 / wget / rpc.php? action = get_list
Answer:
{
status : 1 ,
msg : "Active tasks list" ,
tasks : [
{
url : " mirror.yandex.ru/linuxmint/stable/17/linuxmint-17-cinnamon-dvd-64bit.iso" ,
progress : 95 ,
id : 10910
}
]
}

Conclusion "ps -ax | grep wget "and" cat /tmp/wget{RND}.log.tmp ":

data: ~ # ps -ax | grep wget
10910 ? S < s 0 : 34 wget --progress = bar: force --background --tries = 0 --no-cache --user-agent = Mozilla / 5.0 ( X11; Linux amd64; rv: 21.0 ) Gecko / 20100101 Firefox / 21.0 --directory-prefix = / DataVolume / shares / Public / Downloads / --limit-rate = 2048k --output-file = / tmp / wget9720.log.tmp --max-redirect = 4321 http: // mirror .yandex.ru / linuxmint / stable / 17 / linuxmint- 17 -cinnamon-dvd-64bit.iso
11378 pts / 0 S + 0 : 00 grep wget

data: ~ # cat / tmp / wget9720.log.tmp
- 2014-08-24 12 : 32 : 04-- http: // mirror.yandex.ru / linuxmint / stable / 17 / linuxmint- 17 -cinnamon-dvd-64bit.iso
Resolving mirror.yandex.ru ... 213.180.204.183, 2a02: 6b8 :: 183
Connecting to mirror.yandex.ru | 213.180.204.183 | : 80 ... connected.
HTTP request sent, awaiting response ... 200 OK
Length: 1286799360 ( 1.2G ) [ application / octet-stream ]
Saving to: `/ DataVolume / shares / Public / Downloads / linuxmint- 17- cinnamon-dvd-64bit.iso`

95 % [ =================================== > ] 1 , 225 , 527 , 078 1.37M / s eta 35s
data: ~ #

Features and client settings

“New html5 tags” are not used, but css3 properties are used to design the progress bar of downloads and adaptations. The design is made in a minimalist style without images (the exception is favicon). If there are no tasks in the center of the page, there is a field for adding a download address, if there are tasks - this field is shifted to the top of the page, and the tasks are located below.

All requests are asynchronous (without reloading the page). Page design is responsive:


The state change is also displayed in the tab (window) header:


At the bottom of the page is a javascript bookmark (“Download this”), by moving it to the browser bookmarks panel, you can add new tasks with one click (the active tab will be added when you click; if the tab with the video file is open and this “bookmark” is clicked on the bookmarks - a task will be added to download this video file):


All javascript document code is located in the core file. js . In its upper part are the main settings:

• “UpdateStatusInterval = 5 * 1000 ” - the data refresh interval on the open tab (be careful with this parameter on weak servers);
• “DebugMode = false ” - debug mode, debug information is displayed in console.log;
• "Prc = 'rpc.php' " - the path to the server-side script.

I don’t see a special point in describing the functional moments, but I’ll say that the functions are divided into logical groups, the script is not minified, comments take place.

When you click on F5, a forced update of tasks occurs, the page is reloaded only by clicking on the update button in the browser.

Installation

• Download or clone the latest version of the repository;
• Unpack in a directory accessible "from the outside";
• Change the path to " define ( 'download_path' , BASEPATH . '/ Downloads ' ) ; " in "rpc.php";
• Open in the browser, check the performance. In case of errors - ask a question .

Change history

As changes will update this post

• It became too lazy to update, here is a link to the current changelog
• 0.0.9 - Added the ability to specify the name of the saved file, by adding the line "-> filename.ext" to the URL in the GUI (the full view of the request will be "htttp: //shosthost.io/oldfilename.zip -> newfilename.zip ") Added additional input validation
• 0.0.8 - Added checking the directory for temporary files, added the ability to add multiple tasks to the GUI, minor improvements (mainly in the visual part)
• 0.0.7 - A test for existence and permission to write to download_path ( http://goo.gl/I8gYoK ) has been added to the test function
• 0.0.6 - Ps aux call changed to ps -ewwo pid, args to speed up parsing (issue # 8)
• 0.0.5 - Security Updates, fixed error message when adding a task, functions return not just the result true | false - but a description, added support for working with startup parameters from the console (terminal), added a server testing function (in gui - instead of url pass the word "test"), other minor fixes
• 0.0.4 - Added automatic deletion of log files from wget, minor fixes
• 0.0.3 - Githab Release

What to expect:

• Add authorization and access by key;
• Logging (implemented);
• Automatic (or semi-automatic) cleaning of the spent log files (partially implemented);
• Browser extension (implemented);
• Detailed settings for tasks, such as the path to download and the name of the saved file (partially implemented);
• Deletion of not finished tasks;
• ...

Link to the project: https://github.com/tarampampam/wget-gui-light