Your application on Node.js with storage in Dropbox is just
Despite the fact that robots remain my main hobby, I spend a lot of effort to stay in the trends of my main path - programming. By the will of fate, I recently managed to get acquainted with Node.js, I found out about its express web framework, made friends with a new template engine Jade, and to top it all connected it all with a folder in Dropbox.
In this post I will try to briefly describe how you can organize a web service for storing files using only free solutions.
All interested - please under the cat.
So, we will need:
If everything should be clear with the first two points, then I would like to dwell on the third one. I have already mentioned that everything should turn out to be free, and I am not going to back down from my words.
In the process of my “floundering” in the Node.js world, I came across a number of platforms ready to provide Node.js server at our disposal for free. Personally, I experienced two of them: Heroku and Nodester. As a result, I still stopped at the second, although, to be honest, this decision is not justified by anything.
To register in Nodester you need to get a coupon. You can do this on their website or on the command line via nodester-cli. I received a coupon the day after the request was sent. It is very fast, although I do not exclude that I was just lucky.
We have to start with something. To do this, we will create a folder in any place convenient for us (I have habr-nodebox called) and the package.json file in it:
Fields name, version, author - just give some information about the project and can be changed without any problems; node is the version of Node.js used in the project; The dependencies section lists all used third-party modules. As I already mentioned, the project will use express and jade. The dbox plugin, as the name implies, will be used to work with Dropbox. I tried another plugin called dropbox, but, unfortunately, it did not allow to authorize the application, because it implemented the old Dropbox API, which used / token. Currently, Dropbox uses the oauth standard for authentication.
After saving this file in the command line, call:
')
If everything was written correctly, then npm will download all modules mentioned in dependencies and install them in the current directory.
In addition, we will create two more folders public and view. The first will be used for static files (CSS, JS and others), while the view will be used for Jade templates.
If we want to be able to add some files to Dropbox, we need to perform several actions, the first of which will be to get the key and the secret string for our application. To do this, go to the application page of our Dropbox account through a browser and create a new application there. In the Access type field, set the value of the App folder (the application will have limited access only to its own folder in Dropbox).
On the app page, write yourself somewhere App key and App Secret. That's actually the first step already passed.
To automate the next steps in authorization, I suggest writing a small script for the same node.js. The script is as follows (dbox-init.js in the folder of our application):
In order to run the script just type in the command line:
The script runs along with you all the stages of oauth authentication in Dropbox and helps to get all the keys we need. The steps are as follows:
At this preparatory work passed, we can proceed to writing the application itself.
We turn, in my opinion, to the most interesting - writing the controller. The main actions are: view all files, add new ones, receive existing ones. I will not torment and immediately provide the code (web.js).
I think a little clarification of what is happening here will not hurt.
Announcement of the main building blocks of our application:
Initialization of our web application. Here we set the basic parameters: the paths to static files (public), the directory for templates (views), the engine of these templates (jade), disable the main layout, to simplify writing a little and get along with one template, and finally pass to our application bodyParser, which will parse the body of incoming requests.
And now for the main magic. Next will follow the three main handlers of our service.
This code is responsible for the main page. On it we will display a list of files in our Dropbox folder, so we make a request through the client and get the meta information. It contains information about all files in our application. We pass this information to our template engine, which will render the page.
The method written above will process file requests. Everything is very simple - we get the file name and send the request to Dropbox. The received answer is redirected to the user.
Almost all the work for us did express. The file sent in the post request body to the server has been temporarily saved to the file system. We were presented with all the necessary information for us in the req.files ['file-input'] object, where file-input is the name attribute of the input element of the form in html. We just have to take the file from the file system and send it to Dropbox. After that we will be redirected to the main page.
At the end we set the port for our application, the default value will be 5000. It remains only to write one simple page using the jade template, and we’ll do this.
The first acquaintance with Jade for me personally was painful. Plain html is somehow closer.
However, the discomfort quickly passed. The second page was written without hostility. In general, you get used quickly. I recommend to try.
To discuss jade convenience is certainly great, but it's time to show the code. To do this, create an index.jade file in the views folder and write in it:
I tried to make the template more transparent and understandable. Of course, to achieve outstanding stylistic results with this will not work, but you will not sacrifice anything for the sake of clarity.
We just created a simple HTML page with a list of files in our folder. To do this, we went through the each loop through all the file entries. Let me remind you that content metadata was carefully obtained for our template in the controller. Each element of the list is a link leading us to the controller method for queries of the form “/: path”. Following the list is a form for downloading new files. It consists of two input elements, one for the file and one for the form submission.
That's actually our application and it is ready, it remains only to start it. Maybe someone noticed when he read that all the keys to Dropbox were written as variables of the process.env [] array. This is done in order not to leave them in the code and be able to publish the application without fear of being compromised. In order to transfer your values ​​to the process.env array, it is enough to write them in the form of key = value before calling node. The command line should get something like this:
On services such as Nodester, it is possible to install process.env, so this was the only way, without any problems, that came to my mind.
The result, after adding several files, will look like this.
That's all, if someone has a desire to run such code online - just upload it to any Node.js server. I tested on Nodester - everything worked, I think the rest will not cause problems either.
Code can be found here.
In this post I will try to briefly describe how you can organize a web service for storing files using only free solutions.
All interested - please under the cat.
Prepare a bridgehead
So, we will need:
- Node.js installed on the local machine
- Dropbox Account
- Application Node.js server (if you want to start the service not only locally)
If everything should be clear with the first two points, then I would like to dwell on the third one. I have already mentioned that everything should turn out to be free, and I am not going to back down from my words.
In the process of my “floundering” in the Node.js world, I came across a number of platforms ready to provide Node.js server at our disposal for free. Personally, I experienced two of them: Heroku and Nodester. As a result, I still stopped at the second, although, to be honest, this decision is not justified by anything.
To register in Nodester you need to get a coupon. You can do this on their website or on the command line via nodester-cli. I received a coupon the day after the request was sent. It is very fast, although I do not exclude that I was just lucky.
Create a project
Locally
We have to start with something. To do this, we will create a folder in any place convenient for us (I have habr-nodebox called) and the package.json file in it:
{ "name": "habr-nodebox", "version": "0.0.1", "node": "0.6.17", "author": "andbas", "dependencies": { "express": "2.5.x", "jade": "0.26.x", "dbox": "0.4.x" } } Fields name, version, author - just give some information about the project and can be changed without any problems; node is the version of Node.js used in the project; The dependencies section lists all used third-party modules. As I already mentioned, the project will use express and jade. The dbox plugin, as the name implies, will be used to work with Dropbox. I tried another plugin called dropbox, but, unfortunately, it did not allow to authorize the application, because it implemented the old Dropbox API, which used / token. Currently, Dropbox uses the oauth standard for authentication.
After saving this file in the command line, call:
')
npm install If everything was written correctly, then npm will download all modules mentioned in dependencies and install them in the current directory.
In addition, we will create two more folders public and view. The first will be used for static files (CSS, JS and others), while the view will be used for Jade templates.
Meanwhile in Dropbox
If we want to be able to add some files to Dropbox, we need to perform several actions, the first of which will be to get the key and the secret string for our application. To do this, go to the application page of our Dropbox account through a browser and create a new application there. In the Access type field, set the value of the App folder (the application will have limited access only to its own folder in Dropbox).
On the app page, write yourself somewhere App key and App Secret. That's actually the first step already passed.
To automate the next steps in authorization, I suggest writing a small script for the same node.js. The script is as follows (dbox-init.js in the folder of our application):
var dbox = require("dbox"), stdin = process.stdin, stdout = process.stdout; ask('App key', /^\S+$/, function(app_key) { ask('App secret', /^\S+$/, function(app_secret) { var app = dbox.app({ 'app_key': app_key, 'app_secret': app_secret }); app.request_token(function(status, request_token){ if(request_token){ console.log('Please visit ', request_token.authorize_url, ' to authorize your app.'); ask('Is this done? (yes)', /^yes$/, function(answer) { app.access_token(request_token, function(status, access_token){ console.log('app_key: ' + app_key); console.log('app_secret: ' + app_secret); console.log('oauth_token: ' + access_token.oauth_token); console.log('oauth_token_secret: ' + access_token.oauth_token_secret); console.log('uid: ' + access_token.uid); process.exit(); }); }); } }); }); }); function ask(question, format, callback) { stdin.resume(); stdout.write(question + ": "); stdin.once('data', function(data) { data = data.toString().trim(); if (format.test(data)) { callback(data); } else { stdout.write("It should match: "+ format +"\n"); ask(question, format, callback); } }); } In order to run the script just type in the command line:
node dbox-init The script runs along with you all the stages of oauth authentication in Dropbox and helps to get all the keys we need. The steps are as follows:
- the script requests the App key and the App Secret (those that we received earlier) and generates a link based on them
- we copy the link into the browser and authorize the application to work with our dropbox account, we receive a notification that the application is authorized
- to the question of the script about whether we passed the authorization boldly write “yes”
- we receive and write to the secluded place the data necessary for authorization, specifically: app_key, app_secret, oauth_token, oauth_token_secret and uid
At this preparatory work passed, we can proceed to writing the application itself.
Into the battle: we sketch the controller
We turn, in my opinion, to the most interesting - writing the controller. The main actions are: view all files, add new ones, receive existing ones. I will not torment and immediately provide the code (web.js).
var express = require('express'), app = express.createServer(express.logger()), fs = require('fs'), dropbox = require("dbox").app({"app_key": process.env['dbox_app_key'], "app_secret": process.env['dbox_app_secret'] }), client = dropbox.createClient({oauth_token_secret: process.env['dbox_oauth_token_secret'], oauth_token: process.env['dbox_oauth_token'], uid: process.env['dbox_uid']}); app.use(express.static(__dirname+'/public')); app.set('views', __dirname + '/views'); app.set('view engine', 'jade'); app.set('view options', { layout: false }); app.use(express.bodyParser()); app.get('/', function(req, res) { client.metadata(".", function(status, reply) { res.render('index', { content : reply }); }); }); app.get('/:path', function(req, res) { var path = req.params.path; client.get(path, function(status, reply, metadata){ res.send(reply); }); }); app.post('/', function(req, res) { var fileMeta = req.files['file-input']; if (fileMeta) { fs.readFile(fileMeta.path, function(err, data) { if (err) throw err; client.put(fileMeta.name, data, function(status, reply) { res.redirect('/'); }); }); } else { res.redirect('/'); } }); var port = process.env['app_port'] || 5000; app.listen(port, function() { console.log("Listening on " + port); }); I think a little clarification of what is happening here will not hurt.
var express = require('express'), app = express.createServer(express.logger()), fs = require('fs'), dropbox = require("dbox").app({"app_key": process.env['dbox_app_key'], "app_secret": process.env['dbox_app_secret'] }), client = dropbox.createClient({oauth_token_secret: process.env['dbox_oauth_token_secret'], oauth_token: process.env['dbox_oauth_token'], uid: process.env['dbox_uid']}); Announcement of the main building blocks of our application:
- express - as mentioned earlier, the web framework
- app is the actual web application itself.
- fs - file system interface
- dropbox - factory to create Dropbox client
- client - Dropbox client simplifies working with API
app.use(express.static(__dirname+'/public')); app.set('views', __dirname + '/views'); app.set('view engine', 'jade'); app.set('view options', { layout: false }); app.use(express.bodyParser()); Initialization of our web application. Here we set the basic parameters: the paths to static files (public), the directory for templates (views), the engine of these templates (jade), disable the main layout, to simplify writing a little and get along with one template, and finally pass to our application bodyParser, which will parse the body of incoming requests.
And now for the main magic. Next will follow the three main handlers of our service.
app.get('/', function(req, res) { client.metadata(".", function(status, reply) { res.render('index', { content : reply }); }); }); This code is responsible for the main page. On it we will display a list of files in our Dropbox folder, so we make a request through the client and get the meta information. It contains information about all files in our application. We pass this information to our template engine, which will render the page.
app.get('/:path', function(req, res) { var path = req.params.path; client.get(path, function(status, reply, metadata){ res.send(reply); }); }); The method written above will process file requests. Everything is very simple - we get the file name and send the request to Dropbox. The received answer is redirected to the user.
app.post('/', function(req, res) { var fileMeta = req.files['file-input']; if (fileMeta) { fs.readFile(fileMeta.path, function(err, data) { if (err) throw err; client.put(fileMeta.name, data, function(status, reply) { res.redirect('/'); }); }); } else { res.redirect('/'); } }); Almost all the work for us did express. The file sent in the post request body to the server has been temporarily saved to the file system. We were presented with all the necessary information for us in the req.files ['file-input'] object, where file-input is the name attribute of the input element of the form in html. We just have to take the file from the file system and send it to Dropbox. After that we will be redirected to the main page.
var port = process.env['app_port'] || 5000; app.listen(port, function() { console.log("Listening on " + port); }); At the end we set the port for our application, the default value will be 5000. It remains only to write one simple page using the jade template, and we’ll do this.
Jade pattern - it's all indenting
The first acquaintance with Jade for me personally was painful. Plain html is somehow closer.
However, the discomfort quickly passed. The second page was written without hostility. In general, you get used quickly. I recommend to try.
To discuss jade convenience is certainly great, but it's time to show the code. To do this, create an index.jade file in the views folder and write in it:
!!! 5 html(lang="en") head title habr-nodebox body each item, i in content.contents div a(href="#{item.path}") #{item.path} - #{item.size} div form#upload-form(action="/", method="POST", enctype="multipart/form-data") input#file-input(name="file-input", type="file") input#submit(value="Upload file", type="submit") I tried to make the template more transparent and understandable. Of course, to achieve outstanding stylistic results with this will not work, but you will not sacrifice anything for the sake of clarity.
We just created a simple HTML page with a list of files in our folder. To do this, we went through the each loop through all the file entries. Let me remind you that content metadata was carefully obtained for our template in the controller. Each element of the list is a link leading us to the controller method for queries of the form “/: path”. Following the list is a form for downloading new files. It consists of two input elements, one for the file and one for the form submission.
Launch
That's actually our application and it is ready, it remains only to start it. Maybe someone noticed when he read that all the keys to Dropbox were written as variables of the process.env [] array. This is done in order not to leave them in the code and be able to publish the application without fear of being compromised. In order to transfer your values ​​to the process.env array, it is enough to write them in the form of key = value before calling node. The command line should get something like this:
$ dbox_app_key=abc1qwe2rty3asd dbox_app_secret=123asd123asd123 dbox_oauth_token_secret=aaabbbccc111222 dbox_oauth_token=123asd123asd123 dbox_uid=12345678 node web On services such as Nodester, it is possible to install process.env, so this was the only way, without any problems, that came to my mind.
The result, after adding several files, will look like this.
That's all, if someone has a desire to run such code online - just upload it to any Node.js server. I tested on Nodester - everything worked, I think the rest will not cause problems either.
Code can be found here.
Source: https://habr.com/ru/post/146735/
All Articles