Online implementation of localStorage

I want to share how private Safari mode led to the development of a simple key-value storage on Node.js with backup, access to data from specific domains and password protection from writing and clearing the storage.



It all started with the fact that I was given the task to implement a test order in a web application that is built in through an iframe in one popular resource.
')
The task was solved and worked as follows:

1. unauthorized user clicks on the store (link "_blank");
2. In the new window, test products are displayed, and in iframe we redirect the user to the test user profile and wait for the purchase data to appear in localStorage;
3. after making a purchase, we save data about it in localStorage (amount, quantity, store, time of purchase and number of bonuses)
4. in the iframe, when the test purchase data appears in localStorage, we display the information in the “purchase history” block;

Everything worked in most browsers, and even in IE11, but not in Safari, whose security policy (better known as porno-mode) did not allow access to localStorage data of the same domain inside the iframe and outside (in a new window).

It is necessary to store intermediate data somewhere, to bring back developers to the task to create any API for storing data, but I didn’t get permission, I just had to find some online storage, with the possibility of creating a token for each user.

The search led me to the service keyvalue.xyz , it allows you to create a key, write and read data. And so I started for each user who decided to try a test order, create a token and transfer it to the url parameters in a new window, then with a successful test order, write the data to the repository, and already in the iframe periodically requested the data until they appear.

Everything worked, but then a message came from the tester, this time she said that the test order does not work with adblock enabled. So it was, in the console, adblock wrote that the request to the resource was blocked. I appealed to the developers of the service, with a request to make a mirror, they did not answer, I tried to make a mirror through nginx (proxy_pass), did not help either, most likely because of the cloudflare filter.

It was not nice, you had to get out of the situation.

I decided to write a simple key = value storage similar to localStorage, with backup, access from a specific domain, password-protected write access and a convenient library for working with it.

Development

It’s easy to write simple rest api on Node.js using express, I chose MongoDB for storing data, because there is no rigid structure and you can change the structure of the document with just one line of code in the diagram and of course that mongodb can work with large documents size (100-200GB).

It does not make sense to talk in detail about development, it is very simple and most of us have already used the express framework.

Let's start with the basic storage requirements:

1. Creating a token
2. Token Update
3. Getting value from storage
4. Getting the entire repository
5. Data recording
6. Remove item
7. Cleaning storage
8. Get a list of backups
9. Restore storage from backup

The token scheme is quite simple, looks like this:

const TokenSchema = new db.mongoose.Schema({ token: { type: String, required: [true, "tokenRequired"] }, connect: { type: String, required: [true, "connectRequired"] }, refreshToken: { type: String, required: [true, "refreshTokenRequired"] }, domains: { type: Array, default: [] }, backup: { type: Boolean, default: false }, password: { type: String }, }) 

Extra options:

token	Used to access the repository
connect	Property for storage token token
refreshToken	Update token in cases if you need to update a token or a token somewhere lit up, for example in git commit
domains	An array of domains, access to the repository, which is allowed. The HTTP header is used for verification.
backup	If set to true, then every 2 hours will be executed. backup of the entire repository, that is, several backup copies are always available during the day, to which you can roll back
password	Set password for write and delete

Examples of requests will be given using the axios library, in contrast to the curl command, its code is rather concise and understandable.

 axios.post('https://storage.hazratgs.com/create', { domains: ['example.com', 'google.com'], backup: true, password: 'qwerty' }) 

As a result of execution, we will get a response with a token that can be used to write and read data from the repository:

 { "status": true, "data":{ "token": "002cac23-aa8b-4803-a94f-3888020fa0df", "refreshToken": "5bf365e0-1fc0-11e8-85d2-3f7a9c4f742e", "domains": ["example.com", "google.com"], "backup": true, "password": "d8578edf8458ce06fbc5bb76a58c5ca4" } } 

Writing data to the repository:

 axios.post('https://storage.hazratgs.com/002cac23-aa8b-4803-a94f-3888020fa0df/set', { name: 'hazratgs', age: 25, city: 'Derbent' skills: ['javascript', 'react+redux', 'nodejs', 'mongodb'] }) 

Getting item from storage:

 axios.get('https://storage.hazratgs.com/002cac23-aa8b-4803-a94f-3888020fa0df/get/name') 

As a result, we get:

 { "status": true, "data": "hazratgs" } 

The rest of the examples you can see on the project page in GitHub. .

You can clone the repository and deploy the repository at your place, detailed instructions in the project repository.

For convenience, JavaScript library and Python library are available.
The storage itself is available at storage.hazratgs.com

An example of working with the library

And so we already have a repository and a library to work with it, let's install the library:

 npm i online-storage 

Import to project:

 import onlineStorage from 'online-storage' 

I want to note, since we do the localStorage implementation online, in the project code we return the object, not the class by default, in order to work with one data source for the entire project, if you need several objects, you can import the OnlineStorage class itself and create as many objects as possible.

Create a token:

 onlineStorage.create() 

I must say that onlineStorage is an asynchronous method, like almost all methods of an onlineStorage object, so the best option would be to use the syntax async / await.

After creating a token, it is written to the token property and then substituted if necessary, for example, a data record:

 await onlineStorage.set({ name: 'hazratgs', age: 25, city: 'Derbent' skills: ['javascript', 'react+redux', 'nodejs', 'mongodb'] }) 

reading data:

 const order = await onlineStorage.get('name') // hazratgs 

property removal:

 await onlineStorage.remove('name') 

Now we can safely say that we have online implementation of localStorage and even more, so localStorage works only with strings, and our repository works to store strings, numbers, objects, and logical types.

Conclusion

As a result, we have data storage with backup, access from certain domains and password protection, but I want to say that this storage as well as localStorage is not secure and is not intended for storing application basic data on which the health of the project, user data and much more that could harm your application.

Use it only for irrelevant and publicly available data, as in my example, to transfer test purchase data from a window to an iframe.

Problems can arise due to the fact that the token, like all javascript, we pass to the client and it’s not difficult to get the data of the entire repository, this is not a problem of our repository, any api-key transferred to the client becomes public and although we have some protection in the face of working with certain domains and passwords, all this can most likely be circumvented.

In order to hide a token, of course, you can write a wrapper over api on your server, but this is already, it is easier to set up your database.

Please do not scold strongly, this is my first publication and contribution to open source.
I will be very grateful in helping to fix vulnerabilities, tips, pull requests.