Another implementation of Data Processing

Imagine a tool with which you can transform any data into ... any data.

The most accessible example is Images. What if I want to get a picture of a certain size and put a watermark somewhere (also of a certain size) somewhere.

And here is one example of how this might look like :
')


PS: This is a review article, if it will be interesting, I will further describe the approaches in more detail.

Motivation

Recently, in November 2018, I needed to resize a photo ...

Online services because of their "features" (advertising, strange interface, etc.) made this process as painful as possible.

Standard Windows 10 tools seemed to me not obvious. I had to download GIMP and from there I had to do everything I needed - to change the size of the photo.

In my defense:

1. Problem time ~ 1AM (one in the morning)
2. There was no working laptop at hand
3. Windows at home for game only. Now sometimes I write code with an apple keyboard

And then another programmer decided to create another cost ... Tool.

Requirements

This Tool should be all crutches crutch:

1. Simple UI
2. Simple API
3. Convenient organization of project entities
4. Minimum buttons and settings
5. The ability to implement their ... "standards"
6. Open Source: Backend and Frontend

And most importantly - all who understand what they want should be able to use it.

Several possible sequences:

1. Picture, Picture Smaller, Filter
2. Download the contents of the URL, leave only useful content, leave only text
3. Download the picture, determine that there is any object on it (template matching)

Implementation

Recently I read on Habré. Before microservices you need to grow ... and decided to follow this. There have already been a couple of attempts to make such a thing through “beautiful” (in my imagination) structures: each function (like template_match , md5 ) as a microservice optimized for solving its problem. In the end, everything will come to this, but not today :)

Total available:

1. Frontend without beautiful wallpaper
2. Backend API
3. Celery (in local development I use Redis, and in test and prod RabbitMQ)
4. Backend workers

I have to say: I am very mediocre familiar with Frontend, however, thanks to Stackoverflow and Google search.

More examples

All examples can be found here , they are combined into "projects".

One of them: extracting text from an HTML page



Attention : the site is in read-only mode, but the result will be returned to you. He will not be able to save your changes to Project and Pipeline.

Small explanation

List of what can transform the data: here

What this project is not:

1. Data storage
2. Bitcoin miner

The project is very similar to:

1. Zapier
2. Yandex Nirvana
3. Parse.com (a couple of years bought by Facebook)