Creating a REST API on Falcon

Translation of the article Create a scalable REST API with Falcon and RHSCL by Shane Boulden.

In this article, we will create a REST API based on the Python Falcon framework, test performance and try to scale it to cope with the loads.

To implement and test our API, we need the following components:

• Falcon
• PostgreSQL
• Taurus

Why Falcon?

Falcon is a minimalistic web framework for building a web API, according to the Falcon website, it is up to 10 times faster than Flask. Falcon quick!

Start

I assume that you have already installed PostgreSQL (where are we without it). We need to create a database orgdb and user orguser .
')
This user needs in the PostgreSQL settings in the pg_hba.conf file to register password access to the newly created database and issue all rights.

Database configuration complete. Let's move on to creating our Falcon application.

API creation

For our application we will use Python3.5.

Create virtualenv and install the necessary libraries:

$ virtualenv ~/falconenv $ source ~/falconenv/bin/activate $ pip install peewee falcon gunicorn 

Create the file 'app.py':

 import falcon from models import * from playhouse.shortcuts import model_to_dict import json class UserIdResource(): def on_get(self, req, resp, user_id): try: user = OrgUser.get(OrgUser.id == user_id) resp.body = json.dumps(model_to_dict(user)) except OrgUser.DoesNotExist: resp.status = falcon.HTTP_404 class UserResource(): def on_get(self, req, resp): users = OrgUser.select().order_by(OrgUser.id) resp.body = json.dumps([model_to_dict(u) for u in users]) api = falcon.API() users = UserResource() users_id = UserIdResource() api.add_route('/users/', users) api.add_route('/users/{user_id}', users_id) 

Now we will describe the models in the 'models.py' file:

 from peewee import * import uuid psql_db = PostgresqlDatabase( 'orgdb', user='orguser', password='123456', host='127.0.0.1') def init_tables(): psql_db.create_tables([OrgUser], safe=True) def generate_users(num_users): for i in range(num_users): user_name = str(uuid.uuid4())[0:8] OrgUser(username=user_name).save() class BaseModel(Model): class Meta: database = psql_db class OrgUser(BaseModel): username = CharField(unique=True) 

We have created two helper methods for setting up the 'init_tables' and 'generate_users' applications. Run them to initialize the application:

 $ python Python 3.5.1 (default, Sep 15 2016, 08:30:32) Type "help", "copyright", "credits" or "license" for more information. >>> from app import * >>> init_tables() >>> generate_users(20) 

If you go to the orgdb database, you will see the created users in the orguser table.

Now you can test the API:

 $ gunicorn app:api -b 0.0.0.0:8000 [2017-12-11 23:19:40 +1100] [23493] [INFO] Starting gunicorn 19.7.1 [2017-12-11 23:19:40 +1100] [23493] [INFO] Listening at: http://0.0.0.0:8000 (23493) [2017-12-11 23:19:40 +1100] [23493] [INFO] Using worker: sync [2017-12-11 23:19:40 +1100] [23496] [INFO] Booting worker with pid: 23496 $ curl http://localhost:8000/users [{"username": "e60202a4", "id": 1}, {"username": "e780bdd4", "id": 2}, {"username": "cb29132d", "id": 3}, {"username": "4016c71b", "id": 4}, {"username": "e0d5deba", "id": 5}, {"username": "e835ae28", "id": 6}, {"username": "952ba94f", "id": 7}, {"username": "8b03499e", "id": 8}, {"username": "b72a0e55", "id": 9}, {"username": "ad782bb8", "id": 10}, {"username": "ec832c5f", "id": 11}, {"username": "f59f2dec", "id": 12}, {"username": "82d7149d", "id": 13}, {"username": "870f486d", "id": 14}, {"username": "6cdb6651", "id": 15}, {"username": "45a09079", "id": 16}, {"username": "612397f6", "id": 17}, {"username": "901c2ab6", "id": 18}, {"username": "59d86f87", "id": 19}, {"username": "1bbbae00", "id": 20}] 

Testing API

Evaluate the performance of our API using Taurus . If possible, you must deploy Taurus on a separate machine.

Install Taurus in our virtual environment:

 $ pip install bzt 

Now we can create a script for our test. Create a bzt-config.yml file with the following content (do not forget to specify the correct IP address):

 execution: concurrency: 100 hold-for: 2m30s ramp-up: 1m scenario: requests: - url: http://ip-addr:8000/users/ method: GET label: api timeout: 3s 

This test will simulate web traffic from 100 users, with an increase in their number for a minute, and keep the load for 2 minutes 30 seconds.

Run the API with one worker:

 $ gunicorn --workers 1 app:api -b 0.0.0.0:8000 

Now we can run Taurus. When you first start it will download the necessary dependencies:

 $ bzt bzt-config.yml -report 

After installing the dependencies, our console will be displayed with the progress of the test:



We use the -report option to upload results to BlazeMeter and generate a web report.

Our API does an excellent job with 100 users. We reached a throughput of ~ 1000 requests / second, without errors and with an average response time of 0.1s.



Great, but what if there are 500 users? Change the concurrency parameter to 500 in our bzt-config.yml file and run Taurus again.



Hm Looks like our lone worker can't handle the load. 40% of errors is not the case.

Let's try to increase the number of workers.

 gunicorn --workers 20 app:api -b 0.0.0.0:8000 

Looks better. There are still errors, but the bandwidth has increased to ~ 1500 requests / second, and the average response time has decreased to ~ 270 ms. Such an API can already be used.

Further performance optimization

You can configure PostgreSQL for iron with PgTune .

That's all for today. Thanks for reading!