Geekly Articles each Day
📜 ⬆️ ⬇️

How to prevent frequent releases from breaking your API, or writing auto tests for an open API and sending a result to Telegram bot

image

Foreword


Our team develops financial tools, including open payment APIs, and like many projects that work on continuous integration practice, we simultaneously began to think about how to improve the test coverage of the project and achieve the maximum stability of our code with fairly frequent changes (we sometimes install updates to the grocery environment several times a day). This is especially important in three aspects:


In this article I would like to share my experience and show an example of how we develop tests for API interfaces that include both server-server interaction and work through a browser. For the demonstration, I will give a simple example of testing the payment process with a bank card through our payment gateway with sending the test result to Telegram.

You can download the project with sample tests on GitHub: https://github.com/cloudipsp/auto_tests.git
')
Documentation for our tested payment API: https://www.fondy.eu/info/api/

Prepare the environment


We use the Robot Framework to develop tests, and although this framework has its own RIDE development environment, it is significantly inferior to PyCharm in terms of convenience and capabilities.

RIDE

image

To begin development, we will install

  1. virtualenv

    pip install virtualenv setuptools

  2. free version of PyCharm Edu
    https://www.jetbrains.com/pycharm-edu/download/
  3. for PyCharm we put plugins
    intellibot:
    http://plugins.jetbrains.com/plugin/7386?pr=pycharm111

    Robot Framework Support:
    http://plugins.jetbrains.com/plugin/7415?pr=pycharm99

    while for the current version of PyCharm Edu 2.0.4 I had to install the robot framework version 0.14.2, since the last 0.15 was not compatible
  4. we clone the project with github - this item can be skipped if there is a desire to do everything from scratch:

     git clone https://github.com/cloudipsp/auto_tests.git

  5. install dependencies:

    for a start we have enough such libraries:

    robotframework==2.9a1
    selenium
    robotframework-selenium2library
    requests

    Create a pip-requires.txt file with this content, activate virtualenv and install

     cd auto_tests pip install -r pip-requires.txt


Development: autotests without a browser



For example, take the type of purchase on the 3DSecure card when the card is entered on the merchant side (API section https://www.fondy.eu/ru/info/api/v1.0/4 ).



For simplicity, we exclude Step 2. - redirect in the browser (we will test it in the following example). For a test card, this redirect occurs on the page - a banking emulator, which always returns the same result - the password is entered correctly.

In this case, we will have 2 steps:


Initial Settings Files


In the documentation link https://www.fondy.eu/ru/info/api/v1.0/2 take test cards on which we will test. By the way, all test payments can be seen in the merchant's demo account: https://www.fondy.eu/mportal/#/account/demo .



Now create the robot files:

cards.robot

 *** Settings *** Documentation A resource file with test credit cards. Imported once in resource.robot *** Variables *** #Name Card Number Exp Month Exp Year Cvv2 @{3dsApproved} 4444555566661111 01 24 238 @{no3dsApproved} 4444555511116666 01 24 238 @{3dsDeclined} 4444111166665555 01 24 238 @{no3dsDeclined} 4444111155556666 01 24 238

merchant.robot

 *** Settings *** Documentation A resource file with test merchants. Imported once in resource.robot *** Variables *** ${TestMerchant} 1397120 #(Test merchant)

variables.robot

 *** Settings *** Documentation Variables used in all tests. Imported one time in resource.robot *** Variables *** ${API SERVER} api.fondy.eu ${JSON} application/json ${XML} application/xml ${FORM} application/x-www-form-urlencoded

resource.robot

 *** Settings *** Documentation A resource file with reusable keywords. Resource variables.robot Resource cards.robot Resource merchants.robot Library helper/utils.py Library requester.py

Protocol specifications


In order to be sure that the request to the API and the response from it comply with the specifications, create a file specifications_settings.py, which will contain the structure of the parameters described in the documentation. For example, the parameters in the documentation



will correspond to the structure:

 PAY_SERVER2SERVER_3DS = { 'request_step1': { "order_id": { "required": True, "type": "string", "size": 1024 }, "merchant_id": { "required": True, "type": "int", "size": 12 }, "order_desc": { "required": True, "type": "string", "size": 1024 },

full specifications_settings.py
 PAY_SERVER2SERVER_3DS = { 'request_step1': { "order_id": { "required": True, "type": "string", "size": 1024 }, "merchant_id": { "required": True, "type": "int", "size": 12 }, "order_desc": { "required": True, "type": "string", "size": 1024 }, "signature": { "required": True, "type": "string", "size": 40 }, "amount": { "required": True, "type": "amount", "size": 12 }, "currency": { "required": True, "type": "string", "size": 3 }, "version": { "default": "1.0", "required": False, "type": "string", "size": 10 }, "server_callback_url": { "required": False, "type": "url", "size": 2048 }, "lifetime": { "required": False, "type": "int", "size": 6 }, "merchant_data": { "required": False, "type": "string", "size": 2048 }, "preauth": { "default": False, "type": "boolean", "required": False }, "sender_email": { "required": False, "type": "email", "size": 50 }, "lang": { "required": False, "type": "string", "size": 2 }, "product_id": { "required": False, "type": "string", "size": 1024 }, "verification": { "default": False, "type": "boolean", "required": False }, "card_number": { "required": True, "type": "string", "size": 19 }, "cvv2": { "required": True, "type": "string", "size": 3 }, "expiry_date": { "required": True, "type": "date", "size": 4, "important": False, }, }, 'request_step2': { "order_id": { "required": True, "type": "string", "size": 1024 }, "merchant_id": { "required": True, "type": "int", "size": 12 }, "pares": { "required": True, "type": "string", "size": 20480 }, "md": { "required": True, "type": "string", "size": 1024 }, "version": { "default": "1.0", "required": False, "type": "string", "size": 10 }, "signature": { "required": True, "type": "string", "size": 40 }, }, 'response_3ds': { "response_status": { "type": "string", "required": True, "size": 50 }, "acs_url": { "type": "string", "required": True, "size": 2048 }, "pareq": { "type": "string", "required": True, "size": 20480 }, "md": { "default": "", "type": "string", "required": True, "description_en": "", "description_ru": "", "size": 1024 }, }, 'response_final': { "order_id": { "type": "string", "size": 1024 }, "merchant_id": { "type": "int", "size": 12 }, "amount": { "type": "amount", "size": 12 }, "currency": { "type": "string", "size": 3 }, "order_status": { "type": "string", "size": 50 }, "response_status": { "type": "string", "size": 50 }, "signature": { "type": "string", "size": 40 }, "tran_type": { "type": "string", "size": 50 }, "sender_cell_phone": { "type": "string", "size": 20 }, "sender_account": { "type": "string", "size": 50 }, "masked_card": { "type": "string", "size": 19 }, "card_bin": { "type": "int", "size": 6 }, "card_type": { "type": "string", "size": 50 }, "rrn": { "type": "string", "size": 50 }, "approval_code": { "type": "string", "size": 6 }, "response_code": { "type": "int", "size": 4 }, "response_description": { "type": "string", "size": 1024 }, "reversal_amount": { "type": "amount", "size": 12 }, "settlement_amount": { "type": "amount", "size": 12 }, "settlement_currency": { "type": "string", "size": 3 }, "order_time": { "type": "time", "size": 19 }, "settlement_date": { "type": "time", "size": 10 }, "eci": { "type": "string", "size": 2 }, "fee": { "type": "amount", "size": 12 }, "payment_system": { "type": "string", "size": 50 }, "sender_email": { "type": "email", "size": 254 }, "payment_id": { "type": "int", "size": 19 }, "actual_amount": { "type": "amount", "size": 12 }, "actual_currency": { "type": "string", "size": 3 }, "product_id": { "type": "string", "size": 1024 }, "merchant_data": { "type": "string", "size": 2048, }, "verification_status": { "type": "string", "size": 48, }, "rectoken": { "type": "string", "size": 48, }, "rectoken_lifetime": { "type": "time", "size": 19, }, }, }


Next, create functions.

The function that will run through the specifications specifications_settings.py file and create a request in the format JSON, XML, FORM from a set of all data of different types, finishing them up to the maximum length.

Build required parameters dict
  def build_required_parameters_dict(self, merchant_id, currency, spec, spec_dict, response_url=None, *args, **kwargs): self.merchant_id = merchant_id request_params_specs = getattr( specifications_settings, spec)[spec_dict] # for requests with cards if args: kwargs['card_number'] = args[0] kwargs['expiry_date'] = int(str(args[1]) + str(args[2])) kwargs['cvv2'] = args[3] request_params = {} for param in request_params_specs: if param in kwargs.iterkeys(): request_params[param] = kwargs[param] elif param == "signature": request_params[param] = '' elif param == "currency": request_params[param] = currency elif param == "payment_systems": request_params[param] = 'card' elif param == "response_url": request_params[param] = response_url elif param == "merchant_id": request_params[param] = merchant_id elif param == "delayed": request_params[param] = "n" elif param == "order_desc": request_params[param] = 'test' + randomStr(size=7, chars=string.digits) elif param == "order_id": request_params[param] = self.order_id # for 3ds requests elif param == "pares": request_params[param] = TEST_PARES elif param == "md": request_params[param] = self.md # any other parameters elif request_params_specs[param]["type"] == "email": request_params[param] = "test@fondy.eu" elif request_params_specs[param]["type"] == "string": request_params[param] = randomStr( request_params_specs[param]["size"], param).encode('utf-8') elif request_params_specs[param]["type"] == "url": request_params[param] = "https://" + randomStr( request_params_specs[param]["size"] - 12, param).encode('utf-8') + ".com" elif request_params_specs[param]["type"] == "int": request_params[param] = randomStr(request_params_specs[param]["size"], "", string.digits) elif request_params_specs[param]["type"] == "amount": request_params[param] = randomStr( 5, "", string.digits) elif request_params_specs[param]["type"] == "boolean": request_params[param] = randomStr( 1, "", "YN") self.request_params = request_params


the function of directly HTTPS POST sending data to the API:

Send request
  def send_request(self, content_type, url=None, data=None, protocol=False, **kwargs): requests.packages.urllib3.disable_warnings() print "*HTML* sending request" print "*HTML* content_type=%r, url=%r, data=%r, kwargs=%r" % (content_type, url, data, kwargs) if data is None: data = self.request_params if self.order_id == '': data['order_id'] = 'test' + randomStr( 10, "", string.ascii_letters) else: data['order_id'] = self.order_id data['signature'] = "" data['signature'] = build_signature(self.request_params) self.save_order_id_from_server(data['order_id']) post_data = self.build_request(content_type, data) print "*HTML* POSTREQUEST %s" % (post_data) self.response = requests.post( url, headers={'Content-Type': content_type}, data=post_data, verify=False).text print "*HTML* POSTRESPONSE %s" % (self.response) return self.response


We also need a function to validate the response from the API, which will verify all received parameters with the specifications_settings.py specification file:

Verify response status
  def verify_response_status(self, spec, spec_dict, content_type, response=None, request_params=None, status='approved'): try: if response == None: response = self.response print "*HTML* response %s" % (response) if request_params == None: if self.request_params: request_params = self.request_params print "*HTML* REq_par %s" % (request_params) response_params_specs = getattr( specifications_settings, spec)[spec_dict] print "*HTML* REsponse_par_spec %s" % (response_params_specs) errors_list = [] error = False response_params = parse_response(self.response, content_type) print "*HTML* REsp_par %s" % (response_params) for param in response_params_specs: if response_params[param] is not None: if response_params_specs[param]["type"] == "string": if len(response_params[param]) > response_params_specs[param]["size"]: errors_list.append('Error: size of param ' + param + ' is ' + str( len(response_params[param])) + ' but max is ' + str( response_params_specs[param]["size"])) error = True elif response_params_specs[param]["type"] == "int": if len(str(response_params[param])) > response_params_specs[param]["size"]: errors_list.append('Error: size of param ' + param + ' is ' + str( len(str(response_params[param]))) + ' but max is ' + str( response_params_specs[param]["size"])) error = True if response_params[param] != "" and not str(response_params[param]).isdigit(): errors_list.append( 'Error: param ' + param + ' is not integer') error = True else: errors_list.append('Error: param ' + param + ' is missing') error = True if request_params.get(param) is not None and request_params.get( param) != "" and param != 'signature' and response_params.get(param) is not None: if (response_params_specs[param]["type"] == "string" and request_params.get( param) != response_params.get( param)) or ( response_params_specs[param]["type"] == "amount" and int( request_params.get(param)) != int( response_params.get(param))): request = 'request:' + str(request_params.get(param)) response = 'response:' + str(response_params.get(param)) order_id = 'order_id:' + \ str(response_params.get('order_id')) errors_list.append( 'Error: param ' + param + ' is not equal in request and ' 'response\n request=%s\n response=%s order_id=%s' % ( request, response, order_id)) error = True if response_params_specs.get('signature') is not None: params_sign = {param: response_params.get(param, "") for param in response_params_specs if param != 'signature'} params = collections.OrderedDict(sorted(response_params.items())) params_sign['signature'] = build_signature(params_sign) if params_sign['signature'] != params["signature"]: errors_list.append('Error: signature invalid in response ') error = True if response_params.get('order_status') and response_params.get('order_status') != status: errors_list.append('Error: invalid status in response ') error = True except Exception as e: errors_list.append("final %s" % e.message) error = True finally: if error: raise Exception("*HTML* Errors:\n %s" % errors_list) else: print "*HTML* test passed OK"


And the last function to save the response from the API in step 1 is to pass the parameters to step 2:

Save md pareq and acs url for 3ds
  def save_order_id_from_server(self, order_id): self.order_id = order_id print "*HTML* Order_id %s" % (self.order_id)


Now, based on these functions, we can build the pay_with_3ds_card.robot test script:

 *** Settings *** Documentation A test suite containing tests related to server-server complete purchase with 3ds card. Test Template Server-server full purchase with 3ds card Should Pass Test Timeout 15 seconds Default Tags smoke 3ds Library DebugLibrary Resource ../resource.robot *** Variables *** ${specificatons} PAY_SERVER2SERVER_3DS ${req_dict_step1} request_step1 ${resp_dict_step1} response_3ds ${url_step1} https://${API SERVER}/api/3dsecure_step1/ ${req_dict_step2} request_step2 ${resp_dict_step2} response_final ${url_step2} https://${API SERVER}/api/3dsecure_step2/ ***Test Cases *** merchant_id currency content_type credit_card USD_JSON_Approved ${TestMerchant} USD ${JSON} @{3dsApproved} USD_XML_Approved ${TestMerchant} USD ${XML} @{3dsApproved} USD_FORM_Approved ${TestMerchant} USD ${FORM} @{3dsApproved} UAH_JSON_Approved ${TestMerchant} UAH ${JSON} @{3dsApproved} UAH_XML_Approved ${TestMerchant} UAH ${XML} @{3dsApproved} UAH_FORM_Approved ${TestMerchant} UAH ${FORM} @{3dsApproved} EUR_JSON_Approved ${TestMerchant} EUR ${JSON} @{3dsApproved} EUR_XML_Approved ${TestMerchant} EUR ${XML} @{3dsApproved} EUR_FORM_Approved ${TestMerchant} EUR ${FORM} @{3dsApproved} RUB_JSON_Approved ${TestMerchant} RUB ${JSON} @{3dsApproved} RUB_XML_Approved ${TestMerchant} RUB ${XML} @{3dsApproved} RUB_FORM_Approved ${TestMerchant} RUB ${FORM} @{3dsApproved} GBP_JSON_Approved ${TestMerchant} GBP ${JSON} @{3dsApproved} GBP_XML_Approved ${TestMerchant} GBP ${XML} @{3dsApproved} GBP_FORM_Approved ${TestMerchant} GBP ${FORM} @{3dsApproved} *** Keywords *** Server-server full purchase with 3ds card Should Pass [Arguments] ${merchant_id} ${currency} ${content_type} @{credit_card} Build required parameters dict ${merchant_id} ${currency} ${specificatons} ${req_dict_step1} @{credit_card} Send request ${content_type} ${url_step1} Verify response status ${specificatons} ${resp_dict_step1} ${content_type} Save md pareq and acs url for 3ds ${content_type} Build required parameters dict ${merchant_id} ${currency} ${specificatons} ${req_dict_step2} Send request ${content_type} ${url_step2} Verify response status ${specificatons} ${resp_dict_step2} ${content_type}

This human readable script will test all 3 supported JSON, XML, FORM request formats for 5 different currencies: USD, UAH, EUR, RUB, GBP

Run the tests in virtualenv:

 (tests) E:\work\fondy\auto_tests>pybot server-server-tests



Development: Autotests with a browser and Telegram


Now we will add a robot file in which we will write all the HTML elements with which we will work: fill in or analyze
ui_repository.robot :

 *** Settings *** Documentation Variables used in all tests. Imported one time in resource.txt *** Variables *** # Checkout page ${CHECKOUT_BUTTON} css=.btn-lime ${CVV2} id=cvv2 ${EXPIRE_YEAR} id=expire_year ${EXPIRE_MONTH} id=expire_month ${CARD_NUMBER} name=card_number ${CARD_NUMBER_FIELD} id=credit_card_number ${3DS_SUBMIT_BUTTON} xpath=//button[@type='submit'] #Response page ${ORDER_STATUS} css=.field_order_status .value ${TABLE_RESPONSE} id=table_response

we will add the Selenium2Library library and the browser opening function to the resource.robot file:

 *** Settings *** Documentation A resource file with reusable keywords. Resource variables.robot Resource cards.robot Resource merchants.robot Resource ui_repository.robot Library Selenium2Library Library helper/utils.py Library requester.py *** Keywords *** Open Browser For Empty Page [Arguments] Open Browser about:blank Maximize Browser Window

Add the browser name to the variables.robot file: FireFox.

 *** Settings *** Documentation Variables used in all tests. Imported one time in resource.robot *** Variables *** ${API SERVER} api.fondy.eu ${RESP_URL} https://${API SERVER}/test/responsepage/ ${SERVER} fondy.eu ${BROWSER} FireFox ${JSON} application/json ${XML} application/xml ${FORM} application/x-www-form-urlencoded

The specification file is now replenished with a new set of parameters from the documentation https://www.fondy.eu/ru/info/api/v1.0/3 . These specifications are different in that the card details are not transmitted by the merchant, but they are entered on the side of the payment gateway, after the redirect from the merchant site:

specifications_settings.py
 # -*- coding: utf-8 -*- PURCHASE_FIELDS_REDIRECT = { "request": { "order_id": { "type": "string", "required": True, "size": 1024 }, "merchant_id": { "type": "int", "required": True, "size": 12 }, "order_desc": { "type": "string", "required": True, "size": 1024 }, "signature": { "type": "string", "required": True, "size": 40 }, "amount": { "type": "amount", "required": True, "size": 12 }, "currency": { "type": "string", "required": True, "size": 3 }, "version": { "default": "1.0", "type": "string", "required": False, "size": 10 }, "response_url": { "type": "url", "required": False, "size": 2048 }, "server_callback_url": { "type": "url", "required": False, "size": 2048 }, "payment_systems": { "type": "string", "required": False, "size": 1024 }, "default_payment_system": { "type": "string", "required": False, "size": 25 }, "lifetime": { "default": "36000", "type": "int", "required": False, "size": 6 }, "merchant_data": { "type": "string", "required": False, "size": 2048 }, "preauth": { "default": False, "type": "boolean", "required": False }, "sender_email": { "type": "string", "required": False, "size": 50 }, "delayed": { "default": True, "type": "boolean", "required": False }, "lang": { "type": "string", "required": False, "size": 2 }, "product_id": { "type": "string", "required": False, "size": 1024 }, "required_rectoken": { "default": False, "type": "boolean", "required": False }, "verification": { "default": False, "type": "boolean", "required": False }, "verification_type": { "default": "amount", "type": "string", "required": False, "size": 25 }, "rectoken": { "type": "string", "required": False, "size": 40 }, "receiver_rectoken": { "type": "string", "required": False, "size": 40 }, "design_id": { "type": "string", "required": False, "size": 6 }, "subscription": { "default": False, "type": "boolean", "required": False }, "subscription_callback_url": { "type": "url", "required": False, "size": 2048 } }, "response": PAY_SERVER2SERVER_3DS['response_final'], }


I will not describe all the test script files in detail, it’s pretty easy to figure them out, I’ll only give one:

pay_with_checkout_url_3ds_approved.robot

pay_with_checkout_url_3ds_approved.robot
 *** Settings *** Documentation A test suite containing tests related to recurring api transactions with token. ... Card with 3ds. Suite Setup Open Browser For Empty Page Suite Teardown Close Browser Default Tags 3ds approved Test Template Checkout With 3ds Should Pass Resource checkout_resources.robot *** Variables *** ${specificatons} PURCHASE_FIELDS_REDIRECT ${req_dict_step1} request ${resp_dict_step1} response ${url} https://${API SERVER}/api/checkout/url/ ${checkout_url} ${EMPTY} ***Test Cases*** currency merchant_id message content_type credit_card USD_JSON_Approved USD ${TestMerchant} approved ${JSON} @{3dsApproved} USD_XML_Approved USD ${TestMerchant} approved ${XML} @{3dsApproved} USD_FORM_Approved USD ${TestMerchant} approved ${FORM} @{3dsApproved} UAH_JSON_Approved UAH ${TestMerchant} approved ${JSON} @{3dsApproved} UAH_XML_Approved UAH ${TestMerchant} approved ${XML} @{3dsApproved} UAH_FORM_Approved UAH ${TestMerchant} approved ${FORM} @{3dsApproved} EUR_JSON_Approved EUR ${TestMerchant} approved ${JSON} @{3dsApproved} EUR_XML_Approved EUR ${TestMerchant} approved ${XML} @{3dsApproved} EUR_FORM_Approved EUR ${TestMerchant} approved ${FORM} @{3dsApproved} RUB_JSON_Approved RUB ${TestMerchant} approved ${JSON} @{3dsApproved} RUB_XML_Approved RUB ${TestMerchant} approved ${XML} @{3dsApproved} RUB_FORM_Approved RUB ${TestMerchant} approved ${FORM} @{3dsApproved} GBP_JSON_Approved GBP ${TestMerchant} approved ${JSON} @{3dsApproved} GBP_XML_Approved GBP ${TestMerchant} approved ${XML} @{3dsApproved} GBP_FORM_Approved GBP ${TestMerchant} approved ${FORM} @{3dsApproved} *** Keywords *** Checkout With 3ds Should Pass [Arguments] ${currency} ${merchant_id} ${message} ${content_type} @{credit_card} Get and set checkout url ${merchant_id} ${currency} ${specificatons} ${req_dict_step1} ${RESP_URL} ${content_type} ${url} @{credit_card} Go to ${checkout_url} Input and submit checkout ${merchant_id} @{credit_card} Confirm 3ds ${merchant_id} Response page should be displayed Check transaction status ${message}


To send the results to the Telegram, we need 2 files: the listener and the sender.

PythonListener.py
 from telegram_sender import * class PythonListener(object): ROBOT_LIBRARY_SCOPE = "GLOBAL" ROBOT_LISTENER_API_VERSION = 2 def __init__(self, count=0): self.ROBOT_LIBRARY_LISTENER = self self.count = count self.stat = None def end_suite(self, name, attrs): self.stat = attrs['statistics'] return self.stat def log_file(self, path): print self.stat test = Telegram() test.telegram_article(self.stat)


telegram_sender.py
 import telegram from helper._settings import * from telegram.ext import Updater class Telegram(object): def __init__(self, token=None): self.token = token or default_token self.updater = None self.bot = None def update_bot(self): self.updater = Updater(token=self.token) self.bot = telegram.Bot(token=self.token) self.updater.start_polling() self.bot.getMe() self.bot.getUpdates() def telegram_article(self, status): self.update_bot() # chat_id = bot.getUpdates()[-1].message.chat_id # add this string to update all telegram users chat_id = default_user self.bot.sendMessage(chat_id=chat_id, text=status) self.updater.stop()


Also we will write in the _settings.py parameters for Telegram bot:

 default_token = None # Put Your bot token to this variable default_user = None # Add user chat id

How to get a token and chat id, you can read, for example, here and here .

Now we actually run the browser tests. The result should come in telegrams:

 (tests) E:\work\fondy\auto_tests>pybot --listener PythonListener.py checkout-tests



Afterword


I hope this article will be useful both for auto testers and developers. In the next article I will try to tell you what to do if there are already several thousand tests - how to parallelize them, how to collect metrics about the speed of the tests, how to develop tests that interact with the database.

Source: https://habr.com/ru/post/302736/

More articles:


All Articles