Robot is testing SAP ERP

At AlfaStrak we use SAP ERP as a process loss settlement system. And it just so happened that we are modifying it a little bit, this inevitably leads to errors in the code. If errors reach a productive system, this is bad. This should be avoided; one of the ways is regression testing. In this article I will talk about how exactly we carry out the "regress" for SAP, because we do it (eh!) Outside the box.

It all started a few years ago. In those years, we actively used regression testing, but could not do it at SAP - the tools we used with SAP didn’t work, the testers team didn’t want to learn the tools “sharpened” for SAP tools. I can’t remember exactly why, but I took it as a challenge (it was before I switched to engineering) and decided to “study” the question.

The results of the study (as well as "doing") - in this article (below), I’ll briefly say this: we automatically test SAP (and its immediate environment), we do it quite effectively (in all senses), we have not spent a single ruble on licenses and learning, our approach is simple and quite reproducible. And we do not use any SAP tools for automated testing of SAP (except in the place where we have integrated into its transport system).

Large strokes

There is a danger of going into details and losing readers, I will try not to do this (as an author, I know all the details).

It all began with a study, communication with SAP, our partners and Mr. Google.

The result of this communication was as follows:

• SAP has tools to automate testing (we looked at eCATT, intently SAP CBTA)
• they require substantial immersion (especially SAP CBTA)
• there may be restrictions if it is necessary to go beyond SAP (it’s not in the “vacuum” that lives with us)
• there are third-party products (for example, HP), but we have no competence for them, just like purchased licenses
• there is a way to "control" SAP outside (this idea was suggested to me by Konvista, thanks a lot to her for it)

Since I personally didn’t have any knowledge of SAP, I decided to try to dig in the last direction - the management of SAP not from SAP. Quickly enough, Google provided the document "SAP GUI Scripting API for the Windows and Java Platforms", which served as an excellent start for this initiative. A quick test on your favorite python showed - it works.

Then it was a matter of small things - to find a suitable framework for testing automation. Since python was my favorite language, the Robot Framework quickly got into consideration. And, actually, after I got on the list and was tried, then only he remained on the list. I was bribed by the fact that “it” immediately worked (running ahead - and it still works, I don’t regret my choice for a minute).

A small pilot showed that SAP is perfectly "controlled" by a robot (I will call the Robot Framework such a Russian word): quickly, synchronously, predictably and reliably. In this case - I emphasize - we use the SAP documented way of interacting with the SAP GUI (not some kind of “crutches”).

Architecture

(yes I will be allowed to use this word here)

How it all works:

• the script runs on the server (the word "server" is used here in the sense that this computer serves our requests for testing. In this particular case, it is more correct to use the word "client" because the script controls the testing process)
• the script interacts with the server component of the robot running on the SUT (system under test) via the Remote mechanism of the robot;
• This server component calls methods of the SAP management library.
• SAP GUI handles these requests (in synchronous mode, this is important)
• the results of executing queries or simply "jingles" are returned to the script on the "server", where they are used in the testing process

Technically

• "server" is a virtual machine for Ubuntu
• SUT is the workplace where the SAP workplace is installed and configured (in our case it is a Windows computer or a virtual machine)
• SAP's management library is written in python (there are a few - a couple of hundred lines)
• A "script" is a "program" in a language that the Robot Framework understands.
• the robot (as such) implies a command line, since the developers at ABAP were not used to this, I made a WEB interface that provides, in particular, a collective work (about it just below).

What is wonderful

Actually, what did we get, besides the absence of a licensing charge?

A lot of things, if briefly about the main thing:

Russian language

The script looks like this:

At the very beginning of the journey (probably, during the pilot) I thought - and what will we break the language and invent some incomprehensible-even-to-us-words? The robot involves the creation of your own keywords (sorry for the terminology - so it is called). So why don't we invent them right away in Russian? I asked the testers' community (somewhere in the depths of the Internet) - I was "zashkali": it's dangerous, why, who said, etc. I still went my own way - we have everything in Russian (variables, words, only the control structures of the robot remain, but they need to be hidden inside keywords - if you see English, then it’s time to refactor).

What good is the Russian language (except for clarity without a dictionary) - the script can be shown "not to an IT person", a business person. You can write this script directly with it (I won’t even try to go into the topic ATDD here - this is a separate and big post, I will write someday).

Full and total control and extensibility

I know exactly what happens during the testing process. There is no magic at all - everything is very clear. I do not know how anyone, I like this.

About extensibility - functionality can be developed in any direction than we actively used:

• it turned out to come up with your own "language" of test scripts, understandable to business
• it was possible to automatically check whether the fields in the interface were filled in correctly at the end of the test (for this, in particular, they separated the robot variables into the launch parameters and interface parameters, made it possible to determine which interface element should have the trigger parameter)
• breakpoints can be added to tests; during breaks, see the values ​​of variables
• implemented a mechanism for generating file templates and putting them into the execution process - without this, testing such an animal as a PES would be impossible (and we tested it in a fully automatic mode - from the creation of the application to the statement processing)

Having our own web interface has added more opportunities for expansion - for example, we can afford to modify the language of the robot (we thought up our own conditional operator, for example - we and our business users didn’t like the standard "Run keyword if"). This became possible because the source code files of the tests are generated for the robot by the web application.

Ease of entry

Typically, testers do not have knowledge of the SAP infrastructure and, especially, SAP programming. To master the robot and our improvements to it for them is obtained in a couple of weeks.

User Instructions

We also stumbled upon the "William of ours ..." - for documentation. It is not a secret for anyone - there is usually no documentation on the system, and even if there is, it becomes outdated very quickly. Users transmit the rules of working with the system "word of mouth". If the automatic test code is a description of how to use the system according to the author, then why not convert this into an instruction?

Of course, on this fragment it’s hard to see the details, it’s important that the instructions are generated and updated in fully automatic mode, including screenshots. The instruction is always relevant (for auto-tests always work). In the case of SAP, the screenshots generally work well - in SAP-e you can always find an interface element - a rectangle whose coordinates will be relevant to the current place in the test code, this one (invisible to the eye) is control and is used as a parameter to the keyword "take a screenshot" (this word , of course, it works only with the appropriate test run mode - why just use electricity).

We formatted the instructions through Sphinx (I am sure many have learned the color scheme), so they are available both in the online manual and in printed form.

A little about Robot Framework

Nevertheless, it is impossible to say nothing at all about the robot - it will turn out too incomprehensible and superficial. I will try briefly.

The main idea of ​​this framework is the ability to create your own testing language (in the robot's terminology, the executable unit is the keyword - a keyword). The framework provides

• program execution environment (programmers - do not be offended) in this language
• functionally rich libraries (from working with strings and lists, to ssh and selenium)
• reporting (in the process of using, there was not even a thought to write any report)
• a simple paradigm of forming a program from keywords (a bit peculiar, but you can get used to it), there are such things as variables, parameters and results of "calls" (keywords)
• simple and powerful extensibility - it is very easy to create our own library in python (for example, we worked with Excel in this way), both local (that is, executable in the same place as the test code) and remote (for example, which manages the SAP GUI)

The process of creating a test is quite straightforward.

• form a sequence of actions and translate it into terms of the robot (below will be specificity about interaction with the system under test)
• Repeated sequences refactor into their (new) keywords
• we start the test, we look - as works, we correct, we improve
• debugging is provided by breakpoints (with the ability to view variables — provided by the architecture, more precisely, the use of the robot’s remote library)

The result is not even a program (see examples above), but rather a formalized sequence of actions, which, by the way, describes the use of the program as it was intended by the author (see ATDD above).

Interaction with the system under test

In our case, we test at the user interface level (i.e., our tests interact with the SAP at the GUI level — they press buttons, fill in text fields, etc.). It is generally recognized that this way of writing tests is not the best. I partly agree with this, but I am ready to listen and discuss the best way to test SAP GUI applications (like our SAP FS CM).

There is such a guru - Robert Martin (he is "uncle Bob" - the author of "Clean coder", if anyone read), we talked a little about this with him. We agreed that if it is not very difficult to do, it does not change very often (breaking tests), then it’s okay - you can also test it via the interface.

For my part, I can say this: in the case of SAP GUI, there are not so many options for implementing the user interface. If you need to add the ability, for example, to track IMEI code, you can do it in almost one way - by adding the appropriate field to one of the tabs. So, I think that all the nuances of this “addition” can and should be thought out by the customer (as this field will be called in the interface, width, order of use, etc.). And he can do it directly in the script, which will then automatically test it. If you think up the revision to the end does not work out (as they say - well, you do it, and we will see), then it is better to not do this revision. And to do what turns out to think. Well, this is me again in ATDD climbed ...

Returning to the interaction with the SAP GUI: we did, as I already wrote, about 200 lines in python and about 10 different interface management functions (such as "go to bookmark", "fill in the field", "press the button"). This set was formed practically in the first days and subsequently did not expand much. To the credit of SAP, I note that everything works very quickly - the eye does not have time to keep track of how the interface is “blinking”; in order to slow down, delays were inserted into the control cycle (in cases where visual monitoring is necessary). Also note the advantages of synchronous work - nowhere in the code you have to wait for anything, if, for example, the button is pressed, the action corresponding to the button is completed (for example, the loss is saved).

def get_ctrl_attr ( self, uid, attr ): """     (  )   (  )  exception =  (   log) """ ctrl = self._get_ctrl(uid) try: retText = getattr( ctrl, attr ) except: raise AssertionError("  {1}   {0}".decode("utf-8").format(attr,uid)) return retText 

Slight comments on the code

• shows a fragment of the SAP GUI management library
• a library is an object, methods are directly accessible in tests (in keywords), for example, the above method can be called like this: "res = get ctrl attr $ {UID} text" (using robot notation)
• in case of an error, the text of the exception will be visible in the log of the robot (for this you don’t need to do anything - just to “raise” the exception

The code is very simple, it pleases.

Running tests

Following the logic of the robot, individual tests are united in our projects. Test or project can be run manually from the web interface. Also, the regression testing process is integrated into the SAP transport system:

• the owner of the product at the end of the business testing stage approves transport requests for transfer
• approved requests "move" one by one to the pre-productive environment, where, according to the settings, one or another set of tests is launched (more precisely, projects)
• if the test result is positive, the request is released and goes to the production environment
• if errors occur, an email is sent to the author of the request with reference to the testing report (generated by the robot), the request does not go any further

Important - the web interface significantly reduces the threshold for entering the testing process: it is easy to start the test — press the "start" button. At the same time (due to the use of the Robot Framework) all the advantages of the test file representation (version control, command line and related automation, etc.) are retained.

Not a single SAP

We successfully used our development for testing not only the SAP GUI, I had a small development (a simplified interface for registering a loss) in python and django. Since all the basic points we have already been implemented, all that was needed was to write the browser management library (the same as I had for managing the SAP GUI, only through Selenium). And it turned out great:

• the tests still work on the Linux virtual machine (on the same one - the SAP and not the SAP tests live in the same instance)
• the browser "flashes" at the tester's workplace (full visual control without any tricks)
• standard reports, familiar tools

In the tests of this system, I went further (in the direction of ATDD) - visually visible elements (tags and placeholder's) were formed initially in the tests, they were coordinated with the business and from there they "got" into the source code of the system itself (a bit of DRY was obtained) write code without first writing a test. Cool?

Of course, many tools have been developed for web applications, but I cannot say that I experienced some inconvenience in the process. It turned out well here ...

Summing up

The SAP world is great and contains, it would seem, everything that is necessary for the complete happiness of developers. But this does not mean at all that it is necessary to walk only along those paths that the SAP and its ecosystem “pierced”. It is useful at the beginning of the way you need to ask yourself the question: do I really want to do "like everyone else"? We in AlfaStrakhovaniyu ask ourselves and not only in IT.

And, once again, in programming everything is possible, the question is only time and money (motivation).