State final certification in schools - not all like the exam

We have already told our readers about how the ABBYY TestReader solution has helped us to get on our feet. A few years ago it seemed to us quite logical (and at first even simple) using the experience gained, to process the results of the State final attestation of 9th grade (GIA 9th grade).

So, what did we have back in 2008? A well-established and, in general, reliably functioning USE results processing machine, which operates on the basis of the ABBYY TestReader Network entry software.

What did we want? Apply our technology again on GIA-9 and become even more attractive for investment .

It would seem, what is easier? But, having already set out on this path, we found several differences between the exams, and this radically determined the further development of events.

The fact is that the Unified State Exam is carried out centrally by the Ministry of Education and Science and Rosobrnadzor according to a single scenario, the response forms are also treated the same way. The costs are divided between the federal and regional authorities as follows: the center provides technology and services, and the regions, in fact, organize an exam. The implementation of GIA-9 is entirely given to the regions, which means that it falls on their budget and is carried out in the manner that the regional authorities decide. Unites GIA in Russia only that examinations pass according to the uniform schedule and on identical tasks.
')
As you might guess, the regions of Russia are very different in the amount of available financing and some, clearly, try to save. As a result, “piecewise” automation is observed: someone still boils conducts an exam in the old manner: students write by hand, and examiners check “by hand”. Those regions that, at their own peril and risk, are independently trying to convert the exam to a test form, face many problems. One of them is the need for a large amount of printing forms and tasks (officially they are called CIM - test and measurement material) for students. If for the USE forms are ordered centrally at the expense of the federal budget, then within the framework of the GIA this is done by the regional authorities.

At best, every subject orders a print at a print shop, at worst, prints forms on an ordinary printer, copier, and maybe even on a risograph .

Since for each region, it was first of all considered to save and make everything simpler, it was pointless to suggest color forms for the GIA. Therefore, we proposed black and white forms that are easily replicated on various devices.



And now let's imagine that we are a subject of the Russian Federation. We need to provide forms for up to ten thousand schoolchildren, each of whom passes the compulsory examinations in Russian and mathematics and other subjects. Each student needs a set of two forms of answers and one form with tasks for each exam. Total, we need to print ... count, if you are interested. When you count, do not forget that these students study in different schools, which in turn are in different cities, villages and ... islands. Therefore, when typing, you will have to figure out which school how many forms in Russian, mathematics and other subjects to carry. And if in the village of “Tmutarakan” there is an unfortunate who suddenly wants to hand over literature, then it would be nice to provide him with an appropriate kit. Oh, and do not forget to reserve for excited students who have accidentally spoiled the form. Well, they counted? So we spat on this bad job.

Instead, we decided to add a special printing station to our TestReader, which will consider and print it for us. It was supposed to be a new component of a software system for entering forms, designed for mass high-speed printing of machine-readable forms, as well as forms with tasks.

How we made the print station

This system was supposed to:

1. Be simple enough for operators in the regions to deal with it.
2. To be reliable - otherwise the exam will fail.
3. Be able to quickly print exam materials on laser printers
4. Provide materials for all TES (points of the exam) in the required amount
5. Minimize printing costs
6. Save information security
7. It is necessary to print in the slowest graphic mode, so as not to disturb the formatting of the forms.

At the preparatory stage, we talked a lot: with specialists from the Federal Testing Center, with people in the regions who provide exams. At some point it seemed to us that we were tired of everyone talking :). But how else to find an option that fits all regions? We came up with different ideas, then we looked for flaws in them and came up with new ideas. In the process there appeared such “memory cards” .



At a certain point, we had an understanding of what functions the print station should have, and we started to design. According to the plan, 90% of the functionality should have been related to the planning and collection of sets, and only 10% should be taken up by automatic printing directly.

Already at this stage, we realized that the development will have to order outsourcers. Examinations should be carried out in strictly defined terms, and our developers did not have enough resources - after all, at exactly the same time we need to prepare upgraded software for the unified state examination. We wanted to assign the code to the long-time partner of ABBYY - Novosibirsk company ATAPY . When we started interacting with outsourcers, we took into account the following:

1. They do not understand the subject area (that is, in the state examinations).
2. We already knew that they could write the appropriate code, because they had collaborated with them repeatedly.
3. They are located in Novosibirsk, so there is a time difference.

In order for outsourcers to better deal with our order, we:

1. Made the most detailed TZ, which included ready-made algorithms for the program. In fact, we did not need programmers from the outsourcer, but coders who would do everything exactly as we needed.
2. Themselves completely drawn absolutely all the windows and dialogs of the program and described all the actions that should occur when you click on each of the "buttons".
   
   So, for example, the main window should look like this:
   
   
3. We have made a common complaint base with outsourcers. I must say, at first, this caused dissatisfaction with both our programmers (we still left a small part of the development for ourselves) and Novosibirsk colleagues. However, we insisted on our own and subsequently made sure that we were right.
   When we worked on this project (we recall, it was several years ago), cloud solutions in this area were not so widespread, so we used our own internal development - a clone of the base that we use in ABBYY.
4. They left behind them and developed modules on their own that were to be responsible for the interaction of the printing station with the “environment” - large external systems (databases, etc.). These modules are called "adapters". Thus, we tried to protect outsourcers from interaction with adjacent systems as much as possible.
   As a result, our partners did everything well, and we were pleased with the result.

After the printing station was ready, it was tested, and then technical documents were written to it: the operational documentation (what functions are in the program) and the corresponding instructive manual.

Do you want technical details?

We have them.

Development language - C #. The database of the system is MS SQL Server 2000/2008 (since it is also used in the exam). Platform - Microsoft .NET 2.0. An external visual component TreeList manufactured by DevExpress was purchased for the printing station. Everything works on a platform of Windows XP, 2000. Installed cryptographic protection from the company Aladdin.

The productivity of the printing station on inexpensive computers of that time with Pentium 4 and Celeron processors (namely, they were installed in 2008 for most users of our solution) - up to 15-20 pages per minute (in graphical mode!). This is the maximum speed: if the printer is old and slow, it will certainly be less.

What it looks like from the user's point of view

The subject of the Russian Federation conducting the GIA loads all the necessary data on students into the system: for this it uses the existing databases for each of the subjects of the Russian Federation with lists of schools. The system sees in which locality, city and district the school is located and how many students (in this case, ninth-graders) study there. And if there is no such data, you will have to enter this data: set by default, extrapolate data about 11th graders, enter it manually or in any other way. So we are determined with the number of ninth graders. Then we introduce errors, reserves, stocks, etc. - we complete the planning stage - we choose the printing method - Start! Oh, and would be happy, but with the seal is also not so simple.

The region is given the opportunity to choose a printing method from the following main options:

1. We print the whole set in the printing house (forms of answers and tasks)
2. We print in the printing house only forms of answers, and the task - on printers (for example, in the regional capital or regional information processing centers).
3. We print the whole set on printers in the capital of the region.
4. We print tasks on the printer in the capital, and the forms of answers on the field just before the exam
5. We print the whole kit on the field on the printers just before the exam
6. Other combinations suitable for the region are possible, but they are rarely used.

If with printing in printing houses and on printers in the capital of the region it is more or less clear (armed guards, fingerprint input), then it is worth further clarifying about “printing on the ground”. Since it is required to preserve the confidentiality of tasks before the exam, in the regional capital a “conditional” printing of sets into files that are encoded with a password is performed. These files are sent to schools in advance, and after issuing the password immediately before the exam are printed on any available laser printer.

Passed the exam. Next come the ABBYY TestReader scanning technology. Here we have preserved the exam processing circuit as much as possible: the forms filled out by students are scanned, recognized, verified, etc. After that, without interaction with the federal centers, points on exams are put on the spot (another difference from the EGE).

Results and lessons

• At the moment, 50 out of 83 constituent entities of the Russian Federation have chosen our technology for conducting GIA in their regions.
• The project was completely investment from our side, and it quickly paid off.
• Since the launch of the project in September 2009, the station practically did not require improvements.
• From the beginning to the end, the project took 1.5 years: 10 months - an analyst (of which 2 months - preparation of a detailed TK with pictures), 5 months - development, the rest - testing and preparation of documentation.
  
  Line-up:
  ABBYY : two analysts (the work on the project took about 50% of their working time), a project manager, 1 programmer (1 working month), a distribution kit specialist (2 weeks), a tester (1 month), a technical writer (3 weeks).
  From the outsourcer : project manager (our order took about 20% of his working time), programmer (5 months), tester (2 months), interface designer (2 weeks).
  
  
• We did not use special systems of collective work, so it took a lot of time e-mail correspondence and telephone calls. And we will not advise anyone to use Outlook and Excel as a Task-manager.
• As it often happens in working with outsourcers, we could not avoid stretching the development time: we initially planned to cope in 3 months - we managed to do it in 5.
• But on the other hand, the idea with the most detailed technical assignment, including all interface elements, worked 100% - the outsourcers did everything exactly as we thought. It turned out well.
• The idea with adapters was also successful. It allowed the two development teams to work independently and connect parts of the program already at the final stage. We will use adapters for future system updates for the exam.
• And finally - GIA was even more interesting and complex project than the exam. We liked.