At the moment, on the Internet, unfortunately, there are not so many articles and reports about the CTF information security competitions (and even about the competitions and competitions in general) written on behalf of the organizers themselves. In order to correct this misunderstanding at least a little bit, we ( PeterPen , information security team from St. Petersburg State University) would like to describe our experience in conducting the individual CTF competition StepCTF 2015 , held on May 11 and 12, 2015. In the program: how to prepare, how to hold the competition; also tell about the lessons learned, in order to warn and / or make life easier for potential organizers.

What is CTF?

CTF (Capture The Flag) - information security competitions held in “game” form: individual participants or teams compete using the whole arsenal of existing knowledge and skills to extract the “flags” (hence the name), for which winning points are awarded. CTF, as a rule, is of two types:

• Classic: each team is given a server with a set of services that need to be maintained in working condition so that the jury can upload flags to the services. Of course, these services have a set of vulnerabilities, exploiting which, you can steal these very flags.
• Task-based / jeopardy: as the name suggests, such a scheme is in many ways similar to the well-known television game “Own Game” (“Jeopardy!”). There are several categories, each of which has several tasks of varying complexity. The more difficult the task, the more points it will bring. Unlike the classics, task-based CTFs are both command and individual.

Introduction

PeterFen's team has been organizing CTFs not for the first time: in 2013 and 2014 we organized individual online contests SpisokCTF 2013 and SpisokCTF 2014, respectively, both at the LIST conference held at St. Petersburg State University. We received the first experience of preparing for the competition, wrote our platform for jeopardy CTF, managed to fill more than one cone and this year decided to continue the tradition, but in a slightly different format: to arrange an online CTF competition on the Stepic educational platform.

The choice of Stepic as a platform turned out to be quite spontaneous and risky, since Stepic in itself was not tailored to the format of the competition. In favor of the new platform said that our old is already outdated and required improvement, and one of the members of the organizing team is the current developer Stepic.
')
At the general discussion, they finally decided: “why not Stepic?”. Indeed, almost everything needed was already there, including basic things like news, comments, user management, etc. If we briefly examine the structure of educational materials on Stepic, then the largest unit is the course. A course consists of modules (parts of a course, for example, by weeks), each module consists of lessons, and a lesson, in turn, consists of a sequence of steps, or "steps". A step may be a page with text, video or one of the types of tasks, such as choosing the right option from the proposed ones, a task for programming, a task in which you need to enter the correct answer in the text field, etc. Now let's try to make a CTF-competition from this structure of materials . A course is a competition, modules are categories of tasks, lessons in a module are tasks in the appropriate category, one step in a lesson is the task itself. From the material, all that was missing was the rating table of participants, which was added to Stepic prior to the launch of the competition.

Development and infrastructure

Just a few numbers at once: the competition, which lasted 36 hours, was attended by 193 participants from different cities of Russia and the CIS, who were offered 21 tasks, divided into 6 categories.

This year, the team of organizers (almost all of them are students, graduate students and graduates of St. Petersburg State University mathematics) were scattered geographically: tasks were invented in Russia, Belarus, Switzerland. This did not cause serious difficulties in the second decade of the 21st century: the wonderful Slack service was used for communication and discussion of tasks, a private repository was created on Github for the tasks themselves (at the moment it is very public ). The repository immediately approved a single format for assigning tasks, so that the developer, who is not the author of the task, knows, for example, where the description lies and where the solution lies.

What we lacked was a single repository with all the agreements, rules, utilities used, etc. Even with a team of developers of 9 people, I had to explain the same things several times (where to fill in, how to deploy services, what is a Docker) just because some people connected to the development a bit later than others. For such purposes, a simple wiki would be enough: the githabovskaia, we think, would be a great success.

An unpleasant feature of Stepic was the openness of the course content before it began. This is a common practice in mass online courses, but it didn’t fit our format: we don’t want to show the list of categories and the names and number of tasks before the competition. To get around this restriction, two courses (competitions) were actually created: one - public, in which participants were enrolled, and the second - private, where the organizers added tasks before the start. Before the very start, the modules and lessons from the private course were simply copied into the public one.

In Stepic, a scoreboard was also added - a ranking table of participants, sorted by descending points. It was not there, because for online courses it is usually not needed.



The ability to attach files (“attachments”) to Stepic was convenient. Previously, we searched the Dropbox directory for storing files related to tasks, and in the case of StepCTF we stored them as attachments to a competition course or to specific tasks. It turned out conveniently: tasks and files are in one place, on one platform, accessible to all organizers.

For interactive tasks - those that accept connections over the network, for example, web tasks - we have created a separate server: the simplest VPS from Hetzner. So that everything was more or less safe and resistant to failures, each interactive task was also wrapped in a Docker container, which was automatically restarted when the service crashed. The use of containers allowed us to share the responsibility of developing the service and running it on the server. The developer was fully responsible for preparing the Docker image with the right environment to start his service: he used his favorite linux distribution, installed the necessary versions of languages, frameworks, libraries and other dependencies. With this approach, there was no need to install or update globally on the shared VPS server — there was order. For convenience and order, I still had to pay a considerable amount of time spent studying Docker and preparing working images.

Tasks

Good tasks (in this context, convenient short Anglicism “task” also caught on) is the main goal of any task-based F. You always want them to be interesting, diverse, and so that participants can discover something new during or after the decision. And this is the main challenge to the organizers, especially in entry-level competitions: I want to give something like a SQL injection or a Caesar cipher, but then you realize that in the simplest form, tasks of this kind have already been offered more than a dozen times in other CTFs . And, of course, it is precisely in preparation for the upcoming competitions that the fantasy begins to fail, which increases the number of spent nerves. Ideas of tasks can occur spontaneously, even when there are no competitions on the horizon, therefore a reasonable step is to constantly collect interesting ideas and design them in the form of specific tasks on the eve of the CTF.

As is usually the case, we divided the task into categories, so that the participant at least approximately understood what he was dealing with. In our case, we have identified six categories:

• Admin (4 task) - administration tasks in individual Linux terminals. They deserve a separate description, so about them - just below.
• Crypto (4 task) - tasks related to cryptology.
• Forensics (3 task) - tasks in the field of digital forensic analysis.
• Stegano (3 task) - steganography: hiding information in pictures, videos, etc.
• Reverse (5 tasks) - tasks for reverse engineering (or reverse engineering, if “in Russian”).
• Web (2 task) - it is required to find the flag by exploiting the vulnerability in the web application.

A different number of tasks in some categories may be evident: for example, there were as many as five tasks for reverse, and the web turned out to be deprived. There is a distribution (not too uniform) of the preferences and interests of the developers of the competition.

Admin tasks are done using a special type of Linux tasks for the Stepic Linux Challenge. In this task, the participant receives an individual linux server running in the cloud and a web terminal connected to it in his browser. When executing commands in the terminal, it is necessary to solve the proposed task. More details about how to arrange Linux-tasks for Stepic, have already been written . The technology is pretty cool, well suited for creating live admin tasks for task CTFs. In StepCTF, we offered participants several Linux administration tasks in which they needed:

• To find an opportunity in the terminal to read the flag.txt file on the server, in which standard well-known utilities, such as cat, less, tail, head, strings, vi, have been removed.
• Install and configure an Nginx TLS web server using the certificate issued by us.
• Recover deleted file from process memory.
• Configure the mail server and accept the flag letter from the testing system.

Assessing and determining the complexity of tasks is a sore subject of task CTFs. What seems simple to the author of the assignment may simply be inaccessible to the logic of the average person. As an example, the Passgen task, rated by the author at a minimum of 100 points, remained unresolved much longer than desired, while the web tasks of 200 or 300 points were decided by the participants rather actively. Thus, it is important to reasonably evaluate the tasks not only within the same category, but also as a whole: it is not the case if the reverse by 100 is several times more complicated than the task of the same cost, but from another category.

The main reason for dishonestly priced tasks is the lack of diligence of the organizers, what is there to hide. In our case, several laid out tasks were not really checked by anyone except the author, which led to the need to correct mistakes and change the cost of the task right during the contest. Already after the completion of StepCTF (better late than never) we have formed the following idea, which we dubbed “task-review”: the task is not published on the competition website until a certain number of organizers have passed it (say, two or three). This approach will motivate both solving other people's tasks and kicking people so that they decide yours.

Next time we want to try to apply alternative ways of estimating tasks. For example, there is an option to change the cost of tasks dynamically: in the simplest case, at the beginning assign all the tasks the same cost, and then reduce it with an increase in the number of people who decide. Or you can opt out of the cost in general and declare the winner the one who solved more tasks in less time (a popular approach in sports programming contests). We are still going to think about this; I want the evaluation system to motivate to solve problems more difficult, while avoiding the imbalance introduced by the author’s subjectivity.

In general, the competitions turned out to be a little higher than what was intended: in the announcement, we wrote that the CTF would be for beginners, but the dynamics of problem solving, as well as the feedback received after the competition, showed that it was difficult for beginners.

Flags

A couple of words about the flags - for the sake of what the participants decide the task. Having found the flag (in fact, just a string of characters), the participant can pass it on the corresponding page of the task, receiving points in exchange that raise the participant in the overall rating. According to past experience, we decided to fix the flag format (we chose regular expression / STCTF # \ w {6,} # /) in order to make life easier for the participants, which we reported through the Stepic alert system shortly after the start of the competition. However, this was not enough: we received more than one letter in which participants stated that they had decided the task and found the flag, but it was not accepted by the testing system. In fact, it turned out that people either did not see the message about the format of the flag, or they confused some similar characters (0 and O, capital i and lowercase L, etc.) when rewriting the flag in the answer field. We learned a lesson from this: a single flag format, announced in advance and in a prominent place, is good; zeros and other “problem” characters in flags are not very.

During the competition

Assume that all the tasks have been prepared, an acceptable number of participants have been registered, and the CTF has finally begun. This is not a time to relax, because the time has come to answer the questions of the participants, to publish tips, and also to correct mistakes in tasks that, unfortunately, can appear after the start. In the case of StepCTF, there were three ways to communicate with the organizers: comments on Stepicke, IRC chat on freenode.net and email. We should mention the first option separately, if only because the participants “left” the flags several times in the comments instead of the specially designed text field. We hope that this did happen by mistake, and not by malicious intent. Messages of this kind were promptly deleted by the organizers, but as a defect in the Stepic graphical interface, we recorded this behavior. The guys from Stepic confirmed that this happens during the course of the students, and promise to correct this defect in the interface in the future.

A variety of communication options is, of course, convenient for participants, but it complicates the work of the organizers a little. It is necessary to simultaneously monitor all options and try to respond promptly. When the number of participants is much larger than the number of organizers, then it becomes extremely difficult to answer everyone personally. The use of collective communication tools like comments on Stepic or IRC chat allows you to store the answers to the questions asked and not to answer the same thing twice. Nevertheless, we are firmly in the opinion that we should confine ourselves to some one way of communicating with the organizers, then the participants' questions will not be missed and ignored, and the organizers will live easier.

It is important before the start of the competition to agree on the organizing team about the time at which each organizer and author of tasks will be available for online communication. This is especially important when competitions last more than a day. There should not be a situation where the author of the task went to sleep, and there are bugs in his task, the participants complain about it, and no one except the author is able to correct them or answer questions. Ideally, at any time, one of the organizers should be online, and a method of prompt communication with any of the authors of the task should be established. Authors, in turn, should conscientiously support their tasks and promptly undermine on demand, even if it is 4 o'clock at night (the situation becomes more complicated when the organizing team is distributed, as it was in our case).

Another important point: you must promptly inform the participants about any changes and updates during the competition. For example, posting new tasks, correcting old ones, or adding prompts should be accompanied by sending appropriate notifications. Generally speaking, you need to stipulate the rules in advance: for example, participants have the right to know beforehand how many tasks they are waiting for and when they will be published.

After competition

It would seem that after the end of the competition, the organizers can safely breathe out and go to rest. It is important, however, to collect feedback from participants and arrange a debriefing - to document the mistakes made and the problems encountered, until nothing is forgotten and not lost. It may not be easy to gather strength, because there is no deadline ahead. A vivid example of this phenomenon is this article, completed later than we would like.

Conclusion

Most of the lessons that we learned (for example, about flags or task-reviews) were given in the relevant sections, in conclusion, I would like to highlight the main points.

• To hold competitions on the Stepic platform is possible. In this regard, StepCTF 2015 was a pioneer, but even taking into account all the difficulties, we can firmly say that the experiment was a success. Stepic provides a free platform for everyone, where you can conduct both courses and competitions, so this is a good option if you have ideas for tasks, but you don’t want to develop and maintain your platform.
• Much (or rather, very much) depends on the composition of the organizing team, their motivation and ability to take responsibility. Discipline and streamlined workflow is necessary both during the competition and in preparing and summing up the results. As a result, many of the problems that surfaced during the competition can be avoided; for example, most of the errors in the tasks could be caught if they were previously resolved by other organizers.
• Despite all the above, we should not forget that CTF is friendship;) Both the participants and the organizers want to get a charge of fun and new knowledge. It is impossible to avoid all mistakes and punctures; You can minimize their number, but the rest should not affect the quality and atmosphere of the competition.

It's nice to see that the CTF movement is actively developing, and an increasing number of teams are taking the liberty to hold their own competitions. However, many teams face the same tasks and difficulties during preparation. Therefore, we urge not only to participate in the development of new CTF-competitions, but also to share the experience of organizing and conducting them.

Links

• StepCTF 2015 competition page
• Repository with Tasks Terms and Solutions
• Stepic Linux Job Article
• VK team PeterPen

Authors of the article: rev112 psviderski