How we used Git, CI and code review in the learning process
The Academic University constantly introduces new approaches to learning. Programs, tasks and the process itself change in such a way as to provide the student with the most complete and relevant knowledge, and the teacher - the opportunity to try more effective methods. So in the last semester, instead of taking DZ in Java "on A4 and according to GOST," we with bintree decided to do everything "like a big uncle": use Git, CI and code review. In this article I will share with you the problems that have arisen, their solutions, the pros and cons of this approach, as well as some considerations for the future.
The question of preparing and checking home and control tasks arose before us as soon as we learned that we were going to lead Java. By that time, we had time to visit students at different universities and try on ourselves many approaches to verifying remote sensing, for example, “bring it on a flash drive” or “print on a piece of paper” (I won’t even bother listing the minuses of these two). At that time, the approach that was practiced at the AU seemed to be the most convenient for us: the student sends a solution on e-mail and enters into correspondence with the teacher. However, in this approach we did not like everything:
Another way to do homework in an AU was to use a shared repository; there was a separate directory for each student where he did homework. This approach partly eliminates the routine, but since the teacher still checks the solution manually and writes comments by mail, the student still has to wait for an answer in order to understand that there is something wrong with the solution.
')
A little thought, we decided that a bunch of "VCS + public CI-server + tool for review code" will save us from all the above problems, and the use of third-party services will also eliminate the need to configure all this.
It was decided to organize the repository as follows: for each task, start a separate branch, which contains a README with a description of what needs to be done, as well as the Maven project with the backbone of the solution and tests for it. A student at the beginning of the semester makes a fork of the repository and then simply switches to the desired branch, makes changes, commits and sends a pull request. PR is automatically compiled and tested; if the tests are not passed, then the student is required to complete the work; if everything is good, then the teacher checks the code and leaves comments or closes the PR and puts the assessment.
In contrast to the approach with one repository and the directory for each student, such an organization eliminates potential problems when merging changes, keeps the repository clean, and also protects against vandalism, although, fortunately, there has been no precedent for this in AU.
With the VCS service, everything was simple and straightforward: Github. Historically, both we and the majority of students had accounts there; this was the key argument. In addition, there it was decided to do a review of the code and leave comments.
With the CI service, everything was a little more complicated. Initially, we decided to use Travis CI and until some time it suited us all. However, over time, our requirements have changed: in addition to the open tests that are available directly from the repository with the assignment, we wanted to do some more closed ones - so that the students themselves would think about various subtleties and boundary cases. We created a closed repository, changed the build so that this repository was added as a submodule to the main one, and added encrypted variables with a username and password so that Travis could clone the closed repository. Everything was fine exactly until one of the students tried to make a pull request. It turned out that encrypted variables are inaccessible from PR builds and we didn’t check it. A quick search of the remaining CI services brought us to Semaphore CI. Unlike Travis, it supports downloading secret SSH keys that can be used in a build to access private repositories, which is exactly what we needed.
The first problem we encountered was, oddly enough, Git itself. The guys didn't always manage to send their solutions to Github. The reason for this was that after updating its fork and switching to a new branch, the main repository was installed as a remote, and not the fork. This subtlety is worth mentioning separately.
The second problem was the lack of commit and PR naming rules. As a result, already after the first homework, we received a pile of PR with names like "fix", "1st hw", "my homework", etc. All this was further complicated by the fact that it was almost impossible to identify a student by username, because not everyone had filled out the profile. For the subsequent tasks, we, of course, introduced strict rules, but for the first time we had to sweat.
The main disadvantage of this approach is that not all tasks can be written meaningful tests. For example, one of the tasks for generics meant writing its own functional java. The amount of code in this task was minimal, and its main charm was to come up with the right signatures for the functions. In order to test it qualitatively, you would have to invent complex hierarchies and check various tricky cases; this would take more time than checking the work manually. Also, tasks for which it is difficult and costly to write tests can be assigned tasks for multithreading.
Quite a lot of people also attribute the availability of other people's decisions to minuses - who wants to write DZ himself, if a neighbor has everything ready and even handed over. Personally, I do not think this is a minus, because anyone who wants to write off will still find a way to do it.
The experience with this approach can be considered positive. Despite some problems and irregularities at the beginning, by the mid-end of the semester everything was settled and worked as it should. It was convenient for us to check the solutions, and I hope that the guys were comfortable with them. In any case, according to the results of the semester, only one expressed dissatisfaction with such a method of putting a DZ in my group of ten people.
A logical development of this approach will be the addition of other useful services, such as automatic style checking or static code analysis. It would also be very convenient to teach Github to automatically assign a reviewer for PR. And in general, there is no limit to perfection.
Problems of standard approaches
The question of preparing and checking home and control tasks arose before us as soon as we learned that we were going to lead Java. By that time, we had time to visit students at different universities and try on ourselves many approaches to verifying remote sensing, for example, “bring it on a flash drive” or “print on a piece of paper” (I won’t even bother listing the minuses of these two). At that time, the approach that was practiced at the AU seemed to be the most convenient for us: the student sends a solution on e-mail and enters into correspondence with the teacher. However, in this approach we did not like everything:
- A lot of routine for the teacher: the solution must be transferred to your computer, run it, test it, update it and write all the comments in the letter.
- Additional waiting for the student and the load for the teacher, because the student can be completely convinced of the correctness of his decision only by sending it to the teacher and waiting for an answer from him.
Another way to do homework in an AU was to use a shared repository; there was a separate directory for each student where he did homework. This approach partly eliminates the routine, but since the teacher still checks the solution manually and writes comments by mail, the student still has to wait for an answer in order to understand that there is something wrong with the solution.
')
A little thought, we decided that a bunch of "VCS + public CI-server + tool for review code" will save us from all the above problems, and the use of third-party services will also eliminate the need to configure all this.
Work organization
It was decided to organize the repository as follows: for each task, start a separate branch, which contains a README with a description of what needs to be done, as well as the Maven project with the backbone of the solution and tests for it. A student at the beginning of the semester makes a fork of the repository and then simply switches to the desired branch, makes changes, commits and sends a pull request. PR is automatically compiled and tested; if the tests are not passed, then the student is required to complete the work; if everything is good, then the teacher checks the code and leaves comments or closes the PR and puts the assessment.
In contrast to the approach with one repository and the directory for each student, such an organization eliminates potential problems when merging changes, keeps the repository clean, and also protects against vandalism, although, fortunately, there has been no precedent for this in AU.
Selection of services
With the VCS service, everything was simple and straightforward: Github. Historically, both we and the majority of students had accounts there; this was the key argument. In addition, there it was decided to do a review of the code and leave comments.
With the CI service, everything was a little more complicated. Initially, we decided to use Travis CI and until some time it suited us all. However, over time, our requirements have changed: in addition to the open tests that are available directly from the repository with the assignment, we wanted to do some more closed ones - so that the students themselves would think about various subtleties and boundary cases. We created a closed repository, changed the build so that this repository was added as a submodule to the main one, and added encrypted variables with a username and password so that Travis could clone the closed repository. Everything was fine exactly until one of the students tried to make a pull request. It turned out that encrypted variables are inaccessible from PR builds and we didn’t check it. A quick search of the remaining CI services brought us to Semaphore CI. Unlike Travis, it supports downloading secret SSH keys that can be used in a build to access private repositories, which is exactly what we needed.
Pitfalls and cons
The first problem we encountered was, oddly enough, Git itself. The guys didn't always manage to send their solutions to Github. The reason for this was that after updating its fork and switching to a new branch, the main repository was installed as a remote, and not the fork. This subtlety is worth mentioning separately.
The second problem was the lack of commit and PR naming rules. As a result, already after the first homework, we received a pile of PR with names like "fix", "1st hw", "my homework", etc. All this was further complicated by the fact that it was almost impossible to identify a student by username, because not everyone had filled out the profile. For the subsequent tasks, we, of course, introduced strict rules, but for the first time we had to sweat.
The main disadvantage of this approach is that not all tasks can be written meaningful tests. For example, one of the tasks for generics meant writing its own functional java. The amount of code in this task was minimal, and its main charm was to come up with the right signatures for the functions. In order to test it qualitatively, you would have to invent complex hierarchies and check various tricky cases; this would take more time than checking the work manually. Also, tasks for which it is difficult and costly to write tests can be assigned tasks for multithreading.
Quite a lot of people also attribute the availability of other people's decisions to minuses - who wants to write DZ himself, if a neighbor has everything ready and even handed over. Personally, I do not think this is a minus, because anyone who wants to write off will still find a way to do it.
Conclusions and future plans
The experience with this approach can be considered positive. Despite some problems and irregularities at the beginning, by the mid-end of the semester everything was settled and worked as it should. It was convenient for us to check the solutions, and I hope that the guys were comfortable with them. In any case, according to the results of the semester, only one expressed dissatisfaction with such a method of putting a DZ in my group of ten people.
A logical development of this approach will be the addition of other useful services, such as automatic style checking or static code analysis. It would also be very convenient to teach Github to automatically assign a reviewer for PR. And in general, there is no limit to perfection.
Source: https://habr.com/ru/post/277597/
All Articles